Presence monitoring using iBeacons

One of the goals of Thuis is optimizing it’s own rule engine based on actual activity in the house. Although time is to short to actually start the optimization, with this blog we’ll start collecting presence data using iBeacons. As a bonus we solve the Welcome Home use case!

iBeacons monitoring, ranging and indoor location

Estimote BeaconsiBeacons (in my case by Estimote) are little Bluetooth LE devices which on regular intervals broadcast their identifiers. Mobile apps can use this as a way of determining their location. They can be used in several ways. The three most common ways are:

  • Monitoring – the app gets a signal whenever it enters the region defined by one or multiple beacons (aka when it receives a packet sent by the them). The app gets a small amount of time to do some work, for example notify the user. This works even when the app is terminated.
  • Ranging – When the app is active it can listen to all beacons around and based on the signal strengths make an approximation of the distance between the beacon and the phone.
  • Indoor location – Estimote developed another layer on top of ranging. Based on several beacons (at least one per wall) and the sensors in the phone it can determine its location within a space. This works quite precise, but it uses more energy than the other methods. Also it can only be used when the app is in use.

I experimented with all three to see which way is most usable for Thuis. Indoor location was a bit of a hassle to set up (it involves walking lots of circles close to the walls through the house, which is not easy when there is furniture!), but the result is impressive. Not being able to use it in the background made me choose monitoring as the main technique.

ps: for the Android users out here: although I’m talking about iBeacons (by Apple) there are very similar technologies available for other platforms. For Android that’s Eddystone, which is the Estimote beacons can broadcast as well.

Presence monitoring

To optimize the Thuis rule engine it needs to have knowledge of what happens around. One of the useful things to now is who is where and when. We’ll use presence monitoring to determine who is currently where in the house.

Hardware set up

Estimote iBeacon settingsWe want to know who is where on a room-level, so who is in which room. To identify different rooms we’ll deploy a iBeacon is every room we’re interested in. We’re using 3 Estimote Location Beacons and 3 older Estimote Proximity Beacons. In the entrance, living room, bedroom, kitchen and office 1 beacon is installed at the center of the outer wall (as far as possible from other beacons). The signal strength of each of them is tweaked ranging from -30dBm (±1.5 meter) in the smallest rooms to -20dBm (±3.5m) in the bigger rooms. Each beacon is configured to broadcast its ID approximately 3 times per second. These values will likely be optimized further over the coming period.

Initial version

I started by using just region monitoring as this is very easy to implement. To make my life easier I started with some structs describing the deployed beacons together with some helper functions. The interface of the struct looks like:

And the beacons are defined as:

All code related to beacons takes place in the class BeaconManager. It sets up monitoring in the init method and implements the delegate methods for the ESTBeaconManagerDelegate. It keeps some information about which is the current region for this phone and for each beacon when was the last time this phone entered or left the region. The initial version just logs information to the console based on activity.

This already works quite well, but as we’re working with wireless signals there can be mistakes. For example a packet from another room reaches your phone and therefor your presence is adjusted. Or some packets get lost and cause your phone to think it left the region.

The latter is the reason that for resetting the currentPresence to outside we use the home region. This region consists all beacons, so the change of a false positive is smaller. It does however still happen every now and then.

Improving accuracy with ranging

To make presence monitoring more accurate we can combine monitoring with ranging. When the app is in the background and enters a region it’s woken up and gets a small amount of time to do some work. We can use this time to start ranging beacons, and with the more detailed data about the beacons around us make a better approximation.

To start ranging we have to adjust the didEnter method, instead of updating currentPresence and lastLeftOrEntered we start ranging:  beaconManager.startRangingBeacons(in: BeaconIDs.home.asBeaconRegion) . To receive the results we’ll implement the corresponding delegate method in which we’ll update the values:

Notice we’ll take the first beacon (with a known proximity) from the given list. The SDK returns them ordered from close to far away, so we’ll always use the closest beacon. Our value of currentPresence got a lot more accurate now!

Based on how it performs I’ll do some further optimizations. One thing I still have to do is make the values of  currentPresence and  lastLeftOrEntered persistent, so they will survive a termination of the app.

Publishing presence

As always we want to publish our data through MQTT, so the other Thuis nodes can use it as well. In MQTT User Interface components for iOS we added MQTT to the app, so we can build on this. Whenever the currentPresence value changes we’ll have to publish a message. This means we have to update both the didRangeBeacons method and the didExitRegion method.

didRangeBeacons is updated like this:

And to detect someone leaving the house we change didExitRegion, here we only publish when the specific region is home:

Note that the name in the topic is still static, I’ll make it configurable in the app later.

Walking around slowly through the house gives the following MQTT messages:

Welcome home

Based on the same events we can welcome a user home as well and directly give him a useful action. I could directly turn on some lights for example, but I rather give the user the choice. So we’ll send the user a notification with an action. We’ll start with a single action, but later multiple actions can be added depending on for example the time or person.

When we get home we often watch an episode of a TV series (currently we’re watching The Mentalist, very nice show!), so the action of choice will be turning on the home theatre system.

Sending a notification is easy. In the BeaconManager we create a function for it:

We’ll call it from the didEnterRegion method when we enter the home region. To avoid getting too many notifications in case of exiting and entering the region by accident we’ll add a cool down period of 5 minutes. This looks as follows:

The result is you receive this notification when you arrive home:

Notification: welcome home!

To make it work there is one more thing to do and that’s implementing another delegate method, this time in the AppDelegate:

So when the notification action is used it will publish a MQTT message which enables the Home Theater scene and you can directly start watching your favorite series. How the home theater scene works will be the subject of the next blog!

 

Installing a Z-Wave dimmer

One thing I forgot to show you earlier is how to install Z-Wave devices into your house. In this post I’ll install a dimmer module in the bedroom, which will be part of the wake up light.

Materials used

The materials I’m going to use are the following:

Used materials

  • Fibaro Dimmer 2 – the actual Z-Wave dimmer
  • Fibaro Bypass 2 – needed when using LED lights
  • Philips Dimmable LED – 6W/470lm (comparable to 40W), warm white
  • Jung 535 EU – 2 push buttons
  • Some electrical wire

Installing the dimmer step-by-step

Now we have all materials, let’s start the installation. There are two possibilities to connect the dimmer, depending if you have 2 or 3 wires available. In our case we just have 2 wires available behind the outlets, so we’ll have to consider the following wiring diagram:

Wiring Diagram

We’ll go through the installation step-by-step.

0. Cut the power

For your own safety always cut the power when working on your lights. Find out in which group your light is placed and disable the power for this group.

1. Prepare the dimmer

Prepare the dimmer and push buttons by connecting them together. It’s best to do this now, as while you’re not working on the wall yet you have plenty of room.

Prepare the dimmer

2. Install the dimmer

Now install the dimmer and buttons in place and connect the live and switch wires. Push everything inside to check if it fits, but don’t close it with screws yet.

Install the dimmer

3. Install the bypass

As we’re using a LED light bulb we need to use a bypass. This makes it possible for the dimmer to function on low loads. It’s best to install the bypass close to the light itself, so we’ll install it above the light. (in my case this is also the only place where it fits, as space behind the buttons is very tight) 

Installation is easy as you just have to connect both ends to either side of the bulb (direction doesn’t matter). Here I’m using a screw terminal. The black and blue wires will go to the light itself.

Install the bypass

4. Connect the light

Now connect the light itself and screw in a light bulb.

Plug in light bulb

5. Include the light in Z-Way

The wiring is ready. You can now enable the power again. To use the light we have to add it to the Z-Wave controller: Z-Way. For this you go to http://server-ip:8083 and then move to Settings > Devices. Here you click the Add new button. Click the Start inclusion button. You’re controller is now in inclusion mode. To include the new dimmer you have to activate it; for this dimmer you can triple click the button connected to S1.

Z-Way inclusion

After a while inclusion should be finished and you can test the light!

In my case the interview part of the inclusion could not finish successfully. This is a known problem with this dimmer and Z-Way and hopefully will be fixed soon. The basic features of the device are luckily working as they are supposed to.

6. Test!

And there was light:

And there was light!

By clicking the S1 button you can turn on and off the light. By holding it you can dim it. A nice surprise was that this is the first combination of dimmer/LED which doesn’t make any noise while dimming!

7. Finish up the buttons

Now that we verified the light works, the only thing left is finishing up the buttons. Screw it on its place and click the buttons in their socket.

Install buttons

Add the new light to Thuis

The physical part is done, but we of course want to use the light within Thuis. For this we have to do a few steps as well. As described in Publishing activity from Z-Way to MQTT we name the device and add it to a room. We also add the tag mqtt as that’s how I configured the MQTT app for Z-Way. The light is now controllable through MQTT.

Next is to add it on the Java side in the Core. This is described in Core v2: A Java EE application. We add the device by adding a line to the Devices class:

The bedroom light is now ready to be used in any rules, or to be controlled from other devices as the iOS app.

 

Final implementation UI design

From my holiday location at the coast of Bulgaria I finished up the implementation of the UI of Thuis. It works nicely on both the iPhones and the iPad on the wall. This blog will give you a demo and bring you up-to-date with some of the changes I made since the last post.

iPad on the wall

Adding the Slider Tile

I already implemented the UICollectionViewController and several tiles in the last blog post, however it was lacking the SliderTile implementation. For this several changes were needed, for example a tile should be able to span multiple columns to create enough space for the slider. For this a new property is added to Tile called columns. It now looks like this:

UICollectionViewController can handle cells of different sizes,  so making sure they display correctly is luckily easy:

Now we can actually implement the SliderTile itself. First we add the slider to the UI:

TileCollectionViewCell.xib with Slider

The TileCollectionViewCell class got some more functionality now. To make things easier I moved some of the logic from the TilesCollectionViewController to here. It now takes care of deciding what to show and what to hide. For example the value will be hidden when there is no value (for buttons) or when a slider is used (the slider already visualizes the value).

To handle changes of the value when someone uses the slider a delegate method had to be added. The slider is linked to the delegate method in the TileCollectionViewCell, but this delegates the actual work to the SliderTile. This class also has some properties defining the slider, such as the minimum and maximum allowed values, and takes care of updating the slider when a MQTT message arrives.

Adding a optional postfix to values

Some of the InfoTiles show the weather, and for the temperature it’s a lot clearer when we include a degree sign next to the value. So I’ve added an optional postfix to the value of a Tile. This is a simple String property which is added at the end of the value before display:

Changes to support iPad

The iPhone and iPad have a very different screen size and of course we don’t want to duplicate any code, or add checks everywhere. Luckily for us Apple solved most of this by providing adaptive layouts and layout constraints. We’ll use the latter to slightly change the layout of the TileCollectionViewCell to adapt to the bigger screen. Apple differentiates between devices by using so-called size classes, which are very well described in The Adaptive Model. There are two changes we have to make: change the icon size and the width of the slider. The icon will be 45 points on compact-width devices and 70 on regular-width devices. The slider will get a width of 200 points on compact-width devices and 400 on regular-width devices.

Determining the size of the cell was already working well on all devices as it’s using a size based on the size of the superview. You can see the implementation of the slider above.

Demo

The implementation of the design Marina made for the Thuis app is now implemented, and how better to show it then with a demo:

In the demo you can see the different screens on both an iPad and an iPhone. At the bottom you can see the MQTT messages being send and received. Notice that when clicking a button a message is send to the set-topic (for example Thuis/device/living/moodTop/set) by the iOS app. Then a message is received on the status topic for that device ( Thuis/device/living/moodTop), which is sent by Zway.

When a scene is triggered (in this case Mood, which are the mood lights in the living room) you see the Core sends a message for each device in that scene. The status is also reflected on the iPhone. The slider for the main light in the dining sends out messages every time when you pause for a tenth of a second.

This should give you a good overview of the app and its design. Off to the next use case!

MQTT User Interface components for iOS

How useful would a home automation system be where you could only interact through MQTT messages? Not very, so that’s why we’ll start working on the User Interface today. The goal is to create some generic UI elements, which can be linked to the MQTT topic. The element will display the current status clearly and will let the user interact with it.

For now there will be three types of UI elements, which can be placed in a grid of tiles. These elements are:

  • Button – a simple push button. The background visualizes the state and tapping it will toggle the state.
  • Slider – a slider to be used for e.g. a dimmer module. The position on the slider shows the current state. Minimum and maximum values can be defined.
  • Info – a tile without any interaction. Just displays the current state of for example the temperature.

Each tile can have some properties defining how it looks. Options include a title, an icon, a value and the selected state.

ps: if you’re familiar with Swift and UIKit, you might notice that – for the purpose of learning – I’m using Swift 3 and iOS 10. They are still in beta and will be available in fall. 

MQTT Client

Before we can define the UI elements, we need a connection to MQTT. For this we’ll use Moscapsule, a MQTT library written in Swift. Making a connection is quite simple. We do this by creating a configuration, adding some callback blocks and opening the connection:

As you can see we don’t do anything yet in the callbacks, except for logging. We’ll arrange that later; let’s firstly create the UI elements themselves.

Tiles: data model

All types of tiles share some attributes. One of these is an array of MQTT topics to which this tile will be subscribed. The common attributes are defined in the base class Tile, including an enum for annotating topics and a protocol for making a tile selectable:

For each type of Tile a subclass exists. They describe any additional properties needed, plus they define what happens when a MQTT message is received. Let’s take the button as an example:

User Interface

The model doesn’t say anything about how the tile will look on screen, it just focusses on managing the status. So what about the UI? For that we use the TileCollectionViewCell, which is a subclass of UICollectionViewCell. It is defined using a nib-file and structured by using Stackviews:

TileCollectionViewCell.xib

The corresponding class is a simple subclass of UICollectionViewCell which takes care of the border radius and hides the value label if there is no text defined (due to the Stackview that results in the icon being centered above the title). The heavy lifting happens in the TilesViewController, which is a UICollectionViewController. It has a property to hold a list of Tile objects. To high light a few parts: in viewDidLoad all tiles are subscribed to their respective MQTT topics:

It also subscribes itself to the same topic, so it can reload the UI when new messages are coming in:

The only part missing is the interaction. When you select one of the cells it will publish a message to the defined ‘set’-topic and update the internal state of the tile:

Putting it together

To put it together we create multiple instances of the TilesCollectionViewController and add them to a UITabBarController. Each instances overrides a single method defining the tiles. For the icons we use a library which enables us to use IonIcons instead of images. The home screen displays the most-used tiles:

This brings us the result of this blog post: a working iPhone app which communicates with Thuis through MQTT.

Screenshot

When messages arrive on the status-topics, the selected-status of the button-tiles and the value of the info-tiles are updated. When one of the button-tiles is tapped a message is send to the set-topic with the new status. As described in Publishing activity from Z-Way to MQTT this is already picked up and executed.

You probably do notice that it looks very plain yet: we’ll fix this in a next blog post. We’ll also add some more controls, like slider, which can be used for dimmers and a volume control.

Status update

I’ve been busy with a few other things, so that’s why it has been silent on the IoT front. As I also just past half of the blog posts, I thought this would be good time to reflect on the plan and report the latest status. In this blog I’ll go through all projects/libraries and use cases presented in the plan and show the status. Some to-do’s are marked italic: these might happen after the challenge deadline.

Open Source Projects

These are the projects that I’m making available as open source on my GitHub account during this challenge. They are all set up so that they can be reused by others in their own home automation projects.

Zway-MQTT

80%

Z-WavePlusProduct_150As described in Publishing activity from Z-Way to MQTT messages are published for each status change. It also subscribes to the responding topics, so you’re able to turn on and off devices through MQTT as well. The topics and devices used are completely configurable. With this all major functionality is done.

To-do:

  • Publish a message on scene activation (e.g. used for each secondary push button on the wall)
  • Get it published on the Z-Way App store, already uploaded June, but still no response
  • Publish energy usage

Zway-MQTT on GitHub

Chef-Zway

100%

ChefMaking sure I can use Chef to fully install the Raspberry Pi 3 I needed to update a few recipes, but also create a completely new one for Z-Way. This was a major hurdle and took more time than expected, but I learned a lot from it. In Cooking up the nodes: Thuis Cookbook you can learn more about it.

Chef-Zway on GitHub

Plex-client

30%

plexPlex doesn’t allow me to add a plugin directly in the server, but there is an API and WebSockets for status messages. For now only some research has been done and some models set up. It will be implemented as Java library, likely with a similar set up as I’m using for integrating Java and MQTT.

To-do:

  • Set up projects
  • Implement models and client code for API
  • Implement models and client code for WebSockets
  • Forward events through CDI and MQTT

CEC-CDI

50%

The library for using CEC (Consumer Electronics Control) in Java was developed about 10 months ago and already performs the most common functionality (monitoring stands-status, turning on/off devices, changing volume and changing outputs). However it should still be integrated with the remainder of the system.

To-do:

  • Integrate library with Core
  • Forward messages through MQTT

CEC-CDI on GitHub

MQTT UIKit

20%

interfacebuilderThe work on the MQTT UIKit for iOS is just started and will be the subject of the next blogpost. The goal is to provide reusable UI elements which update their status by subscribing to a MQTT topic and are able to send messages as well.

To-do:

  • Implement several UI elements (button, slider, info)
  • Integrate them with MQTT
  • Build an app around them

Use Cases

Light when and where you need it

80%

Multi-Sensor

Sensors are placed in both the kitchen and the entrance room. The Core knows about them and as described in Core v2: A Java EE application rules are defined to turn the lights in those rooms on and off depending on movement and time. This works pretty well already!

To-do:

  • Further optimize the rules
  • See if improvements can be made by using iBeacons

Welcome home

The implementation of this use case is mostly dependent on iBeacons being in place. As they are finally delivered, I can start setting them up.

To-do:

  • Set up iBeacons
  • Integrate them with the iOS app (which will publish MQTT messages about their current location in the house)

Home Cinema

30%

Home Cinema

The Z-Wave hardware for the home cinema is in place (using a 6-socket PowerNode), so they can be turned on and off. As mentioned above the Plex and CEC libraries are work in progress. When these are there we can make a full integration.

To-do:

  • Finish Plex and CEC libraries
  • Set up a Raspberry Pi for the CEC communication
  • Integrate them with the Core
  • Add and integrate a DIY ambilight

Mobile & On-The-Wall-UI

15%

iPad on the wall

Work on the iOS UI elements is just started. Further development of the iPhone and iPad apps depends on this. The iPad is already mounted on the wall though!

Todo:

  • Finish MQTT UIKit
  • Create an app for the dashboard (both mobile and On-The-Wall)
    • Provide basic actions (turning on/off device, triggering scenes and providing information)
    • Show the current movie playing in Plex and pause/start
    • Add speech commands
  • Create a custom app for the kitchen (either iPad or web)

Wake-up light

Work has not started on the Wake-up light, mainly because one of the required components (the MOVE) is not delivered yet. As it’s an Indiegogo project, it’s not certain when it will be delivered. I’m not counting on it for the duration of the challenge. I will start on the wake-up light with only bedroom light gradually turning on.

To-do:

  • Sleep Cycle doesn’t have a web hook available yet, so it’s still needed to set up a Philips Hue bridge
  • Install Z-Wave dimmer in the bedroom
  • Install and integrate the MOVE

Manual override

80%

Wall Switches

Most lights can already be switched manually using the buttons on the walls. Some of them should however be switched using the secondary button, which does a scene activation. I have to add support for this to the Zway-MQTT.

To-do:

  • Add support for secondary buttons in Zway-MQTT

Energy monitoring & saving

10%

For energy monitoring I only did some research yet. InfluxDB seems to be a good candidate for storing the data. As time is running out, I’m not sure if I’ll be able to fulfill this use case.

Todo:

  • Let Zway-MQTT publish energy usage
  • Integrate YouLess to record total energy usage of the house
  • Create reports based on the usage

Summary

Up to now I mostly set up infrastructure and backend code. It’s now time to really start implementing the selected use cases. This is what I will focus on for the upcoming weeks. Although there is only 1 month left until the deadline, I’m confident I can implement most of them on time!