Smart home occupancy and presence is one of most useful pieces of information in the contextual smart home, allowing it to accurately determine if someone is home, who is at home and also know which rooms they are in. This information is used by the Home Control System to make informed decisions and ultimately be able to predict events and take pre-emptive action. Occupancy and presence are one of the key defining features of the smart home. Without them it is pretty much impossible to move from simple home automation to a truly smart home. The real power comes from a single, logical model that incorporates multiple layers for location, proximity, occupancy and presence data though. When viewed as a whole, it is possible to infer much more and thus enable many more powerful features.
A key objective for our home is to be able to automatically update the status, which then alters things like target temperatures for rooms, etc. The biggest challenge is achieving the right balance between manual control (i.e. us telling the house we are out), scheduled control (e.g. a learning pattern that defines when we are likely to be in) and detected occupancy or presence of people in our home.
Devices like the Nest thermostat use a PIR to detect the presence of people and modify their behaviour but, a single PIR sensor is a very compromised approach, especially if mounted in a 'quiet' area. It is also not linked to other smart home sensors, devices and services. A 'whole home' approach is always going to provide a more useful and accurate solution, especially when it is linked to all other smart home capabilities and uses a hybrid technology approach to utilise occupancy and presence data from the all the technologies found in our smart home.
Firstly, some definitions. The words occupancy and presence mean different things to different people. To be clear, these are the definitions we are using:
Whilst presence is very desirable, it assumes people are carrying some kind of device or technology that allows you to identify them (short of using feature recognition, which is not really accurate) and this is very often not the case. Occupancy detection is much easier and it is just as important in the smart home. This is why we have adopted a model that combines both. Our software models identified individuals and their known devices and also supports an 'Unknown' person or many unknown people for occupancy. Unknown people can occupy more than one zone/room at a time, whereas identified individuals cannot.
At its simplest level, occupancy and presence detection in the home enables more advanced control of things like heating and cooling, hot water heating, etc. Occupancy applied at a zone level enables basic functionality such as automated lighting that doesn't switch off whilst someone is still in the room (e.g. in a toilet or bathroom).
The best possible outcome is providing reliable and accurate presence detection, to enable intelligent personalisation in the smart home. This could be entertainment that is personalised user experience and follows you through the house, personal environmental settings such as temperatures, lighting and even lighting colour. It can also include information and communications being delivered to our current location. Ultimately, we see our home making predictions and taking pre-emptive action to improve quality of life. We would also like it to deliver some personalised user experience serendipity.
The bottom line though, is that a smart home with occupancy and presence is incredibly efficient and convenient. The added intelligence and behaviour also adds some subtle style and a luxury feel.
Our contextual smart home also tracks how long people have been at home or away from home, as this is also very useful context. Knowing that the whole family has returned from holiday after two weeks will result in some very specific actions occurring.
Our approach is based upon a combination of many years work leading a team of researchers on future communications and online presence, combined with much research in our own @smartest_home. For many of the same reasons that we have adopted a hybrid technology approach to our Home Control System, we have taken a similar approach with presence in our home. This ensures a whole home context approach using all the information available for all of the technologies (and suppliers of equipment) in our home.
The initial focus was on passive detection of people and this allows us to have occupancy for 'Unknown' people. We can use door door contact sensors, PIR sensors, etc. to detect people and their location. This approach allows us to do real things with occupancy immediately and to add and extend as new technologies are then added.
Not everyone carries a Smartphone. We have guests and children in our home that have no technology to help identify them. It is essential that we therefore include passive approaches to occupancy detection as well as "active" presence detection. This approach is also extensible and future-proof. Apple's iBeacon technology is an example of a relatively new technology that could extend our presence and location capability.
Much like services with on-line presence, it is quite hard to be sure that someone is in the house (c.f. available on-line) with 100% certainty. The '100%' number also implies a guarantee that we can't really deliver on. Equally, we can't provide a guarantee that someone is not in the home because they may have just walked in before they could be detected (though door contact sensors make this extremely unlikely). Because of this we are modelling the probability of an individual person (or an unknown person) being in a particular room or zone as a probability between 1% and 99%.
We can know when a device associated with a person is connected but, even this is historical data (albeit very recent history) as it is based on when the check was last made. Because of this, we treat all occupancy data as being a historical. We model this as a significance that decays over time, so we store the time of each presence update using Java system time (in milliseconds). The decayed value can thus be easily calculated knowing the start value and the elapsed time since the event.
Presence also has different meaning depending on what sort of sensor or device was used to capture the event. We assign a different rate of decay to each source type and we are refining our model over time, to optimise these decay rates.
Ownership of a device enables us to associate its presence to an individual. As an example, we can see when our son's laptop is connected to the home network and we know that because only he uses this device, we can be sure it is him present in our home. There could be situations where he has left it on though but, we have assumed for now that this doesn't happen.
The biggest challenge with this ownership model is that when someone really does leave their 'owned' device on and at home, then they will incorrectly be shown as being at home and the house as being occupied. Because of this we have had be extremely cautious in assigning device ownership and we do not use this approach for example with children's personal Smartphones.
Whilst occupancy can be achieved in near real-time, presence is subject to greater time delays and there is some latency in knowing whether a space is occupied or unoccupied by a specific person.
Accuracy is more important than timeliness. Our software models and algorithms are aim to allow us to say with almost 100% certainty whether our home is occupied or not, assuming you give us some time to answer the question. In the real-world of smart home automation this is good enough, especially when using whole home context.
Location is an area where we are currently doing a lot of research. Mobile devices in the home provide occupancy, presence and proximity data but, we also model and are using them for location. The objective is to not just know that someone is within a zone but, to be able to pin-point their location within it.
One of the biggest challenges with occupancy and presence is in providing an accurate view of how many people are in the house. Occupancy and presence down to room or zone level goes a long way to answering this question but, it is a lot more complicated than this. We have now reached a point in our research where we can do this with a high degree of confidence but, we still have more to do.
Doors (or more specifically the door contact sensors) are the cheapest (if wired) and best source of occupancy events in the smart home but, they don't allow us to identify people. We have yet to find an example where we can link a specific person to a particular door with any level of confidence. The more doors with contact sensors, the more useful this information is. In our @smartest_home every single internal and external door has an invisible (hidden in the door frame or the door itself) contact sensor.
Doors become more important when you model relationships between doors and zones. Our 'wasp in a box' model only works if you do this. Some doors can also provide occupancy information within a zone, e.g. an ensuite bathroom door within a bedroom.
The use of door contact sensors like this obviously assumes that they are actually being opened and closed by people, i.e. not being moved by drafts, etc. It is also possible to have 'virtual doors' in the smart home, using beam break sensors. Breaking the beam is the equivalent of opening a door.
PIR sensors are a type of motion sensor and are the second best source of occupancy data in the smart home. They are also the most frequent source of occupancy events and we can see over 7000 such events in day from our wired PIR sensors (when our house is occupied all day). Because this volume of events gives the best view of occupancy, we have mostly implemented mostly wired PIR sensors. They give the best performance and best reliability. This eliminates any potential battery issues too.
Microwave motion sensors are similar to PIR sensors but use microwaves and the Doppler effect to detect movement. They have the advantage of being much more sensitive to movement.
Pressure mats are another simple source of occupancy data. They can be used on stairs (narrow versions are available for this specific application) but they are particularly effective when used close to external doors.
They should not be too close to a door though, or whilst entering the house the door won't be closed before the pressure mat is activated. In this respect the ideal installation for occupancy data is different to the one typically used when these are installed purely for security purposes.
In our contextual smart home, we are also using Tile trackers for detecting the presence of people. Typically, these are connected to keys which are always carried around by family members.
It is possible to use the signal strength and algorithms to more accurately locate people in the home with this technology but, people don't tend to carry keys around with them whilst in our home.
We are also using cheaper BLE tags to track people and objects coming and going from our home.
We have developed our own very low-cost bed occupancy sensor and have found it to be a highly accurate and useful source of both occupancy and presence data. It provides a very clear indication of a bed being occupied and works on both single and double beds.
Our cameras have movement detection capability and we also use this to trigger events around our home. We are also researching the use of cameras for face recognition.
Automated locks can often use technologies like RFID to identify people but, they can not be used for internal presence and occupancy as they are located outside the home. They can be used if events can be linked to provide added context though and they do provide reliable proximity information.
When someone presses a smart light switch in our contextual smart home to turn a light on/off or adjust its brightness, this generates an event that indicates occupancy with a very accurate location associated with it.
The smart mirrors are all a source of occupancy information in our contextual smart home and updated the nest occupancy model. So if a person is detected in front of our ensuite smart mirror, our smart home knows that the ensuite, main bed room, upstairs and house zones are occupied.
We use a Raspberry Pi to monitor Bluetooth and identify devices in range.
We also monitor Wi-Fi and Ethernet networked devices to detect the proximity of an individual's device(s). This has proved amazingly accurate in the case of the adults in our home (who always carry a phone with them) but is not a lot of use with our children (who do not).
We have other Wi-Fi network connected devices in our home that can signal the proximity of people such as iPods, tablets, etc. We also have Ethernet connected devices that have know locations and can provide occupancy information. In total, we now have over 100+ IP-networked devices in our home.
The most important events are when a devices has just appeared online (at home) but, in practice Smartphones appear to regularly connect and disconnect from Wi-Fi networks whilst stationary in the home. Our algorithms can handle these small breaks in connectivity but, this has the effect of adding latency to presence results.
We are currently using the presence of our Sony PlayStation 4 games consoles on our home network to signify occupancy of our lounge and ensure that lights stay on, etc. when the PS4 is in use.
Our @smartest_home also spots our Wi-Fi connected TomTom Go 6200 satnav as we pull up onto the drive and uses this to turn on the drive lights.
In our smart home, connected appliances like our smart toothbrush are also sources of timely occupancy information.
Our Ender 3 Pro 3D printer is a source of occupancy information, when it is switched on or off.
At CES2015 the Wi-Fi Alliance Wi-Fi Aware which would enable indoor location based services:
During our research and development we have also developed some bespoke occupancy sensors for specific applications:
We have used the JSN-SR04T waterproof, ultrasonic distance sensor to provide very 'narrow beam' occupancy detection in very localised environments or small spaces such as a shower cubicle.
We have also used the Sharp 2Y0A21 infra-red distance sensor to provide very 'narrow beam' occupancy detection in very localised environments.
We have already modelled zones in our home. This project required us to revisit some of our thinking on zones though, as occupancy and presence needed a more detailed understanding of the relationships between them:
Zones have an attribute that defines whether they are 'Open' or 'Closed'. There is also an attribute to model whether they are occupied (wasp in the box). Our zone model also supports the concept of 'nested zones'. This is because we have rooms within rooms. If someone is in the 'Main ensuite Bathroom' then they are also in the 'Main Bedroom' zone and also within the 'Upstairs' zone and thus also within the 'House' zone.
Some zones can't be closed and a flag is used to define this attribute. This means a zone that has no doors or beam break sensors that define its boundaries.
Our design has to model the fact that doors are two-way devices. Opening a door or breaking a beam-break detector has implications on the zones on both sides of it.
Our whole smart home is built around events, which are both human and machine readable and the basis for our unified communications protocol. These allows us to use technology abstraction and implement a hybrid technology Home Control System.
Occupancy and presence are also event driven and we have an event type of 'Presence' to support them. As part of this project we have added three new zones: 'Bluetooth', 'LAN' and 'WiFi'. The event object is the source or occupancy and presence events, which equates to the MAC address of a networked device.
One simple and clever technique we use in our home is the 'wasp in a box' algorithm. If we detect a specific person in the home or an unknown person, we can assume they are still inside our home until one of the external doors (front, back and conservatory doors in our house) is opened. We model this in our presence system and capture and store this information. Once a perimeter door is open, we can not know for sure who has left the house and have to assume that everyone has.
Our testing has shown that this simple algorithm massively improves our occupancy knowledge. It completely solves the main issue with occupancy detection, which is when people go to sleep at night, there is no obvious movement (and therefore no events) to update occupancy.
This algorithm only works if you have contact sensors on all of the perimeter doors. The great thing about it though is that it can be implemented very cheaply, with a single sensor in the home (PIR or door contact sensor) and a sensor on each perimeter door. The more sensors you have though (especially ones just inside the perimeter doors), the better it works. Up to a point obviously!
This same approach can also be applied to an individual room (e.g. a toilet) and used to ensure a convenience light doesn't turn off with someone still in it.
This approach also works with our 'nested zone' model, so that we can use occupancy of one room to infer occupancy of a zone or room that encompasses it.
This algorithm is incredibly simple but very powerful. It means that when you walk into a room and shut the door, if my smart home then detects movement, it knows the room is occupied until the door is opened again. In practice, this means you never get left in the dark, even if you stand perfectly still. Applied at the home level, it means you can know your home remains occupied whilst all exits are closed.
The algorithm:
Our presence model assumes that we have a number of known devices and that each device has a number of interfaces, each with a unique ID (MAC address), e.g. LAN, Wi-Fi, Bluetooth, etc.
In our home we have many devices and some of them can be considered unique to a person in our household and assigned an owner. Many of them are not significant and are not assigned an owner. Quite a few are significant as sources of occupancy information but are not owned or uniquely associated with a person. These devices are assigned an owner called 'Unknown'.
Many of these devices are mobile devices such as Smartphones, tablets and laptops. Some have a fixed location that can be used to determine which zone (or room) they are in. Some can even be associated with a very specific location within a zone. We are still trying to work out the best way to model location within a room or zone and will be experimenting with different solutions in the coming weeks. The key thing we want to be able to infer is distance between devices and known objects, i.e. centimetre level proximity.
For now, we have assumed that if a device has a zone or location it is not specific to a given interface. In future we may add a zone and location associated with each device interface though.
One of the key things we want to be able to do is know is when someone has returned to an empty home and how long the house has been empty. This information will be used to make certain announcements.
Our implementation and experiments so far have shown that we can use proximity to infer presence with a very high degree of accuracy and this hugely improves the high-level view of who is at home. This is based on the simple premise that if a known person is in proximity of the home for more than a few minutes, then they are probably within it. This is where the wider context of the contextual smart home helps.
The challenge is in defining what period of time must pass before we can do this. The shorter the time period, the more timely the presence information but, there is a higher chance of making the wrong assumption. We are currently capturing real-world data, to determine the optimum time period.
Our research has shown that things like occupancy and presence need to be tried and tested in a real home to get a full understanding of how well they work and implications and issues. Whilst you can simulate elements of this in a lab, it is very hard to model numerous zones and the relationships between them, whilst collecting high volumes of occupancy and presence data with a real-world distribution.
It also shows the value of a 'whole home' approach to occupancy and presence. Our approach provides complete context and understanding, to get an accurate view of house occupancy and also detailed zone views. Whilst you can buy 'smart' devices that claim to know if you are home or away, we would question whether this is really the right way to get an accurate answer to this question.
Whilst many think presence is the most important aspect of a smart home, we have shown that occupancy, presence, location and proximity are all linked and deserve equal weighting in the smart home. It is the combination of all four concepts that really make a smart home smart.
As with most of our projects, this isn't about the technology and software. None of the stuff here is exposed to people living in our home and none of it needs to be. Accurate and timely presence detection and occupancy knowledge enables a smart home to be much more efficient and much more intelligent in its behaviour. It simply makes our contextual smart home more usable and much nicer place to be.
In undertaking this project, we have collected some really interesting statistics about our home and how we use it. The appearance of devices on networks and the detection of people in rooms has enabled us to build a detailed profile of usage (down to zone/room level) over time. We can see variations throughout each day, a typical working week and also expect to see variations across the seasons in each year. We can see things like who is at home most often and spot patterns in behaviour and usage. These are all valuable inputs that can be used to add learning capability to our smart home and this is now a focus of our research work.
