/building · 2020

Apartment Lighting Automation

raspberry pi · camera · python

I set up Raspberry Pi cameras with an incremental object detection server that controlled my Noon smart switches. I ran some interesting experiments trying to create a recursive self-improving home automation system.

A Raspberry Pi Zero with a camera module and two lenses, taped to a ceiling

I had spent many years playing with Arduinos and motion sensors, which use infrared beams to answer the question "did something move?". Raspberry Pis and the Raspberry Pi camera module allow you to ask a much more powerful question: "what moved?".

The reason is because when a human moves and then the camera no longer detects a human for a few frames, you can be pretty certain that the human has left the room and you can turn off the lights. This simple heuristic produces a thoroughly reliable "smart home" system without any further complexity.

Synthesizing multiple camera feeds#

I installed eight Raspberry Pis with RPi camera modules and TensorflowLite around my apartment, and had them streaming their video feed to a centralized Mac Mini server. The Mac Mini used a reverse-engineered client to hit Noon Home's APIs directly.

An aside - if you worked at Noon Home around the years 2015 - 2020, why did all your API server names involve Star Wars references?

Two Raspberry Pi boards and a camera module mounted on a bracket, with cables running along a wall
A compact Raspberry Pi with a camera module and two lenses, mounted flat against a ceiling
On the left is a rudimentary setup where I literally screwed Raspberry Pis into the drywall. On the right, a rudimentary bracket-mounted unit that I built from off-the-shelf hardware.

Incremental detection#

Running a detector on every frame of every camera is the obvious approach, but it is also a waste of a Raspberry Pi. Instead, I modified the TensorFlow image classification model to operate on the subset of pixels that are distinct from a differential dataflow calculating the linear regression of RGB values from previous frames. Little did I know that I was building a highly neutered version of modern LLM technology at the time.

The expensive work only ran on the segments that changed over a certain period of time, and the rest of the scene keeps whatever the detector concluded about it last time.

That is what makes the useful behavior affordable. A person who has not moved in twenty minutes is still a person in the room, and the lights stay on, because the state is attached to the scene rather than being re-derived from motion.

The switches are Noon, which handle the actual mains side and expose the room to the network, so nothing in this project touches a wire in a wall.

cd /building