For the SmartHome project, we’ve taken a step back to better understand the problem space and come at solutions based on research and evidence. As a team we spent last week interviewing people in their homes, with questions on a theme of freedom and independence. We choose early teens (12-15yrs) and post-retirement folks as a demographic that might have useful insights into this topic.

As a software engineer this has been an interesting process. There’s a temptation to jump into any new project with both feet and start hacking on code; starting with simple successes and working iteratively to add features and meet requirements. And we’ve done some of that in order to get familiar with what we expect will be our prototyping platforms - the raspberry pi and ESP8266. But the more we looked at the problem the more it became clear that we weren’t yet sure what technical questions the project would ask, let alone how to solve them. In the meantime, our team was trying to figure out a better vision for the smart home that would align with Mozilla’s values and potential solutions in a broad and confusing product space. None of us were in our comfort zone, so we made a determination to put roles aside and roll up sleeves and muck in. We’ve posted craigslist ads, we’ve had Skype calls, house visits, coffee shop rendezvous; interviewed, transcribed and now begun to process input from 15 different people.

It turns out that a curious mind, a knack for spotting patterns, analysing outcomes for the motivations and circumstances that produced them - these are the stock-in-trade of any software engineer - and they are skills that work just as well in user research and exploratory product definition as they do in software development. And perhaps more important than that, before we are engineers, we are people. We have families, jobs, aspirations, frustrations and concerns. We were young and hope to grow old. Talking to people this last week has been a great reminder that it is people that solve problems, not code.

With no keyboard or pointer inputs, ensuring the Smart Mirror can be restarted and booted up entirely automatically was high on my priority list. Once installed, I can’t startx or click on any icons; it needs to bring up all the backend services and the dashboard to leave it in a working state without any user intervention. That lead me down a merry path and was (for me) the trickiest part of this project.

Here’s the moving parts:

• The kernel and OS itself, with networking and other key systems
• The display and window manager - the subsystems that allow me to put my dashboard up on the screen
• The mosquitto message broker
• The gpio listeners
• The web server
• The browser, which should load up my dashboard URL

I went through lots of helpful posts and projects on creating linux kiosks to figure out potential approaches. While the mirror isn’t really a kiosk - a kiosk usually has keyboard/pointer/touch user input - its a reasonable match up. After a few false starts trying Firefox/Iceweasel and Chromium kiosk options, I settled on the approach outlined in this Dashing-Pi page. This eschews the LXDE desktop environment entirely and uses nodm and the matchbox window manager to boot into the browser with the minumum of unecessary fluff inbetween.

Orchestrating startup is a bit fiddly even so. First, nodm is configured to startup with/as the ‘pi’ user. The rest of the graphics/display related startup is then in a script copied to /home/pi/.xsession, which starts up the matchbox display manager, and the Uzbl browser to load the dashboard. For the backend pieces, Raspian uses the init.d system, so we install scripts in /etc/init.d/ to start up mosquitto, pm2 (which manages the node.js server(s)) and scripts to relay GPIO events as MQTT messages for the rest of the system.

That done, I can plug the thing in and in just a minute or so it brings up the dashboard on screen and responds to sensor events. The Uzbl browser is a wrapper around WebKit. It supports commands via a socket, which means that once up, I can ssh to the rPi and remotely refresh the page, navigate to other URLs and so on which has proved valuable during development as I have none of the traditional inputs (e.g ctrl+r on the keyboard) to accomplish this otherwise.

One of the goals of this project is to get my hands dirty with integrating data and events coming from hardware (sensors attached to the raspberry pi or functions of the pi itself) and content in the browser. The raspberry pi ecosystem leans heavily on python, so while I was already using node.js for the web server, I didn’t want to get tied into a system which limited my implementation choices for publishing or consuming data. I wanted lightweight, language-agnostic messaging.

My first thought was to simply use the filesystem: writing to some files and watching for changes. That seemed fraught with potential problems and reinventing too much of the wheel. So I went shopping for messaging solutions.

After looking into ZeroQ and ZeroRPC, I settled on Mosquitto and MQTT. The Mosquitto project is a message broker (think central hub for dispatching messages to subscribers) that implements the MQTT protocol. As well as the broker (available as a raspbian-ready package) it also has command-line clients for publishing and subscribing which proved useful. There are MQTT implementations for both python and node.js (as well as a bunch of other languages.) The pub-sub paradigm turns out to be a great fit here. My “hello world” was to use the RPi.GPIO python module to watch for rising-edge button events on the GPIO pin I had jumpered a momentary button to, and publish a message over MQTT when the button is pressed. Logged into the raspberry pi, I run:

1

$ mosquitto_sub -t sensors/buttonup

.. to see those messages from the broker. In node.js land, the mqtt module provides the same functionality. We connect to the host/port the broker is on, subscribe to some topics and register callbacks for when messages arrive.

Finally, to bring the browser into the equation, I ended up making an adapter to subscribe to MQTT topics, and use socket.io to relay to the dashboard page. There is some (new) websocket support in mosquitto, and probably other more direct ways to accomplish this, but this works for now. As a proof of concept, I handle ‘gpio/button’ events from socket.io, and dispatch a ‘hardbuttonup’ event on the window object. A listener for this event toggles a class to flash the screen blue. This opens up a lot of opportunities as I have flexibility at every stage: the button is a stand-in for any sensor or input device that can toggle a GPIO pin high/low. The MQTT message produced can be caught by any program running on this device, or even anywhere on the network potentially. Turning this message into a DOM event enables a seamless tie-in so you can use existing frameworks to respond to the event.

Its a bit subtle, but in the video I’m clicking the button on the breadboard, and the display is flashing blue.

There are a few key requirements and moving parts to this project that I wanted to explore early on. I began at the end - the UI I wanted to show on the mirror. I initially tried out Atlasboard but quickly found it did more than I needed in some areas, and fought me to do what I wanted in others. To KISS, a ground-up implementation was going to be easier in the long run.

To get started I created a static page with a simple dashboard/widget system. It had a fixed pre-defined number of slots - a 3x3 grid of rows and columns - making layout and addressing really easiy with CSS and getElementById calls. This gave me a straight-forward way to get stuff on the screen. The widgets were a rudimentary base class implementing a init/update/render (and optionally poll to update and re-render) lifecycle.

On the server-side I made an “socket-board” express app to do reverse proxying, and keep things like API keys out of the client-side code. The name alludes to my eventual goal to hook it up to websockets for sending data between the client and data and events on device. All my feed/api requests from the browser go through the express app, and it sends on their response. To that end I created a basic config library and a request-forwarding middleware.

At this point, the dashboard runs anywhere with node.js and a modern browser.

Between the initial implementation and this screenshot, I’ve added a widget for the local IP address, which is particularly useful during development.

With the winding down of the Firefox OS / smartphone project, and a new interest in IoT and connected devices, I thought it would be fun and instructive to go out and build something connected. I settled on the “smart mirror” concept as a nifty thing that would be actually useful at home, and fun to make. I first saw this via hackaday featuring a Magic Mirror project. Its a doable project and should drop me in the middle of some more or less unfamiliar territory: getting hardware talking to software, lots of linux hackery, sensor inputs and gpio, and orchestrating moving parts up and down the stack. It should also be a good test platform to build on, to explore different technologies and interactions.

So, I have in mind a dashboard of sorts, displayed on a monitor/tv behind a two-way mirror. It should collect together some local and remote data of a sort that might be useful to know before you head out into the world: date/time, weather, appointments, actual (vs forecast) conditions and so on. As there wont be any keyboard/mouse/touch inputs, any direct user interactions (not sure yet what they would be if anything) would need to be hands-off - maybe voice input? some kind of presence or simply proximity detection to activate it?

I’m keeping notes as I go about this and I’ll post a series of updates with progress, dead-ends and observations. Some decisions I’ve already made - I’ll start out running this on a Raspberry Pi, the dashboard will be an HTML page in a fullscreen browser - others I’ll figure out along the way. In the past I’ve found that there is no substitute for actually making something to understand what the nature of the challenges will be, and its usually not what you expect. So, I expect to fall into all the traps, trip on all the hurdles and generally fumble my way though. Sounds like fun :)

• Smart Mirror: Data channels
• Getting Started on the Front-End
• Starting Up

I found this article stub in my drafts folder, dated back to 2014. I had apparently intended to write about the cancelling of the Firefox Metro (Firefox for Windows 8) project, but never put pen to paper. 18 months later, I cant think anything better to say than “Alas, Metro.”

I’ve been enjoying responses to Crystal Beasley’s post about how she got into tech/development. It got me thinking about the diversity of backgrounds and paths people take to progamming and tech work; I know it never ceases to amaze me. Here’s my story.

This past weekend I was in Toronto for the Mozilla summit. It was one of three venues where mozillians - staff and volunteers - gathered to talk about the Mozilla project, what we’re doing, where we are going and most importantly, to meet members across the breadth of the community.

I started at Mozilla almost 2 years ago now, when I joined to work on the now-shelved Pancake project. Now I work on Firefox for Metro. I had at least dim awareness of all the projects and intiatives discussed over the last few days, but a few things were brought home to me. One is that success - which I’ll define as an open, universally accessible web - is far from assured. In many ways the open web is a solution to a problem lots of people don’t know they have, and as such it is vulnerable to erosion through public apathy as much as commercial and governmental influence. The right to a free and open platform for publishing, sharing, doing business and all the other ways the web has become critical infrastructure for our society - is not written or enshrined in any law. It is a thing we must all work at preserving and building.

Furthermore, Mozilla is just one of many communities that share these values. While we must compete with software giants, we must shouldn’t resemble one. We must guard against sacred cows and not-invented-here syndrome. We must proactively reach out and connect up with groups that share our values and vision, and frame our work as a collective effort, not a “Mozilla project.” This is especially true when many people only understand Mozilla to be a software company that produces a browser; nuances like it being a non-profit, mission-driven open source project can get lost in the mix.

Seeing the open web vision being driven forward by so many people, across so many countries and in so many ways was invigorating though. I’m proud to be playing a part in this and look forward to the future we can build.