Interactive Light Feedback

Introduction

As part of my PhD research, I investigated interactive light displays, a novel way of presenting visual feedback in the space surrounding a device. This keeps small screens free for showing content and allows visual feedback if a device has no screen. Interactive light displays use small light sources, like the ones shown below, to illuminate surrounding surfaces with light. When used as feedback, we call this interactive light feedback [1].

Interactive Light Displays

Interactive Light Display
An interactive light display for a mobile phone, which has 60 LEDs.

My first prototype interactive light display used a flexible strip of LEDs (Adafruit NeoPixels) wrapped around a mobile phone. This display has 60 lights which can be controlled independently of each other. This means that each can have a unique hue and brightness, opening up a large design space for interactive light feedback. The LEDs are controlled by an Arduino microcontroller and they can be updated hundreds of times per second, allowing smoothly animated changes in light. This display is described in an Interact 2015 demo paper [1] and was used in my CHI 2016 paper [2].

Interactive Light Display
An example of an interactive light display which varies its brightness.
A photo of the HaptiGlow system, an ultrasound haptics device with LEDs around the front edge.
The HaptiGlow system, an interactive light display for an ultrasound haptics device.

Interactive Light Feedback for Gestures

During my PhD I investigated how interactive light feedback can be used by in-air gesture systems. Users require lots of feedback to be able to gesture confidently and effectively (see this rant), but small devices (like mobile phones) are limited in their ability to present feedback. Interactive light feedback allows such devices to present information visually, without affecting the content on screen. This feedback can also be noticed more easily from a greater distance away, meaning users can gesture without first approaching their phone.

The Gesture Thermostat, featuring an interactive light display.
The Gesture Thermostat uses an interactive light display to give feedback about gestures.

I also investigated the use of interactive light displays for household controls, allowing simple devices with no display to give feedback without adding screens or changing their form factor. By giving these controls an expressive way of giving feedback, they can use other input modalities which require feedback to remain usable. For example, speech and in-air gesture input could be used, with users getting interactive light feedback about their actions. One of these projects, the Gesture Thermostat (pictured above), is described in a workshop paper from NordiCHI 2014 [3] and a poster paper from Interact 2015 [4]. This has been an effective way of giving gesture feedback.

For a demonstration of the Gesture Thermostat and its feedback, see:

Hardware

The prototypes I’ve used in this work have used Adafruit NeoPixel LED strips (link), controlled by an Arduino Uno.

References

[1] Interactive Light Feedback: Illuminating Above-Device Gesture Interfaces
E. Freeman, S. Brewster, and V. Lantz.
In Interact 2015 Demos. 2015.

[2] Do That, There: An Interaction Technique for Addressing In-Air Gesture Systems
E. Freeman, S. Brewster, and V. Lantz.
In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems – CHI ’16, pp. 2319-2331. 2016.

[3] Illuminating Gesture Interfaces with Interactive Light Feedback
E. Freeman, S. Brewster, and V. Lantz.
In Proceedings of NordiCHI ’14 Beyond the Switch Workshop. 2014.

[4] Towards In-Air Gesture Control of Household Appliances with Limited Displays
E. Freeman, S. Brewster, and V. Lantz.
In Interact 2015 Posters. 2015.