Traditionally, fiction creates and induces emotions and empathy through words and images. By using a combination of networked sensors and actuators, the Sensory Fiction author is provided with new means of conveying plot, mood, and emotion while still allowing space for the reader’s imagination. These tools can be wielded to create an immersive storytelling experience tailored to the reader.
To explore this idea, we created a connected book and wearable. The ‘augmented’ book portrays the scenery and sets the mood, and the wearable allows the reader to experience the protagonist’s physiological emotions.
The book cover animates to reflect the book’s changing atmosphere, while certain passages trigger vibration patterns. Changes in the protagonist’s emotional or physical state triggers discrete feedback in the wearable, whether by changing the heartbeat rate, creating constriction through air pressure bags, or causing localized temperature fluctuations.
Our prototype story, ‘The Girl Who Was Plugged In’ by James Tiptree showcases an incredible range of settings and emotions. The main protagonist experiences both deep love and ultimate despair, the freedom of Barcelona sunshine and the captivity of a dark damp cellar.
The book and wearable support the following outputs:
- Light (the book cover has 150 programmable LEDs to create ambient light based on changing setting and mood)
- Personal heating device to change skin temperature (through a Peltier junction secured at the collarbone)
- Vibration to influence heart rate
- Compression system (to convey tightness or loosening through pressurized airbags)
Created for a class called "From Science Fiction to Science Fabrication," the Rodent Sense project links its wearer to the world of animals.
We drew on Umwelt theory to imagine how a human might make sense of the world if given the opportunity to to switch between various animal sensory inputs and augment (or diminish) their senses in particular ways. For the demo, we focused on allowing the viewer to see through the eyes of a hamster.
As hamsters can only see 2 inches in front of their eyes, the view offered to the wearer is a quite distorted one. To create this experience, we attached stereoscopic cameras to a carriage and hamster-ball device that the hamster pulled, and processed the resulting video feed so that it could be seen by the viewer in 3d while wearing an Oculus Rift. The carriage and wheels were laser cut from 5mm mirrored acrylic and 25mm clear acrylic.
Ultrasound, a common imaging modality used for maternal care in the developed world, is a safe, effective, and non-invasive means of identifying pregnancy complications that contribute to maternal mortality. However, the high cost of ultrasound devices and the scarcity of ultrasound training are two major barriers to adoption in resource-constrained environments.
From 2010 - 2012 I worked on a project to build a portable, low-cost ultrasound system for midwives and mothers in low-resource regions. My role in this group was leading design research and fieldwork in Uganda and Kenya, interface design, grant writing, data analysis, and project management. Our system, composed of off-the-shelf hardware and a custom user interface, included an integrated help and tutorial system to help midwives in remote areas answer diagnostic questions when a radiologist may not be reachable.