Making/breaking

Explorations in narrative biosensing

Commercial biosensors can measure a host of physiological processes, including heart rate, blood oxygen levels, activity, cycles of sleep—even emotions. They promise self-improvement by providing clear metrics and recommendations that can lead to better health, performance, and self-care practices. Yet, and perhaps especially when it comes to the body, discrete data is not always an accurate—and definitely not a comprehensive—account of what is really going on. But what is going on? How can we leverage biosensing technologies to produce more subtle, open, and expressive accounts of the body and its many (ambiguous) signals?

To answer this question, we designed and built the Bodyteller, a device that records electrodermal activity and translates it into poetic texts. The texts were then shared between us for a period of four weeks. In this period, we exchanged these "body-poems," which, far from revealing our bodies, thickened and blurred them, creating a multilayered encounter with poetic texts that ultimately pointed toward new possibilities for representing biosensing data in everyday life.

Gabrielle's early sketches of the interaction.

Body-poem from July 22, 2021.

Final interaction sketch. The sender sends biosensing data to a server using HTTP. The data is then parsed and translated into poems, and sent to a thermal printer at the reader's house.

A printer jam caused the lines to print on top of one another. The process of exchanging body-poems was very much shaped by these technical and physical quirks.

Circuit assembly at home.

Our intention was to highlight the body's production of polysemic data and to create a context in which this ambiguity could be received, shared, and interpreted. Over nine months, we prototyped systems to translate biosignals to text, exploring the resulting design possibilities through text swatches—textual samples processed with different sentiment-analysis mappings, layouts, and source materials (including a physics textbook and a book of philosophy). The final system consists of a distributed set of microcontrollers equipped with off-the-shelf electrodermal activity biosensors, a locally hosted Web application server, and a variety of digital and physical data-output modalities.

To generate a body-poem using our system, the sender—that is, the person whose biosignals will be used to generate the text—wears an off-the-shelf electrodermal activity (EDA) sensor on their fingers or wrist. This sensor measures the electrical conductivity of the skin; we choose to use EDA data due its increasing prevalence in commercial wearables to track stress and categorize mood. This data is sampled once a minute and sent to a server running on a Raspberry Pi 3B+. The data is then mapped onto lines taken from a book of poetry using sentiment analysis. The results are a series of lines arranged according to the biosensing data.

In the summer of 2021, we lived with the Bodyteller for four weeks, exchanging 15-line poems, five days a week. We both kept journals during this phase, in which we wrote down the date, time, current mood, and a few lines to a paragraph of what was going on in our lives to provide context for each body-poem, either as sender or receiver.

One question that loomed over the entire process of producing and exchanging the body-poems was: What do they mean? On the second week of data collection, Blair changed the mapping of the sentiment analysis code so that the body-poems would yield only "happy" lines, which raised questions of authorship, seen here in this excerpt from Blair's journal: "[H]aving control over the mapping makes the whole experience feel more like coauthorship, where before I was considering Gabrielle the 'author' of the poems (insofar as the data relates to her biodata) and myself the reader."

The coauthoring between the "emitting body" (the sender), the reader, and the algorithmic transformations that happen between the two is built into this interaction from the start. Scholar Ashley Clayson describes writing as always being distributed across tools, written artifacts, and bodies [1]. This distributed quality was particularly evident in the Bodyteller interaction: Not only the writing process but also the meaning of the poems unfolded through the interaction between bodies, algorithms, and printed texts. The body-poems were less a fixed information than a process of constant engagement and interpretation with the biosensing data and the evolution of their meaning over weeks.

Journal excerpts. Left to right: Blair's journal excerpt, the body-poem for that day, Gabrielle's journal excerpt.

Overview of the hardware.

Two body-poems. From left to right: body-poem from July 21, 2021 (full view and close up) and body-poem from July 30, 2021 (altered by Blair).

  References

1. Clayson, A. Distributed writing as a lens for examining writing as embodied practice. Technical Communication Quarterly 27, 3 (Jul. 2018), 217–226; https://doi.org/10.1080/10572252.2018.1479607

  Authors

Gabrielle Benabdallah investigates the role technology plays in shaping perception and the possibilities of knowledge production. With a focus on language technologies and digital fabrication, she designs and builds systems that question practices of sensemaking with and through machines. [email protected]

Blair Subbaraman is interested in how computational tools might engage creative and expressive goals. His research practice draws methods from engineering and design, and usually involves building things. [email protected]

Copyright held by authors

The Digital Library is published by the Association for Computing Machinery. Copyright © 2022 ACM, Inc.