Florian Mueller, Josh Andres, Joe Marshall, Dag Svanæs, m. c. schraefel, Kathrin Gerling, Jakob Tholander, Anna Martin-Niedecken, Elena Segura, Elise van den Hoven, Nicholas Graham, Kristina Höök, Corina Sas
In late 2017, 23 researchers and academics from Europe, Australia, and the U.S. gathered for a week to discuss the future of body-centric computing. Dagstuhl, a nonprofit center for computer science research located in a rural area in Germany, hosted the seminar in a picturesque 18th-century castle.
The goal of the seminar was to discuss the future of this novel area of interaction design: what it means to design interactive technology when centering on the human body. This area evolved in part with the emergence of movement-, physiological-, and bio-based sensors and actuators, after which followed the blooming of technologies such as wearables, quantified-self systems, and movement-based interactive systems (e.g., exertion games). These technologies bear a close relationship to the body: They may be worn (wearables), carried and kept close to the body (mobile technology), or involve body movements or physiological responses as their main interaction modality (e.g., exertion games, quantified-self systems). They stand in stark contrast to technologies and applications within the previously prevalent desktop computing paradigm, which involved interacting with computers in a way that required minimal bodily engagement and rather static positions, such as sitting still. The motivation for the seminar stemmed from the realization that, until today, most work in this area has taken an instrumental perspective, focusing on achieving objectives such as “most miles jogged this week” to reward athletic performance. However, theories and perspectives such as phenomenology and user experience design can help us extend this focus on performance to also include the subjective embodied experience of the user engaged in such interactions.
Though the concept of embodiment has been in the spotlight in human-computer interaction (HCI), the body—the actual corporeal, pulsating, and felt body—has been notably neglected in theory and design work. This may be because we lack the knowledge and vocabulary to access, articulate, and ultimately design for these highly subjective and elusive bodily experiences that go beyond external sensorial interactions. The challenge further increases when a key design material is the vast amount of data reported by various sensors. Another issue is that HCI researchers typically have limited direct knowledge of or training in how the body works as a complex system (i.e., in-bodied knowledge). In order to drive an agenda that supports instrumental as well as experiential, embodied, and in-bodied perspectives of the active human body, the seminar brought together leading experts in the field. They discussed key questions around the use of interactive systems to support both instrumental and experiential perspectives to pioneer new approaches for what we frame as a future of body-centric computing.
The aim of body-centric computing is to design products and services that reposition the role of the body from the periphery to the center of the interaction, thus becoming body-friendly. This means solutions that seriously consider, respect, extend, enhance, or facilitate the user’s body and associated bodily experiences. This contrasts with solutions that ignore, neglect, and disregard the role of the body to perceive, act, and construct meaning—in short, to be in the world.
|Dag Svanæs attaching the tail to Corina Sas.|
|Walking very slowly through the forest, in silence.|
The core application domains for body-centric computing are health, well-being, sports, and entertainment. The health domain includes prevention, rehabilitation, disease management, and cognitive/physical performance. Well-being supports pleasure and connectedness, or uses embodied interaction to facilitate cognitive offloading. Sports, including insights from sports science, view the body as a site for performance, training, learning, and improvement. Entertainment, including gaming, aims for full bodily immersion in interactive experiences.
The seminar began with talks by all attendees, in which they presented their work in the area, the theoretical perspectives that guide their work, and a description of their most and least favorite body-centric computing projects. After the presentations concluded, no slides were shown for the remainder of the week; all activities were conducted either as a roundtable, standing up, or exercising both indoors and outdoors, fitting with the seminar theme. The program included group activities with supplemented whole-body movement practices to support a “brain as part of the body” approach. Optional morning and evening activities, such as playing golf, jogging, cycling, hiking, or slacklining, further supplemented this use of the physical in support of the cognitive/social dimension. A selection of some key activities and methods, and the body-centric reflection they facilitated, is outlined below to illustrate the seminar and its ethos.
|VR on a swing.|
|Plunder Planet cooperative multiplayer setup.|
In an interactivity session, participants tried out each other’s body-centric computing systems through hands-on experiences in order to reflect on the instrumental as well as experiential perspectives these designs actively supported.
Joe Marshall from the University of Nottingham set up a swing where users wear a head-mounted display (HMD), through which they see a virtual world that responds to the movement of the swing. This involved interesting mappings of action types in the physical world (swing) to effects in the virtual world. This work elicited questions around how movement can be deliberately fed back to the user in an altered fashion to elicit novel entertainment experiences.
Anna Lisa Martin-Niedecken from the Zurich University of the Arts presented her dual flow-based fitness game Plunder Planet , which adapts to players’ abilities in real time, provoking questions around the role of technology in allowing people to experience their bodies in an individual and social context.
Lifetree by Patibanda et al.  is a VR game aimed to train proper breathing technique. Participants who tried the game said their experience prompted reflection on the role that technology can play in promoting well-being, while technology often presents challenges for health.
|Designing a chair without chairs.|
|Collaborative working session at Dagstuhl.|
Florian “Floyd” Mueller from RMIT University in Australia presented various headphones that use noise-canceling, in-ear, over-ear, and bone-conducting technologies in order to raise the question of whether we could say one is more body-centric than another.
Dag Svanaes from the Norwegian University of Science and Technology presented an interactive tail  that moves in response to the hip movement of the user, controlled via sensors and actuators. This work elicited the phenomenological question about the role of technology in extending human bodies.
Perttu Hämäläinen from Aalto University in Finland presented his work on using AI to predict and animate movement in virtual worlds. This work provoked questions around how AI can inform the movement of embodied systems such as exoskeletons or the tail mentioned above, and whether AI can accurately or evocatively represent human motion.
Methods for how to design body-centric computing were not only heavily debated but also tried out through design exercises. To better connect with our inner self and be able to access and use our bodily experiences in design, we engaged in several somatic-sensitizing activities, led by Thecla Shiphorst (Simon Fraser University, Canada). Some activities revolved around helping us alternate between and work with different foci of attention. First, participants were invited to close their eyes to help them shift their attention to various body parts and somatic sensations, emphasizing a first-person, attentive, inward-looking experience. In contrast, other activities revolved around acting in the world without focusing on any object or people, letting our attention drift as we moved in the world. In particular, we were to maintain this type of attention while walking very slowly in silence through the surrounding forest for about 20 minutes.
This shared exercise of diffuse attention aimed to facilitate a heightened awareness of the body in the world. It was done at a slow pace to allow the participants to use their bodies as a tool to access and a site to cultivate their sensorial and felt experiences. Participants later compared the act of diffusing attention to the type of cross-eye concentration required to see random-dot autostereograms. After this exercise, participants were asked to pick three elements from their experience to “imprint” in their bodies. This was done by focusing on one particular element while gently pinching a part of one hand using the index finger and thumb of the other hand. In doing so, participants placed on their bodies a tactile cue for each experiential element selected. These bodily imprints were later accessed and utilized by participants in a group task: designing an interactive, shape-changing chair. These imprints were meant as lived-experience inspirations and as reminders of those experiential elements, moments lived, and their bodily states. They were intended to transport participants to their previously lived state of slowness, appreciation, and cultivation, which can be inspirational body-centric design qualities.
The design outcome, the chair, was not as important as the process itself. All the design teams realized how difficult it is to articulate felt experiences and applauded this bodily imprinting method for how it helped them articulate and agree on particular experiential qualities they wanted their design to elicit. The activity functioned as a way of defining clear and focused design qualities.
Most participants appreciated the slow forest walk, which afforded them a fruitful, design-conducive state. Many commented on how easily and quickly they decided on particular experiential qualities to design for, and how in sync they felt with their design teams. Many related this to the prior bodily activity they all shared, which created an experiential common ground that facilitated internal communication and joint design work. Many speculated that walking slowly together in silence created a richer design vocabulary than a more traditional brainstorming or design-discussion activity. This shared activity helped build a community and a shared sense of trust, which helped materialize the collective experience in the design process (e.g., facilitating the taking of risks) and outcomes. Participants also noted that, throughout the exercise, they were at ease combining things during the design phase. That might not have been the case if we were in a critical state. The “anything goes” state of mind is perhaps more productive than being “critical, smart, and the brilliant designer you always want to be.” However, for some participants, the slow walking did not work so well; they had difficulty not being goal-oriented, letting go, and being in the moment. This shows that these activities might not work for everyone and in any state of mind, or that some people might require more time to appreciate such activities.
In a critical examination and discussion about design outcomes from body-centered design processes, it was noted that they often fall into a “meditative,” “inward-looking,” “self-reflective” genre (e.g., the somaesthetic yoga mat ). Participants pondered which types of sensitizing pre-design activities other types of aesthetic bodily experiences would require. If you are designing for a dance club, would slowly walking in the woods help? How? Likewise, what would happen if these designers had designed those chairs after jogging? It was also noted that this particular activity focused on designing from our experiences (i.e., the brief to design a chair was given only after the silence activity), whereas in reality, the brief usually comes first; hence, the design process might therefore unfold differently.
Though the concept of embodiment has been in the spotlight in HCI, the body—the actual corporeal, pulsating, and felt body—has been notably neglected in theory and design work.
Participants also observed that two lines of conversation emerged when reflecting on the activity: One was about designing for the human body from the lens of what is good for the human, while the other is about designing new experiences that are experiential and not necessarily good for the human body. However, they do not need to stand in opposition, as Mueller and Young highlighted previously .
Participants also tried out taking on a first-, second-, and third-person perspective when designing for the human body . The first-person perspective is concerned with the personal felt experience of the body; the second-person perspective is concerned with the interdependencies between bodies; and the third-person perspective is concerned with an external, more objective view of the body. Subsequent discussions highlighted that the third-person perspective is prevalent in most of today’s available wearable technologies, with a few emerging systems also considering the first-person perspective. The second-person perspective is probably the hardest to grasp, with limited exemplars available as guidance.
There are many open questions around body-centric computing. One that arose: how to articulate bodily experiences for design. At the moment, there is limited knowledge of how to communicate, share, and articulate bodily experiences. Following from here, we acknowledge the need to understand how much knowledge about the human body one needs in order to design for body-centric computing. Of course, one might argue the more the better; however, one position was that it is also impractical to expect all designers to take kinesiology courses. Others questioned how impractical that would actually be. Is there some basic knowledge that might be sufficient, or would we always need to bring in experts? For example, HCI curricula currently teaches aspects of the vision system and Fitts’ Law about our perceptual and performance capacities. Doesn’t our new interest in body-centric computing require us to add more specific knowledge about other relevant physiological, chemical, hormonal, and neurological processes? If so, how do we provide HCI students and designers with the necessary knowledge to draw from and interpret their own bodily experiences with confidence? Overall, there is a need for more methods and concepts for personalized and adaptive body-centric systems based on fine-grained knowledge of the human body.
|Using VR to disrupt awareness of space to force relearning of how to navigate in the physical world.|
Another question that emerged is how to respect experiences of body changes. For instance, how do we design for restricted movement? This question arose from the discussion around whether physical restriction could be used as a means to facilitate empathy for people living with physical disabilities.
In essence, how can we design body-centric computing systems that affect movement, such as physical restrictions, to facilitate limits across life course and experience? And how can designers create these experiences and ensure they are accurate (if they are meant to reflect a “real” experience)? Such an exploration could, for example, lead to exertion games in which the player starts off with a physical constraint, like a distorted view that impacts orientation in the virtual world along with coordination of movement in the real world, and then through leveling up experiences increased empowerment. This progressive approach could be a way to design for transformation and development yet acknowledge that real-life strength and flexibility develop over months rather than individual exercise sessions. Technology could make this larger timespan more immediately visible and exploratory.
There are also many open technical questions around body-centric computing. A particularly pertinent one was detecting when people touch each other, and to what extent. Certainly, solutions already exist; however, they are often not very portable, requiring tethering and power. Developing systems that can unobtrusively sense human touch and that are mobile and can be worn for long periods is still technically very challenging. Furthermore, a shortage of prototyping tools for body-centric computing is also limiting the field.
Finally, another important underexplored area concerns the ethics around body-centric computing. Moving the body comes with certain risks; however, these risks have value in and of themselves, and interaction designers need to be aware of how to deal with them . Participants argued that the alternative is to design for stillness, which might not facilitate immediate injury but ultimately leads to obesity and unhappiness, something that all participants aimed to avoid.
We thank Dagstuhl for their extensive support and all participants for contributing and taking pictures.
1. Martin-Niedecken, A.L. and Götz, U. Go with the dual flow: Evaluating the psychophysiological adaptive fitness game environment “Plunder Planet.” Proc. of International Conference on Serious Games. Springer, 2017, 32–43.
Florian “Floyd” Mueller is director of the Exertion Games Lab at RMIT University, Australia, where he engages body-centric computing to help facilitate personal growth. Previously, he was a Fulbright Scholar at Stanford University and has spent time at the MIT Media Lab, Microsoft Research, FXPal, Xerox Parc, University of Melbourne, and CSIRO. firstname.lastname@example.org
Josh Andres leads user experience and design at IBM Research - Australia; his research in HCI investigates the interplay between a user’s exertion and actuation-enabled systems to facilitate new user experiences for the active human body. He is a Ph.D. candidate at RMIT’s Exertion Games Lab. email@example.com
Joe Marshall is a senior research fellow at the Mixed Reality Lab, School of Computer Science, Nottingham, U.K. His work investigates games and artistic installations that respond to the user’s full body. firstname.lastname@example.org
Dag Svanæs is full professor in interaction design at Norwegian University of Science and Technology, Trondheim, Norway, and adjunct professor at the IT-University of Copenhagen, Denmark. His research focuses on designing with and for the living body, using phenomenology as an inspiration. He is manager of the NTNU UX Lab and currently leads a research project on exergames. email@example.com
m.c. schraefel is professor of computer science and human performance at the University of Southampton, as well as a certified strength and conditioning coach nutritionist, and movement coach. m.c. leads the WellthLab in Human-Systems Interaction, whose mission is to explore the research and design of interactive computation systems to help #MakeNormalBetter and enhance quality of life for all. firstname.lastname@example.org
Kathrin Gerling is an assistant professor at KU Leuven, Belgium, where she is part of the e-Media Research Lab. Her work broadly falls into HCI and physical computing. She is particularly interested in how interfaces can be made accessible for audiences with special needs, and how interactive technologies can be leveraged to support well-being. email@example.com
Jakob Tholander is an associate professor at Stockholm University, Sweden. He is a researcher in human-computer interaction and has studied various aspects of interaction design and body-centric computing ranging from tangible technologies, to sports and recreation, to musical performance. Jakobth@dsv.su.se
Anna Lisa Martin-Niedecken is a senior researcher at the Zurich University of the Arts focusing on game design. With her background in sports science, her R&D work mainly focuses on movement-based games. She is particularly interested in how technology shapes gameplay/bodily experiences in individual and social play contexts. firstname.lastname@example.org
Elena Márquez Segura is a postdoctoral researcher in the Social Emotional Technology Lab at the University of California, Santa Cruz. Her research and design focus on movement-based collocated social interaction. Recently, she has been working in the domain of social wearables and social VR. email@example.com
Elise van den Hoven is a full professor in the Faculty of Engineering and IT at UTS, associate professor in the Department of Industrial Design at Eindhoven University of Technology (TU/e), and has two honorary appointments. She leads the Materialising Memories research program, using design research to support remembering activities. firstname.lastname@example.org
Nicholas Graham is a professor at the School of Computing, Queen’s University, where he performs research in game design and development. His group focuses on exercise video games (exergames) for children with disabilities such as cerebral palsy. email@example.com
Kristina Höök is a professor in interaction design at the Royal Institute of Technology (KTH). Her research focuses on soma design, affective interaction, bodily engagement, and aesthetics. firstname.lastname@example.org
Corina Sas is a full professor in HCI and digital health at Lancaster University, U.K. She has expertise in technologies for well-being and health, and currently leads AffecTech, a Marie Curie innovative training network on personal technologies for affective heath. email@example.com
Copyright held by authors. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.