Authors:
Tyechia L. Thompson, Eric Lyon, Wallace Lages
Stories are ubiquitous, transcending traditional media and books, permeating our self-talk and scientific publications. Immersive technologies are no exception. Storytellers and artists are using spatial audio, virtual reality (VR), and augmented reality (AR) more frequently to create and share narratives about the past, the present, and the future. What sets immersive technologies apart from their predecessors is their ability to create embodied experiences that evoke deeper and more enduring emotional responses [1]. They are also unique in the way they can influence empathy through perspective taking and immersion [2].
Immersive technologies, however, are not without their ethical dilemmas. Empathy can inadvertently lead to unjust behaviors [3]. Moreover, as with other digital media, those who create or disseminate content using these technologies have opportunities to shape discourse, influence culture, and accumulate wealth. It is crucial, therefore, that we approach the rise of immersive technologies with caution and intentionality, ensuring they do not accentuate existing disparities [4]. Instead, they should be harnessed as tools to create opportunities, amplify underrepresented voices, address issues of access, and foster mutual understanding among diverse communities.
Listening to Multiple Voices
With the support of a Digital Justice Seed Grant from the American Council of Learned Societies, we assembled a group of practitioners and researchers in storytelling and immersive technologies to explore these concerns. In selecting this group, we aimed to achieve a diverse representation of background, experience, academic focus, and artistic practice. Our advisory board consisted of Dr. Riham Alieldin, a physician and medical educator at the University of Rochester; Bryan Carter, director of the University of Arizona Center for Digital Humanities; Damon Davis, a multimedia artist, musician, and filmmaker; Al Evangelista, a dancer, choreographer, and assistant professor of dance at Oberlin College and Conservatory; and Ashley Shew, an associate professor of science, technology, and society at Virginia Tech. Their collective expertise spans film, visual arts, opera, podcasts, dance, science, medicine, Africana studies, community building, arts administration, disability studies, and gender studies.
It is crucial that we approach the rise of immersive technologies with caution and intentionality.
Our journey as a group began with a visit to Virginia Tech in August 2022, where the board members met in person for the first time and experienced immersive performances at Cube Fest 2022: Afrofuturism (https://artscenter.vt.edu/performances/cube-fest-2022.html), a three-day festival exploring the frontiers of immersive media and Black futurism. They also toured multiple state-of-the-art technology facilities to create a common ground for the follow-up discussions. Over the next 12 months, we held group and individual conversations around empathy, narrative, access, and ethics. Some of these conversations are captured in the seven-episode podcast series The Inside Story, in which we also explored the practical question of how to build an institute to support technical competency and promote awareness of the ethical aspects of immersive storytelling.
Three Concerns for Immersive Storytelling
As we explored several questions around representation and empathy, three recurring issues that affect immersive storytelling emerged. One is how to create genuine experiences that connect people to a culture as welcomed guests, not exploiters. Our conversations with Damon Davis pointed toward the importance of attentive listening, individual authenticity, and mutual understanding, particularly in community engagement. He emphasized the necessity of meeting people where they are and facilitating empathy through new myths and paradigms. This includes avoiding being extractive of underrepresented narratives and engaging with individuals directly to understand their needs. Ashley Shew urged creators to reconsider the concept of empathy and first evaluate values within the community. She also highlighted the need to move from emotional experiences to action-based engagement and to acknowledge the context, history, and power relations within empathy discussions. At the same time, Al Evangelista pointed out that risk is inherent in art, and that intimacy, even when simulated through sound and eye contact, raises questions about community, hierarchies of information, and ethical curation. He cites his Dancing Lab project as an example of the interplay between ethics, risk, and power in community artmaking.
 | From left to right: Amanda Hodes, graduate research assistant, and grant principal investigators and advisory board members Bryan Carter, Ashley Shew, Riham Alieldin, Tyechia Thompson, Damon Davis, Eric Lyon, Wallace Lages, and Al Evangelista at Virginia Tech. |
A second issue is related to the need to properly contextualize immersive stories. The more participatory and realistic the experience is, the more necessary a debrief may be to create a healthy distance from the narrative, especially if the experience is emotional or triggering. This requires that storytellers create comprehensible immersive narratives, even when users interact with them in a less than ideal way. Implementing onboarding processes before the immersive experience and debriefing processes afterward can be especially beneficial for users new to immersive technologies. Riham Alieldin discussed the phenomenon of the "disintegration of empathy and compassion" among medical students as they progress in their training. While simulations that offer real patient perspectives can improve interactions between medical professionals and patients, they may cause users to feel like they "know" the experience for themselves, potentially leading to generalizations. Alieldin stressed the importance of debriefing to avoid generalizations about people's experiences, and the challenges of separating novelty effects from genuine, long-term changes.
A third concern focuses on access. Immersive technologies are still expensive, which limits access to a small segment of the population. Additionally, there is a skill gap for certain types of production. How can we enable competency building in underrepresented communities so they can tell their own stories? The group discussed ways to support these creators, ranging from workshops to integrating competency building as an outcome in larger project grants. From his experience engaging with participants unaffiliated with academic institutions, Bryan Carter shared the importance of building relationships that can support project integrity and equitable resource sharing in the face of a partner's unique ethical perspectives. This includes considering ways to create accessibility in immersive technologies, including the use of multiple modalities to accommodate individuals with special needs.
Conclusion
Immersive experiences in spatial audio, VR, and AR offer a potent medium that can revolutionize how we tell stories, connect with one another, and experience the world. As adoption increases, we need to maintain an ongoing conversation about ethical and equitable uses of these unique technologies. Although there are not clear answers to many of the challenges ahead, we should start by acknowledging the power and responsibility of storytellers, technologists, artists, and scholars so that we can harness immersive stories as a transformative tool toward inclusion and equity.
References
1. Wincott, A., Martin, J., and Richards, I. Telling stories in soundspace: Placement, embodiment and authority in immersive audio journalism. Radio Journal: International Studies in Broadcast & Audio Media 19, 2 (2021), 253–70.
2. Cummings, J.J., Tsay-Vogel, M., Cahill, T.J., and Zhang, L. Effects of immersive storytelling on affective, cognitive, and associative empathy: The mediating role of presence. New Media & Society 24, 9 (2022), 2003–26.
3. Jackson, L. Empathy reifies disability stigmas. Keynote speech, Interaction 19, Feb. 6, 2019, https://interaction19.ixda.org/program/keynote--liz-jackson/
4. Franks, M.A. The desert of the unreal: Inequality in virtual and augmented reality. UC Davis Law Review 51, 2 (2017), 499–538.
Authors
Tyechia L. Thompson is an assistant professor of English at Virginia Tech. Her areas of research focus include Africana studies, digital humanities, manuscript and archival research methods, and immersive storytelling. She explores the digital realm and literature, highlighting the interconnectedness of culture, technology, and human expression. [email protected]
Eric Lyon is a professor of practice at Virginia Tech. He is a composer, computer musician, spatial music researcher, audio software developer, and curator. His current work focuses on spatial audio for high-density loudspeaker arrays, and automated sound design. [email protected]
Wallace Lages is an assistant professor at Northeastern University with a joint appointment in the Department of Art + Design and the Khoury College of Computer Sciences. His research focuses on user experience of augmented and virtual reality applications, leveraging methods from disciplines such as design, engineering, and psychology. [email protected]
Copyright held by owners/authors
The Digital Library is published by the Association for Computing Machinery. Copyright © 2024 ACM, Inc.
Post Comment
No Comments Found