ACM Interactions

During the pandemic, when some learners are expected to be remote, conventional videoconferencing tools like Zoom or Microsoft Teams are used by necessity, but they are not without their limitations (e.g., bandwidth and Internet accessibility requirements). These tools, initially designed for corporate use, can be extremely useful when everyone is remote. However, we have learned that videoconferencing tools are not ideal for hybrid classroom discussions, group work, design activities (such as sketching and diagramming), or creating and manipulating physical artifacts.
 
Emerging telepresence technologies have the potential to alleviate the problem of social interaction for learners who are unable to attend in person [1], attenuating the limitation of inaccessibility and creating a more inclusive classroom environment. Concretely, these technologies allow the interchange of nonverbal signals between participants that greatly influence the effectiveness of in-person communications [2]. 

While these technologies have positive affordances, they also present limitations [3]. For instance, many commercially available MRP units do not have hands to manipulate objects; many robotic arms cannot move independently around the classroom; and wearing VR headsets for long periods of time may disrupt the sensory system and/or cause eyestrain. Likewise, these technologies have open privacy challenges for students and teachers that must be analyzed and discussed.

This article highlights key findings from our workshop on emerging telepresence technologies in hybrid learning environments. The workshop was held at the 2022 ACM Conference on Human Factors in Computing Systems (CHI), to learn about HCI research in this space and promote future telepresence research to meet the needs of remote learners.

Emerging Telepresence Technologies in Hybrid Learning Environments: CHI 2022 Workshop

To discuss the impacts, affordances, and limitations of different emerging telepresence technologies that can be used in various learning contexts, a group of interdisciplinary researchers including Houda Elmimouni, John Paulin Hansen, Susan Herring, James Marcin, Marta Orduna, Pablo Pérez, Irene Rae, Janet Read, Jennifer Rode, Selma Sabanovic, and Verónica Ahumada organized and conducted the first workshop on Emerging Telepresence Technologies in Hybrid Learning Environments [4].

The workshop was held in a hybrid format with five participants attending remotely via Zoom and four participants in person. A large screen in the room displayed the Zoom participants for the in-person participants and a Meeting Owl 3 camera provided the remote participants views of the physical meeting room and the in-person participants. After brief introductions, the workshop started with a keynote talk from Laurel Riek, an internationally recognized roboticist. Following the keynote, papers on the following topics were presented: 

• Paper A: Robots as teachers; robots as students
• Paper B: Investigation of an ungrounded haptic force device that could be used for remote movement instruction
• Paper C: Extended reality (XR): Exploring use of the Owl, a telepresence system focused on real-time immersive capture, delivery, and rendering of physical spaces and interactants
• Paper D: Assistance recruitment interactions between remote and local users of mobile robotic telepresence

The afternoon sessions were dedicated to discussion of the challenges and opportunities of telepresence technologies in the context of hybrid learning environments. Ideation and prototyping exercises followed the discussions to cocreate recommendations for future technology and research directions.

Evaluating Current Telepresence Practices in Learning Environments
During the first open session, some of the paper topics were discussed and debated. In particular, the participants explored and evaluated current telepresence practices in learning environments. We compared conventional videoconferencing tools (e.g., Zoom, Microsoft Teams) with emerging telepresence technologies such as MRP, stationary and mobile telepresence robots, robotic arms,  holograms, Cartesian manipulators, and haptic tools. We discussed their affordances and use expectations as well as their actual use and performance in educational and learning environments. The discussions touched on the role of embodiment in helping the remote attendee feel immersed in the distance learning environment and how UX and usability issues can disrupt the attendance and the completion of tasks.

Creating: Design Sprint
The prototyping session of the workshop took the form of a design sprint. Each participant had 10 minutes to sketch eight different ideas that could be used to address any of the challenges previously discussed. At the end, the ideas were put in a shared document and each participant voted for their three favorite ideas.

The result of the design sprint can be grouped in three areas:

• Improve the communication capabilities of telepresence robots. First, by giving them better embodiment capabilities such as a 360-degree representation of the remote user, robotic arms, and stair climbing. Second, by powering them with immersive communication technology (for example, omnidirectional video capture, XR telepresence, directional audio, and remote haptics). And finally, by adding augmentation capabilities to compensate for the inherent limitations of not being present (e.g., using machine learning to recognize people and automatically tag their names).
• Design the classroom for hybrid scenarios. For instance, using screens to show remote users in fixed positions within the classroom, or creating an interactive classroom map for spatial awareness of the local environment. This includes creating specific tools for the hybrid experience (for example, a physical microphone with a digital twin, used to “pass the floor” between students; connected tablets to share drawings and text; a mini-chat companion app to have a common side-talk channel for everyone; or even having everyone participate in the physical space via virtual means).
• Change the teaching strategy to actively include the remotes. For instance, paring them with local students, giving locals access to the remote user view, creating a “karma” system to gamify courtesy with remotes, and ensuring time-zone awareness across platforms.

Challenges and Opportunities
Telepresence technologies present many opportunities for novel applications and research. During our workshop, we explored the idea of moving beyond a single product to thinking about the whole classroom more broadly. The ideation exercise further prompted us to examine what aspects of interaction might matter—spatial audio, three-dimensionality, gaze, pointing, back-channeling, whispering—and how elements of meditated interaction (such as digital or robotic) could be blended to give remote and local users more options to express themselves and engage meaningfully with one another. 

There were several challenges identified in our workshop. On the technical side, the current technology is still limited, and those limitations may have social repercussions in learning activities. When connectivity lagged in our workshop, the audio was unclear and the visual resolution low. The remote participants’ experience was disrupted, and they missed out on what was happening in the physical space. We also explored issues of equity. Workshop participants acknowledged that for telepresence to work well requires time, money, and effort; this includes an appropriate space with good WiFi, screens, and cameras, and detailed plans that include telepresent learners. Promoters of telepresence technologies must consider how these challenges will be met to ensure equitable access. 

Future Directions for Research and Practice
Future directions for research in telepresence should include studies on the immersive capabilities of 360 cameras. We believe the ability to independently view the complete physical space will remarkably increase feelings of presence for the remote viewer. Just as people can view an entire room without leaving their seats, the 360-degree camera affords the remote user this same capability. Future studies may also explore how interactants feel about a 360 camera. For example, how would it feel if you were speaking with someone on a telepresence unit and they turned their “head” 180 degrees to talk to someone behind them? As humans, we do not have this capability. Would it be welcome? Would it disrupt acceptance of the remote user?

Additionally, we expect future studies may include:

• User privacy and data-sharing issues
• Social guidelines for all participants in the telepresence experience
• Accessible controls that are adaptable by end users to accommodate personal and physical preferences
• Ethics surrounding how to present equal inclusion of remote participants by respecting their audiovisual rights in the physical space

Conclusion
The number of telepresence researchers in the HCI community is increasing in the post-pandemic world as the global community has adapted to work-from-home, learn-from-home, and socialize-from-home activities and experiences. Our emerging telepresence workshop was an effort to provide a place for telepresence researchers to engage in transfers of knowledge, share works in progress, cocreate, and problem solve. Topics that emerged in our workshop included: views on robots as educators and learners, the potential use of haptic touch/force to communicate feedback on physical learning activities (e.g., dance), desktop telepresence technologies that relay immersive capture, delivery and rendering of physical spaces and interactants, and exploration of assistance recruitment interactions between remote and local users. 

As our workshop took place during a Covid surge, some participants were unable to travel to attend. We were fortunate, however, to have researcher expertise in both HCI and telepresence technologies to conduct our workshop with a hybrid approach. As a hybrid learning approach was our desired area of study, we were excited to synthesize our topics of discussion with our workshop-participant experiences in learning via telepresence. We used several forms of digital media to interact within our workshop: Zoom, Meeting Owl 3, Google Docs, cell phones (cameras and text), and shared screens. Our workshop ideation and prototyping exercises found that HCI researchers valued the use of a 360-degree camera (remote user to view the physical room and people), a detailed agenda on progression of learning activities with identification of technologies throughout the learning exercise (e.g., use Zoom now, transition to Google Docs now, etc.), and directed plans for paired learning between remote users and interactants. Additionally, as many participants were logging in from different time zones, we suggest an optional time stamp for awareness of local times. Overall, we were encouraged by the hybrid learning experience and excited to participate in ways like other remote learners in hybrid spaces. Our discussions and topics raised several challenges, however, that need to be addressed for equitable and inclusive use of telepresence technologies. Future HCI research has great potential to address these challenges, create technologies, and promote social practices that facilitate equitable learning through sociotechnical inclusive environments.

Endnotes
1. Ahumada-Newhart, V. and Olson, J.S. Going to school on a robot: Robot and user interface design features that matter. ACM TOCHI 26, 4 (2019), 1–28.
2. Grondin, F., Lomanowska, A.M., and Jackson, P.L. Empathy in computer-mediated interactions: A conceptual framework for research and clinical practice. Clinical Psychology: Science and Practice 26, 4 (2019), e12298.
3. Rae, I. and Neustaedter, C. Robotic telepresence at scale. Proc. of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017, 313–324.
4. Elmimouni, H. et al. Emerging telepresence technologies in hybrid learning environments. CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM, New York, 2022, 1–5.

Houda Elmimouni

Pablo Pérez

Andriana Boudouraki

Fatma Guneri

Verónica Ahumada-Newhart