Demo Hour

DEMO HOUR: Mergereality

Mergereality is a design exploration that aims to explore multi-device hand-gestural interactions in augmented reality (AR). We present two demos here: dragging sticky notes from an iPad to AR, and pulling 3D models from a computer display. We prototyped the concepts with an Oculus Rift VR headset and Zed Mini camera as a passthrough AR solution and coupled it with Leap Motion for hand tracking. By merging the boundary of digital and physical, we demonstrate an interaction strategy that harnesses physical affordances to assist digital interaction in AR with hand gestures.

Wu, S., Byrne, D., and Steenson, M.W. 'Mergereality': Leveraging physical affordances for multi-device gestural interaction in augmented reality. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020.

Shengzhi Wu, Google
wushengzhi891215@gmail.com

Daragh Byne, Carnegie Mellon University
daragh@cmu.edu

Molly Steenson, Carnegie Mellon University
steenson@cmu.edu

Dragging a sticky note from an iPad into AR.