DEMO HOUR: Mergereality
Issue: XXVII.5 September - October 2020Page: 15
Digital Citation

Authors:
Shengzhi Wu, Daragh Byne, Molly Wright Steenson
Mergereality is a design exploration that aims to explore multi-device hand-gestural interactions in augmented reality (AR). We present two demos here: dragging sticky notes from an iPad to AR, and pulling 3D models from a computer display. We prototyped the concepts with an Oculus Rift VR headset and Zed Mini camera as a passthrough AR solution and coupled it with Leap Motion for hand tracking. By merging the boundary of digital and physical, we demonstrate an interaction strategy that harnesses physical affordances to assist digital interaction in AR with hand gestures.
Wu, S., Byrne, D., and Steenson, M.W. 'Mergereality': Leveraging physical affordances for multi-device gestural interaction in augmented reality. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020.
Shengzhi Wu, Google
wushengzhi891215@gmail.com
Daragh Byne, Carnegie Mellon University
daragh@cmu.edu
Molly Steenson, Carnegie Mellon University
steenson@cmu.edu
![]() |
Dragging a sticky note from an iPad into AR. |