XII.2 March + April 2005
Page: 42
Digital Citation

Interaction challenges in human-robot space exploration

Terrence Fong, Illah Nourbakhsh

back to top 

In January 2004, NASA established a new, long-term exploration program to support the President's "Vision for U.S. Space Exploration." The primary goal of this program is to establish a sustained human presence in space, beginning with robotic missions to the moon in 2008, followed by extended human expeditions to the moon as early as 2015. Moreover, the program places significant emphasis on the development of joint human-robot systems:

NASA will send human and robotic explorers as partners, leveraging the capabilities of each where most useful. Robotic explorers will visit new worlds first, to obtain scientific data, assess risks to our astronauts, demonstrate breakthrough technologies...Human explorers will follow to conduct in-depth research, direct and upgrade advanced robotic explorers, etc.
— NASA Vision for Space Exploration

A key difference from previous exploration efforts is that future space exploration activities must be sustainable over the long term. Experience with the space station has shown that cost pressures will keep astronaut teams small, and that care must be taken to extend the effectiveness of these teams well beyond their individual human capacity. Thus, in order to reduce human workload, costs, and fatigue-driven error and risk, intelligent robots will have to be an integral part of mission design.

bullet.gif Space Robotics

Although robots have previously been used for scientific purposes (e.g., geology) in space, robots will now also be called upon to perform non-scientific work. In particular, robots will be used for a wide range of tasks (under human control and autonomously) ranging from on-orbit assembly and maintenance of spacecraft to construction and maintenance of surface habitats to prospecting and processing of raw materials. The intricate nature of these tasks may require teams of robots capable of monitoring their own progress with a high level of autonomy.

Considerable research has already focused on developing human and robot systems for planetary surfaces. Scant attention, however, has been paid to joint human-robot teams. Such teams, however, are attractive for numerous applications. Robots could support long-duration manned excursions by scouting, surveying, and carrying equipment. Robots could assist in site preparation, sample collection, and transport. Finally, robots could be used for field labor, inspection and maintenance, and servicing of equipment and structures.

bullet.gif Construction

Construction, both in-space (on-orbit) and on planetary surfaces, is fundamental to exploration. Because structures are expected to be large and heavy, robots will be needed for a range of construction tasks including material transport, equipment positioning, and assembly. For example, planetary outpost construction will involve the transport, positioning and assembly of habitat modules that have been deposited (landed) in a scattered manner around the construction site.


bullet.gif Infrastructure Support

Long-duration exploration will require significant robot support to maintain and service infrastructure. Robots could be used to search for, identify, and repair loose fixtures on interior structures (e.g., inside crewed spacecraft) and outdoor structures (solar array, communication structure, etc). A key challenge will be enabling the robot to perform these tasks as autonomously as possible, requesting human assistance and expertise only when necessary.


bullet.gif Contingency Life Support

Lunar pioneers will encounter hazards and crises requiring new emergency procedures. Given that crew sizes will be small, responding to medical emergencies will require robotic support. For example, robots could provide contingency life support or medical transport.


bullet.gif Science Exploration

Robots have long been identified as potentially useful assistants for science exploration. Since the 1990s, numerous researchers have studied how mobile robots can support the field activities of biologists and geologists. Robots can perform a wide variety of functions: scout, equipment transport, caretaker (monitor scientist during EVA), and co-worker (deploy sensor, survey region, etc.).


bullet.gif In-situ Resource Utilization

In-Situ Resource Utilization (ISRU) is a critical need for sustained exploration. Launching and deploying all the materials (consumables) that are needed for a permanent outpost is so cost-prohibitive, that in-situ resources will have to be used. At the same time, in order for ISRU to be cost effective, humans cannot perform this task unaided. Instead, groups of robots could be used to mine, transport, and even process materials such as regolith, water, etc.


back to top  HRI Challenges

The interactions between humans and robots will be unlike anything that NASA has designed and implemented before. The operation of robot teams will at times be directed from ground control. For example, a lunar rover team may be assigned tactical assignments (e.g. "inspect solar array alpha"). Surface astronauts will also communicate with in-situ robots using voice-based commands, gestures, and wireless digital communication. Rovers, in turn, will communicate with one-another for team-based collaboration and must also develop sufficient self-diagnostic introspection to request human help when appropriate.

Making human-robot interaction (HRI) effective, efficient, and natural is crucial to future space exploration. In particular, we contend that humans and robots must be able to: communicate clearly about their goals, abilities, plans, and achievements; collaborate to solve problems, especially when situations exceed autonomous capabilities; and interact via multiple modalities (dialogue, gestures, etc), both locally and remotely. To achieve these goals, a number of HRI challenges must be addressed.

bullet.gif Multiple Spatial Ranges

Exploration will require human-robot collaboration across multiple spatial ranges, from shoulder-to-shoulder (e.g., human and robot in a shared space) to line-of-sight interaction (human in habitat, robot outside) to over-the-horizon (human in habitat, robot far away) to interplanetary (human at ground control, robot on planetary surface). Although a great many telerobotic systems have been developed during the past 50 years, none currently support multiple spatial ranges (i.e., all existing systems have been optimized for a particular spatial range). The challenge, therefore, is to develop HRI techniques for supporting multiple spatial ranges.

bullet.gif Interaction Architecture

Interaction architectures are structured software frameworks that support human-computer interaction. These architectures typically provide a set of core services (messaging, event/data distribution, etc.), support a variety of interfaces and displays, and facilitate human-centered interaction (as opposed to device-centered). Significant research effort has focused on developing interaction architectures during the past several years, particularly for supporting ubiquitous and context-aware applications, but few are designed for robotics. Moreover, none support the range of interaction scenarios and data requirements we believe will be necessary for exploration.

bullet.gif Peer-to-Peer Dialogue

Human-robot dialogue is an emerging sub-domain of HRI research. At present, most HRI systems rely on explicit dialogue (i.e. task commands), which is directed from a human to one, or more, robots. A few systems, such as NRL's multi-modal dialogue system, use multiple interaction modes (gesturing, natural language, etc.) to improve usability and for disambiguation. In all these systems, however, the dialogue model is essential the same: the human "speaks;" the robot "listens" and perhaps asks for clarification. What we need for exploration, however, is dialogue that works both ways. We need to enable robots to ask questions of the human (so that they can obtain assistance with cognition and perception) and to develop techniques so that robots will be able to make use of "implicit" language and gesturing.

bullet.gif Constrained Interfaces

Human-robot teams will require appropriate user interfaces in order to effectively perform exploration "field labor." Because humans will need to interact with robots in a variety of ways (different levels of autonomy, different spatial arrangements, etc.), a wide range of interfaces will be needed, both inside habitats and in EVA. Moreover, to improve usability and interaction effectiveness, a significant challenge is to develop constrained, standardized user interfaces. Standardized methods will reduce training time and will increase reusability by allowing modular improvements.

back to top  Conclusion

The goal of effective, diverse collaboration between human/robot teams demands broad collaboration between members of disparate fields, including human factors researchers, human-computer interaction experts, roboticists and artificial intelligence researchers. Enabling fruitful collaboration across the philosophical and scientific divides of disparate fields of inquiry can be a daunting task, and overcoming these divides may prove to be the biggest challenge of all.

back to top  Authors

Terrence Fong is the deputy leader of the Intelligent Robotics Group at the NASA Ames Research Center (ARC). From 2001 to 2004, he was a joint post-doctoral fellow at Carnegie Mellon University (CMU) and the Swiss Federal Institute of Technology (EPFL). At EPFL, he served as deputy group leader of the Virtual Reality and Active Interfaces Group was co-investigator for virtual environment telerobotic field experiments at NASA ARC from 1990 to 1994. He received his B.S. and M.S. in Aeronautics and Astronautics from MIT and his Ph.D. in Robotics from CMU.

Illah Nourbakhsh is an associate professor of Robotics in The Robotics Institute at Carnegie Mellon University. He received his Ph.D. in computer science from Stanford University in 1996. He is co-founder of the Toy Robots Initiative at The Robotics Institute. He is a founder and chief scientist of Blue Pumpkin Software, Inc. Illah recently authored the MIT Press textbook, Introduction to Autonomous Mobile Robots.

back to top  Figures

UF1Figure. Artist conception of humans and robotic assistants building a space telescope.

UF2Figure. Artist conception of human-robot lunar operations

UF3Figure. Artist conception of robot medical transport

UF4Figure. Artist conception of robot medical transport

UF5Figure. Firestarter concept for robotic regolith collection

back to top 

©2005 ACM  1072-5220/05/0300  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2005 ACM, Inc.

Post Comment

No Comments Found