Crossing the thresholds of indignation and inclusiveness

XV.2 March + April 2008
Page: 67
Digital Citation

FEATUREWhat robotics can learn from HCI

Aaron Powers

As the robotics field grows and becomes competitive, robotics companies are beginning to inject user-centered design methods into their processes. Applying HRI methods to industrial and commercial products introduces new challenges and a focus on cheap, proven methods. The specialty of human-robot interaction (HRI) is a growing group of roboticists, social scientists, and designers, but the field of industrial practitioners is still small. Robotics has yet to reach the transition point that Don Norman talks about in The Invisible Computer, where the level of performance exceeds users’ needs [1]. For that reason, the robotics industry to this point has focused on technology rather than user experience. As we see robots become ubiquitous consumer products, that focus is starting to change.

At iRobot we have one practitioner of HRI (that’s me). iRobot has begun the transition from a technology-centered company to a user-centered company, as we grow from research robots toward commercial products. As we focus more on product development, we transform many research methods from the HCI and HRI fields into practice. Additionally, robotics companies provide a good opportunity to put HCI principles into practice. Because robotics companies like iRobot are growing quickly and shifting toward commercial products, the field is too new to have an ingrained process. HRI can become the framework for development of commercial robots.

Ethnography is the most popular investigative method being adopted in commercial HRI. Detailed ethnography studies helped iRobot learn about the culture of the PackBot users in the military and about the homes and cleaning patterns of Roomba owners. The open-ended approach of ethnography allows a series of short studies to explore varied topics and build a baseline of knowledge. Interacting with a humanlike robot is, in some ways, like the intersection of a new culture into an old one, making ethnography an excellent method of research and evaluation. For example, you may have seen a Roomba push an empty trash can around the room or catch computer cables as it vacuumed. Environment and context can be crucial factors in the success of a robot, and the ethnographic method is effective at discovering their influence.

Certainly, we run experiments in industrial HRI, but running formal experimentation is rare. It is much more common that we need quick, effective, “discount” techniques because projects or decisions are often on a tight deadline. Just as “discount” techniques have garnered widespread commercial use in the usability domain, they are needed in HRI as well.

To understand what principles of HCI will have the most impact in HRI, iRobot ran a series of systematic evaluations of several of iRobot’s teleoperated robots, which are driven by remote control. iRobot has several teleoperated robots, such as the PackBot, the R-Gator, and the recently announced ConnectR. To study teleoperation, we collected many hours of observations and documented more than 700 one-line “stories” from the observations. For example, users commented that powering the robot through remote control was difficult because their vision was limited. The video stream that users use to drive the robot had a low frame rate and lagged by less than a second. We’re using these stories to identify issues and prototype new ideas. By watching the videos, we noticed that when a team is working with a robot, they would often point where they were going to before they would drive there. So we prototyped a laser-pointer robot—operators use the laser pointer to put a dot on the ground in front of the robot, and the robot will drive forward toward the dot.

We organized the large list of “stories” into several areas, and we found four key areas where there are many challenges in HRI—these are the areas that we will focus on improving for next-generation HRI.

Situational Awareness. Especially during teleoperation, users need to know the internal states of the robot, the robot’s position in the environment, and the environment. For example, good cameras help users understand the robot’s position and state.

Robot Control and Movement. Robots are capable of complex movements, and it is important to be able to clearly and effectively command the robot to do what you want it to do. For example, controls to drive the robot and move its arm need to be flexible enough to complete the task, while remaining accessible for human operators.

Controller/UI. Teleoperated robots follow a client/server model in which the controller interface is a client that can operate independently of the robot. This area has many of its own challenges, like ergonomics, because the operator is working separately from the robot itself.

Communications. Communications between the controller and the robot create limits on the robot’s behavior, such as how far away you can send the robot.

Using these stories as a basis for our future work, we’ve looked at HCI and HRI theories and defined a list of key HCI/HRI principles to focus on. This list of “heuristics” was developed from three core sources: Jakob Nielsen’s classic list of usability heuristics [2], Ben Schneiderman’s core principles [3], and Jean Scholtz’s methods for evaluation of intelligent systems [4]. Certainly, this is an untested, initial list—there is room for research in this area.

Many of these principles are not unique to HRI, but their relative value of weighting is slightly different from other HCI communities. During our evaluations, we found the most space for improvement in the areas of “required information should be present and clear,” “prevent errors if possible, if not, help users diagnose and recover,” and “use metaphors and language the users already know.” That’s why these three are on the top of the list.

Sholtz’s work on intelligent systems adds a new spin to some of these universal HCI principles in the context of HRI.

“Design should be aesthetic and minimalist” has been the most important interaction design principle used in iRobot’s projects intended for use in homes, like the Roomba. Since Roomba is a consumer product, a simple user interface keeps the cost of the robot down while keeping its operation simple. As the product matures, the Roomba team is taking on a broader ethnographic approach, including more in-home studies.

Sholtz also suggests that HRI developers “make the architecture scalable” and “support evolution of platforms,” because robotics is still an immature medium and the robots are often required to do much more than they were designed for. In short, if you don’t make it easy for the system to grow, it will be outdated very soon.

“Simplify tasks through autonomy.” The fewer tasks the user is required to assist with, the more he can focus on high-level planning of his task. As mentioned earlier, it is simpler to drive the robot to a location by pointing with a laser pointer than navigating the robot to the location by remote control. Similarly, to simplify telemanipulation, where users control each joint of the robot arm, we are testing haptic interfaces that will automatically adjust the joints of the robot to move the robot’s arm into the desired place.

“Allow precise control.” Although it is important to use autonomy to simplify things, the robot still must be able to accomplish difficult and complex tasks. Designers can’t predict all the tasks or uses of a commercial robot, once it is in the hands of the user. For example, when teleoperating a robot like the PackBot for Explosive Ordnance Disposal, the user may need to be very careful or complete an action in a specific way when using the robot’s arm to manipulate objects. There are many things that robots do not know, and so it is often important that users have the capability to exert precise control over the robot and its arm.

At iRobot, the ninth principle, “create a positive brand image,” is crucial because we’re focusing on industry and commercial usage. If branding and name recognition become part of a robot, we can expect brand to influence users’ perceptions of robots, and their perceptions of the robots may change how they interact with one.

“Strive for a natural human-human interaction.” People work in the physical world, and so interfaces that also work in the physical world are the most effective, the simplest, and the most natural. If you work with robots as your teammates, you want to be able to talk to them and gesture to them just like you would to another person—you don’t want to drop your task and pick up a laptop. We have begun several small projects allowing users to gesture or speak to robots and to allow users to give commands without using any additional hardware. While general-purpose gestural interfaces are a long way off, we consider other ways to reduce interface burden on users who are multitasking such as using head-mounted displays and familiar gaming controllers (similar to the Playstation 2 controller).

As robots become more complex and as markets become more competitive, the robotics industry is sure to see a growth in its need for HCI and HRI specialists. Similarly, HRI techniques must be expanded and improved to be relevant and useful in the commercial development space. Robots are quickly becoming a staple in many homes around the world, and they open up a whole new world of possibilities for interaction design.


1. Norman, D. The Invisible Computer, Cambridge, MA: MIT Press, 1998.

2. Nielsen, J. and R.L. Mack eds., Heuristic evaluation, Usability Inspection Methods, New York: John Wiley & Sons, 1994. Available at

3. Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction, Boston: Addison Wesley, 1997. Available at

4. Scholtz, J. “Evaluation Methods for Human-System Performance of Intelligent Systems,” Proceedings of the 2002 Performance Metrics for Intelligent Systems Workshop (PerMIS), Gaithersburg, MD, 2002. Available at

5. Jenson, Scott, The Simplicity Shift, Cambridge University Press, 2002.


Aaron Powers

About the Author

Aaron Powers is the lead human-robot interaction researcher at iRobot, Corp. iRobot is known for its commercial successes both with the Roomba line of robotic vacuum cleaners and the PackBot line of military robots.





©2008 ACM  1072-5220/08/0300  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.


Post Comment

No Comments Found