People: the way I see it

XIV.4 July + August 2007
Page: 46
Digital Citation

The next UI breakthrough, part 2

Donald Norman

back to top 

In my previous column I discussed the reemergence of command line language. Once these were the ways we used our operating systems and applications. Now they are reemerging within search engines. They are hidden and not easy to learn about, but I expect them to grow in power and, over time, become the dominant means of interaction.

In this column I will talk about a second trend, one that also has much earlier origins: the return to physical controls and devices. In the theoretical fields that underlie our field, this is called embodiment: See Paul Dourish's book, Where the Action Is. But the trend is far more extensive than is covered by research on tangible objects, and somewhat different from the philosophical foundations implied by embodiment, so I use the term "physicality."

Physicality: the return to physical devices, where we control things by physical body movement, by turning, moving, and manipulating appropriate mechanical devices.

We have evolved as physical creatures. We live in a complex, three-dimensional world filled with physical objects. We are analog beings in an artificial world of digital devices, devices that abstract what is powerful and good from the physical world and turn it into information spaces, usually in arbitrary ways. These new approaches put the body back into the picture. They require us to control through physical action rather than virtual, which means through mechanical devices, not electronic or graphic.

At one point, when digital circuits took over the control of such mundane objects as automobile radios, physical controls were removed. Ugh. The most recent advances in automobile radios and other audio equipment is to reintroduce knobs for tuning and loudness control. Even BMW, in its attempt to replace all knobs, buttons, and controls with a single control knob and complex menu hierarchy, has been forced to bring back physical switches and knobs.

Perhaps the most dramatic example of this trend is the Nintendo Wii game machine. Here, physical movement is the major method of interacting with its video games. The Wii has completely changed the game world: kudos to Nintendo! Tablet computers are slowly inching toward respectability, because the joy of being able to write and draw directly on a page or upon images is powerful, especially when coupled with a machine that also allows the more standard mouse-based pointing and typing inputs to work just as before: The result is the best of all worlds.

Physical devices have immediate design virtues, but they require new rules of engagement that differ from what we are used to with the typical mouse movements and clicks of the traditional keyboard and mouse interface. Designers have to learn how to translate the mechanical actions and directness into control of the task.

As we switch to tangible objects and physical controls, new principles of interaction have to be learned, old ones discarded. With the Wii, developers discovered that former methods didn't always apply. Thus, in traditional game hardware, when one wants an action to take place, the player pushes a button. With the Wii, the action depends upon the situation. To release a bowling ball, for example, one releases the button push. It makes sense when I write it, but I suspect the bowling-game designers discovered this through trial and error, plus a flash of insight. Not all of the games for Wii have yet incorporated the new principles. This will provide fertile ground for researchers in HCI.

Physical devices, what a breakthrough! But wait a minute, isn't this where the machine age started, with mechanical devices and controls? Yup. Just as command line interfaces—now available in the quasi, natural language format now used within search engines—is also a throwback to earlier times but with improvements, so too is the return to physical controls a throwback to the earlier mechanical era, with improvements. It's about time we returned to our roots, to something intended for people.

One interesting implication of the movement toward physical interfaces is that the dominant discipline for the technology of human interaction might also move from computer science back to mechanical engineering (which is really where it started many years ago). New disciplines will have to be learned, for example control theory and mechatronics. Mechatronics is the combination of mechanical engineering mechanisms with electronics (and computer science). Where does one learn mechatronics? CS departments shun the mechanical, so you won't find it taught there. The social sciences shun the engineering, so you won't find it there. Mechanical engineering departments almost always teach it, but devoid of contact with human beings. Training in mechatronics is common in forward-looking design schools, because they know that the design of future things will include a hefty dose of intelligent mechanics and electronics, and in the best design schools, consideration of the people for whom the designs are intended is always key.

Part of the future of design is that of smart, intelligent devices, where almost everything will have a microprocessor built in, plus motors, actuators, and a rich assortment of sensors, transducers, and communication devices. If the future is a return to mechanical systems, mechatronics is one of the key technological underpinnings of their operation. Mechatronics taught with an understanding of how people will interact with the resulting devices. Taught with an understanding of all the critical areas of design: mechanical, computer, and electrical engineering; business; social sciences; and aesthetics. But where is one to gain skills in all of these areas? Within the university, each component is a separate discipline, sometimes not even on speaking terms with the others, a social separation that unfortunately can persist into the workplace. Not in the arts or sciences, for they are often dismissive of both applications and business. Similarly, business schools lack the emphasis on technology and aesthetics and, in far too many cases, on the social side. Design schools and departments have their own deficiencies, sometimes attempting to cover the entire gamut, but without the depth that comes from within the discipline. Fortunately, many individuals have put together the requisite skills. It's time for our educational institutions to catch up.

Physicality: the return to mechanical controls, coupled with intelligent, embedded processors and communication.

That is one path back to the future.

back to top  Author

Donald A. Norman

About the Author

Don Norman wears many hats, including cofounder of the Nielsen Norman group, professor at Northwestern University, where he codirects the Segal Design Institute, and author of The Design of Future Things (November 2007). He lives at

back to top 

©2007 ACM  1072-5220/07/0700  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2007 ACM, Inc.

Post Comment

No Comments Found