ACM Interactions

A journey ended with the publication in January of my book, From Tool to Partner: The Evolution of Human-Computer Action.

The beginning

Ron Baecker’s 1987 Readings in Human-Computer Interaction quoted from prescient 1960s essays by Vannevar Bush, J. C. R. Licklider, Douglas Engelbart, Ivan Sutherland, Ted Nelson, and others. I wondered, “How did I work for years in HCI without hearing about them?”

When Ron invited me to work on a revised edition, more questions piled up. Our discussion of the field’s origin oscillated between human factors and computer science as though they were different worlds, yet the Human Factors Society co-sponsored and participated in the first two CHI conferences. Then they left. Why? Similarly, management information systems researchers presented interesting work at early CSCW conferences, then vanished. Another mystery: NSF and DARPA program managers who funded HCI research never attended our conferences. Readings in HCI devoted a large section to Stu Card and Tom Moran’s keystroke-level and GOMS models, the most praised work in CHI. Why did almost no one, including Card and Moran, build on those models? While working in AI groups at MIT, Wang Laboratories, and MCC, I wondered why two fields about technology and thought processes seemed more often at loggerheads than partners. 

There were other mysteries. For example, in 1984, my fellow software engineers loved the new Macintosh, even though we worked for an Apple competitor. We agonized as the Mac flopped for a year and a half. Apple slid toward bankruptcy. Then Steve Jobs was fired and the Mac succeeded. What was that about?

I located people who were involved in these matters. Eventually, I found convincing answers to every question that I had begun with and other questions that arose along the way (Convincing to me, anyway.) I wrote a short encyclopedia entry on HCI history, an article for IEEE Annals of the History of Computing, handbook chapters, and other essays. I edited and sometimes authored history columns for Interactions magazine. Finally, the book.

Could you benefit from reading the book?

The book is primarily about groups of researchers and developers who contributed to HCI, coming from computer science, human factors, management information systems, library and information science, design, communications, and other fields. By stepping back to get a larger picture, you might see how your work fits in, where you might find relevant insights, and where you are less likely to.

A better name for our field might be “human-computers interaction.” Machines changed radically every decade. We all know Moore’s law, but when I read a paper that describes the object of study as “a computer,” I unconsciously picture my current computer. This makes it easy to fail to see when an advance was primarily due to new hardware; for example, the Mac succeeded when “the Fat Mac” arrived with four times as much memory, soon followed by the faster Mac II. Human-computer interaction in 1970 and 2000 were both human-computer interaction, but no one would confuse human-mainframe interaction and human-smartphone interaction.

Patterns emerge. Some topics wax and wane, and then wax again. Artificial intelligence had summers and winters that affected HCI. Interest in virtual reality surged and receded: Alphaworld in the mid-90s, Second Life in the mid-2000s, and now Oculus Rift, HTC Vive, and Hololens. Another recurring focus of research and development was desktop video-conferencing. Was it inspired more by a pressing need or by telecoms interested in selling bandwidth?

However, there was steady progress. It took longer than many expected, but we collectively built the world imagined by Vannevar Bush, J. C. R. Licklider, Douglas Engelbart, Ivan Sutherland, Ted Nelson, Alan Kay, and others. In the 1960s, a few engineers and computer scientists used computers. Yet a common thread in their writing was of a time when people in diverse occupations would use computers routinely. We’re there. Ivan Sutherland wrote a program that changed a display to create a brief illusion of movement and speculated that someday, Hollywood might take notice. Thirty years later, Toy Story was playing in theatres. Some might quibble over whether every detail has been realized, but we are effectively there.

The concept “from tool to partner” is owed to Licklider, who in 1963 forecast that a period devoted to human-computer interaction would be followed by a period of human-computer partnership, which in turn would end when machines no longer needed us. For many years, we interacted with computers by feeding in a program, typing a command, or pressing a key, after which the computer produced an output or response and then waited for the next program, command, or key. Today, while we sleep, software acts on our behalf. It accepts incoming email, checks it for spam, and filters it in response to prior instructions. Software considers our history and location in selecting search query responses and advertisements. My favorite application of recent years eliminates the need to type a password: When I start up my tablet, it turns on the camera and sees that it is me! A dull sentry (“I know your caps lock key is on and you typed the caps lock version of your password, but turn off caps lock and try again”) is transformed into a cheerful colleague (“Jonathan—Hello! Welcome.”) as I quickly go on my way.

Of course, issues arise. The book identifies challenges that will long be wrestled with. Like Samantha in the film Her, my software is interacting not only with me—it is also interacting with the owners of the sites I visit and the advertisers who buy space. Software can mislead us by not being consistent in ways that people usually are. It can be an expert at chess, abysmal at checkers, and unable to play dominos at all. A sensor can be highly expert at differentiating gas odors while neglecting to suggest that I avoid lighting a match to look. And we steadily move into a goldfish bowl where all activity is visible, with positive and negative repercussions.

Keep it impersonal?

How do you approach writing about a time much of which you lived through? The “new journalism” of the 1960s showed that personal experiences can reveal hidden complexities and the emotional impact of events. But I wanted to write a history, not a memoir. My ideal is Thucydides, who, more than halfway through History of the Peloponnesian War, stunned me long ago: “Meanwhile the party opposed to the traitors proved numerous enough to prevent the gates being immediately thrown open, and in concert with Eucles, the general, who had come from Athens to defend the place, sent to the other commander in Thrace, Thucydides, son of Olorus, the author of this history, who was at the isle of Thasos, a Parian colony, half a day's sail from Amphipolis, to tell him to come to their relief. On receipt of this message he at once set sail with seven ships which he had with him, in order, if possible, to reach Amphipolis in time to prevent its capitulation, or in any case to save Eion.” Thucydides then briefly describes reaching Eion in time but how his opponents out-maneuvered his allies and convinced Amphiopolis to join the other side. His small part played, he disappears from his history without another word about his experiences.

I compromised by including some personal experiences in an appendix. For some, they could convey the significance of events that otherwise can seem remote. That said, a history is a perspective; it is never wholly objective. An author decides what to omit and what to include. For example, I do not cover the evolution of key concepts, theories, and applications to which so many people contributed. When I cite specific work, it is usually where the boundaries of the fields working on HCI were defined or transcended. Prior to the appendix, my technical contributions get less attention than Thucydides’ campaign.

A second short appendix lists resources that you could consult in writing another history of HCI, perhaps a conceptual history. Please! While writing, I shifted from hoping to wrap everything up to hoping to live to see other HCI histories. We have been privileged to participate in an exceptional period of human history. Technology evolves so rapidly that those not yet active will struggle to understand what we were doing, yet what we are doing will shape their lives in subtle ways. We owe it to them to leave accounts.

Jonathan Grudin