When I was in school, history was presented as an immutable timeline, stretching from the dawn of writing to about World War II, after which it was too controversial for children. My forays into the early days of HCI have revealed less constancy; history changes as our perspective changes. We continually rewrite history.
A well-attended event at CHI 2008 was “Usability Evaluation Considered Harmful,” featuring a critique of CHI reviewing practices by Saul Greenberg and Bill Buxton . They argued that three HCI landmarks, featured in most HCI histories, omitted studies of use and therefore would have fared poorly at the hands of CHI reviewers. They wrote: “Usability evaluation, as practiced today, is appropriate for settings with well-known tasks and outcomes. Unfortunately, [it fails] to consider how novel engineering innovations and systems will evolve and be adopted by a culture over time.” Greenberg and Buxton stress that the CHI community needs to be far more liberal in considering what makes a valuable contribution.
Agreed, but on careful examination each of these early contributions has more to say about HCI history and practice than is generally noted.
Greenberg and Buxton wrote:
“In 1945, Vannevar Bush introduced the idea of cross-linked information in his seminal article “As We May Think,” which in turn inspired Hypertext and the World Wide Web. Bush described a system called ‘Memex’ based on linked microfilm records. Yet he never built it, let alone evaluated it. Bush’s vision wasn’t even correct: it was constrained to knowledge workers. He certainly never anticipated the use of linked records for what are now the mainstays of the Web: social networking, e-commerce, pornography, and gambling. Even if he had done a usability evaluation, it would have been based on tasks not considered central to today’s culture.” [Italics added.]
The Atlantic Monthly published Bush’s essay soon after another in which he used quarry mining as a metaphor for the pursuit of knowledge in the arts and sciences. Atlantic Monthly, a magazine of literary criticism, political analysis, and fiction, won two of the top four O. Henry short-story prizes in 1945. World War II was drawing to a close; “As We May Think” was accompanied by essays such as “For the Record: Buchenwald,” “Japan’s Secret Weapon,” “Should Jews Return to Germany,” and “Keeping the Country at Work.” Fiction by Vladimir Nabokov, Jessamyn West, Roald Dahl, and Raymond Chandler appeared that year, along with essays on Jefferson, Melville, Rainer Maria Rilke, and Tom Paine. In Nabokov’s story, a 90-year-old man writing in approximately the year 2015 about the strange world of 1945 reported that all technology self-destructed and vanished in an unspecified but spectacular way in the 1970s.
The Memex was a conceptual sketch of a huge, mechanical, microform-shuttling device that couldn’t be built even now. Had it been plausible, it would have been affordable only for elite knowledge workers, so Bush got that right. Rather than a failure of imagination, Bush’s vision of an information device that would be used by people who were not engineers was far ahead of its time. It was a key reason his writing inspired future generations and resonates with us today. And of course, user tests were neither a concern of Atlantic Monthly nor a possibility for a device that did not exist.
Bush’s focus on the expansion of knowledge in this essay and his “quarrying knowledge” essay published two months prior was part of a movement that began much earlier. In the 1930s science fiction pioneer H. G. Wells wrote and campaigned for a “World Brain” or “World Mind,” a microform-based, Web-like project he had outlined in 1905: “These index cards might conceivably be transparent and so contrived as to give a photographic copy promptly whenever it was needed, and they could have an attachment into which would slip a ticket bearing the name of the locality in which the individual was last reported. A little army of attendants would be at work on this index day and night… An incessant stream of information would come, of births, of deaths, of arrivals at inns, of applications to post-offices for letters, of tickets taken for long journeys, of criminal convictions, marriages, applications for public doles and the like. A filter of offices would sort the stream, and all day and all night for ever a swarm of clerks would go to and fro correcting this central register, and photographing copies of its entries for transmission to the subordinate local stations, in response to their inquiries…”
Wells’s contemporary, the Belgian Paul Otlet, went further. In the early 20th century, he assembled millions of documents and images into a real cross-linked information repository, with a human staff providing a commercial information-retrieval service .
Greenberg and Buxton:
“In 1963, Ivan Sutherland produced Sketchpad, perhaps the most influential system in computer graphics and CAD. Sketchpad was an impressive object-oriented graphics editor, where operators manipulated a plethora of physical controls (buttons, switches, knobs) in tandem with a light pen to create a drawing. No evaluation was done. Even if it were, it would probably have fared poorly due to the complexity of the controls and the poor quality display typical of this early technology” [Italics added.]
Perhaps our standards have changed, Sutherland wrote of Sketchpad: “The user group experience showed that relatively new users with no programming knowledge could produce simple drawings with the system if a skilled user (myself) prepared the building blocks necessary For example, a secretary designed and drew an alphabet with the aid of a 10×10 raster of points to use as end points.”
Sutherland was using a $10 million machine (in today’s dollars) as an individual tool, prior to the formulation or recognition of Moore’s Law. He knew Sketchpad would have to prove useful for high-end knowledge work to impress anyone, so he had circuit designers use it to design circuits. The experiment was, in his words, “a big flop.” Sutherland identified and reported why it failed, which very likely contributed to the CAD successes that soon followed at Lincoln Labs. Rather than a failure of user testing, Sutherland’s thesis is an exemplary illustration of the value of reporting negative results.
Greenberg and Buxton may under credit Sutherland, but it’s true that CHI is disposed to reject papers that report negative results. One program chair said that reporting negative results would “run the danger of discouraging people to go down a certain path… That could be amazingly damaging… Imagine not wanting to go down a negative result path even if we don’t fully understand why.”
I can’t imagine such timid technology developers. The image I see is the glistening lake that once covered the La Brea Tar Pits, luring thirsty mammals to their doom. After a creature thrashed about and was pulled under, the lake closed over it and calmed, as enticing as before, with no warning left behind for the next mammal to come along .
Greenberg and Buxton:
“In 1968, Douglas Engelbart gave what is arguably the most important system demonstration ever held in Computer Science. He and his team showed off the capabilities of his NLS system. His vision as realized by NLS had a profound influence on graphical interfaces, hypertext, and computer supported cooperative work. Yet Engelbart’s vision was about enhancing human intellect rather than ease of use.”
This is correct. Engelbart felt that systems should be designed to be complex and difficult to learn, but powerful when mastered, like natural languages. User testing for him was an examination of human factors issues such as error and fatigue, not usability. Greenberg and Buxton note that reviewers who carp about ease of use could have missed the big picture and deprived us of this visionary work.
The best thing that could have happened to Engelbart would arguably have been pressure to consider ease of use.
The thoughtful system-builder should wonder, “How is it that not long after the most impressive computer science demo in history, instead of opening the doors of the U.S. Treasury and shipping money to SRI, Engelbart’s funding was cut and his work plunged into relative obscurity?” This should alarm anyone working on a prototype system or a new product.
The answer, at least in part: NLS had under-emphasized ease of use.
Before the 1968 demo, NLS was used by its developers. They were skilled users to whom initial ease of use didn’t matter. Engelbart’s funders at ARPA (Advanced Research Projects Agency) were impressed with the demo; they briefly increased funding, installed NLS in their headquartersand found it “baroque” and difficult to use. Vietnam had drained the economy, an AI winter was setting in, and Engelbart lost his funding .
Key Engelbart collaborators moved from SRI to Xerox PARC, interested in applying his concepts to personal computers rather than to the more expensive minicomputers that ARPA and Engelbart had favored. They brought along a lack of concern for ease of use. A slide shown at CHI’85 depicted a superpowered tricycle, symbolizing disdain for Apple’s focus on simplicity in the Macintosh. NLS and the Xerox systems did not succeed in the marketplace; the Macintosh did.
We owe a huge debt to Engelbart and his colleagues, as well as to Sutherland and Bush, for their conceptual innovation, remarkable systems, and inspirational writing. But we can learn yet more from them by looking carefully to see what was not practical, what did not work, and why their projects did not always go as planned.
A tip of the hat…
...to Saul Greenberg and Bill Buxton for discussions of these issues and to Joseph Reagle for pointing me to H.G. Wells.
1. Greenberg, S., and B. Buxton. “Usability evaluation considered harmful (some of the time).” In Proceedings CHI 2008, 111120. Florence, Italy: April 510, 2008. The paper presentation was followed by a panel discussion.
2. Atlantic Monthly articles can be accessed at www.theatlantic.com. Some, including “As We May Think,” are free, others require a small payment. For Bush’s predecessors see W. B. Rayward’s H.G. Wells’s “Idea of a World Brain: A Critical Re-Assessment,” JASIS 50 (1999): 557579, <http://people.lis.uiuc.edu/~wrayward/Wellss_Idea_of_World_Brain.htm>; and Alex Wright’s “Forgotten Forefather: Paul Otlet,” Boxes and Arrows, 10 November 2003. <http://www.boxesandarrows.com/view/forgotten_forefather_paul_otlet>
3. For Sutherland references see the interactions March+April 2006 Timelines column. The program chair quotation was in response to my emailed proposal that the conference solicit and carefully review negative results.
4. Thierry Bardini’s Bootstrapping and M. Mitchell Waldrop’s The Dream Machine cover Engelbart’s SRI experience and relationship with ARPA. The latter describes the end: “With ARPA headquarters now using NLS on a day-to-day basis, moreover, those deficiencies had become painfully apparent. So in 1975 [Licklider] terminated the SRI contract.” Larry Tesler described attitudes at SRI and early PARC in “A CHI Retrospective: 1962 1982,” June 24, 2004. The slides are at www.nomodes.com. The Mac as tricycle was presented in Engelbart’s closing keynote at CHI’85.
Jonathan Grudin is a principal researcher in the Adaptive Systems and Interaction group at Microsoft Research. His Web page is http://research.microsoft.com/~jgrudin
©2008 ACM 1072-5220/08/0900 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.