It's not just what you know, it's what you know about what you know.
"Information visualization" is a special category of user-interface design. Tables, forms, charts, maps, and diagrams also have to solve how best to use metaphors, mental models, navigation, interaction, and appearance to make themselves usable, useful, and appealingthe catechism for good user experience. Lately, with new search and retrieval techniques popping up from Google, Yahoo, and other major Web-oriented companies, it may seem that much progress is being made. In some ways this is true, but advanced visualization (and sonification) techniques have made the rounds for yearsin some cases for decadesin specialized professional circles of finance, medicine, the military, and sciences, without achieving widespread recognition or use.
There is much visual information available about information visualization. One report is Woolman's Digital Information Graphics, a graphically rich catalog of innovation, circa 2002. Some techniques are decades old, like Inselberg's parallel coordinates paradigm, which he first showed me at a conference in 1981. Professor Marti Hearst from the University of California at Berkeley teaches a summary of techniques in her course on information visualization, similar to others taught around the world. Conferences that focus on the latest techniques include the symposia organized since 1990 by the IEEE and others like the upcoming Diagrams 2006. In some ways, information visualization is a healthy, robust subset of the industry. Nevertheless, we have a ways to go.
Even today, most computer users do not have a diagramming tool or advanced data representation tools as readily at hand as they do word processors or spreadsheets. That's unfortunate, because many people actually do their best thinking in nonverbal ways, and explaining and describing may best be done by verbi-visual or perhaps even verbi-sonic means, especially dynamic techniques. Admittedly, many PC users rely on Visio, a diagramming application, or one of its competitors, like SmartDraw. However, not all PC users are familiar with them, and they are not available on all platforms.
It is true that general-purpose, thinking tools are becoming more robust, sophisticated, and easier to learn and use. James Fallows reports on some of these in a recent New York Times article. He cites Devonthink Professional, a data-collection and -retrieval system that enables one to enter information easily into a database of email messages, Web pages, text or number files, pictures, PDFs, and anything else that can be "copied with a keystroke or mouse-dragged into storage." One can tag or classify information as it goes in, or later. Retrieving information through fast searching is similar to the Apple Macintosh's built-in Spotlight, or Google's desktop search utility, but its special feature is a "show also" button that displays files or text related to the search semantically in ways that are more sophisticated than those generally available for Web searches. Another product cited for special merit is Tinderbox, which shows visual connections among facts or ideas that one may have entered as quotes, themes, phrases, texts, pictures, diagrams, and other forms. One can view the spatial display as text, outline, flow chart, or link diagram.
Products like these just cited, the hyperbolic browser from InXight, which is based on research developments years earlier at Xerox PARC, or some of the experiments developed in Microsoft's Research group, like the Data Mountain metaphor, reported on by George Robertson, et al., all seek to provide new spatial syntax for displaying entities and relationships. Many of these techniques have concentrated on visual display without sound. Others have focused purely on data sonification, like Kaper, et al., while a few have experimented using both. Alice Preston and Susan Fowler surveyed the sonic scene at a UPA workshop in 2004 and provide a good citation list of examples. Although many innovative approaches to information visualization/sonification have been tried, none has followed the path toward nearly universal, ubiquitous usage that, for example, spreadsheet software achieved during the 1980s.
What will bring these esoteric approaches into the mainstream? What tool will almost every person have on her/his computing/communication device in ten years, one that is taken for granted and comes with almost every device? Whatever form it takes, it seems likely that it must account for several challenges.
The popular tools have some kind of historical precedent upon which to build new functions. Word processing, which far exceeded the characteristics of the familiar typewriter, was very much based on the metaphors of the established mechanical or electro-mechanical devices of earlier years. The same is true of the spreadsheet, which was built on the familiar ledger used for centuries by bookkeepers. Don't forget that when Gutenberg and others invented movable type and developed the printed book we know today, they considered that they were simply improving upon manuscripts, and the nature of the new invention as we understand it today did not take hold for some decades.
What can we find already in use that can be used as a basis for a new information visualization/sonification experience? Certainly Euler diagrams, flowcharts, and other basic diagramming techniques are a start.
Another challenge is the ability of the tool to make content entry, local editing, and global editing (batch mode) easy to accomplish. One of the frequent challenges is facilitating users' skills in making complex Boolean searches without a detailed knowledge of Boolean logic. How can the logic of querying be indicated more effectively by visual means?
Still another challenge is the tool's ability to handle large quantities of information. As a rule of thumb, I have considered the challenge posed to me several decades ago in a project for Professor Andries van Dam from Brown University in thinking about the future of hypertext: As a result of clever Boolean searches, no matter how clever, one may still be required to view and compare 500 retrievals. How can one view 500 entities displayed within one scene, in which each entity has seven, plus or minus two, attributes that may be in seven plus or minus two, states? Let's also assume that we have only a few seconds to evaluate this scene, make a decision, and move on to the next action. For me, this is an archetypal challenge of information display.
A final challenge is that the technique must be workable on a variety of platforms: both fixed desktop displays, as well as mobile environments. At the moment, this means big screens and little screens. All must be able to take advantage of the winning approach. Being compatible with Web 2.0 also doesn't hurt.
As we move forward, let us not forget Herman Hesse's fantasy novel, The Glass Bead Game (Magister Ludi), written originally in 1943. He envisioned a future society in which all knowledge had been converted into music. Those who could hear and play appropriately were masters of the society, in control of its, and their own, destiny. It is my belief that those who have mastered superior information visualization and sonification techniques will be in a position to manage their lives, and others' lives, more powerfully and effectively. Let's hope that power is shared by many, not just a few.
My own fantasy of the future of information visualization, by the way, is one that I envisioned more than 20 years ago: I sketched a future technique that was simply an all-purpose mosaic of pixels that had been coded for spatial position, color, dynamic, and even sound characteristics. What resulted was an abstract representation of many entities in a general-purpose visualization display. Of course, one had to know and remember the semantics to be able to "read" the image. With careful design and training, it could be a workable solutiona simple, efficient mechanism displayable on most hardware platforms.
Whether this technique, or any other, achieves universal, ubiquitous status remains to be determined. One thing seems certain, however: Our future toolkit must employ a mechanism of revealing structures and processes that our current widespread software applications lack.
3. Del.icio.us (a bookmarking site) visualization sites of interest: http://www.solutionwatch.com/252/visualizing-delicious-roundup/ (General roundup), http://www.ivy.fr/revealicious/index.html, http://www.browsedelicious.de/, http://kevan.org/extispicious
16. Preston, Alice, and Susan Fowler (2004). Workshop on Sound in User Interfaces, UPA 2004. Web summary at this URL: http://www.fast-consulting.com/upa%20sounds%20and%20graphics/UPA2004-AWResults.htm.
17. Robertson, George, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten van Dantzich (1998). "Data Mountain: Using Spatial Memory for Document Management" Proc. UIST 1998, San Francisco, p. 153ff. http://www.microsoft.com/usability/UEPostings/p153-robertson.pdf
Aaron Marcus and Associates, Inc. (AM+A)
About the author
Aaron Marcus is the founder and president of Aaron Marcus and Associates, Inc. (AM+A). He has degrees from both Princeton University and Yale University, in physics and graphic design, respectively. Mr. Marcus has published, lectured, tutored, and consulted internationally for more than 30 years.
©2006 ACM 1072-5220/06/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.