“Oh, cool. It looks like a giant insect.” This might be your first thought when you look at Image 1 on the facing page and read the caption. If you have ever been to a Burning Man festival, you may also start reminiscing about the sun, the dust, and all the awesomeness and creativity you encountered there.
But what you are not so likely to think of is the overpowering way in which the vehicle moves: one slow, giant step at a time with sinister pauses in between. Or how the ground shakes under your feet as the vehicle walks past you, since it happens to weigh several tons. Or how it walks back and forth slowly across the festival area in a straight line like a caged animal, since there is no way of turning it. Or the spine-shattering bounces of the vehicle’s riders. Or the deadpan demeanor of the caretakers surrounding the vehicle who brush bystanders out of the way to prevent fatal accidents.
These are things you would remember if you were there and saw the vehicle in the desert sand. They are also things you tend to notice when you watch a video clip showing the vehicle in action.
In our academic field of inquirywhether we call it interaction design, experience design, or new mediamuch of what we talk about is centered on experience. We design artifacts to catalyze new use experiences or to improve upon existing ones. We study existing artifacts and practices, again, increasingly with an eye toward use experiences. We do all this to construct and communicate knowledge, the signature task of an academic. But when we communicate the knowledge we have constructed, we do so almost exclusively in the medium of text, with a few images.
There seems to be room for some improvement in our choice of communication media. Christine Satchell reminds us interaction design has a proud tradition of using impulses from other fieldsartistic as well as scientificto improve upon its design and research methodologies. Doing the same for knowledge communication should not be entirely foreign to us. And indeed, it is easy to note how design-oriented researchers within our field are starting to use richer media to disseminate their results and how effective that form of communication may be.
A case in point might be the work developed at NYU’s Media Research Lab on “Frustrated Total Internal Reflection,” which is used in the biometrics community for fingerprint sensing (the first patent was granted in 1965). Jefferson Han and his colleagues applied this principle to build a multitouch, high-resolution input technique for large touchscreens, which they presented in a paper at UIST in 2005.
I have to confess that I didn’t hear about the work at the timeI wasn’t able to attend the conference; I didn’t notice Han’s paper when browsing the conference proceedings; and I never heard about it in professional conversations or through my online channels. Yet my colleagues and I claim to be scholars in interaction design, and we teach innovative interaction techniques to students every day.
Han and his colleagues moved on to create a set of compelling demos, illustrating the experiential qualities of multitouch interaction with a large display. When Han presented the demos at TED2006 and the talk was published online as a video, the impact was suddenly phenomenal. As I recall, everybody talked about it, and at least 10 separate people shared the video link with me. Research strategies were reformulated, future scenarios rewritten, and whole new families of commercial products were developed and released in record time. Today it’s difficult to believe that the idea of multi-touch smartphones was more or less unknown only five years ago.
At press time, Han’s 2005 paper has been downloaded almost 12,000 times from the ACM Digital Library, and it is cited 500 times, according to Google Scholar. These are very strong numbers for a five-year-old scientific paper. The TED2006 demo video, on the other hand, has been viewed nearly 350,000 times on YouTube alone (it is also available on ted.com and other servers).
What the multitouch case suggests is that video can play an important role in the communication of experiential knowledge within our field. Interaction techniques, the temporal dynamics of interaction, multimodal interfaces, and complex real-world use contexts are a few examples of research fields where text plus a few images does not do full justice to the knowledge constructed in the research process. This is, unsurprisingly, better understood in the communities of design research and artistic research, where there is a lively and ongoing discussion on different ways to incorporate the artifacts as parts of knowledge contributions.
But hang on: Didn’t I communicate (or try to communicate) the experience of the giant insect vehicle at Burning Man in my second paragraph, using text plus an image? No, not really. I verbalized it and tried to evoke it using emotionally loaded adjectives, present tense, and other linguistic devices. I talked about the experience. Being there and seeing/feeling/hearing/smelling the vehicle within the whole ambience is having the experience. Similarly, playing with a large multitouch screen is something quite different from reading about what large multitouch screens look and feel like. Watching a video is not the same as playing with it, to be sure, but it has the potential to communicate more of the experiential knowledge embedded in the design than a paper could. In the words of design researcher Nigel Cross, the research question you approach when designing as part of a research process is “How can we design an X (that does Y, or has the property Z)?” It seems quite obvious that an answer expressed in a medium that accommodates visual, auditory, and temporal aspects is a richer answer.
A video forum has the potential to make the knowledge gained from scientific communication richer and more useful, by providing a deeper understanding of the experiential aspects of the published contributions.
This is not to say that we as an academic community haven’t tried to do justice to experiential knowledge using our default communication medium of text plus a few images. In scientific articles concerned with interaction experiences, it is not uncommon to see temporal representations such as storyboards or textual narration. Some scholars aim at operationalizing experience through systematic evaluation methods. Others feel that the qualities of interaction experiences can and should be articulated, for instance through criticism. My position is that there are even greater communicative benefits to be had for our community by going beyond the default medium.
But when we think of video in interaction design and experience design, we can’t help thinking of slick demos and sales pitches, of persuasion and rhetorical manipulation. As pointed out by Jeff Bardzell, the common practice of showing close-ups of the detailed touchpoints of a specific interaction has the effect of diminishing the viewer and dragging them up close to a point where it is hard to maintain a critical distance. Similarly, the faceless voiceover of many online, screen-based demos serves to induce uncritical belief (the omniscient narrator, not to be questioned).
It would be naïve to call for honest or transparent video in scientific communication, but a partial solution to the dilemma might lie in evoking the conventional criteria for scientific communication. Specifically, I am thinking of the notion of “criticizability.” What this means is roughly that a scientific argument must be constructed in such a way that there is a clear path from goal to means to claim. At every point along the path, it should be possible for peers to assess the grounds for decisions taken and conclusions drawn. In some scientific communities, this is called internal validity; in others, the terms “repeatability” and “generalizability” are more common. But they all ultimately amount to the requirement that knowledgeable peers should be able to judge for themselves whether they choose to believe the argumentative construction of the contribution at hand.
I am afraid that all got a bit technical, but the simple point is that video in scientific communication should be criticizable, just like articles, conference papers, and monographs. Similarly, conventional quality criteria like novelty and relevance would be as important for scientific video as they are for the more established forms of scientific communication.
On a related note, every medium has its genres. In the medium of text plus a few images, there is a venerable genre of academic writing that all aspiring researchers spend years mastering. The video medium does not really have an equivalent genre of “academic editing,” but what we can and should do is to start experimenting with how related video genres can be adapted in the service of scientific communication. For example, as pointed out by John Zimmerman, there are established formats for documentary and news video that would suggest, among other things, that a screen-based demo can be presented through shots of an identifiable person talking, interwoven with footage of the demo screen while the person is still talking in order to remedy the faceless-voiceover problem mentioned earlier.
As you have surely guessed by now, the reason I am writing this is that I am involved in creating a forum for scientific video communication within our field of inquiry. More specifically, the aim is to launch an archival publication for video articles in interaction design, experience design, and new media as part of the online presence of ACM Computers in Entertainment. The magazine will be peer-reviewed, and acceptance is selective. I work with an outstanding panel of co-editors (see below, and together we plan details like submission formats, author guidelines, review criteria, and how to launch and operate the new video forum. By the time you read this, we should be in the middle of soliciting contributions: innovative and scientifically sound video articles to be reviewed and possibly published in ACM CiE.
An obvious final question to ask is whether we really need another academic publication. After all, there are already more conferences and journals than most of us would care to stay abreast of. From the reader’s point of view, I think the answer is clear. A video forum has the potential to make the knowledge gained from scientific communication richer and more useful, by providing a deeper understanding of the experiential aspects of the published contributions. From the point of the view of the author/videomaker, however, it is not as clear that the new forum will be an attractive choice of publication venue for a knowledge contribution. I think there are three crucial factors here. First, the new forum needs to be academically credible. If you publish a video article, it must be something you can put on your résumé of proper scientific publications, rather than having to lump it with general-interest magazine articles and public talks. A properly peer-reviewed forum, which is known to reject inadequate submissions and which resides within an archival scientific journal such as ACM CiE, might just do the trick. Second, the idea of publishing a video article might appeal to the more design-oriented researchers in our field as an innovative possibility and a way to engage in unconventional forms of expression (which, in my experience, is attractive to many designers). Third, the idea of communicating in videoand the necessary craft skills to produce videois less foreign to our field than to many other scientific communities, which might mean that the initial threshold is not intolerably high.
The idea of a scientific communication forum for video articles is certainly not new. There are successful examples in other communities, where the media-specific qualities of video are used to enhance the quality of scientific communication (e.g., the Journal of Visual Experimentation’s www.jove.com, which builds on the benefits of video to communicate experimental procedures in the life sciences). In our own field, I remember seeing my first conference video track at CHI ‘89, which was my first CHI and where the video program was broadcast nonstop on internal TV channels in the official conference hotels. And you could, of course, buy the VHS cassette by mail order after the conference. Today most of our conferences feature video tracks along with demo tracks, exhibitions, and other ways to get a more hands-on experience of the contributions presented at the conference. A common practice seems to be to publish video contributions as short papers with supplementary video clips. What we seem to lack, though, is an archival-quality forum for sustained and scholarly communication on scientific topics that are primarily audiovisual. I suggest the time is ripe to create one.
Contact me if you would like to find out about the current status of the work or if you want to help in creating the forum. Even more important, contact me if you were inspired by this to start thinking about how to publish your work as an archival video article.
Jonas Löwgren is professor of interaction design and co-founder at the School of Arts and Communication (K3), Malmö University, Sweden. He specializes in cross-media products, interactive visualization, and the design theory of digital materials. Löwgren has taught interaction design courses at universities and in companies since the early 1990s and initiated the influential two-year master’s program in interaction design at Malmö University in 1998. He has published some 50 peer-reviewed scientific papers and three books, including Thoughtful Interaction Design (with Erik Stolterman, published by MIT Press).
FORUM CO-EDITORS: Jeffrey Bardzell, Indiana State; Carl DiSalvo, Georgia Tech; Jodi Forlizzi, Carnegie-Mellon; Kristina Höök, Mobile Life and Stockholm University; Jon Kolko, Austin Center for Design; IIpo Koskinen, Aalto University; Christine Satchell, QUT and University of Melbourne; Erik Stolterman, Indiana State; Jeremy Yuille, RMIT; John Zimmerman, Carnegie-Mellon
©2011 ACM 1072-5220/11/0100 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.