In 1991, Pierre Wellner introduced the world to the DigitalDesk . This was the first attempt at revolutionizing office work by integrating the best of the tangible and digital realms. Since then, technology has developed: The Internet has connected us, and large interactive screens are fast becoming a fixture of the modern workplace. Yet despite the romance between computer and desk, the details of a possible relationship have remained elusive and curiously vague.
Interactive desks are an idea with a rich history in research. Much of this has focused on short, walk-up-and-use scenarios that focus on evaluating specific aspects of interaction. Subsequently, this has led the technology away from the office and into niche roles suited to specific problem domains. Even today, 21 years later, it is rare to find studies that look at the longer-term impact of interactive desks for general productivity [2,3]. As a result, we lack a contextualized understanding of the interactive desk and its potential roles in the modern office. Given the importance of and trust placed in computers today, are we ready for interactive desks to be considered a serious replacement for the personal computer? Or, critically, are they expensive toys that lack significant transformative benefits?
Curious about the possibilities, late one evening in December 2010 I carried several lengths of wood, a projector, and some tools into my office with the intention of building myself an interactive desk. The striking construction that greeted my surprised (and slightly bemused) colleagues the next morning marked the start of a year that I would spend using it as my primary means of interacting with the digital world for both work and pleasure.
The desk (shown here) was based on a desktop computer, a 24-inch monitor, and a projection pointing down at a simple white table. Interaction was possible using LED-tipped styluses in addition to a wireless keyboard and mouse.
The experience shed light on some of the elusive missing details: a glimpse into the subtle affordances an interactive desk can offer once you let it influence how you work . Here, I reflect on my experience and these different influences, and offer my perspective on the future.
Shortly after I sat down to begin work, one of the first things to become apparent was that the limited pixel density of the projection (approximately 20 DPI, relative to the approximately 96 DPI of a standard monitor) influenced what it was possible to use the desk for. For instance, while I was able to adjust to reading short bursts of text on the desk, I was never able to comfortably use it for longer, focused reading.
Prolonged exposure to bright projected light soon became unpleasant, especially in the winter evenings. Fortunately, selecting a black desktop background and turning down the brightness of the projector alleviated the issue. However, if a window was maximized to the desk, it would again flood my visual field with light. Similarly, the differences in DPI between the monitor and the desk meant that windows that were relatively small on the monitor underwent a jump in size when moved onto the desk. Although I understood why it happened, this was unexpected and annoying.
An oft-cited criticism of top projection is occlusion; if I were to stand up and lean over the desk, it would cast a shadow over my interface. However, as I was mostly seated while working, this was rare. Until we can explore possibilities such as the e-ink desk, top projection offers some interesting qualities. For instance, it lacks a bright backlight, has the ability to overlay onto existing objects such as paper, resists spilled coffee, and of course, allows the desk to act as an impromptu seat.
For most tasks, the fast and precise interactions offered by the mouse and keyboard vastly superseded the stylus. While their ease of use is undoubtedly affected by practice and the design of the UI, there are underlying reasons why they remained my preferred input method. First, after long periods of time, it became tiresome to repeatedly use the big muscle groups in my back, arm, and shoulder to perform tasks that could be achieved with a mere flick of the wrist using the mouse. Second, the mouse and keyboard could be used to interact with both the vertical and horizontal display planes. Since the monitor lacked a touch screen, the same could not be said for the stylus. Last, the adaptability and generality of mouse and keyboard meant it was easy to flip from task to task without worrying about interrupting my workflow to swap interaction modality.
To enable more ways of arranging the digital items in my workspace, I added the ability to rotate and scale certain application windows. This both looked good and offered a solution to the size jumps caused by DPI differences between the screens. Naturally, shrinking windows made them harder to read, but this was not always a big problem for applications that displayed video or featured large UI controls.
Larger display sizes are often considered a high-value feature , and I was keen to see what kind of impact the desk would have on how I worked. As you might expect, technical issues like projector resolution meant that certain tasks were better suited to a given surface. The desk provided an excellent area to organize my thoughts and keep track of my schedule using a projected to-do list inside of a virtual sticky note. It was a space for peripheral awareness, peripheral applications, visual organization, subtask triage, group review, and the temporary storage of files and notes. By contrast, the monitor was an area for focused tasks, such as reading, writing, programming, and Web browsing.
The physical separation of the surfaces could also be used to reflect a logical separation within and between different aspects of work processes. For example, I would often take note of, organize, and sort through various things (thoughts, sources, images, etc.) on the desk before integrating them in a more complete form using the monitor, where I had better control over details. This is not dissimilar to habits observed using virtual desktops or multidisplay solutions [6,7]. However, unlike a virtual desktop, the contents of my periphery remained immediately visible, and unlike multiple displays, the desk space offered more creative ways to arrange the contents of my focal and peripheral zones.
Part of the value of being able to creatively arrange your working environment stems from epistemic actions: the act of modifying your environment to put yourself in a better position to think, solve problems, and extract information from your surroundings . The desk expanded the palette of such actions, allowing me to mix the physical and digital, juxtapose items, and play with layouts by changing position, size, color, and orientationwith the layouts all remaining in view and sharable with those around me.
An example of this in practice is document planning done by distributing thoughts over sticky notes. Physically grouping them by relevance allowed me to use color to indicate additional details like belonging within a specific theme. Other examples include projecting icons onto a physical inbox as a reminder that they should be dealt with soon. In a collaborative context, placing and orientating objects toward a person can be used to imply ownership or attract attention.
Most of what I perceived as a benefit of the desk stemmed from its role as an output device rather than an input device.
Through these kinds of habits, I came to see the desk as an extension of my thought processesa memory aid, like any desk, but also integrated more fluidly with a digital context. I could quickly manipulate, save, and share with none of the cumbersome qualities of physical objects, but with all the transience, speed, and connectedness of the digital world.
In terms of personal expression, I was able to decorate my workspace using virtual and physical items to suit my taste. My desk played host to an array of clutter: bottles, mugs, paper, icons, sticky notes, laptops, and digital fish, which would swim around unintelligently, appearing to hide behind my monitor and beneath my keyboard. Even with a setup as simple as mine, there was scope for interplay between the physical and virtual. For example, I placed my recycle bin over a hole in my desk so that as files were dropped into the hole, they were simultaneously deleted.
Taking this concept to the next level, Lui et al. recently demonstrated how a USB stick embedded within a glowing ball can be used as a way of playfully sharing files in an office . They argue that this is far from trivial: There are characteristics to such interactions that will be key to supporting the next generation of office workers. Other such features include the ability to share data directly between the desk and personal devices [10,11]. This is suggestive of a much broader design space, a function of which would be to support a diverse technological ecosystem.
Horizontal surfaces are known to promote collaboration. Indeed, the desk was brilliant for facilitating quick discussion and collating ideas. We would often flip up to the monitor to Google information. However, like most multitouch devices, the desk interface was not multi-user. Being unable to distinguish different users at the application level has limitations, but the most severe limitation in my experience was that the operating system supported interaction with only one application at a time. This had the effect of forcing everyone at the desk to be constantly aware of how and when those around them were interacting. It severely restricted the number of ways in which collaborative work could take place.
Having the interface assume a single user makes sense for personal computers and mobile devices. However, as computers become more exotic and increase in size, this assumption begins to break down. Most applications designed for interactive desks work around the issue by replacing the desktop environment with a full-screen application that offers a complete interface. However, my work relied on being able to use a diverse range of applications for different tasks. If the computer as a general-purpose platform is something we wish to preserve, this will need to change.
Although there were many technical limitations, none were severe enough to prevent me from accepting the desk into my workflow or achieving my goals. Nevertheless, the influence of these limitations on how I interacted and established practices was very high. Factors like low resolution were silent governors over how I did what I did. While this would probably change given better technology, I remain convinced that my experience would not have been as positive had it not been for the failsafe of the vertical monitor. However, as I was the developer of the system, my tolerance to problems is considerably higher than would normally be the case.
Most of what I perceived as a benefit of the desk stemmed from its role as an output device rather than an input device. Even with limited UI controls, the size of the desk encouraged me to spread information over my working environment and throughout multiple applications. Indeed, different tasks invoke different cognitive processes, and it is around a user interface that many of these processes are often structured. Giving people control over their environment allows them to construct processes that are illustrative of any personal requirements and the task they are doing.
It is worth noting that much of my experience was dependent on my willingness to customize the desk, reflect on my work processes, and experiment with the interface. Not all office workers will have the inclination, or be in a position, to do so. This perhaps presents a complex design challenge.
Overall, the exploratory and metaphorical nature of the traditional computer desktop exists comfortably within the real world. My understanding of the interactive desk remained within the world of mice and windows, prioritizing generality over intuitive interaction techniques. Given the opportunity to change the interface, I would first expand the palette of virtual items. A new, enhanced set could take advantage of more layout capabilities and the understanding that can be implied by physical surroundings.
I suspect the greatest initial benefits will come from enhancing window-management capabilities rather than providing more intuitive interaction techniques with content.
Despite being peripheral to the current commercial focus on portable and personal devices, the improvement of multi-user support, window management, and display technologies has immediate scope. Yet, even given these improvements, I suspect the interactive desk needs to offer people more.
Computers first existed to automate calculations, and later, to draw better graphs and communicate, without the need for an office clerk. For those things, my desk was no better or worse than a normal computer. It was not a silver bullet for more productive output. But perhaps it shouldn't be. After all, productivity involves skills and creativity honed through education and experience.
What the desk represented for me was an ability to take the digital world off the screen and combine it with the physical world. This changed how the computer appealed to my human side: my ability to interpret a space, operate within it, and from that, construct and share understanding.
Predicting the future is a risky business, but I can't help but wonder if the interactive desk is a transitional vision: a piece of the puzzle that hints at a large, diverse, technological ecosystem that has yet to emerge. For now, it feels to me like a technology displaced in time, able to find a home in niche roles, but not as the replacement for the personal computer that I had hoped for when I first switched it on.
Assuming we choose to continue blending the virtual with the physical, perhaps there will be a race between creating the first accessible, general-purpose interactive desk and the development of houses with interactive work surfaces and wallpaper. In the latter world, would the role of the interactive desk be to provide a hubnot a computer in the traditional sense, but a platform that can be appropriated by different devices and information sources so that each might represent its own interest in a form that is reactive to context, sharable, and malleable on a large interactive canvas?
My belief is that if you give people the tools, they will use them to build ideas. As our technological ecosystem develops, I suspect that providing access to novel technology like the desk will help us to continue developing an understanding of the broad range of scenarios and contexts that it will one day support.
Thanks to all who reviewed this work, and in particular to Mark Ward, who drew the infographic. This project is supported by the RCUK-funded Centre for Doctoral Training, HighWire (highwire.lancs.ac.uk) - grant reference EP/G037582/1.
John Hardy is a second-year Ph.D. student studying digital innovation at Lancaster University. He researches original computer interfaces and is the technical director of the STExcalibur project.
©2012 ACM 1072-5220/12/11 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.