Ambient intelligence: the next generation of user centeredness

XII.4 July + August 2005
Page: 21
Digital Citation

From information design to experience design

Norbert Streitz, Carsten Magerkurth, Thorsten Prante, Carsten Röcker

back to top 

It seems like a paradox but it will soon become reality: The rate at which computers disappear will be matched by the rate at which information technology will increasingly permeate our environment and determine our lives. This notion of the "disappearing computer" [5] is one of the starting points that determines our work. Another one is the shift from information worlds to experience worlds. This was a consequence of our work on innovative office environments where we explored the range of social processes that should be supported with information technology and the shift to a new application domain, i.e. games and entertainment in the context of home environments.

The Disappearing Computer. The increasing ubiquity of computers and related devices (e.g., sensors) and their diffusion into our environment requires a rethinking of the complex interplay between technology and humans. The often quoted observation by Mark Weiser, "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it," [9] set the stage for the vision of an unobtrusive, calm technology. Since then, the effort to make technologies disappear into the background has been an ongoing endeavour involving a series of international initiatives and innovative program. One prominent example is the proactive initiative "The Disappearing Computer" (DC) launched and funded by the "Future and Emerging Technologies" strand of the IST programme of the European Commission (

As computers disappear from the scene, become invisible, and disappear from the perception of the users [5], a new set of issues is created concerning the interaction with computers embedded in everyday objects resulting in smart artefacts: How can people interact with invisible devices? How can we design implicit interaction for sensor-based interfaces? How can we design for transparency and coherent experiences? One way of tackling these problems is described in the following examples. It is characterized by returning to the real world as the starting point for design and trying to exploit the affordances that real-world objects provide.

From Information Design to Experience Design. Our initial work on innovative office environments focused on support for "productivity-oriented" processes and was reflected in the design of so called Roomware® components (see Figure 1) as, for example, large interactive walls and tables [4, 8] and innovative multi-user interfaces and software support for brainstorming in electronic meeting rooms [3]. In these developments, we speak of the "mental disappearance" of computers in smart artefacts because they are not perceived as computers anymore, due to smooth integration, but rather as "walls" and "tables" that are interactive. This is complemented by our notion of "physical disappearance" of computer components due to miniaturization.

Addressing the office environment as an integrated organization located in a physical environment and having particular information needs, both at the organization's collective level and at the worker's personal level, we learned that more "experience-oriented" processes are important as well [6]. Examples are awareness of what is going on in an environment, team interaction, and informal communication. This corresponds to an extension and augmentation of our perception and experience of the physical and social environment. It can be reflected in a shift of the focus from developing not only "information worlds" but also "experience worlds." As a consequence, this requires complementing information design by experience design. This "experience-oriented" approach is even more prominent when designing games and leisure activities for future home environments.

Different Qualities of Experience and Perception. In order to address experience design in a more systematic fashion, we investigated and classified the different types of experiences. At a first level, we distinguish between a) direct experiences using our human senses and b) indirect and mediated experiences.

Direct experiences are based on perception using the human senses vision, hearing, touch, respective tactile and haptic sensation, smell, taste, as well as the equilibrium sense. The indirect and mediated experiences can be more complex. Here, we distinguish between perception of "invisible" things, perception using aggregated parameters, and social experiences. The latter categories are not exclusive but have some overlaps.

A typical example of an indirect and mediated experience of invisible things is radioactivity conveyed by the sound of a Geiger-counter. Other invisible parameters are certain components of pollution, traffic over computer-networks, etc. Another aspect is to "look into" real objects, making them "transparent" and identify certain internal components or their ascribed characteristics in a corresponding virtual world (e.g., the "data shadow" or Web page of a physical object or the playing piece example described below). Vice versa, one has the possibility of assigning real experiences (e.g., haptic and tactile properties) to virtual objects. Aggregation of parameters is used in case of conveying higher level properties as, for example, the atmosphere or ambience of a room or building; activities, moods of situations, emotions... Finally, we think of social and group experiences based on communicating closeness, connectedness, awareness, interaction, and—at a higher level—perhaps even friendship.

The task of experience design is now to identify the relevant parameters for a given situation, collect their corresponding raw data, aggregate them—if necessary—at a defined level and in some cases map them to an intermediate representation which is then used for the final mapping to output/display parameters on the different dimensions that are accessible to the human senses. This implies defining certain codes sometimes involving metaphors that tap the everyday background knowledge and experiences of people. In the following sections, we illustrate this by describing examples of designing different types of experiences taken from our work.

Communicating Awareness via Ambient Displays. Our work on creating smart environments for augmenting social architectural spaces in office settings is an example of supporting experience-oriented social processes. The context was provided by the project "Ambient Agoras" ( being part of the EU-funded "Disappearing Computer" initiative mentioned above.

As the guiding metaphor, we chose the notion of the Greek "agora" (marketplace). Here, we investigated how to turn everyday places into social marketplaces of ideas and information where people could meet, interact and experience the "feeling of the place" (genius loci). Design recommendations for the workplace frequently conclude that informal communication, awareness about ongoing activities in the local work environment and a sense of community play vital roles in the workplace. Teams that share the same physical environment generally benefit from increased informal awareness. The situation changes dramatically when we are confronted with the situation of remote teams in physically distributed locations. In this case, the particular design goal is to convey the experience of people being present in the respective locations as well as the mood and the atmosphere. The communicated experience should provide awareness about chances for initiating spontaneous encounters—similar to chance encounters in the hallway or in lounge areas where people have coffee or tea. The support system should facilitate opportunities for informal communication that can take several different forms in a follow-up activity. We concentrate here on conveying experiences between different locations.

We considered that a smart environment based on a calm, ambient technology would best support the informal social encounters and communication processes. Ambient displays exemplify this approach and go beyond the traditional notions of the typical displays found on PCs, notebooks, PDAs, and even our interactive walls and tables. As our version of an ambient display, we developed the Hello.Wall for the Ambient Agoras environment [6, 7]. It is a 1.8-meter-wide by 2-meter-high smart artefact with integrated light cells and sensing technology for identifying people and artefacts. Figure 2 shows how this "display" facilitates communication via dynamically changing light patterns. The Hello.Wall provides awareness and notifications to people passing by or watching it. Different light patterns correspond to different types of parameters. Using abstract patterns allows distinguishing between public and private or personal information. More details can be found in Streitz, et al. [7].

To evaluate how the Hello.Wall and its supporting artefacts could facilitate awareness and communication between two remote teams, we ran a living-lab evaluation between two sites: our lab at Fraunhofer IPSI in Darmstadt, Germany, and our project partner's lab at Electricité de France in Paris, France. We used dynamic light patterns to communicate different types of parameters (Figure 3): the presence and number of people at the opposite site, their general mood, the presence and availability of specific team members, and their interest in communicating with the remote team. The evaluation showed that participants could learn to identify and interpret the Hello.Wall patterns correctly in a short period of time. The participants perceived the Hello.Wall as an appropriate means of establishing awareness of people who were working at remote sites. We are now applying this knowledge to other domains, including home entertainment and extended home environments.

Interactive Hybrid Games. Smart artefacts also play an important role in the design of so-called hybrid or pervasive games. The leitmotif of these future entertainment systems is to bring together the best aspects of physical, virtual, and social dimensions. The result is a new gaming genre that blends game elements from all these dimensions using ambient intelligence technologies. No longer is the graphical user interface with mice, keyboards, and joysticks the primary interaction interface, but players interact directly with each other and may use dedicated tangible or graphical interfaces to interact with the digital world [1, 2].

In effect, there is a unique set of experience qualities that characterizes hybrid games.

Social Quality—Players meet face-to-face for playing hybrid games and exploit rich means of communication including non-verbal hints such as mimics or gestures. Social dynamics can emerge much more easily than in traditional computer entertainment, where the notion of a player is only conveyed through a computer display.

Haptic Quality—Similar to board games, the primary interfaces in hybrid games are tangible artefacts such as smart playing pieces or game boards. The haptic quality of these artefacts provides natural interaction means. Additionally, many physical pieces are nice to touch, to look at, to assemble, or to collect (see Figure 4).

Multimodal Stimulation—Multimodal stimulation adds to the immersiveness of the gaming experience and can effectively be realized with an ambient digital logic. Changing the ambience of the players to reflect the status of the game can vastly enhance the atmosphere of the gaming experience. The modes to stimulate are manifold, ambient light or sound or even creating wind with a simple fan can add to the multimodal stimulation of the players (see Figure 5).

Real-world Parameters—In the same way as ambient intelligence can stimulate the gaming experience by altering physical aspects such as light or sound, these real world properties can also provide an input for the virtual parts of a game. For instance, the background noise level in a room might influence how efficiently digital desk workers in a respective game perform their tasks. Or the light in a room might affect how well digital flowers bloom.

Virtual Attributes of Physical Artefacts—Arbitrary virtual properties that differ from game to game can be assigned to physical artefacts. In one game, an artefact might be associated with attributes such as intelligence or dexterity, in another game, certain behaviours or alignments might be put into the artefact's digital representation providing for context aware enrichments. Even dedicated GUI artefacts might be used to peek into the physical artefact and to modify its virtual state (see Figure 6).

Conclusions. In this contribution, we presented different examples of how we designed experiences by making use of selected parameters and indicators for processes and other characteristics for situations and objects. The application domains range from office environments to interactive hybrid games.

back to top  References

1. C. Magerkurth, M. Memisoglu, R. Stenzel, N. Streitz (2004 a). Towards the Next Generation of Tabletop Gaming Experiences. Proceedings of Graphics Interface 2004 (GI'04) (London, Ontario, Canada, May 17-19, 2004.) AK Peters. pp. 73-80

2. C. Magerkurth, M. Memisoglu, T. Engelke (2004 b). Augmenting the Virtual Domain with Physical and Social Elements. ACM Transactions in Computers in Entertainment. 2 (4), October 2004, pp. 12 - 22.

3. T. Prante, N. Streitz, P. Tandler (2004). Roomware: Computers Disappear and Interaction Evolves. IEEE Computer, December 2004. pp. 47-54.

4. N. Streitz, J. Geibler, T. Holmer, S. Konomi, C. Müller-Tomfelde, W. Reischl, P. Rexroth, P. Seitz, R. Steinmetz (1999). i-LAND: An interactive Landscape for Creativity and Innovation. Proceedings of ACM CHI 1999 Conference (Pittsburgh, May 15-20, 1999.) pp. 120-127.

5. N. Streitz, P. Nixon (2005). The Disappearing Computer. Communications of the ACM, 48 (3) March, 2005. pp. 33-35.

6. N. Streitz, T. Prante, C. Röcker, D. van Alphen, C. Magerkurth, R. Stenzel, D. A. Plewe (2003). Ambient Displays and Mobile Devices for the Creation of Social Architectural Spaces: Supporting informal communication and social awareness in organizations. In: K. O'Hara, M. Perry, E. Churchill, D. Russell (Eds.), Public and Situated Displays: Social and Interactional Aspects of Shared Display Technologies, Kluwer. pp. 387-409.

7. N. Streitz, C. Röcker, Th. Prante, D. van Alphen, R. Stenzel, C. Magerkurth (2005). Designing Smart Artifacts for Smart Environments. IEEE Computer, March 2005. pp.41-49.

8. N. Streitz, P. Tandler, C. Müller-Tomfelde, S. Konomi (2001). Roomware: Towards the Next Generation of Human-Computer Interaction based on an Integrated Design of Real and Virtual Worlds. In: J. Carroll (Ed.), Human-Computer Interaction in the New Millennium, Addison-Wesley. pp. 553-578.

9. M. Weiser (1991). The Computer for the 21st Century. Scientific American, September 1991, pp. 94-104.

back to top  Authors

Norbert Streitz
Fraunhofer IPSI

Carsten Magerkurth
Fraunhofer IPSI

Thorsten Prante
Fraunhofer IPSI

Carsten Röcker
Fraunhofer IPSI

About the Authors:

Norbert A. Streitz is the head of the "AMBIENTE-Smart Environments of the Future" research division at Fraunhofer IPSI, Darmstadt, Germany, where he also teaches in the Department of Computer Science at the Technical University Darmstadt. His research interests include cognitive science, computer-supported cooperative work, human-computer interaction and interaction design for ubiquitous computing. Streitz has a Ph.D. in physics and a Ph.D. in psychology. He is the chair of the Steering Group of the EU-funded initiative "The Disappearing Computer."

Carsten Magerkurth is the deputy head of the "AMBIENTE-Smart Environments of the Future" research division at Fraunhofer IPSI, Darmstadt. His research interests include ubiquitous computing, user-interface design, and pervasive gaming. Magerkurth has a diploma degree in psychology and is now finishing his Ph.D. dissertation in computer science.

Thorsten Prante is a scientific staff member in the "AMBIENTE-Smart Environments of the Future" research division at Fraunhofer IPSI, Darmstadt, where he also teaches in the Department of Computer Science at the Technical University Darmstadt. His research interests include context-aware information management, user-interface design, and computer-supported cooperative work. Prante has a diploma degree in computer science and is now finishing his Ph.D. dissertation in computer science.

Carsten Röcker is a scientific staff member in the "AMBIENTE-Smart Environments of the Future" research division at Fraunhofer IPSI, Darmstadt. His research interests include ubiquitous and ambient computing, awareness and privacy in sensor-based environments. Röcker has a diploma in electrical engineering and is now finishing his PhD dissertation in computer science.

back to top  Figures

F1Figure 1. Second generation of Roomware® components

F2Figure 2. The Hello.Wall—an ambient display showing continuously high-quality aesthetic patterns conveying presence and mood parameters

F3Figure 3. The Hello.Wall patterns express: (a) three different levels of mood and (b) three different levels of presence—low, medium, and high.

F4Figure 4. Physical game boards and playing pieces conveying haptic qualities

F5Figure 5. Multimodal stimulation with ambient intelligence technologies

F6Figure 6. Displaying virtual properties of physical smart artefacts

back to top 

©2005 ACM  1072-5220/05/0700  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2005 ACM, Inc.

Post Comment

No Comments Found