Ambient intelligence: the next generation of user centeredness

XII.4 July + August 2005
Page: 50
Digital Citation

Interaction contextualized in space

Marco Susani

back to top 

"Architecture has to be an object of our memory, and that memory is a form of architecture."—Louise Bourgeois, artist and sculptor

Original interactions with personal computers focused on doing (writing a document, making calculations on a spread sheet...), and its physical context was a single individual user sitting in front of his or her computer, isolated from the surrounding physical space. Today, the merging of computer technologies with telecommunication technologies and wireless network access, calls for a completely different paradigm: computers, telecom, and networks are not only a complex set of tools to "do." They are, above all, communication media to connect people, collaborative spaces to meet others, and tools to access published, dynamic, ubiquitous information.

These sets of interfaces are clearly not exclusively mediators between machines and individuals, but much more often the interfaces are "places" to inhabit, or "orders" that facilitate access to knowledge. And, clearly, the user is not alone, but is part of many communities of different scale.

The "desktop" and the "point and click" in that digital space start, in fact, to feel the "competition" of other models of interaction, namely the information architectures organized around email and IM, that today often dominate when compared with desktop access to files and folders.

Still, the personal computer limits its interaction to the screen and input devices, and is agnostic of the physical context of the user.

The interaction with mobile wireless media has intrinsically different characteristics: It is focused on communication, it is inherently social and, being mobile, it is highly contextualized.

A kind of translation of the conventional PC GUI paradigm is dominant in the interaction with wireless media like the cell phone: primitive graphic menus and icons arranged in the small screen of the mobile device.

But that is only the most superficial aspect of the interaction with devices like cell phones: There are multiple other aspects of interaction with cell phones that are less tangible, yet more important, than the keypad and screen UI. The whole experience eco-system of the interaction with wireless media includes the combination of the interaction with the physical device, with the services, and with the physical environment.

The two following scenarios emphasize the contextual aspects of interaction with wireless mobile media, taking advantage of innovative interaction technologies.

The main inspiration for these innovative interaction paradigms is the need to update the interfaces to this new situation; to take a drastic departure from the existing interfaces and build on the characteristics of wireless media:

  • connecting information to the physical space
  • creating and editing information through mechanisms of social sharing in physical proximity

These scenarios are also based on emerging technologies for mobile media; on the side of the terminal, a complete integration of cellular technologies with bluetooth, RFID, and Wi-Fi, and, on the side of services, the diffusion of location-based content and mobile blogging.

Point and Click at Real Space. The Hearing Hand scenario anticipates the widespread diffusion of RFID tags, and similar proximity of RF technologies, in public environments like shops and public transportation. There are already cases of RFID technologies in Asia being used as payment systems and tracking systems for supply-chain management. The first are based on conventional cell phone-form factors, and the latter are so far limited to industrial solutions.

The Hearing Hand is based on a Hand Wearable form factor that has the ability to "point and select" to artifacts in the physical space (such as magazines, clothes tags, store shelves, public transportation billboards), and access visual and audio information contextualized around these physical artifacts and spaces.

Collective Wisdom around Things. This scenario is about smart tags, and what they may enable people to do in the future. Smart tags can be attached to anything—items of clothing, food cans, food itself, and can be programmed to provide information about the items to which they are attached. The tags can also take note of their environment, recording changes in temperature, etc. This latter feature is of particular importance for food items—shops and customers can know the age of a product and the conditions that it was subjected to on its way from producer to the shop.

This scenario explores how smart tags may enable the creation of digital word-of-mouth-similar to the spreading of information and knowledge by human word-of-mouth-in the context of agricultural products. Information and knowledge are attached to items to which digital technologies allow access. In the case of the fruit and vegetables, this will happen on the field where they grow. This will be related to location-based agricultural data, but with additional further information related to annotations, previous experiences, and common sense, word-of-mouth on the material culture needed to grow the produce in the best way.

Experiments on connecting collective wisdom with digital systems on the field have been undertaken in India since the 1990s. The fully-fledged scenario will allow replication in a digital environment, of the wisdom of traditional agricultural word-of-mouth know-how. This would create a kind of "knowledge aura," a container of "collective wisdom" around the produce, and it is an interesting potential of future material knowledge systems.

The integration of these information system tags that are able to tell, at any moment, about of the life of any item of produce, will allow the quality, the origin, and the value to be tracked in real-time. Produce that is fresh or older, in season or off, near or far from its origin, or with or without a track record of its growth, will have different values.

The whole idea of advertising as building attraction before and separate from the experience of purchase can be revolutionized by a constant flow of information at the point of purchase. Even more intriguing is the idea that deeper information about a product, connected to similar experiences of purchase and use—what is known as a recommendation on a Web site—will be readily available at any moment that a product is encountered. The lure and temptations that attract a buyer will come directly from the product, from the background voices of people that bought and used it, and by the life history of the product; a kind of very detailed, dynamic, digital certification of authenticity.

The idea of an aura of knowledge that is associated with products or produce takes a further twist when it as a networked knowledge that is the sum of what comes from different products (or the produce in a recipe), combined with the annotations describing the experiences of people. Instead of extracting knowledge from books (recipe books, for example) to make a pie, digital information can dynamically manage the cloud of information around all the ingredients while one prepares the food. It is the model of apprenticeship returning, enhanced by all the potential of networked systems. Any apple, and the sugar, and wheat—every item—will bring to the moment the information about where it came from, how it was grown, and how it has been manipulated. This can be compared with the genetic code embedded in all living things. Instead of genetic information, the tags will provide a rich description of the life of the product and how it relates to other items.

The Technical and Social Components of These Scenarios. The scenario we propose is an extension of what we can already forecast with the introduction of smart tags. One sure fact is that smart tags are being adopted for functional uses, such as tracking goods and managing the supply chain, like keeping track of components in a factory or between suppliers and factories. Less certain is the fact that Smart Tags may be adopted as a standard, a kind of smart replacement of other ways of tracking goods, like the universal UPC bar codes we see on today's products. A standardization of codes, technologies and data, in addition to ensuring an efficient cross-company use of the tags, would also allow what is the main point of this scenario: the building of a knowledge system over the functional system. In other words, this would be the equivalent of building the culture of book over the technology of the print. The most important components needed to enable these scenarios are:

An evolution of smart tags. Today, smart tags come in one main form that is extremely cheap, requires no power, and holds one piece of information, the identification code of an item. Tomorrow, other more sophisticated forms will complement this: slightly more expensive versions may hold more data and be recognized at a distance (today's smart tags are only readable in close proximity, several centimeters from the reader). If tags would hold more data, we can imagine they could also "execute" actions, like small computers embedded in objects.


An evolution of smart tags readers. Readers are machines that are able to `talk' to the Smart Tags and read their data. Today these are special handheld machines, but we can foresee the diffusion of these readers in many objects we use everyday, like cell phones, cash registers, or the diffusion in environments, like offices or factories, but also in public places like shops or transportation systems. Also, readers that could better identify products in a space would allow an easier interaction with goods, a more natural way to access information, such as pointing at a product on a shelf and requesting, "I want information about this product..."

A standardization of protocols. Present systems are proprietary, so they work only with a certain type of tag and the right reader. In addition, the data is not "universal," as it refers to an archive on the Internet, but belongs to one single company or entity. When any data in tags could universally be interpreted, the creation of a true ecosystem will enable the scenario just described.

A true, complete functional ecosystem. Since all tags would refer, via their readers, to information on the Internet, the potential of the system to manage rich information is immense. But in order for this to happen, a truly interoperable system is needed. Universal data definition, that we mentioned already, is only the first step. What is needed is a series of enablers and protocols and meta-tag definition that may crate this ecosystem. And, of course, a series of accessible data repositories that would allow anybody to know everything about...the life of the apple.

A knowledge ecosystem built over the functional system. This aspect is only in part technical, and implies contractual, legal, and cultural changes. The "knowledge" eco-system would comprise the recording, storage and availability of information about goods. To give an example, today's nutritional information is very limited; it refers to an apple, not to this apple. Seen with the eyes of the possible future we described, the information we access today with a conventional label of, for example, packed food, is extremely primitive. And it is not networked and "tailored" to the user. There is no way to connect it with other goods, or to go deeper and "ask more" about one aspect. Another part of this system, of course, would be also an aspect of information selection and filtering, because the scope of this scenario is definitely not the one to be surrounded by more information, but by better information.

A dynamic, distributed, "grass-roots" knowledge system. There is another aspect of the present information connected to goods: It is managed with the model of a "broadcast." Like information in TV and mass media, information about goods is circulated with the model of one (the producer) to many (the consumers). But what if this social model would be broken by the real essence of the technologies we just described, that are networked, distributed, pervasive, and "peer-to-peer?" What if we could "hear" the word of mouth about how the apple was grown? What if, attached to goods, we could access informal annotations of other people on how to cook a recipe? It would be like a kind of blog accessible through interaction with "real life" goods in a shop and products on an assembly line. It could be a dramatic change in the way critical "material" knowledge is generated and circulated, more similar to the ancient model of the oral tradition than the model in the age of mass media.

Social Space as Mediator of Experiences. In the beginning of the Internet era, the virtual communities blurred the geophysical and territorial dimension of the community. The social space of these communities in the global village opposed the old idea of "place" in architecture. It was "anywhere:" a generic and neutral entity, abstract from the physical territory.

With the first diffusion of wireless networks, the space of the city also risked losing the role of social places where people meet and interact with the public space: When mobile and ubiquitous computing can connect people to their private, remote info-spaces, the risk was that these individual info-spaces may disconnect humans from the material space of the city, or from any physical shared space (a pub, an office, etc.).

All design choices in these scenarios have been based on the opposite concept: that information has no meaning if it has no connection with material, public space, and if it doesn't have mechanisms for the social sharing of information.

The first consideration (connecting information to the physical space) has influenced all the interaction design proposals for what regards the "front-stage" interaction (how people access information, and interacts with the system), while the second (creating and editing information through mechanisms of social sharing) has influenced the interaction design proposals for what regards the "back-stage" interactions (how information is managed, filtered, and pushed, how information is connected to people, events and locations).

The synergic overlapping of digital information on the physical and social territory creates a hybrid, a richer and original space for social interaction. Interactions in this space happen at different scales between people and their communities, physical artifacts, and the space itself and are strongly contextualized. A "third space" emerges, created by the synergy between the physical and the virtual space, that has peculiar characteristics: It is fluid, dynamic, intangible, but it can be "inhabited" and serve as a catalyst of social relationships.


If such scenarios become reality, space becomes the main structure to navigate and interact: The space of mediated interaction is a social digital space overlapped to physical public spaces.

Mobile and ubiquitous computing change from "Anywhere, Anytime" into "In This Specific Place, In This Specific Moment, for this Specific Person", and digital communication is re-connected to the shape of the territory, to the "Genius Loci."

Urban architecture would again become a container of individual and social experiences, a real-time and real-space system of social relationships, and in part also a collective memory to allow the sedimentation these experiences into a shared contextual word-of-mouth.

back to top  Author

Marco Susani
Motorola PCS

About the Author:

Since October 2000 Marco Susani has been the director of the Advanced Concepts Design Group of Motorola PCS, in Cambridge, Massachusetts, working on the future of mobile networked communication. His works have been shown in numerous exhibitions and honors include the ID magazine Concept Design Award in 2001. He has co-authored the books Interface Design, Seamless Media, The Solid Side, Service Design, Presence, and New Media for Older People and he is currently preparing a book on social aspects of mobile wireless communication. Susani is on the advisory board of the Interaction Design Institute in Ivrea, Italy, and has been visiting a professor and external examiner at the Royal College of Art in London, and a professor of New Media at the University of Siena, department of Communication Science.

back to top  Figures

UF1Figure. Tags would "certify" the breed of seeds


back to top 

©2005 ACM  1072-5220/05/0700  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2005 ACM, Inc.

Post Comment

No Comments Found