Daniela Petrelli, Luigina Ciolfi, Dick van Dijk, Eva Hornecker, Elena Not, Albrecht Schmidt
[T]he museum’s preoccupation with the information and the way it is juxtaposed to objects ... immediately takes the museum visitor one step beyond the material, physical thing they see displayed before them, away from the emotional and other possibilities that may lie in their sensory interaction with it. Sandra Dudley 
It sounds banal to say that a core feature of any digital artifact is its lack of materiality, but when dealing with traditional cultural heritage this becomes a serious issue, as materiality, authenticity, or “aura” cannot be transferred to the digital. A digital copy of a Le Corbusier drawing supports analysis to a greater level of detail than its paper counterpart, but the feeling of being in the archive, the emotion of touching the same paper as the master, and the smell of dust and years past are what makes the experience unique and unforgettable. Emotion, affect, and sensation are essential parts of the experience of heritage, “[y]et museums’ preference for the information over the material, and for learning over personal experience more broadly and fundamentally conceived, may risk the production of displays which inhibit and even preclude such affective responses” .
The “information over object” approach has influenced the use of digital technology in cultural heritage ever since computers started to populate the exhibit floor. The intent has been to provide in-depth information and to support different learning styles. Indeed, visitors spend more time on site if technology is available, but a close observation shows friction between the technology and the heritage context. To begin with, the carefully prepared content is rarely looked at in full; interactive games are often for a single user while others queue; and visiting together can involve sharing the earplugs of the audio guide, one each.
Whatever the form of heritage , some physicality and materiality is usually more conducive to social enjoyment and sharing. Science museums exploit tangible and bodily interaction as an effective way to engage visitors to explore concepts, ideas, and objects. More traditional museums instead tend to showcase multiple historic or artistic artifacts, and “handling sessions” are special events limited to objects that can sustain being touched. Indeed, preservation concerns may prevent heritage artifacts from being experienced in a tangible way, although the importance of tangibility and physicality is recognized: The physical qualities of an object have been conveyed to visitors via haptic devices and via extremely faithful reproductions, or through “open drawer” displays where visitors physically reveal parts of an exhibit, making the gesture meaningful in the process of discovery. Sites such as historic buildings or outdoor spaces can exploit the full-body experience and engage multiple senses, for example, the uneven stairs in an old building and the smell of wood and dust. It is not unusual for these types of heritage to build upon this opportunity and stage enacting events, for example, Roman soldiers battling or a real-life character play. A further way to engage visitors is to diversify the offer on the basis of different audience typesin other words, to personalize the visit (see Figure 1).
There is an opportunity for interaction design to take advantage of the visitors’ physical experience with cultural heritage and to integrate technology into it instead of creating a parallel and detached digital experience . This needs the right sensibility and is not without challenges. The design of digital into physical has to consider the complex ecology of cultural heritage with the conflicting goals of curators, visitors, and technology providers:
- Visiting is done in groups, but different heritage attracts different groups, and different groups follow different visiting paths. For example, small children lead the discovery with characteristic energy, while older people are more easily fatigued.
- Even if curators aim for visitors to learn, visiting is often just a way to spend a day out; it could be a restorative experience or merely a time filler.
- Digital media can enrich the experience but can easily divert attention, thus preventing contemplation and reflection.
- Digital models can recreate and contextualize exhibits, but this may contribute to diminishing the perceived value of the original (e.g., its “aura” and authenticity).
- Digital media often targets visitors’ cognitive abilities via quizzes, games, or detailed information, thus preventing them from engaging in what is essentially an affective experience: the visit.
- Digital media determine the pace of the visit and induce visitors to follow their digital guide, thus potentially missing exhibits they may enjoy more.
- Interactive technologies can offer a great user experience, but screens and apps create new barriers that distract and disengage visitors from the actual content on display.
- Interactive technologies often interfere with social interaction within visitor groups (e.g., audio guides tend to isolate visitors in their individual “audio bubbles” and small-screen devices are hard to share in larger groups).
This list is not exhaustive. Experience and previous research show that the various stakeholders involved, from museum management to curators to public relations professionals to technology providers, have very different goals when introducing computing in exhibitions. In particular, companies creating the software and mobile applications aim to sell the same solution to different places, which often leads to an experience that is not convincing to visitors because it is too generic. In contrast, experiences that are optimally crafted are expensive, time consuming, and hardly portable. On a practical level, heritage institutions become dependent on these companies for not only the creation but also the maintenance of installations and for the updating of content.
To put the physical back at the center of the cultural heritage experience, we must enable curators, artists, and designers to create manageable networks of adaptive smart exhibits that make it possible for visitors to “feel the heritage” and for staff to convey the values of their institution. Our vision is of a cultural space with smart objects, each with their own digital content embedded therein, which will be revealed if and when conditions are right, for example, when visitors have reached the proper time in the storyline, or a group of them is acting in a certain way, or another smart object is close by. While technically this has been possible to a certain extent for some time, to fully achieve this goal and make smart tangible objects sustainable for heritage institutions, curators, exhibition designers, and artists need a simple hardware and software platform that allows them to conceive, design, make, and maintain interactive artifacts.
Figure 2 illustrates the stages envisioned for creating the visitor experience: (clockwise from right) 1) the author retrieves digital content; 2) snippets of content are organized in a network, each node controlled by a context-of-use condition, and then downloaded to a smart interactive device; 3) an ecology of smart objects interact with each other, the visitors, and the space and provide personalized content in context; 4) the smart objects have an online shadow that logs the visit for further use, such as connecting with online heritage repositories or simply to social media, or, from the curator’s side, analyzing the visiting patterns and improving the exhibition.
By shifting the focus from the audience to the curators, we aim to foster creativity and facilitate the creation of novel ways of using and communicating heritage. We explore the possibilities of this novel approach within the meSch project that stands on three pillars: tangible interaction, personalization, and do-it-yourself.
To bring heritage to the center of the visitor’s attention, we advocate tangible interaction. Technically, the digital enhancement can be implemented as technologies that surround the object (external interaction) and technologies embedded in the object (internal interaction).
External interaction places the exhibits within a space enhanced by technologies and makes the surroundings interactive. Examples of such spaces are reactive projections that illuminate and bring to life specific elements, or dynamically generated sound and audio content to attract the attention of passing visitors or to create an atmospheric soundscape. This approach is suitable for large-scale originals that cannot be altered or replicated, such as a statue, or outdoor heritage sites, such as ruins or a building. An interactive space invites social dynamics and brings the original into the ongoing dialogue between visitors and heritage, for example, when the quality of a projection depends on the synchronous movement of the visitors.
On the other hand, artifacts, such as replicas of artwork or historic objects, can be digitally augmented by physically embedding into them a computing device (e.g., microcontroller, phone) as well as sensors and actuators that enable (internal) interaction. By augmenting exhibition artifacts, we take advantage of the engaging power of the physical object, enriching it with the new opportunities that arise from digital media and smart technology. The challenge is to design the computing device in a way that 1) fits multiple objects in size and shape, 2) includes customized sensors and actuators, and 3) plays back content according to the concepts of the curator. Hardware platforms such as Arduino, Gadgeteer, and Raspberry Pi are a starting point, but much work is needed to create hardware and software toolkits that lower the threshold of use and allow interested parties to create digitally augmented artifacts with only minimal technical knowledge.
To create an engaging experience, the curator or exhibition designer must have the ability to design interactions that span multiple smart objects in the exhibition and allow their interlinking in meaningful ways. At visiting time, elements in the track created by the curator will be progressively experienced by the visitors, each in their own way, creating something unique and personal.
Personalization can occur at two levels: personalization of content, when different content is offered to different people (as shown in Figure 1) or presented to offer different interpretations; and personalization in context, when the decision of which snippet of content to deliver and how is made on the basis of the current situation. For their in-depth understanding of the collection and its multiple interpretations, as well as their knowledge of their audience and of the best way to engage them, curators and exhibition designers are best suited to structure personalized interactions. Hence, it is essential to design technologies that are as easy to use as a content-management system and that support the curator to gather and compose content into a compelling storyline. The second step is to build the most appropriate context in which it will be delivered. An example we envision in a historical war museum is a soldier’s diary, with his recollections, poems, and pictures. Two enemy diaries close by vibrate: Pairing the diaries unlocks complementary maps of the battlefield, one per diarya piece of information available only if the two “enemies” collaborate. Access to the content (maps of the battlefield) would be controlled by the context (the pairing of the diaries). The mechanisms that connect content and context are therefore crucial for a smooth experience.
Developing tools that enable non-technical users to create such complex interactions requires first establishing what makes for a successful exhibition, which content is more interesting for which people, as well as which interaction mechanisms are more engaging and for whom. We then need to establish an understanding of modalities and structures of interaction that can be captured in templates available to curators to populate with content and create interactive exhibits. We foresee many templates for many different experiences, from a treasure hunt for children to a poetry discovery for an older audience. Sustainability must also be part of the design: Templates and interactions must be reusable for different exhibitions. In decoupling content and context, we allow multiple stories for the same interaction (e.g., the mechanism of the paired diaries unlocking content could be used in another type of museum to stimulate collaboration among schoolchildren), as well as the same story for multiple interactions.
Our experience and previous work with heritage show it is not a lack of ideas that is the limiting factor for curators to adopt innovative technology; rather, it is the need for technical knowledge and skills paired with the high costs of interactive exhibitions that put them off. The proof is the success of Open Exhibits, a multitouch, multi-user toolkit for creating interactive exhibits customized to the museum’s needs (http://openexhibits.org/), and its increasing popularity with museums across the world that share their know-how and templates. An editor for composing tangible interaction should abstract hardware components and interaction concepts to such a level that all technical details are hidden from the users and the fabrication of a smart object becomes as simple as building with Legos. Such a toolkit does not exist yet, but this does not prevent curators from imagining what a tangible exhibition could be. At a recent co-design workshop hosted by our project, heterogeneous groups with museum curators, designers, and computer scientists took only three hours to generate concepts and sketch them in hardware. Figure 3 shows the concept and sketch of an interactive bag for a treasure hunt used by children to collect RFID-tagged objects in a museum and get feedback from the bag itself. The ability to quickly assemble physical and functional prototypes is a powerful tool for thinking about smart exhibits, their behavior, and their appearance. By easing the creation of software and hardware prototypes, we expect to enable a paradigm change: from interactives created for museums to interactives created by museums.
Enabling a Paradigm Change
Working with curators and cultural heritage professionals, we see that many are conflicted. They see the potential of digital augmentation and the added value that can be created; at the same time, they see current technologies affecting the values they deeply care for, such as authenticity and appreciation for heritage holdings. The possibility of integrating physical and digital assets makes our approach particularly appealing, as it enables renewed focus on the physical heritage and, at the same time, digital capabilities specifically tailored to fit the curator’s vision.
We aim to empower heritage professionals to create and share their own templates and smart exhibits. For this to become widespread, it is critical to lower the hurdle and cost for the creation of physical artifacts that are digitally enhanced. With current platforms, the main cost is not the hardware; it is the development effort and the skills required. Some first steps toward easier-to-use tools have been made by creating drag-and-drop interfaces to compose hardware components into more complex devices  and by taking these descriptions and automatically generating a fitting case . People with some knowledge and interest in technology can immediately start using these tools after comprehending a few basic concepts, as was the case with one artist taking part in our workshop. But for less technical users, such as the curators, these tools are still out of reach. The design and development of such tools must be done in tight collaboration with heritage professionals. Our approach has the potential to affect cultural heritage in the same way as content-management systems changed website design. Fifteen years back, creating an engaging website required HTML and basic programming skills; today users with little or no interest in the underlying technology are able to create engaging blogs and Web pages focusing solely on the content. We expect the same level of creativity and the same level of quality to be achieved in the coming years for the creation of digitally enhanced artifacts and spaces.
The research described here is part of a shared effort carried out in meSch, Material Encounters with Digital Cultural Heritage. The interdisciplinary team includes Sheffield Hallam University, University of Limerick, Waag Society, University of Strathclyde, eCTRL, DEN, University of Stuttgart, University Carlos III Madrid, Museo Storico Italiano della Guerra, University of AmsterdamAllard Pierson Museum, MUSEON, and FBK. See http://www.mesch-project.eu for details. meSch (2013-2017) receives funding from the European Community’s Seventh Framework Programme, “ICT for access to cultural resources” (ICT Call 9: FP7-ICT-2011-9) under the Grant Agreement 600851.
Daniela Petrelli is a reader in interaction design in the Art and Design Research Centre, Sheffield Hallam University. Her current research brings together material and digital in the context of personal and cultural heritage. Her other interests include multimedia and multilingual information access, and intelligent user interfaces. She has a diverse background in fine art, computing, and social research.
Luigina Ciolfi is a reader in communication in the Communication and Computing Research Centre, Sheffield Hallam University. Her research is focused on the human experience of technology in the physical world for both individuals and groups, and on notions of place and embodiment. She has studied and designed technology for heritage sites, urban spaces, and work settings.
Dick van Dijk is creative director at Waag Society. Part of his role there is creating interactive concepts, strategizing design research and user involvement, and monitoring the development of the actual “thing.” He is mostly interested in the crossover between virtual and physical interactions and in creating a narrative space, a place for imagination.
Eva Hornecker is intrigued by the possibilities of tangible and embodied interaction and enjoys taking a broad but conceptual view of what it may mean for our lives. Her second research passion is studying visitor interactions with museum installations to determine what makes them engaging. Her career has led her on a pinball trajectory between Germany, Austria, the U.K., and New Zealand.
Elena Not is a senior researcher scientist in the Intelligent Interfaces & Interaction group at Fondazione Bruno Kessler (FBK) in Italy. She has a background in computer science and a particular interest in the application of ICT technologies to cultural heritage. Her research is focused on the adaptive information presentation with special emphasis on personalization, discourse structure, and integration with other modalities.
Albrecht Schmidt is a professor of human-computer interaction at the University of Stuttgart, working on new user interfaces. Albrecht has a background in computer science and media informatics and is interested in novel platforms and technologies for interactive systems. He is on the editorial board of IEEE Computer magazine and edits the column on invisible computing.
Figure 1. Museums produce a large variety of material tailored to specific visitors: the colorful treasure-hunt brochure and the graphic story for children; the trekking for teenagers; and more reflective material for older vsitors.
Figure 3. A concept from the co-design workshop: an interactive bag where children can collect smart items around the museum and get feedback when putting them in. The initial sketch is shown on paper and in hardware.
©2013 ACM 1072-5220/13/07 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.