XXI.5 September + October 2014
Page: 70
Digital Citation

House memory

Tao Dong, Mark Newman, Mark Ackerman

back to top 

In the past few years, we have seen a wave of new "smart" consumer products that make everyday places aware of our activities. There are thermostats that adjust temperatures based on occupancy [1], doors that alert us when we forget to close them, and "beacons" that track our indoor locations [2]. With recent advances in sensor technologies and the Internet of Things, every corner of our world is slated to gain some capability of capturing our activity traces.

As everyday places become more aware of what we do, an enormous volume of activity traces can be captured and potentially amassed over the long run. Yet the narratives surrounding those technologies mostly focus on short-term gains in efficiency and comfort; few have considered the long-term value of those captured traces. We are concerned that traces will be discarded prematurely, since the perceived risk to privacy easily outweighs the as yet unclear benefits. Thus, it is important to ask: How might we, or rather our future generations, find digital footprints left in a place useful in the long term?

back to top  Insights


Our research and design work [3] suggests that activity traces captured by everyday places can be a new form of cultural heritage for people who come after us. Here, we will use several design concepts to illustrate the idea that those traces, no matter how unremarkable they appear to be, can potentially serve as resources for our descendants to connect with us and our times.

Admittedly, it is difficult to anticipate how people in the far future might interpret, use, and act on the traces their predecessors left. Nonetheless, we believe today's designers can gain inspiration by looking closely at comparable practices of interpreting historical traces in the present (which is, after all, the future of the past). Accordingly, we examined how people today make sense of and make use of the traces they found in the places where they live. What we learned from our fieldwork inspired and grounded our designs.

To be more specific, we studied how current homeowners interpreted and used previous occupants' traces of inhabiting and appropriating their homes in the Midwestern United States. We chose to examine houses—especially old houses—over other types of places for two reasons. First, every owner of a house leaves long-lasting personal and family marks on the house through, for example, remodeling and reconfiguring the space, and these marks often carry significant meaning. Second, the domestic environment is becoming one of the most instrumented spaces that capture and hold new types of traces.

We visited 20 houses and conducted semi-structured interviews with the homeowners. Each interview started with several factual questions such as: When was your house built? How long have you been living here? We then asked participants what they knew about the house's history, who had lived there before them, and how they learned about the house's past.

We further situated our interviews through a show-and-tell-style exercise in which participants gave us a home tour with an emphasis on changes made to the house over the years either by them or previous owners. When participants were telling us about specific home repair or improvement projects, we probed how they dealt with the traces of prior work (or lack thereof) when they were trying to modify or appropriate the place to fit their own needs and tastes. We asked questions such as: Was there anything surprising or unexpected in this project? What guesswork did you have to do? What did you learn about the house from this project? If participants had records about the house, we asked about whether those records were useful in these projects. The home tour and walkthrough of artifacts and records also occasionally took the conversation back to the broader historical background of the house and the neighborhood.

Our fieldwork led us to a few insights that have implications for designing potential ways of using digital traces left in a place in the long term. We describe them here.

The traces passed on and the ones left behind. What carries traces, and where can traces be found? These were the first questions we were trying to understand.

Some traces were carefully preserved and passed on in formal records. Lisa, one of our participants, received the original architectural plan of her 100-year-old house during the closing (see Figure 1). Those drawings guided the rehabilitation of her house, which, after being divided into a three-unit apartment, had lost much of its character. For example, the plan revealed the original doorways for pocket doors that had been walled off in a previous remodeling.

Many other traces were discovered and used in a way that could hardly have been anticipated by their creators. For instance, an antique postcard, collected by one of Lisa's friends, clued her in to the house's original exterior features (see Figure 2), and an old wedding photo, possessed by a daughter of the second family who owned the house, helped Lisa learn the detailing of the original staircase.

More striking, some traces were left behind in the house itself, and they can be just as informative about its past adaptations if one knows what to look for. Linda and John learned a lot about how their house evolved over more than 150 years as they ran into covered flooring, overlaid wallpaper, and other remnants of old materials when they were demolishing parts of their house, which was in great disrepair when they bought it (see Figure 3). They called those "shadows of what had been there before."

Traces of prior appropriation of our participants' homes were diverse in their forms, origins, and intended purposes. Those traces were found and made use of long after they were initially created partly because they were left in place or embedded in things that carried either informational or sentimental value. This echoes and extends Daniela Petrelli and Steve Whittaker's work describing how common objects become reminders of people, experience, and places as they are integrated into everyday life [4].

Connecting with the past through traces. David Kirk and Abigail Sellen suggest that sentimental objects kept in a household can connect younger family members with a past they did not experience by themselves [5]. We found that traces, often unexpectedly encountered rather than held dearly within a family, had a similar evocative power.

In our study, discovering old construction and decorative materials in the house provided some participants rare opportunities to learn and reflect on the lifestyle and aesthetics of an earlier period. For example, Robert was surprised when he found a nice hardwood floor covered under linoleum, which he considered ugly, in his kitchen (see Figure 4). That discovery made him wonder about changing tastes over the years: "I know, it's because in the '70s, everybody wanted it to look a certain way. Every few decades, we have a different style [that's] considered appropriate. I guess they must have thought that hardwood looked country or rural or something."

Learning the history of one's house through its historical traces might also help develop one's attachment to it. For example, Linda made a sentimental comment along these lines: "The house was there before you were born, and the house is going to be there after you die—the concept being that it's your job to sort of take care of it while you're there. I think that kind of fits with our gestalt."

The social life of traces. Our study also shows that traces have a social life, and for that reason traces can potentially become part of the collective heritage of a community.

On one hand, traces are often kept by community members and passed on through social interactions. As mentioned earlier, Lisa obtained the antique postcard featuring her house from a local friend, and the wedding photo showing the detailing of the staircase from the 1940s from a former owner's daughter. The social context of those traces, including her town being a close-knit community and the fact that Lisa has lived there for decades, played a role in the traces' rediscovery and reuse. On the other hand, stories of an old house can serve as resources for community members to socialize and build relationships. Lisa felt proud about this: "It's kind of cool because [if] you live in a town long enough, people have stories about your house."

Learning the history of one's house through its historical traces might help develop one's attachment to it.

Furthermore, we found that participants' attitudes toward passing on records and artifacts were influenced by how their predecessor handled that decision. For example, at closing, David was given a letter Albert Einstein wrote to the original owner of the house. When we asked what he would do with the letter, he answered that he would definitely pass on the letter to the next owner. He said, "It's given to us in trust, you know. The previous owner said, 'This has been with the house ever since the first owner. Each owner has passed it onto the next. We're passing it on to you. We encourage you to do that.' So it's a sense of promise that we will do that, that it will stay with the house rather than go with us. So I just feel that's appropriate."

As these examples suggest, the survival and transfer of traces often depend on social norms, ties, and interactions in a local community. In the meantime, those traces help preserve the common heritage of the community and strengthen its identity.

Anticipating future uses through design. Applying the insights from our fieldwork, here we illustrate some possible ways of interacting with decades' worth of traces at some point in the future. The three design concepts described below embody the following principles we learned: leaving traces in their original place and in everyday things, leveraging traces' power to both inform and evoke, and preserving the social context in which traces can be saved, encountered, and reused.

The first concept, Footprints, is a system that overlays aggregated patterns of previous occupants' indoor movements on top of the current floor plan of the house, presumably on electronic paper. The goal of Footprints is to give the current occupants a better sense of the space's prior layout and flow. For example, a trail of footprints through the window of the dining room implies that there used to be a doorway to the outside.

While Footprints mostly serves a practical purpose, Phantoms is designed to evoke users' reflection and imagination by presenting snapshots of previous occupants' indoor locations. As an ambient display, Phantoms plays back snapshots of indoor location traces in a mysterious if not random order. The purpose is not to inform but to evoke the viewer's own memory of the past. To understand its user experience, let us consider the following scenario: Repetitively seeing Phantoms displaying three pairs of footprints in the family room, Harry, a middle-aged man from the 2040s, speculates that the family who lived here at some point might have a routine of watching TV together after dinner. It reminds him of his childhood, when there was still a TV in his parents' house. He cannot help but readjust his glasses, with which he and everyone else watch "TV" nowadays ...

The third design concept, Stewards, addresses the problem that homeowners' interactions with locals who know their houses, as in Lisa's case, are becoming increasingly rare. To facilitate collective curation and interpretation of traces, Stewards creates a closed online micro-community for people associated with a house. For example, previous occupants, guests, contractors, and neighbors can join the community anonymously or under pseudonyms.

In each micro-community, the current owner of the house can learn the house's history and ask about "puzzles" found in the house. But more important, we envision that this can serve as a virtual drawer where homeowners in the future can easily deposit digital traces related to their houses upon or after moving out, just like many of their present-day counterparts did with paper records: simply leaving them behind in a drawer, usually located in the kitchen.

A final reflection. Our fieldwork shows that traces, however humble and trivial they might initially appear, can serve to facilitate a sense of heritage as people find meaning in and through them. Compared with their physical counterparts, digital traces are at once more persistent and ephemeral; they are easier to retrieve and easier to bury. These conflicting properties of digital traces require us to make a conscious and creative effort to imagine and anticipate how the captured traces of our everyday activities might be useful for the people who come after us for both practical and evocative purposes. Furthermore, it is important to consider how we can cultivate a favorable social context for the preservation and reuse of traces.

Our fieldwork also prompted us to reconsider the temporality of the privacy question. The public discourse is increasingly obsessed with the idea of setting an expiration date for our digital footprints, as advocated by writer Viktor Mayer-Schönberger [6], required by European law [7], and implemented by Snapchat. However, we might have a cultural responsibility to allow those traces, even in an aggregated or abstract form only, to be recovered and accessed at some point in the future.

back to top  References

1. Yang, R. and Newman, M.W. Learning from a learning thermostat: Lessons for intelligent systems for the home. Proc. of Ubicomp'13. ACM, 2013, 93–102.

2. iBeacon. Wikipedia, the free encyclopedia, 2014;

3. Dong, T., Ackerman, M.S., and Newman, M.W. 'If These Walls Could Talk:' Designing with memories of places. Proc. of DIS'14. ACM, 2014.

4. Petrelli, D. and Whittaker, S. Family memories in the home: Contrasting physical and digital mementos. Personal Ubiquitous Comput. 14, 2 (2010), 153–169.

5. Kirk, D.S. and Sellen, A. On human remains: Values and practice in the home archiving of cherished objects. ACM ToCHI. 17, 3 (2010), 10:1–10:43.

6. Mayer-Schönberger, V. Delete: The Virtue of Forgetting in the Digital Age. Princeton University Press, 2011.

7. Streitfeld, D. European court lets users erase records on Web. The New York Times, May 13, 2014;

back to top  Authors

Tao Dong is a Ph.D. candidate at the School of Information at the University of Michigan. His current research explores implications of long-term automatic capture of sensor data in ubiquitous computing environments, especially the prospect of using such data as resources for reflection and as new forms of memory.

Mark W. Newman is an associate professor in the School of Information and the Department of Electrical Engineering and Computer Science at the University of Michigan. His research interests are in human-computer interaction and ubiquitous computing, with a focus on support for interaction design and end-user configuration.

Mark S. Ackerman is the GH Mead Collegiate Professor of Human-Computer Interaction and a professor in the School of Information and the Department of Computer Science at the University of Michigan. He has published widely in HCI and CSCW, including about collective memory, social computing, and pervasive environments. He is a member of the CHI Academy.

back to top  Figures

F1Figure 1. The original plan of Lisa's house. It shows the layout of her house when it was built in the 1900s.

F2Figure 2. The antique postcard Lisa received from one of her friends depicts the exterior detailing of her house in 1905.

F3Figure 3. Linda and John found "shadows" of old architectural elements as they tore down and renovated their house.

F4Figure 4. Robert was surprised when he discovered a nice hardwood floor covered under linoleum.

back to top 

Copyright Held by Authors. Publication Rights Licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2014 ACM, Inc.

Post Comment

No Comments Found