ACM Interactions

Authors:
Daria Loi

---

 

There are books that once read stay with you forever. These are books with the astonishing ability to circle back and entrench themselves in your life in the most remarkable, wondrous ways and for the most remarkable, wondrous reasons. Douglas R. Hofstadter's Gödel, Escher, Bach: An Eternal Golden Braid [1] is one such book. This is not a book that can be easily summarized in the space of a paragraph. For the purposes of this column, I will share one key learning relevant to the story I am about to tell: Hidden neurological mechanisms are at the base of our cognition. That is, cognition is like a map created through seemingly unrelated elements, analogies that we unconsciously create and weave together over time. As Hofstadter puts it, "analogy-making is what lies at the core of human thought" [2].

In this column, I explore the interstitial space that is created when research, design, and art collide. As the story unfolds, I will leave space for readers to unconsciously weave together their own analogies and new mind-tapestries.

In the early 2000s, my colleagues at RMIT University and I became fascinated with the idea of crafting experiences that leverage unexpected and unanticipated elements to unlock learning, phenomenological awareness, and reflective practice. The baseline idea—which became clearer later on—was that by playing with anomalous objects and odd experiences, it is possible to dramatically expand creative and interpretive engagement between people, providing platforms where diverse interpretations can be generated.

With symbolic constructivism and phenomenological perspectives at the core of our fascination, the Pea Project [3] was born and subsequently run in Sweden, the U.S., and Australia in 2002 and 2003. In the Pea Project, we approached potential participants at conferences or in educational contexts, invited them to take a fresh pea from a pod, and to hold it while we took a photograph of their hand (Figure 1). Without any explanation, we asked them to fill in a postcard with two questions: What did you see? and What is going on here? Our very puzzled participants were then invited to a second encounter (frequently a workshop), during which we presented a slideshow of all the hand/pea images and gave them a postcard identical to the previous one. To our surprise, a simple pea ended up stimulating reflective awareness, curiosity, creativity, and collaborative engagements. Equipped with several hypotheses and questions from the Pea Project, my colleagues and I embarked on a second experiment. In A Surrealist Encounter [3], we decontextualized a series of objects from Magritte's L'Assassin Menacé (The Menaced Assassin) to elicit creative and collaborative encounters between strangers. The Pea Project and A Surrealist Encounter highlighted for us that people are highly creative beings and that almost anything, removed from its original context, can stimulate creativity and collaboration.

Figure 1. Still image from the Pea Project [3].

Building on those experiments, I started exploring and experimenting with more tools. Reflective Probes [4] were designed to create the conditions for reflective practice to "prosper through activities that take the form of creative, ambiguous, and inspiring artifacts." Playful Triggers [4] (Figure 2) focused on establishing "a bond—a collaborative practice—among participants" and focused on "dialogue-creation, acting as communication (rather than ethnographic/empathetic) devices by activating receptive modes of engagement." I also explored more-experimental tools, such as Primitive Probes [4], which were designed scaffolds for end users to design a probe to use on themselves. This last experiment was ultimately about "stretching the boundaries of who is entitled to design a probe and why/how" [4].

Figure 2. Examples of Playful Triggers [4].

In these explorations, art, design, and research entwined—they blurred the line between who is researched and who researches (or who designs and who uses), enabling me to glean from/through/with participants much more than I had originally expected or planned. That said, I was left with numerous unanswered questions and unexplored opportunities.

In 2019 one such question (and opportunity) re-presented itself: What happens if I design creative tools to gather qualitative data in the context of a traditional research study and then inject data into an artistic process? In other words, what happens if a designer creates tools to enable a researcher to gather data that is then used as a starting point for an artist to make art? What if designer, researcher, and artist are the same person?

Fascinated by this notion, I wanted to compare what would happen if the same data became the starting point for two artists. A research colleague who doubles as a visual artist agreed to participate, and Debris [5] was born.

For this experiment, we focused on people's relationships with nonhuman agents (e.g., smart home systems, intelligent agents, smart robots, voice-based agents, etc.). While I had conducted qualitative and quantitative research on the matter previously, I knew that much more could and should be unveiled, learned, and built upon. Our process started with designing a creative tool kit and research protocols. This was done as a collaborative effort while also building on my past experiences and design tactics. We then used the tool kit and research protocols to interview different users separately and reconvened to analyze the data. During data analysis, one of us shared rich vignettes and details from a specific participant to enable shared understanding and intimacy with their lived experiences. We then discussed the data at length and focused on identifying the experience's essence for each interview to establish a starting point for the artistic endeavor—we agreed to write short elevator pitches for each participant. Once this shared analysis was completed, the art phase began and our paths separated again: While we used the same canvas size, we did not disclose our individual processes, ideas, or trajectories. We unveiled our pieces to each other while setting up our art exhibit at EPIC2019. In that setup, our curated space included art pieces, creative tool kits, researcher notes, photographs taken during interviews, and sketches (Figure 3).

Figure 3. Still captures from Debris [5].

When my colleague and I submitted the work to the conference, our thought was that "the collection of art pieces, objects, visual commentaries, humans-about-machines accounts" would represent "the fragments that are left behind by human interactions with smart agents"—the idea was to offer them to ultimately "prompt reflections, re-connections, and re-discoveries of human-non-human hybrid landscapes" [5]. As I reflect on the experiment, however, it is clear that an additional something else took place and that we learned more than and differently from what we planned or imagined.

Multisensoriality. Asking the same question (e.g., What is your relationship with your robotic vacuum cleaner?) in multiple, diverse ways (e.g., If your relationship with your robotic vacuum cleaner was a word, what would that be?) enables researchers to glean richer, deeper qualitative data. The experiment demonstrated that asking the same question leveraging multisensorial input/output (e.g., If your relationship with your robotic vacuum cleaner was one of these objects, which one would that be?) not only expedites and focuses the analysis process but also enriches the researcher's ability to get to the core of the matter.

Unexpected patterns. While we anticipated some similarities in our creative output (e.g., color scheme, as one of the activities focused on color associations), we were surprised to discover that both of us mixed elements, created analogies, and adopted a radically different art process, yet the results had hard-to-explain visual elements in common (e.g., a circular theme when nothing in the interview suggested it).

The value of artifacts. While we analyzed data objectively and as a team, both of us added a subjective layer in making the art pieces—hidden neurological mechanisms are at the base of our own cognition. The pieces were as personal as they can get, yet participants easily recognized which piece was inspired by their story. Additionally, the pieces became very useful starting points for follow-up interviews, showing the potential of this process to gather deeper insights and enable longitudinal research.

Targeting the core. The agreement to identify and articulate the experience's essence for each participant to aid the artistic practice ended up being surprisingly helpful from a pure research analysis standpoint and boosted our ability to summarize our insights in clear, approachable, influential ways.

This and all prior experiments demonstrated the opportunities that exist when art, design, and research intermingle and collide—opportunities to enrich a researcher's tool kit, a designer's understanding, and an artist's output. For HCI practitioners, the interstitial space created in the midst of these three practices may bring great opportunities. Historian France Morin once said that "artists have the capacity to make a lasting positive impact on people's lives by helping them to see for themselves the dignity, beauty, and sacredness of the activities of their everyday life: the creative spirit, a powerful agent of transformation that lies within everyone" [6]. The design-research-art braid may offer HCI practitioners a golden opportunity to enrich their understanding of everyday life, what they design, and themselves.

  Acknowledgments

I wish to acknowledge Peter Burrows for decades of friendship, inspiration and co-experimentation; Heather McGeachy for being part of Debris; and the many students and participants who made the explorations discussed in this article possible.

  References

1. Hofstadter, D.R. Gödel, Escher, Bach. An Eternal Golden Braid. Basic Books, New York, 1979.

2. Hofstadter, D.R. Analogy as the core of cognition. Language, Cognition, and Computation Seminar Series. Apr. 13, 2005; https://www.media.mit.edu/cogmac/lcc/hofstadter.html

3. Loi, D. and Burrows, P. Magritte and the pea: Anomalous artefacts and the contexts they create. Working Papers in Art and Design 4 (2006).

4. Loi, D. Reflective probes, primitive probes and playful triggers. Proc. of Ethnographic Praxis in Industry Conference 2007, 232–245.

5. Loi, D. and McGeachy, H. Debris: Intermingling ethnography with design and artistic practice. Proc. of Ethnographic Praxis in Industry Conference 2019. DOI:10.1111/1559-8918.2019.01254

6. Morin, F. The quiet in the land: Everyday life, contemporary art, and Projeto Axé. Art Journal 59, 3 (2000), 4–17.

  Author

Daria Loi has been combining design strategy with experience research and innovation to enrich people's lives and humanize technology for the past two decades. She is on the board of directors for DemocracyLab, conjoint professor at Newcastle University Australia, and co-chair of the Participatory Design Conference advisory board. [email protected]

 

Copyright held by author

The Digital Library is published by the Association for Computing Machinery. Copyright © 2021 ACM, Inc.