Forums

XXII.4 July - August 2015
Page: 74
Digital Citation

A personal perspective on research through design


Authors:
Cristiano Storni

What is research through design (RtD)? In my understanding, and not necessarily from the available literature, the first thing to say is that RtD is in the business of knowledge, not design. Of course design plays an essential part, but only in so far as it is instrumental to the study of something else—a resource for the production of new knowledge. So the question is: How can design be used to produce new knowledge? Also, what type of knowledge can it produce, for what purpose, and for whom? There are many ways of answering these questions. Here, I try to answer them by following my personal journey from HCI to science and technology studies and back.

After graduating in HCI under the guidance of Sebastiano Bagnara at the University of Siena, I was offered a Ph.D. position in a program called Information Systems and Organization (IS&O) hosted by the department of Sociology and Social Research at the University of Trento. HCI offered a unique modus operandi, user-centered design, which revolved around designing things, testing them with potential users, and iteratively improving them to satisfy users’ requirements. The general approach was cognitive, borrowing methods from applied and experimental psychology. The Ph.D. program, on the other hand, offered an opportunity to move from looking at individuals interacting with interfaces, to looking at groups and organizations working together using complex information systems. It also represented a move from psychological methods to sociological ones, but the objective of designing something to improve some sort of situation was still there, at least in my understanding at the time.

Insights

ins01.gif

Animated by my experience with HCI, I initially proposed to design a “social computer” for my Ph.D. I innocently thought that a sociology department concerned with IS was the perfect place for exploring such an idea, but the Ph.D. committee of the department could not but laugh at my proposal. In retrospect, I cannot really blame them. I was dealing with sociologists of work and ethnographers, whose approach was qualitative and highly influenced by constructivism. Sociologists neither have nor need the resources for design work; they do not see design as a legitimate way of researching social phenomena. Design is arbitrary and subjective—how could you use it as a reliable means of learning about something else? When used as a method of research, design changes the very reality you are trying to understand in a way that is idiosyncratic and too difficult to measure, raising issues about reflexivity, replicability, the validity of the data, and the generalizability of the research findings. When used as the outcome of sociological research, a designed artifact becomes too difficult to assess: Where is the literature review, the theoretical frame, the empirical data, and the thesis to defend? A design intervention looks like a subjective statement by its author, and even if it has been empirically informed and verified, it does not contribute to an understanding of social phenomena. Sociologists, at least the ones I knew, were concerned with describing and understanding social phenomena, not changing them.

Echoing the Tavistock Institute, I tried to problematize my social computer as an action research tool, but I was still missing the main ingredients. While reflecting, I learned an important distinction that has helped me to clarify RtD. On the one hand, some believe there should be a Cartesian separation between a subject who studies and an object to be studied. This epistemology of separation demands minimal interference with the object of study. The epistemology of interaction, on the other hand, assumes that to gain knowledge about something “out there” you have to interact with it, even at the cost of changing it. In traditional research, when there is interaction between the subject and the object, it is controlled, as in the case of physicists interacting with matter, or it is restrained, as in the case of ethnographers unavoidably interacting with the objects of observation during a field visit. Designing something is obviously incompatible with an epistemology of separation because its purpose is always to change—by design, so to speak—but it is also incompatible with controlled interaction because it is too complex, uncontrolled, and specific.

It was pointless to echo the Heisenberg Principle and the Hawthorne Effect by arguing that sociologists also design their surveys, interview questions, and field visits, which change the reality to be studied. I began to see myself as a Don Quixote. One thing was becoming clear: I did not need to start with an idea about something to design—in fact, I did not have to design anything at all. What I needed first of all was a research question, then an adequate methodology to collect data, and then a framework to analyze the data and answer my question. I had to step back, put aside my design ideas, start fresh, and focus on something to study. Then I had what now sounds like the most obvious idea: What if I study designers instead of acting like one? What if I investigate design practices, ethnographically looking at how designers translate their ideas into tangible artifacts? Bingo! To stay close to design and still achieve my Ph.D., I decided to collect case studies looking at professional designers and their practices.

Let’s go back to my question: How can design be used to produce new knowledge? To some social scientists, design itself is an object of inquiry, something to be studied rather than a resource for the study of something else. Basically, what I ended up doing for my Ph.D. was research (sociological, qualitative, ethnographic) on design. The goal was not to produce a design intervention but rather to acquire knowledge about design processes and practices that could be of interest to others concerned with studying design, as well as to designers interested in scrutinizing their own practices [1].

After my Ph.D., I was hired as a postdoctoral researcher working with Liam Bannon at the Interaction Design Centre (IDC) at the University of Limerick. Interestingly enough, after being the design guy in a sociology department, I was now the social science guy in a computer science department, and this afforded me a different perspective on design and research. I began to study self-care practices and technologies associated with managing chronic disease. To learn about chronic disease, I joined a support group of Type 1 diabetics for almost a year. There, I did the same as I had done with designers during my Ph.D., which involved interviews and observations of individuals with diabetes. In the IDC, we used my study to inform a collaborative design process that led to the development of a mobile journaling app to support the personalization of self-monitoring practices [2].

ins02.gif

What I learned at the University of Trento was still relevant: The goal is not what you design (as I now repeat to my Ph.D. students), but rather the knowledge you produce from the process. How could just designing something qualify as research? You need at least an evaluation explaining whether the design worked as expected or not (and possibly why) and some sort of intellectual framework to guide the production of new knowledge. Indeed, it was only through testing our design prototype that we gained further knowledge. But knowledge about what, exactly? Through our evaluation we could question whether our design assumptions (grounded on our interpretation of the ethnographic data) were adequate. We asked participants to use our prototype for a few weeks and to keep diaries. We then conducted follow-up interviews and further probed them about their self-care practices.


What is produced is knowledge about how a design intervention and a phenomenon interact, accepting that as the two meet, they are both transformed.


Interestingly, the knowledge produced at this stage of the research project was different from the knowledge produced at the beginning of the project obtained by attending a support group and interviewing affected individuals. The knowledge gained at the beginning of the project was descriptive of Type 1 diabetes self-care practices that exist “out there,” independent and separated from me, the researcher. Although open to different interpretations, I do not think other researchers would have generated radically different results from mine. The second type of knowledge generated was different because it was inseparable from our design prototype and how it changed the very self-care practices we were investigating. The prototype extended our ability to interact with diabetics in order to gain further knowledge about their practices, but only in relation to a context that was no longer independent of us and our design, no longer “out there.”

Even though we were conducting research in the healthcare area, our experiment was different from the experiments you might find in psychology or medicine. The knowledge we generated in the latter phase of the project was no longer knowledge about what diabetics typically do, but it provided insights about what they would do if our application were going to be used for real. It was knowledge about our participants as well as our design. Consequently, research questions changed; they were no longer about actual self-care practices, but about the role of technology in improving them. The type of knowledge we produced through our design wasn’t just descriptive of a reality that we altered; it also reflected what we, who were involved in the design process, deemed desirable or problematic, good or bad. In this sense, that knowledge was partially speculative (transcending the actual state of things) and not value-free.

Though RtD is initiated with a research question, which then requires the collection of empirical data and a conceptual framework to make sense of the data, the knowledge that it produces is different from that developed in qualitative social science studies, not to mention the knowledge acquired through scientific methods in the natural sciences. So let me ask the question again: How can design be used to produce new knowledge? And what knowledge? By now, you may have guessed that my answer is, it depends. It depends on your research questions and on your epistemological stance, as well as on the department you work in and its modus operandi. Design can become a resource for the production of new knowledge in different ways.

As far as my experience is concerned, we saw that design can be something to study both ethnographically and quantitatively in social research, but this would not qualify as RtD. In RtD, design is not what you look at, but rather what you need to extend the ability to investigate and acquire new knowledge. However, there is a specific trade-off. Yes, you can learn new and different things—things you would not learn with interviews, field observations, or a large number of questionnaires. The price to pay for this new knowledge is that it cannot be separated from the designed artifact that interacts with the reality under scrutiny. In RtD, design becomes a resource for new knowledge through the empirical effects it produces. It is pragmatic and reflexive, in the cybernetic sense of reflecting (on) itself, as the recently deceased cyberneticist of design, Ranulph Glanville, would suggest [3]. What is produced is no longer just knowledge about a phenomenon; it is knowledge about how a design intervention and a phenomenon interact, accepting that as the two meet, they are both transformed. In this sense, design in research allows you to acquire new knowledge and offers insights into alternative realities. This can be problematic for those concerned about producing objective universal knowledge about certain realities, but it is very useful for those concerned with experimenting with and improving realities.

Echoing debates in the philosophy of science, one would wonder if the knowledge produced in RtD is commensurable with other knowledge produced in the same field, and whether RtD experiments are replicable and their results falsifiable. I am afraid RtD would probably fail all these tests [4]. But that is not the point. If we are interested in how things can be improved and not in how things are, then Feyerabend’s epistemological anarchism is probably a better framework for RtD. Design interventions are too complex; they have too many variables to believe in one superior method, theory, or design to produce knowledge from them. But this does not mean that RtD should not be rigorous; it suggests that the rigor should not come from the way we develop our design intervention, but rather in the way we produce knowledge from it. Considering its character, RtD is rigorous when it is modest, accountable, and generative. It should always be modest because RtD is not good for producing universal laws or general claims. It should evolve in small steps. Its accountability is also of particular importance. Given the high number of potentially arbitrary decisions in RtD, I believe it is always important to look at the motivations behind design decisions. This means explicitly discussing embedded assumptions, rationales, and criteria for inclusions and exclusions (of concepts, of particular types of users, of design features, etc.) so that those enjoying the results can better understand where the knowledge came from. In this vein, a recent call for annotated portfolios [5] seems to point toward an improved accountability of RtD. Finally, I believe that RtD should be generative. RtD is empirical and pragmatic; its value should reside in its effects and the ability to be used by those who come next to ask better questions, propose new ideas, and produce better designs.

I am sure that people from different backgrounds might have much to add to my contribution. Let’s not forget that RtD is intrinsically multidisciplinary and enjoys both conceptual and methodological contributions from other disciplines bringing different assumptions, expectations, and practices to the table for discussion. In this forum I’ve brought mine; I hope it will generate a fruitful exchange.

References

1. Storni, C. Unpacking design practices: The notion of things in the making of artifact. Science, Technology & Human Values 37, 1 (2012), 88–123.

2. Storni, C. Diabetes self-care in-the-wild: Design challenges for Personal Health Record systems and self-monitoring technologies. Information Technology & People 27, 4 (2014), 397–420.

3. Glanville, R. Second order cybernetics. Encyclopaedia of Life Support Systems. 2002.

4. Gaver, W. What should we expect from research through design? Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2012, 937–946.

5. Bowers, J. The logic of annotated portfolios: Communicating the value of ‘research through design’. Proc. of the Designing Interactive Systems Conference. ACM, New York, 2012, 68–77.

Author

Cristiano Storni is a researcher at the Interaction Design Centre and a lecturer in interaction design in the computer science department at the University of Limerick. His research lies at the intersection of social science and design. He has studied science and technology studies (STS), actor network theory (ANT), ethnography, HCI, and participatory design. cristiano.storni@ul.ie

©2015 ACM  1072-5220/15/07  $15.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.

Post Comment


No Comments Found