Columns

XXIV.3 May + June 2017
Page: 20
Digital Citation

What makes a good CHI design paper?


Authors:
William Gaver, Kristina Höök

back to top 

In the last issue, we suggested that the perfect CHI design paper is a myth, not just because it is all but impossible to achieve, but also because there are so many ways to pursue and report design research. In this column, we suggest that a few ingredients seem to be present in all, or at least nearly all, submissions considered by the subcommittee to be successful enough for acceptance to this year’s CHI conference. Each ingredient is a key factor in delivering design practice as design research. We hope that describing them here will help replace the imaginary perfect CHI design paper as a guide for both authors and—equally important—reviewers trying to identify valuable contributions to CHI.

Demonstrate and discuss design work. Not surprisingly, one of the key features of successful design papers is that they describe and discuss something to do with design. We’re always excited to see submissions that explore design practice in some form, whether it’s the development of artifacts; design processes, approaches, and methods; creative ways to engage with the settings for design; investigations of and with materials; or explorations of design’s aesthetic qualities. We recognize, in addition, that design is an expanding field. Thus, “something to do with design” also includes design histories; critical-interpretavist reflections on design; service, infrastructure, and activist design; as well as the whole burgeoning world of open-sourced, crowd-funded, hacker-spaced, cooperatively produced, data-fueled, and user-finished designs. Moreover, we love surprises, so keep sending us the experimental, avantgarde, transdisciplinary submissions that don’t fit elsewhere. Overall, what we really want are papers with a contribution to design clearly at the heart of the work and that will clearly benefit from evaluations by ACs and reviewers from the design community.

Tell us what’s important. Rather than trying to tick all the boxes of the perfect CHI design paper, it is sufficient, and often preferable, to focus on articulating one or two aspects of a project well. Crucial, however, is to tell the reader that this is what you are doing, and to explain why your focus is important and the grounds on which you believe it should be assessed. This is not only to guide readers toward what matters, but also to forestall criticism based on elements that aren’t there or that are less well developed. For instance, you might focus on the development of a design space for an artifact that hasn’t been fully fleshed out; discuss the interplay between designers, makers, funders, and manufacturers of a product you don’t endorse; or critically analyze the gender assumptions of a service design without discussing how it is used. All could be viable topics for a successful CHI contribution. To make them work, however, it is essential that their focus is framed and justified as clearly as possible.

Show links to other work. As design research has flourished at CHI and elsewhere, the quantity of literature related to a given project can ramify outward indefinitely. Now that references do not count toward the CHI page limits, it is not unusual to see papers with 50 or more references. This may be a positive development in recognizing the development of design research as a discipline, but it can be overwhelming to authors and lead to papers overbalanced by their literature reviews or, conversely, with nothing but long lists of uncommented references. Ultimately, an unending expansion of literature reviews could transform CHI design research from a primarily empirical community into an intertextual one. We believe that design papers should not be required to produce comprehensive literature reviews so much as sufficient ones. It is important to situate your work in the field, and this is usually done through references to other key papers. Written work, however, is not the only terrain for design research. Instead, design research can also be positioned alongside other design outputs or practices, including, naturally, those from industry or from practitioners outside the research community altogether.

Include a critical assessment. The idea that design research must be accompanied by a user study is a myth. Nonetheless, it is not enough simply to proffer a designed artifact (or process, or investigation, or even a user study) without any attempt to assess its strengths and limitations, or, alternatively, the effects it might have. Of course, user studies are one way to go about this, and the success of submissions that include them no doubt reinforces the myth. However, many other forms of assessment are also potentially valuable. This might include expert critiques from other practitioners, analyses by cultural theorists, considered accounts by relevant commentators, or even the reflections of the makers themselves. What is important, though, is that assessments are sincere and probing. It is rare to produce a flawless piece of work, and research papers are not marketing tools. One of the hallmarks of research is the ability to learn from failure, so accounts of research must take a balanced view in assessing what they offer.

ins01.gif

Tell us what you learned. A corollary of the need to go beyond merely presenting a design outcome as self-evidently worthy (and, indeed, of the need to frame what is important and link to other work) is the need to explain the lessons learned by producing it. Explaining that can take many forms. Research doesn’t always produce facts: Equally useful are new conjectures, perspectives, suspicions—or, for that matter, support for or contradictions of existing ones. Learning may be equally expressed through theories, mid-level strong concepts, annotations, frameworks, or guidelines. It doesn’t have to be, though. Some may feel that the outputs of design research—its artifacts, processes, aesthetic studies, or engagements with people and places—offer value through their specificity, that generalization in any form dilutes that value, and that design practitioners routinely look to other design outcomes rather than generalizations as inspiration for their own work. This is fine, too, so long as that specificity is identified and what it offers is made clear enough for readers (and reviewers!) to value. What is essential, however it is done, is that the contributions offered by reporting a project are clear.

These, then, are our reflections about the essential features of good CHI design papers. Just as crucial, they imply the things we think are not obligatory (reviewers take heed!). Successful CHI submissions do not have to include user studies (though they do need some form of critical assessment). They don’t even have to tell the story of a complete design project (though they should have something to do with design). They do not need comprehensive literature reviews (though they do need to link to the community). They do not require grand theories or research programs (though they need to explain why they were done). And they don’t have to generalize from the specific designs to a class of systems (though they do need to explain what was learned).

There are endless possibilities for the careful articulation of research about, through, into, by, over, during, or of design, and we earnestly hope the community will continue to produce new and unexpected examples. Our aim here has been to open the doors for these possibilities, both by dispelling the myth of the perfect CHI design paper and by communicating to would-be authors—and reviewers—what we think it takes to write a successful submission.

back to top  Authors

William Gaver is a professor of design and co-director of the Interaction Research Studio at Goldsmiths, University of London. His research on design-led methodologies and innovative technologies for everyday life led him to develop an internationally recognized studio that brings the skills of designers together with expertise in ubiquitous computing and sociology. w.gaver@gold.ac.uk

Kristina Höök is a professor of interaction design at the Royal Institute of Technology (KTH) who also works part-time at RISE SICS. Her research on interaction concepts including social navigation, seamfulness, somaesthetic design, and mid-level strong concepts has been significantly influential on the application of theory to design practice within HCI. khook@kth.se

back to top 

Copyright held by authors

The Digital Library is published by the Association for Computing Machinery. Copyright © 2017 ACM, Inc.

Post Comment


No Comments Found