Susanne Bødker, Kasper Hornbæk, Antti Oulasvirta, Stuart Reeves
Let’s be honest: It’s hard to start a career in HCI research. Working out what path to take is daunting. Our community is ever growing and ever diversifying. Interactive technologies change quickly and good research can be rapidly forgotten. At some point you might ask yourself: What should I study in HCI and how? It’s a difficult question because our field is open-ended and evolves quickly. At the same time, answering it will have a tremendous influence on your career.
This essay distills lessons learned in four workshops focusing on the question of what to study in HCI. Many experienced researchers from different branches and subfields participated in these discussions (16, besides us ). For our own part, we are constantly faced with advising and being challenged by others, be they students or colleagues. We have conducted work in many countries, though mainly in Europe and North America. We span a couple of research generations. We each diverge theoretically and methodologically in our approach to research, as well as in our attitudes toward the role of technology in research. Yet we have found common ground in discussing how we and others go about doing research in HCI—an interest that has fueled our workshops. Eventually we found no answers to the question. Instead, we found a set of shared questions that we think are useful, particularly for new HCI researchers.
But why bother a researcher in the making with even more questions? We see the nine questions below as thinking tools for choosing ways to conduct research. They help find a way to see beyond a single paper and anticipate where our choices and outcomes might take us. At the same time they serve as reminders to carefully consider what is important in our research community. And, imagining our own beginnings as Ph.D. students or postdocs, we came to realize that we wished we had asked ourselves some of the following questions.
1. If you could address just one problem in 10 years, what would it be? In the setting of current funding schemes, career paths, conference deadlines, and so on, you might be lured to the easy research pickings without considering the effort and time needed to complete more meaningful research. Don’t be afraid to do research that is unpopular or hard, with longer perspectives. Often this sort of work is slower to publish, and with the yearly circus of CHI and other conferences, retaining focus is hard when all your colleagues are getting Best Paper awards every year.
The short-sighted selection of problems hampers the whole field. It detracts from our ability to address grand challenges and pursue solid contributions. Most of the deepest contributions to HCI have required long periods of concentrated research on an idea that may have looked impossible or naive in the beginning. There is no guarantee of success and the risks are high. But the payoffs can compensate. Consider some breakthroughs in HCI, such as tangible computing, activity theory, or cognitive modeling: Their influence peaked only after several years of work.
The selection of research problems should be done against longer-term, even career-level goals. It’s hard but worth pursuing. Ask yourself:
- How does this research problem serve your career goal or the main goals of the field?
- What are the steps to be taken, here and now and in a longer perspective?
- Are the problems you pick or the way you frame your research going to allow you years of research on one question?
2. Are you using your unique situation and resources to the fullest? In essence: What may be the benefits of doing your research at a university versus at a commercial research lab, since both offer possibilities and limitations? Why does it need to be done as research, and not as product development or technical development at scale? Commercial research labs may have more resources to build solid technologies or access richer use data, while universities provide flexibility in the choice of research topics and the possibility of longer-term research agendas.
If you are studying for a Ph.D. or doing a postdoc, it helps to know why you are there. Take time to understand the intellectual as well as practical history of your research group (assuming you are part of one), its merits and strengths as well as its weak spots. What distinctive and unique features does it have, and how are you going to exploit that?
For instance, the research group at Aarhus University has long been known for participatory design (PD). Certainly PD is not all what the group does. However, it would seem strange if a newcomer to Aarhus did not somehow make the best of this connection.
What can you do about it?
- Concern yourself with the history and track record of your research group and environment. Ask yourself: What does it do best?
- Build a network of peers who have similar interests and concerns. Ph.D. courses and international conferences and workshops are good places for this.
3. What’s your HCI research genre? HCI spans a bewildering range of research “genres.” To simplify matters, we suggest three ways of splitting genre in HCI:
- You might be a designer, builder, or constructor of interactive technologies.
- You could be conducting empirical studies of newly deployed technologies or existing ones.
- Or you might be trying to develop theoretical accounts of HCI phenomena.
Of course, these overlap, and you may be engaged in all three. While this is a thinking tool, not a definition of HCI, considering where you are within this triangle could help position your work clearly for others.
You can use the triangle to consider things critically: Is my research question solvable without building things (e.g., prototypes)? Is it without a theoretical basis, or for that matter without empirical insight? A project on mobile technologies, for instance in art galleries, can be based on extensive empirical investigations through observations, interviews, and questionnaires. It can involve technological prototypes and even lead to an innovative design. It can be informed by theories of many sorts—for instance, on experience, materiality, or Fitts’ law—some general, some specific.
What can you do about it?
- Explore relevant literature for conceptual/theoretical contributions that can help you get started. Consider whether you need to look beyond HCI’s mainstream for conceptual inspiration.
- Consider the importance of a functional technology to your project, as well as the relevance of less technically ambitious prototypes.
- Think about what kinds of empirical insights you’d need to both open up and consolidate your project.
- Could alternative contributions open new vistas to your topic? For example, if you have been working on an interaction technique but there are no studies of it, what kind of study would most help to develop it further?
4. In one sentence, what is the contribution of your research? Ask of yourself, what is the contribution of my work? Answering this question will brutalize all your carefully nuanced caveats, your hedged arguments, and your considered scholarly verbosity. Yet at the same time it will force you to learn how to start telling others about your research.
Think about how to summarize your contribution before committing to your topic. If you can’t do it now, how do you expect to write up a paper about it later? Sometimes things develop differently, of course, but the question is still useful to ask, even if the answer evolves over time.
Often highly cited papers in HCI are wrapped around one clear contribution that is presented and re-presented throughout the paper so that the reader is in no doubt. This is for the reader, but also so the reader can communicate the contribution of the paper to others. Redundancy is key: From title to abstract, from the introduction to the conclusion, the focus on the contribution drives the coherence of the paper ahead.
What can you do about it?
- Pitch your work as often as you can and present your work well before submission. The contribution here might be imagined, simply because the research is not complete.
- Experiment with new framings for your contributions.
- Be clear about the main contribution or the strongest one if you have several.
5. Is your approach right for your research topic? And is your topic right for your approach? We all like to think our research approach is the Correct One. But we have to acknowledge there are certain research problems that are better dealt with using different approaches. For example, field studies aim to capture the use of technology in particular contexts but might not generalize across situations; lab studies allow for very fine-grain control yet introduce lab-specific phenomena.
Ask yourself, is there a match between my approach and my chosen research problem? You may wish to pick a familiar approach without much concern for what is useful for your particular research question. Research will be easier, but the flaws of the approach may harm the outcome. As reviewers on many papers, we all have seen how such mismatches kill papers. While there is sometimes value to innovating research approaches, don’t do it without clear reasons.
Here are some questions to ask:
- What can you do best with your preferred method?
- What would be the drawbacks and benefits of using an alternative approach?
- Are you more concerned with “sticking to method,” where creating what is needed might be more appropriate?
6. Why is your research interesting? HCI is diverse, encompassing multiple and discontiguous perspectives. For this reason, you can never rely on your work’s presumed “brilliance” to be obvious to everyone. You need to clearly communicate the significance of it to others, but you must also explain why they should care, and at the same time identify who those others are.
If you are really close to the work, it can be hard to consider what might actually be surprising in your research. Try thinking about your findings from one of HCI’s alternative perspectives: Are your results more surprising than you thought? If you don’t reflect on this question, you may end up burying the most surprising things among stuff that only you care about.
As an example, it was surprising to people in HCI when Lucy Suchman demonstrated mismatches between the actual activities of a human user and the implementation of goal-based task decomposition underlying the design of user interfaces. The result shed light on key conceptual assumptions about how “plans” feature in models of both machine and human action.
In order to bring out your research in the best possible manner, you may want to consider:
- Which beliefs or tacit assumptions does your work challenge? If the result is obvious to everyone, how valuable is it?
- Ask a colleague who does not do similar research to look at your work. Have them describe to you what they now know that they didn’t know before. It will probably be different from what you expect.
7. Can you fail in trying to answer the research problem? It is easy to fail in research: Data collection can be fumbled in some way; equipment can break down; users will not do what you “want” them to do; and so forth. But we’re not pointing toward that sort of failure here. Instead, ask yourself whether failures are even possible with your given question and research setup. Consider what a failure for your research would look like. Think about what an interesting failure would be—perhaps not finding a hypothesized difference between interfaces or not confirming the predictions of a performance model.
Answers to research problems may often be a priori (i.e., predictable from the models used). They also almost certainly follow what is given through what is already known in HCI (e.g., on interactive behavior). The answer may be given in the specific setup of a study or form of analysis. If your research cannot fail, it hardly advances or challenges HCI. Ultimately, failures can be highly generative and have the potential of advancing what we know about human use of interactive technology. They can challenge our expectations or the established wisdom in HCI.
Opportunities for failing can be thought about for most research questions:
- In empirical studies, would null results matter?
- For your key expectations, is finding the opposite possible? Would failing be worthwhile or of interest to others?
- If you build on earlier results, is there a potential for exposing those things as incorrect in some way or otherwise challenging them?
8. Will your work open new possibilities of research? Writing down plans can be very boring. Nevertheless, consider making a long-term research plan, one that helps you situate your current work. Consider whether your work is generative and what kind of generativity it offers. If you are building something, are you laying a path for a new class of interactive systems or might it be a dead end? If you are studying a prototype technology in use, do your findings have implications only for the next version of that prototype or are they abstractable as principles for designers working in the particular use case? If you are investigating a mass consumer device, is your study producing alternatives and new perspectives for understanding human action?
You can think about the potential contributions of your research in terms of how it might enable important new research problems; greater problem-solving capacity means that whoever reads your paper might be able to address research challenges with a higher success rate and efficiency, and greater confidence. What power might your research have for others? For instance, usability engineering increased practitioners’ problem-solving capacity remarkably. It offered a simplification of methods that were hard to master in a practical context. Obtaining good results in interface design did not require years of education in empirical methods.
Things to consider:
- Could you write a research-project proposal off the back of your last published paper?
- Consider problem-solving capacity by considering the kinds of outcomes your work would produce.
9. Why do you build/prototype? Doing HCI research does not mean developing products (although we might make something product-like or investigate existing commercial products). Instead, HCI’s technological prototypes serve many purposes: to understand, to elicit, to provoke, to learn, to show feasibility (i.e., an “existence proof”), and more. So it’s important to understand why you are building some technology. If you cannot answer that question, you may spend time completing unnecessary technical work or leaving things too incomplete where more research depth is necessary.
Are you trying to build better interactive technologies or trying to make HCI research better? Or both? The difference between these two is subtle but important. Making better things is not the same as making better products; building new interactive techniques, devices, and systems for research can lead to radical creations of new interactive paradigms (e.g., tangible interaction). Making better research is about introducing new concepts and approaches to how HCI research itself is done (e.g., participatory design). You can do both, but be careful not to confuse them lest you spend time and resources simply doing the wrong things.
What can you do about it?
- Consider whether a particular technology is part of the outcome of your research or whether, while yielding sound technological insights, it is merely a means of your research. If it is a means, decide what it is a means for.
- Apply least-effort strategies to do only what is needed to understand, to elicit, to provoke, to learn, or to show feasible, in general or in the particular use setting you address.
- And, are you putting your work in the right venues that appreciate one, the other, or both research modes?
Deciding what to do research on is strongly governed by our own personal judgment. Such judgment calls are full of dilemmas and tricky trade-offs. You cannot be brilliant at everything, no matter how hard you work. As authors we try to use the above questions to sharpen our own research, yet we recognize that finding great research problems is also about intuition and dialogue with your community. And it’s also about what you find most interesting and fun. Enjoy!
Stuart Reeves gratefully acknowledges the support of EPSRC (EP/K025848/1). The Aarhus University interdisciplinary Center PIT sponsored part of our writing of this paper.
1. Including Patrick Baudisch, Victoria Bellotti, Sebastian Boring, Mike J. Chantler, Torkil Clemmensen, Pierre Dragicevic, Giulio Jacucci, Yvonne Jansen, Jussi Jokinen, Jesper Kjeldskov, Vassilis Kostakos, Jörg Müller, Stefano Padilla, Esben W. Pedersen, Constantin Schmidt, and Mikael Skov.
Susanne Bødker is a professor of human-computer interaction in the Department of Computer Science, Aarhus University, and a co-director of PIT. She works on computer-mediated human activity. firstname.lastname@example.org
Kasper Hornbæk is a professor in computer science at the University of Copenhagen. He works on user experience, shape-changing interfaces, large displays, and body-based interaction. He is also interested in the methodology of HCI, including the role of replications, measures of usability, and solid experimental work. email@example.com
Antti Oulasvirta is an associate professor at Aalto University in Helsinki, Finland. He works on computational user interface design. firstname.lastname@example.org
Stuart Reeves researches interactive technologies for a range of cultural, performance, and public settings. He holds an EPSRC Fellowship and is investigating the work practices of UX professionals. He is also author of the book Designing Interfaces in Public Settings. email@example.com
Copyright held by authors. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.