The Gap Between HCI Research and Practice
A popular definition of HCI as a discipline (ACM SIGCHI Curricula for Human-Computer Interaction, 1996) reflects a duality: HCI is a discipline concerned on the one hand with practice, and on the other, with the research into phenomena associated with this practice. But if one would ask practitioners engaged in the design, evaluation, and/or implementation of interactive computing systems if they benefit from research into phenomena associated with these systems, many would probably agree with the following quote:
"There are very few, if any, research articles published in scientific and academic journals that can be utilized effectively in the practice of HCI design" (anonymous practitioner, present).
Moreover, if one asked researchers engaged in studying behavioral, social, organizational, and other phenomena associated with interactive computing systems whether they are concerned with the actual design, evaluation, and/or implementation of such systems, many would probably indicate that there is not much linkage between their research and the actual practice in HCI. Such responses represent a gap between the practice and research sides of HCI. Does such a gap represent a problem? Yes.
Imagine you are a practitioner asked to examine an application. If you find it ineffective, you’ll need to propose what needs to be redesigned in the user interface to improve its effectiveness. You are then faced with the question of why it is ineffective. In other words, you have a question that requires relevant research findings that can inform you of why or why not things work in your application. Such understanding can drive better design decisions.
The need for research, what kind of research, where to locate that research, and how to use it all depends on how HCI research is defined and how the research question is formulated. In this essay I delineate different levels of research in HCI, and given such taxonomy, how research can inform practice.
Taxonomy of Research in HCI
I propose adopting the following definition of research: any systematic endeavor to find an answer to a practical or theoretical question. Within such a definition, usability tests can be considered a type of research. A common strategy in a case similar to that mentioned above would be to employ some form of a usability test. While usability tests are very good at determining if a product is effective or efficient, or whether it provides appropriate user experience, usability tests are not very good at offering an explanation of why something works or not.
Research findings that can be generalized to a variety of contexts are more likely to provide basic principles that can account for various patterns and phenomena. This implies that there can be different levels of research that can be beneficial for different purposes in various contexts.
Broadly speaking, HCI research can be viewed as a multitiered structure, with each tier corresponding to a different type of research (see Table 1).
The four tiers differ along two interrelated dimensions and can be visualized as a pyramid (see Figure 1).
Level of focus. Research types range from addressing questions focusing on a specific product, to comparing between products, to searching and examining guidelines for a family of products, through to general questions on behavioral, social, organizational, and other phenomena. The higher the research type is in the pyramid, the narrower its focus. Thus, usability testing has a typical narrow focus on a specific product. Research that addresses theoretical questions may not even be concerned about specific products, and its focus is usually broad.
Extent of generalization. The capability to generalize the findings and conclusions of the research to a variety of contexts and problems is directly linked to its level of focus. A usability test of a design has very narrow generalization potential and can be applied primarily to the specific tested product. It can rarely be reused in any other contexts. Conversely, a theory-oriented research has a large generalization potential due to the questions it addresses and the rigorous research methods used. The findings and conclusions of such research can be reused in a variety of contexts.
Use of Research in HCI
Given the proposed taxonomy of research in HCI, how can a practitioner use it to benefit from the varieties of research types? The answer lies in how one abstracts the design problem and defines the research question. A given research question can lead one to find and benefit from the relevant research type.
Imagine you are a practitioner involved in the design of a Web site for browsing via a mobile phone. One of your design challenges is the information/navigation structure for that Web site. This design challenge can be abstracted on different levels and represented as different questions. The following four examples of possible questions are associated with the four tiers in the research pyramid. The questions are not mutually exclusive. Addressing any or several of these questions will lead the practitioner to seek different types of resources that can provide an answer.
1. Given a hierarchical structure, should the navigation structure have five levels and a maximum of ten options in each level? Such problem definition can lead to performing a usability test. Usability testing, being a narrowly focused research type, will address the question only if the product with five levels and ten options in each level of a specific product is effective, efficient, satisfying, or meets any of the usability/design objectives. The generalization power of this test to other products and contexts is very limited.
2. Should the hierarchical navigation structure of this Web site be wide (fewer levels, more options in each level) or deep (many levels, fewer options in each level)? Defining the problem in this way will encourage looking for a comparative study between at least two navigation structuresa wide and a deep oneto determine which one meets the usability/design objectives. Other comparative studies that can be relevant to such a question are studies looking at the impact of different tasks, contexts of usage, or any other contextual factors (e.g., user profile, device type, etc.).
3. Are there some general guidelines on how to design navigation structures for Web sites on mobile phones? The third question will lead to searches for guidelines, standards, style guides, and similar sources. Such guidelines are usually based on the findings of many studies, which could be comparative or could even be usability tests. Guidelines could also be derived from a more general research type that is found at the base of the research pyramid. Research studies that provide guidelines can be generalized/applied to several products or families of products; for example, a given set of guidelines could apply to all cellular phones, or all Web sites viewed on all small-screen devices.
4. Is the type of navigation structure related to human working memory, or is it an orientation and way-finding issue? The fourth question is formulated as a general one that addresses basic aspects of human behavior. In the above example, the design problem can be framed as one related to human-memory limitation, human orientation and way-finding in information spaces, human language comprehension, or a combination of these. Defining the research question in this way will lead one to search for the relevant research studies on the limits of human memory. Such research can facilitate an understanding of the implications of wide versus deep navigation structures, and can fill in gaps not covered by guidelines.
Toward a Common Ground
There is indeed a gap between research and practice in HCI. Primarily, practitioners express difficulties in benefiting from research. It was proposed here that there are different types of research in HCI, and those types were delineated as a taxonomy. The ability to utilize and benefit from any of the research types depends on how a practitioner defines his practical problem as a research question. The abstraction of the question on different levels can lead one to search and find potentially beneficial research that can be applied in the practical arena.
About the Author:
Avi Parush’s professional career in human factors engineering (HFE), usability engineering, and human-computer interaction (HCI) spans over 20 years. He has been involved in user-interface design and testing in a large variety of domains and projects. Parush is the founder and editor in chief of the Journal of Usability Studies. His academic background is in cognitive-experimental psychology and he is presently a professor of psychology at Carleton University, Ottawa.
To submit a research article, please email Carolyn Gale at firstname.lastname@example.org
©2006 ACM 1072-5220/06/1100 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.