Elizabeth Churchill, Anne Bowser, Jennifer Preece
Human-computer interaction is a discipline concerned with the design, evaluation, and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them.
Thomas T. Hewett et al., 1992
Human-computer interaction (HCI) as a field of inquiry necessarily evolves in response to changes in the technological landscape. During the past 15 years, the speed of change has been particularly dramatic, with the emergence of personal mobile devices, agent-based technologies, and pervasive and ubiquitous computing. Social networking has also profoundly changed the way people use technology for work and leisure. Who would have predicted a decade ago that (smart)phones would offer constant access to the Web, to social networks and broadcast platforms like Facebook and Twitter, and to hundreds of specialized apps? Who could have anticipated the power of our everyday devices to capture our every moment and movement? Cameras, GPS tracking, sensorsa phone is no longer just a phone; it is a powerful personal computing device loaded with access to interactive services that you carry with you everywhere you go.
In response to these technological changes, user populations have diversified and grown. Once limited to workplaces and used only by experts, interactive computational devices and applications are now widely available for everyday use, anywhere, anytime by any and all of us. Though complex institutional infrastructures and communications networks still provide the backbone of our digital communications world, HCI research has strongly affected the marketability of these new technologies and networked systems. HCI now has a major role in developing products for what Jonathan Grudin refers to as “discretionary use” by everyday users .
HCI specialists still focus on the issues that gave birth to the field: Are technologies learnable, usable, useful, reliable, comprehensible, ethical? We are still concerned with assessing whether technologies serve, engage, and satisfy people and extend their capabilities, or frustrate, thwart, and confound them. While HCI has always been associated with “usability,” its remit has always been broader and continues to grow: Systems and applications must also be aesthetically attractive and emotionally appealing, and they must provide the right level of challenge and satisfaction for users. Further, many devices are becoming fashion objects: They lead, track, and follow the mercurial world of fashion. Psychological, sociological, and cultural questions remain pertinent: As the technological landscape changes, which human characteristics, capabilities, and traits remain stable and which do not? Which technologies bring advantages to some and disadvantages to others, and how might we level the playing field to provide access and success for all? Which technologies and interaction styles survive the test of time and which do not?
The new issues emerging from the development of new technologies and a continually expanding user population require HCI progressionals: practitioners, researchers, and educators who are willing to keep step with technological progress and master new design and evaluation methods and new technical competencies while maintaining the professional and intellectual values, tenets, and perspectives that are unique to our field. Universities, professional bodies like ACM, and professional trainers must keep abreast of advances. At the same time, an increasing number of companies and universities in countries across the world are embracing HCI and welcome knowledge about how HCI education is structured in regions with an established history in HCI education as an appropriate starting point for building new programs molded to their own needs.
Here, we report on an ongoing project sponsored by the ACM SIGCHI Executive Committee to investigate the philosophies and practices that underpin present and future HCI education. This project is a response to requests from HCI educators and hiring managers for guidance and insight into which HCI methods and skills can or should be taught to new graduates and incoming and current staff members. In addition to providing guidance for those developing courses at the university level, a major incentive was to consider which issues could be covered at SIGCHI-sponsored conferences, through our SIGCHI chapters, and through the provision of online resources.
Our project set out to ask: What are the core issues of the field, and to what extent (and how) should we ensure students are trained and researchers, practitioners, and educators are kept abreast of changes in technology and the emergence of state-of-the-art methodologies, theories, and practices that result from those changes? Over the past several months, through interviews and surveys, we have asked researchers, practitioners, educators, and students in the HCI community: What is HCI now? What will it be in the future? What should the remit of HCI be and what does and should HCI education entail?
A Curriculum for HCI?
There have been several efforts to outline core courses for a program in HCI. In 1988 the Executive Committee of SIGCHI sponsored research that led, in collaboration with several key thinkers on related panels, to a document published in 1992 titled “ACM Curricula for Human-Computer Interaction” (see http://old.sigchi.org/cdg/index.html). Known affectionately as the “Lime Green Report,” this document provided a blueprint for early HCI courses (the quote at the beginning of this article is from it). The report reflected themes and issues that were emerging at the time, laying out areas that were deemed to be central to HCI practice and therefore central to HCI education. It focused specifically on the notion that educational programs should become increasingly “HCI oriented,” not “HCI centered,” and that HCI could be characterized as more of a “sensibility” in need of a specific set of methods and toolswhich, though in development, were less established than in most other parts of computer science. As a result, the authors argued, HCI education needed to continue to include domain-appropriate methods and tools that were familiar to the traditional contexts from which HCI was emerging: primarily computer science, psychology, and engineering. They also argued that we, as educators and learners, needed to embrace new perspectives and new areas of focus; the areas called out were subjects typically taught in art and design schools, and in the information and library sciences. That said, the authors favored HCI aligning strongly with computer science and information systems programs, referring to these as “base disciplines,” and stated: “We have therefore designed frameworks for an HCI orientation within a computer science program and within a management information systems program.” This orientation reflected the archetypal North American education system, but many readers, seeking guidance to shape their own curricula, tailored it to their own cultures.
Although this 1992 curriculum proposal still contains material that is key to our field and we urge readers to review it, we know there are topics that were not covered, areas that are no longer seen as core, and new areas that need to be added. Consequently, we, along with the many people whom we consulted while doing this research, have been asking: What should be the core topics taught in HCI education? We started with conversations and surveys with people in the SIGCHI community in order to develop a base understanding of current perspectives and concerns. Below we present some preliminary observations from our research. The topics and themes we present are distilled from a large volume of interview and survey data that we collected from HCI educators, practitioners, and students from across the world.
A Global Perspective: What Topics are Considered Core to HCI?
HCI is practiced globally. A challenge for us has been to ensure we hear voices from different regions. Therefore, we broadcast our initial survey in English over several months, recruiting colleagues and friends around the world through social networks. To date, we have received survey responses from more than 30 countries for our English-language survey (see Table 1), and conducted in-person and remote follow-up interviews and focus groups with English-speaking HCI professionals and students from at least 10 of those. In addition, we have worked with colleagues to translate our survey into Portuguese and Chinese, and are seeking to broaden our language base and dissemination.
As part of our survey, we asked participants to rate subjects and related fields, topics, and methods that emerged from pilot surveys and interviews in terms of their relevance/importance for HCI. When we analyzed participants’ responses, a number of intriguing similarities and differences emerged. Our surveyed populations differed in which topics they rated most highly. For those answering our English survey, qualitative research methods and approaches is considered a key topic. Portuguese-speaking respondents rated interaction design (as a paradigm, rather than as a set of specific methods) as most important, while Chinese-speaking participants favored mobile design and development. When we asked for more detail, we also saw intriguing convergence and divergence. Table 2 summarizes subjects and related fields, topics, and methods that were rated positively and negatively by our survey participants across the different language groups . While the dominant group is our English-speaking survey responders, we see some intriguing trends when we break out the responses from our Portuguese and Chinese respondents.
Themes: Opportunities and Tensions
HCI has always been about results, insights, methods, theories, and sensibilities. And it has always attracted diverse and sometimes incompatible or competing theoretical and practical perspectives. In recent years, computational tools have become more readily available for recreational and discretionary use. Their omnipresence suggests that this generation of new technologies is easier to use than their predecessors, even though the ecosystem of devices and information flows is getting more complex.
HCI researchers and practitioners increasingly address the broader social implications of the roles technology can play in our lives, in addition to helping make inherently complex technology more accessible and usable. Susanne Bødker nicely frames these shifts in terms of waves of HCI . According to Bødker, these waves reflect the technological status quo: The first wave focused on mainframes and desktops for work, the second on individual and collaborative work within well-established communities of practice, and the third wave focuses on the pervasive and ubiquitous presence of interactive technologies and their appearance in “non-work, non-purposeful, non-rational” contexts. This third wave also includes consideration of cultural and historical embeddedness, emotion, and aesthetics.
Unity in HCI education and training. As a result of all these “waves” coexisting, HCI as a field remains complex. Our remit is growing as we try to honor so many different perspectives and gain skills in so many areas. Therefore, it is not surprising that a common refrain we hear is “We need a mission statement or a value proposition that people can hang their hats on.” Our survey respondents and interviewees call for some form of unity or consensus; there is a desire for “a unified theoretical perspective” and “a common curriculum.” Lack of consistency and common structure in training/courses causes anxiety and leads to much hand wringing in course selection. Concerns are expressed about the lack of a “common valuation of an HCI degree.”
Clearly, some courses are ranked higher in terms of relevance of subject matter and quality of tuition than others, as is the case for all disciplines. However, the problem is perceived to be more far-reaching than the creation of a rank-ordered list of courses would imply. For many in our community (educators, practitioners, students, hiring managers), a lack of consensus and clarity about the value of a degree in HCI is concerning for two related reasons: First, a standardized curriculum or degree would help students select programs and inform industry professionals about what to expect from the students they hire, and second, survey and interview respondents realize that a more unified perspective would make it easier to advocate for the value of HCI as a discipline, noting that “a common language seems to be lacking.” In other words, there continues to be fear that we have failed to clearly state the value proposition of having a strong education in HCI, despite there being clear value associated with the skill sets that students in HCI develop. This is an ongoing concern for HCI as a discipline, as has been articulated by Shackel [4,5] and others since the field’s emergence.
While concern was expressed about sending clear messages regarding the value of HCI and articulating the core perspectives and topics represented within the discipline, survey and interview participants clearly indicated that HCI is, and should be, essentially multidisciplinary. As one survey participant said, “I don’t think a unified theoretical perspective is possible or desirable.” HCI is an interdisciplinary field, drawing on research from psychology, computer science, design, anthropology, information science, and others. Therefore, it is not surprising that many participants identify interdisciplinarity as a key element in HCI education. One usability manager writes, “It is hard to learn the core skills in one place or department.” A student expresses a similar sentiment, noting, “Interdisciplinarity is very important, and not just from an ‘I’m going to include someone else on my project’ perspective…but it’s important that our students be competent in more than one area.” Many participants point to the social sciences as increasingly relevant fields. In the words of one, “I don’t like saying ‘old HCI’ because it is all still current and relevant and we don’t do it right. But the set of interdisciplinary domains that we’re drawing [from] is much more restricted…and social sciences can help.” A related challenge worth noting is the need “to respect different epistemological differences.”
A standardized or flexible curriculum. One theme that emerged early in our research was the question of how structured and how flexible an HCI curriculum should be. One professor pragmatically implies that HCI specialists may have a harder time finding and retaining jobs in times of economic hardship: “The problem is that they’re not going to be deep in everything. It’s not a problem as long as the economy is running fairly smoothly. But for students who have multidisciplinary things…when the going gets tough, they strip it to the bare bones.” A second professor believes that students with diverse portfolios are more likely to succeed, saying optimistically that “[t]he trade-off is: Do we think there should be a body of scholarly knowledge that everyone should know, or are we training them in different areas of practice so they can go out in industry and do whatever? I’m leaning toward the latter because it’s hard for me to stomach sitting in front of them and saying they have to know this stuff because it’s good for them. They’re going to want those experiences to be more valuable.”
All our interviewees and the survey participants were keenly aware of the need to carefully address the future of HCI and to recognize that the field is continually changing, making comments such as “It’s more than just a computer now.” There is considerable awareness that touch, speech, and gesture-based interfaces are the future of interaction: As someone succinctly put it, “a lot less keyboards and mice.” Smarter interfaces that utilize AI techniques for personalization and adaptation and that integrate new developments in areas such as vision processing require deeper knowledge. But diversification and platform fragmentation remain a central concern. As one person notes, “What concerns me is that there are fundamental things you need to know for designing in desktop that are different for mobile apps, Facebook apps, and things on an iPad. There’s rarely any capacity to teach multiple courses on these different kinds of media. It takes long enough to teach screen design for the webpage.” And innovation results in a constant pressure to remain relevant; as one professor told us, “I am currently trying to teach courses that use these new technologies. There aren’t any existing textbooks that really support that easily but that’s okay because there are a lot of interesting papers and exploration areas to draw on.”
Breadth and depth in HCI. Of course, interdisciplinarity brings its own set of tensions. Given a finite amount of available time, there are only so many topics that can be explored in depth. Yet our investigations reveal that both depth and breadth are demanded, or at least expected, from courses but that depth is associated with rigor, and breadth with a lack thereof. Commenting on breadth, one educator believes that “the top trend is to water it all down and out of the curriculum.” Another interviewee states it more strongly: “If you set a requirement and what happens is you get a person who is mediocre in a lot of thingsthat’s something I don’t want to happen.” Another remarks, “The challenge is students might come away with a broad set of skills that they are decent at instead of a few things that they are expert at…HCI education as the core discipline may not emphasize the depth part enough.” A professor summarizes the issues as follows, noting that this is not simply an issue for HCI, but rather a general observation about the Ph.D. process: “What a good Ph.D. student should know in terms of breadththat’s the most problematic piece. Because one of the problems is that when you finish your Ph.D. you are the world’s smallest expert in this particular topic…it’s such a limited point of reference.” Students too are concerned that without depth they will not be considered domain experts, but without breadth the value of big-picture or systems thinking that is central to HCI is lost. However, it is possible that encouraging a broad curriculum may lead to a more unified theoretical perspective because a broad curriculum encourages diverse ideas from “people who are versed in multiple perspectives.”
Technical skill sets. In some instances, depth and rigor seem to be confused with prowess in specific techniques. An enduring question in HCI education focuses on what specific technical skills students should be exposed to and/or required to master. Some believe that every student should learn basic computer programming: “I am discipline-centric enough that I believe everyone, not just those in CS, should know what a loop is.” A user experience architect agrees on the grounds that “being able to know their language and talk to them [programmers] very directly enables you to have credibility.” On the other end of the spectrum, a UX instructor chose to pursue an M.S. in library and information science because “f I had done computer science they would have made me do programming, which I felt was totally irrelevant to HCI.”
A closely related tension exists between computer scientists and non-computer scientists, or “people who program and those who study people who program (i.e., because they can’t program).” According to one computer scientist, HCI “doesn’t always get respect from traditional CS faculty; it’s still viewed as a question of (inessential) aesthetics.” A similar frustration was expressed by those who feel it is tough to get traditional computer scientists to develop the sensibility to understand HCI: “[Y]ou first have them build a system…and then you show them that what they built was not a good interface in terms of usability.” However, with increasing focus on copy-paste programming, sophisticated development libraries, plug-in modules, and scripting environments, it is questionable where formal education in computer science and facility with computer programming actually meet. Computer science as a discipline is having its own issues in this regard, and bold attempts are being made to encourage the learning of computational skills by appealing to a broader range of different learning styles than were traditionally supported in computer science education.
Theory versus practice in HCI. A further issue that is regularly raised is that of the relationship between theory and practice. While half of the people we talked to thought both theory and practice were essential to training in HCI, it was clear that some form of exposure to empirical research through hands-on projects and practical experience through internships and apprenticeships was considered important. Linking these activities back into theory and overarching meta-frameworks is essential. A respondent who critiqued a course that focused heavily on practical application and technical expertise told us that “[t]heir whole curriculum was based around software to the point that graduates would then go to interviews and present their portfolio and say, ‘This is my flash project,’ ‘This is my Photoshop project,’ rather than, ‘This is where I tried to solve this problem, and I used these tools.’”
Again, exactly which theories are fundamental to learning in HCI is contested. Frustration was expressed about a lack of understanding of theory, particularly in the participants’ own chosen discipline. For example, one person said, “I think it’s important to go into the more theoretical basis of HCI. And when I say theory, it’s not just mathematical but the theories coming from the social sciences and from psychology.”
Academia and industry: an either/or? Another recurring theme is a perceived divide between academia and industry, which was of particular concern to students. In the words of one, “I think the divide between practitioner and researcher is growing, and this is becoming apparent in HCI education. While most of the professors are coming from academia, they need to understand the differences between the two worlds in order to best inform their students, who may be going into either world.” Another worries about competing in research: “We are quickly moving beyond the point where a capable student with a team of researchers can compete with industry.”
Many academics mirror these concerns, often on their students’ behalf; the opinions of industry practitioners are divided. While students articulated concerns about making a choice early on between industry or academia, some more seasoned professionals, later in their careers, reflected on the blurring between the two arenas as positive, with opportunities for collaborative engagement on joint projects.
A collaborative effort will be particularly helpful for HCI educators and learners in areas where HCI is starting to be recognized as an emergent field of study in response to growth in interactive technology design and development.
Notably, however, our table of topics rated most and least highly by our different constituencies clearly shows a disjoint between industry and academic professionals (see Table 3). While industry professionals give high ratings to topics such as change management, e-commerce, product development, and product management, and low ratings to topics such as health informatics and ubiquitous computing, academic professionals rate ubiquitous computing highly and change management low. We are loath to derive too many hard-and-fast conclusions from this data, but on the surface it appears that these disjoints reflect the relative amount of importance placed upon near- and medium-term goals for product launch and maintenance versus longer-term goals in industry, and the more agenda-setting and exploratory role afforded to research in academic institutions. Since its incept, HCI has been focused on operationalizing general principles derived from abstract principles. This includes instantiating abstractions into products, which includes producing frameworks and processes (including change management and product development) that are tactically appropriate to enable strategic success. In addition to driving research insights forward academically, HCI is charged with providing better development methods that draw on and honor those insights.
Summing Up, Looking Forward
At the highest level, the results of our research echo the observations of Hewett et al.‘s 1992 report, in which the authors state: “There is currently no agreed upon definition of the range of topics which form the area of human-computer interaction” .
In some respects, this is surprising, given the passage of time and the incredible advances we have witnessed in technology development, plus the growing awareness of the role of HCI, particularly in consumer product development. This awareness has encouraged some companies to employ HCI specialists, which in turn influences universities to develop HCI programs and courses. In other respects, it is not surprising that the fundamental issues that emerged from our research are similar to those identified by Hewett et al.: Our multidisciplinarity is an enduring strength but also a continual challenge. We believe that there is cause for great optimismmore and more people in our discipline feel empowered to participate in discussion of these tensions. In 1992, HCI education was relatively new. Very few people outside of SIGCHI and other sister professional communities felt their voices could, or would, be heard. At that time, most HCI educators knew all the courses and programs that were offered across the world. Now, there are hundreds, if not thousands, of HCI courses. But does this signal success? It depends on whom you ask. For some, the answer is a resounding “yes!” For others, there is concern about disciplinary focus and differentiation, “quality control,” and “brand.” Certainly, we need a deeper understanding of the issues raised here, and issues that are emerging as a result of developments in the technological sphere. We believe that there is no one size fits all solution. There are many paths through our multidisciplinary field, and we can rejoice in its richness and diversity while acknowledging that at times it may be daunting, particularly for newcomers.
We invite you to join us in an ongoing conversation about what is and what should be part of HCI education. We hope you will send us comments and reflections to help deepen and broaden the discussion about current needs, emerging trends, and upcoming challenges that HCI educators and learners are confronting. Educators are also invited to share their syllabi, class notes, and other teaching resources, and students are invited to share their perspectives on past and current courses. We also want to hear from HCI practitioners and those in related fields about their needs for workshops, courses, and meetings at conferences and in local settings, such as at SIGCHI chapter gatherings, or for online courses. We are eager to know which kinds of courses translate well to online contexts and which do not, and why. We invite hiring managers to let us know what HCI-related skill sets they see emerging in the workplace. In addition, we welcome help from educators, learners, practitioners, and hiring managers from around the world in adapting and disseminating our survey further afield.
As we all collaborate on various aspects of HCI education, we hope that an outcome of this research will be to stir interest in community-based sharing of teaching resources. A collaborative effort will be particularly helpful for HCI educators and learners in areas where HCI is starting to be recognized as an emergent field of study in response to growth in interactive technology design and development. In collaboration with educators involved in online course development, a goal for this project is to deliver recommendations for a collaborative social platform for sharing ideas about HCI education. We envision this as a “living curriculum resource” to be shared by all, rather than as a static curriculum recommendation.
So, whether you are a learner, teacher, practitioner, or hiring manager, please join this effort by sending your thoughts on the future of HCI education to firstname.lastname@example.org.
The authors would like to thank all the people who have contributed their time and expertise and are continuing to be involved in this ongoing project. In particular, we would like to thank our interviewees, our survey participants, and our CHI 2012 workshop participants. We would also like to thank our colleagues Simone Barbosa in Brazil and Zhengjie Liu, Junliang Chen, and Yufang Liu from Dalian Maritime University in China for translating and administering the survey. Finally, we thank friends on Facebook, colleagues in the IxDA, UXPA, and British HCI communities, and members of the ethnodesign email distribution list for their help in disseminating our survey. We also thank our colleagues on the SIGCHI Executive Committee for their encouragement and insightful comments on our work.
1. Grudin, J. Three faces of human-computer interaction. 2012; research.microsoft.com/en-us/um/people/jgrudin/.../annals.doc
6. Hewett, T. et al. ACM Curricula for Human-Computer Interaction. 1992; http://old.sigchi.org/cdg/index.html
9. Carroll, J.M. Human computer interaction (HCI). In Encyclopedia of Human-Computer Interaction. M. Soegaard and R.F. Dam, eds. The Interaction Design Foundation, Aarhus, Denmark, 2009; http://www.interactiondesign.org/encyclopedia/human_computer_interaction_hci.html
13. Pemberton, S. The CHI conference: Interviews with conference chairs; http://old.sigchi.org/bulletin/1996.1/conf-chairs.html
Elizabeth Churchill (elizabeth-churchill.com) is director of human computer interaction at eBay Research Labs. Her current research focuses on sociotechnical design, social media, computer-mediated communication, and e-commerce. A regular columnist for interactions, Churchill is also the current executive vice president of ACM SIGCHI.
Anne Bowser is a student at the University of Maryland iSchool, where she is currently pursuing a Ph.D. in library and information science. She is studying how the motivational aspects of games can serve the needs of citizen science and other crowdsourcing initiatives. Bowser is a contributor to Biotracker (www.biotrackers.net).
Jennifer Preece is professor and dean at the University of Maryland’s College of Information StudiesMaryland’s iSchool. A longtime contributor in the field of HCI, her current research focuses on understanding what motivates citizens and scientists to contribute to biodiversity citizen-science projects (see www.biotrackers.net)
As authors we represent the three different perspectives of the participants who contributed to the research that we are reporting: that of hiring manager, student, and educator. All of us bring a deep commitment to furthering a humanistic perspective that is technically grounded to HCI education.
Sidebar: HCI History
As has been well documented in a number of excellent articles [5,7,8,9], HCI emerged in the 1970s as a field of inquiry as a result of the increasing penetration of computers and computational devices into the workplace. First came mainframe computers, which required considerable training to use and much maintenance, followed by the creation of the personal computer. Pcs became increasingly available in offices and were promoted for home use during this time. Due to growing potential for computational devices in all walks of life, and the awareness of the difficulties people had using them, it was not long before the need for “people-oriented computers” was asserted . And thus the field of HCI began to emerge in the mid- to late-1970s, drawing researchers from disparate disciplines, including human factors and ergonomics, engineering, computer science, and psychology. A problem-centered, applied area of research, HCI needed this amalgam of perspectives and opinions, viewpoints, and methods. More a guild of researchers from various disciplines than a discipline in itself, the 1970s and 1980s saw a growing need for this area of focus in research and development. Consequently, calls for courses to train people in methods and for the development of a sensibility around the complexities of understanding human-device interaction followed. During the late 1970s and 1980s, the field gained momentum through the publication of academic texts (e.g., [11,12]), the establishment of university-level courses, and the creation of professional bodies like ACM’s Special Interest Group in Computer Human Interaction (SIGCHI) and its associated conference on Human Factors in Computing Systems (better known to those who read interactions simply as CHI). Of course, the establishment of interest groups and courses and conferences on topics that could broadly be construed as HCI was a worldwide phenomenon. Examples include the Interact conference, which was hosted for the first time in London in 1984, underwritten by the International Federation for Information processing and chaired by Brian Shackel; and the establishment of interest groups like the British Computer Society’s HCI group. Similar efforts emerged in France, Germany, the Netherlands, and elsewhere.
SIGCHI was formed in 1982 by renaming and refocusing an already existing special interest group, the Special Interest Group on Social and Behavioral Computing (SIGSOC). The first CHI conference was held in Gaithersburg, Maryland in 1982 and was chaired by Bill Curtis and Jean Nichols . By the mid- to late 1980s, core texts were being published, broadening the field of HCI and discussing how to more systematically consider peopleusersin the design of interactive systems. Examples include The Psychology of Human Computer Interaction by Stuart Card, Tom Moran, and Allen Newell, which celebrates its 30th anniversary this year; User Centered System Design: New Perspectives on Human-computer Interaction, edited by Don Norman and Steve Draper and published in 1986 ; and Ben Shneiderman’s 1987 book Designing the User Interface: Strategies for Effective Human-Computer Interaction .
©2013 ACM 1072-5220/13/03 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.