Simone Barbosa, Clarisse de Souza
Recently, Brazilian researchers have been receiving mixed signals regarding the recognition of human-computer interaction (HCI) as a relevant area of study in Brazil. On the one hand, HCI was granted considerable prestige when the Brazilian Computer Society launched its Five Grand Challenges for the decade in 2006. One of them was the “universal and participatory access of Brazilian citizens to knowledge and services,” a challenge that cannot be faced without seriously involving HCI research in its many forms. Four years ago, the establishment of such a challenge attracted the attention of several graduate students, who saw an opportunity to make substantial contributions to the Brazilian people and society with their research. It brought about an exciting and prolific moment for HCI in Brazil. On the other hand, in less than three years, these feelings have changed.
HCI in Brazila Bit of History
The HCI research community in Brazil has steadily grown in the past decade. The first Brazilian HCI conference was held in 1998 (IHC 1998), with 15 papers on the program. Up until 2002, we held annual conferences, gathering between 100 and 150 participants every year and bringing renowned international speakers, mainly from North America and Europe. In 2001, at the Latin American Development Consortium at CHI, we decided to join our efforts with other HCI researchers in Latin America to create a series of regional HCI conferences. In 2003, the first Latin-American Conference on HCI, CLIHC 2003, was held in Rio de Janeiro, gathering 250 participants from all over the world. Between 2004 and 2008, we held the Brazilian HCI conference every other year, alternating it with CLIHC, which was held twice in Brazil and twice in Mexico. In 2010 our national HCI conference included 32 papers in three different categories (long papers, short papers, industrial reports), in addition to demos and posters. The community decided to hold annual conferences from 2011 onwards and to have next year’s edition co-located with CLIHC, the Latin American conference, in Pernambuco, Brazil. Also, in 2007 we held the IFIP INTERACT conference in Rio de Janeiro, which brought even more attention and prestige to the area. We found 2006 and 2007 were good years for motivating HCI research in Brazil.
How Graduate Programs Are Currently Evaluated in Brazil
Every graduate program in Brazil, from any research area, is evaluated every three years by a governmental agency called CAPES (http://www.capes.gov.br). Its main goals are to ensure high-quality university programs at both undergraduate and graduate levels. In the most recent evaluation, 40 percent of performance was judged on how many high-ranking publications faculty and students produced for the graduate program. But one major issue was how the publications were ranked. CAPES issued its own rank of “qualified” publications for each of the 40-plus areas in which graduate programs were grouped, in a classification system named Qualis (http://qualis.capes.gov.br/webqualis/).
When CAPES was devising the classification criteria, there was a very strong reluctance to include both computer science journals and conference proceedings in Qualis. At first, CAPES was adamant about considering only journal publications in their ranking. After much discussion, the Computer Science Commission and CAPES have reached a compromise to create an objective measure of the quality of conference publicationsa formula based on citation indices.
Although the ranking for computer science included both journals and conference proceedings, the formula CAPES has come up with makes no distinction between the idiosyncrasies of different research areas. Younger research areas, emerging lines of research, innovative research topics, and localized research to address regional issues get consistently low values in the adopted evaluation. As such, HCI research is unfortunately among the areas that were most strongly affected by the current evaluation.
In the past, the citation indices were considered, of course, but the Brazilian research community could argue for the quality or relevance of a conference to the country, and then have the ranking adjusted accordingly. Whereas then we had room for some qualitative assessment of our publications, now our research community is restricted to the global indices that do not reflect the regional issues that must be addressed by Brazilian researchers.
HCI Evaluation and Graduate Programs in Brazil
In the most recent Qualis ranking established by CAPES in late 2009, only 1.6 percent of the top 25 percent of computer science publications were specifically devoted to HCI. Even globally recognized conferences, such as ACM CHI and IFIP INTERACT, were not among the chosen. This has had a very negative impact on young researchers looking for an academic career in HCI. Compared with colleagues doing research in artificial intelligence, software engineering, or information retrieval, their chances of having “starred” publications on their CV has become considerably smaller. Thus, in competing for a faculty position in Brazilian universities, HCI researchers are not as likely to impress graduate program directors as candidates specializing in areas with higher-ranking and more abundant publication opportunities.
According to the CAPES website, there are currently 43 accredited computer science programs in Brazil. Upon further review of each program’s website, 12 of them include HCI in their research areas or topics. Unfortunately, eight out of the 12 include HCI as part of another research area, such as software engineering, computer graphics, and so on. Only four of them characterize it as a standalone research area.
From 2004 to 2008, there was a high increase (a little more than 50 percent) in the number of computer science-related graduate programs, growing from 28 to 43 accredited programs. Among the 15 recently accredited programs, three indicate HCI as a research area on their websites. As emerging programs, they are even more affected by the lack of prestige. Academic programs may not only hesitate in hiring HCI researchers, who have a low potential of producing “high quality” publications as understood by CAPES, but may even come to prune the “problematic” research areas if the next CAPES evaluation raises a red flag.
The Impact of the Current Evaluation Criteria
The negative effects of the current Brazilian ranking for CS publications have already transcended individual academic careers. The 12-year national conference series on HCI, which has been greatly successful in consolidating a community of productive and creative HCI researchers, features in the second tier of the official evaluation ranking. Therefore, the conference is no longer a priority when Brazilian researchers wish to publish their research. This is especially disturbing when we stop and think about our object of study.
Among all multidisciplinary sub-areas of computer science, HCI is probably the most sensitive to the cultural context in which research is done. The “user experience,” which we all want to understand, improve, and diversify, cannot be dissociated from the user’s social and cultural background. Therefore, doing research in HCI requires that researchers themselves have a deep understanding of the particular sociocultural context in which the object of their investigation is placed. Hence, national HCI conferences have a great value for HCI research communities anywhere in the world, because they congregate people who have an insider’s perspective on the issues that affect the way technology is perceived and used in a particular country or region. The interplay between an insider’s and an outsider’s research perspective is an absolute requirement for quality research in any culturally sensitive area. So, the disqualification of the Brazilian HCI conference series by official academic evaluation criteria may have a devastating effect, not only on individual professional careers but also on ICT development in this country as a whole. We must change this situation quickly.
The overall goal of research in HCI is to improve the quality of interaction between people and digital technologies, with special emphasis on the opportunities for increasing the social value of those technologies. In emerging economies like Brazil, good HCI solutions can contribute to leveraging social change by helping low-income populations with little education use ICT products and services to transform their lives. Thus, the Grand Challenge launched for the Brazilian HCI community in 2006 still stands. All HCI researchers, especially in countries like Brazil, have an acute awareness of their social responsibility, which imposes a particular research agenda, research pace, and even research method that is not necessarily the same as in other countries with different socioeconomic profiles and challenges.
Our government’s goal to motivate Brazilian researchers to play an increasingly relevant role in the international scientific community is of course highly commendable. However, to use impact factors and citation indices of publications as the most important measure of the quality of research is a serious threat to the very mission of socially responsible researchers. It places the country’s own priorities for social development in second place compared with the global scientific interests that guide editorial policies of international publishers. Although the international scientific community is increasingly sensitive to social issues, the number of citations that a particular article receives should not be the prime instrument for promoting and evaluating high-quality, socially relevant scientific research. Many of us recognize the quality of various country-specific publications and learn something from them, but precisely because they talk about a socioeconomic reality that is not the same as ours, we may not have a reason to cite them when reporting the results of our own research. Not all countries have equivalent problems, goals, concerns, values, and priorities. Therefore, HCI solutions in one place may not necessarily work in another. The problems themselves are likely to be considerably different.
Because HCI research in Brazil is mainly carried out in universities, Brazilian HCI faculty have been left with only a fraction of opportunities to show that they can perform well academically. It is highly probable that HCI researchers are indeed an endangered species in this country. This new reality is contradictory even within the country. The Ministry for Science and Technology, for instance, has fully embraced the Grand Challenges of the Brazilian Computer Society and has been creating many funding opportunities for socially inclusive ICT projects. The question is: Who is going to engage in these projects? Without academic prestige, undergraduate and graduate students are likely to develop little or no interest in HCI. Many already believe that a quick training session on design and evaluation techniques is enough to develop good interfaces with a desirable level of usability and accessibility. However, as every HCI researcher knows, it is not that simple.
We must be able to promote and sustain the constant development of local scientific communities capable of producing knowledge and innovation that is in line with each country’s or region’s context. We must also be able to find efficient and effective mechanisms for assessing the quality of our own research, keeping in mind that gaining international visibility and appreciation is necessary but not more important than fulfilling the social role of research in this country. Should we forget this, our research would be exclusively aligned with global research interests and might help address social demands and opportunities that are not necessarily among the priorities of Brazilian citizens.
As we said, the cultural determination of HCI research is not a Latin American idiosyncrasy. All HCI research carries the trace of cultural values of those who do it and those who benefit from it. International scientific exchange is the premier means to increase the awareness of such cultural determination and share different perspectives on problems and solutions. We believe encouraging the development of strong national HCI communities will unfailingly benefit the global community as a whole. So, we invite our colleagues from other countries to join our efforts in bringing together data and testimonies on the importance of HCI as a strategic research area that respects their countries’ characteristics, needs, and culture.
Simone D.J. Barbosa is an assistant professor in the Informatics Department at PUC-Rio, Brazil. She is an active member of the HCI community, and has helped organize several international conferences, including INTERACT 2007 in Rio de Janeiro. She is on the editorial board for Elsevier’s Interacting with Computers and a member of the IxD&A advisory board.
Clarisse de Souza is a proponent of semiotic engineering. She is one of the founders of the Brazilian HCI community and has substantially contributed to the development of HCI in Latin America. In 2010 she received the ACM SIGDOC Rigo Award for her lifetime contribution to the field.
©2011 ACM 1072-5220/11/0500 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.