According to Christoph Bartneck's 2009 scientometric analysis of CHI conference proceedings , Asian researchers' activity at the CHI conference was very small at that point; between 1982 and 2008, only 3.61 percent of papers in the main conference proceedings were from Asian countries, and Japan was the only country listed in the top 10 most active countries. Six years have passed since Bartneck's report. During that time, 2,214 papers have been published in the CHI proceedings. This is larger than the total number of papers that Bartneck analyzed (only 1,945 papers were published in CHI proceedings from 1982 to 2008, according to the ACM Digital Library).
This surprisingly rapid growth of the CHI community has had a significant effect on the proceedings' statistics over the past six years. Here, we aim to share insights from an analysis of the CHI proceedings from 2009 to 2014. Bartneck's analysis was more broadly focused on both quantity and quality, including numbers of citations and best paper awards. Due to limited space and resources, we focus only on the quantitative analysis, including geography, organization, and author statistics.
Generally, our analysis follows Bartneck's method. We employ the idea of credit for the analysis, in which one paper equals one credit. Each author receives an equal share of the credit. For example, for a paper written by four authors, each coauthor receives 0.25 credits . This method is also effective at determining how much a country or an organization contributed to a paper. We analyzed 2,214 papers, so the total amount of credits was 2,214 for each category. We compiled author statistics by using another method; all authors of a paper receive one credit because the analysis is aimed to determine how many papers were authored or coauthored by each author.
Our analysis focused on analyzing main conference proceedings—not extended abstracts. Main conference proceedings consist of papers that have been submitted to the "Papers and Notes" category, which has the most competitive reviewing process and represents the latest important findings in HCI. All data was taken from the ACM Digital Library (ACM DL). For this analysis, we wrote a PHP data-scraping script to retrieve data for the analysis of table-of-contents data in the ACM DL. Data for the analysis included the paper title, author name and affiliation, and the number of coauthors (country name is included in the author affiliation). Data was saved as an XML file. When we retrieved the data, 334 author affiliations were missing, so we manually fixed them. Likewise, if the organization name in the author affiliation was inconsistent (e.g., many authors used double affiliations, making the organization names inconsistent), we also manually fixed these entries. All data processing was done with PHP scripts.
There is limited space here to present the results of our analysis. Details, including the complete list of countries, organizations, and authors, can be found at: http://daisukesakamoto.jp/articles/Asian-Researchers-at-the-CHI-Conference/.
Geography. In general, we determined that 44 countries from all over the world contributed to the CHI conference from 2009 to 2014. The overview distribution is shown in Figure 1. When Bartneck analyzed the CHI proceedings from 1982 to 2008, the U.S. received 67.34 percent of all credits in the main conference proceedings, and the top four countries (the U.S., the U.K., Canada, and Japan) accumulated 88.16 percent of all credits. In our analysis, the U.S. received 49.22 percent of all credits, and the top countries were reshuffled. The top five countries were the U.S., the U.K., Canada, Germany, and France. These five countries received 79.3 percent of all credits. Among the Asian countries, Japan was the highest ranking at sixth (Table 1). Looking at the other Asian countries, South Korea was in eighth place, China was in 12th place, and Singapore was in 13th place (Figure 2). Finally, Asian countries received 183.93 (8.35 percent) of all 2,214 credits, which is an almost 5 percent increase from the previous analysis. This suggests that Asian countries are becoming more active in the CHI community.
Organizations. Next, we analyzed author affiliations. This analysis covered only Asian countries. Our results show that 106 organizations contributed to the CHI conference. The Korean Advanced Institute of Science and Technology (KAIST) in South Korea was the most productive organization in Asia (Table 2). Even though the other South Korean organizations were not in the top 10 list, the 25.43 credits from South Korean organizations indicate that South Korea is a rapidly growing country in the field of HCI research.
The National University of Singapore was in second place, and Nanyang Technological University was in seventh. These two universities received 89.68 percent of Singapore's credits, contributing highly to Singapore's move up in the geographical analysis list (Table 1). Likewise, National Taiwan University received 75.5 percent of Taiwan's credits. Just five Taiwanese organizations contributed to CHI, but this time Taiwan's rank rose from 34 to 17. Interestingly, four Japanese organizations were in the top 10 list, and 10 organizations were in the top 20 list, but its final rank in the geographical analysis dropped from four to six. Actually, Japan is the highest-ranking Asian country. The largest number of organizations came from Japan (34 organizations; Table 3), but the small contributions from each one might have affected the geographical result.
One thing we need to be concerned with is that many authors have double affiliations. For example, in the case of the "JST ERATO Igarashi Design Interface Project" (ranked 10 in Table 2), usually the affiliation would include the author's university (e.g., the University of Tokyo) and this organization. In this case, the same amount of credit goes to both organizations so that the total amount of credits is larger than in the geographical analysis. This is a worldwide trend, not a characteristic of Asian countries specifically. In summary, organizations in Asia received 194.1 credits in total, and the top 10 organizations received 109.64 credits (56.48 percent). The average credit was 1.83 (SD = 3.54).
Authors. A large number of authors from Asia contributed to the CHI conference. In fact, 518 unique authors whose affiliation contained an Asian country's name were found in the proceedings. The average number of publications per author was 1.50, and the median was 1; 388 authors (74.9 percent) published just one paper in six years. Table 4 shows the list of authors who published at least six papers in the six-year period. Further, we determined that 50 authors (9.65 percent) published at least three papers. These results indicate that the active organizations employ productive researchers. However, by considering this data along with the organization results, it shows there is much room for growth. Almost half the credits go to organizations that are ranked below 11. This indicates that there are organizations with the potential to grow in the future. Our analysis targeted a narrow range of years, and we are in the process of rapid growth, so some authors are not included in this analysis. An appropriate method for author analysis might be necessary to understand the stats more deeply.
Limitations. These days, it is common for researchers to leave their home country and work in other countries and on other continents. One researcher may get a tenured position in another country, while another researcher has an internship in a foreign country but will return to his or her home country when finished. The situation of each individual researcher is different, and it is impossible to know all of them from the data we had from the proceedings. A major limitation of this analysis is that it was done with a limited data source. We aimed to understand the research activity of Asian researchers at the CHI conference; however, we can say for sure that this is not a comprehensive analysis.
Another concern is our lack of confidence in the handling of double affiliations. The problem is that the credit should go to the affiliation where the research had primarily been conducted. However, our limited resource for analysis does not allow us to do this. It may be possible to identify where the research was done by carefully reading the papers, but this is unrealistic. We need more collaborators or better methods to conduct this kind of meta research.
Finally, we used a quick-and-dirty method for the analysis. Some data was fixed manually, and some part of the data processing was done manually (copying data to a spreadsheet, etc.), so this analysis may contain errors. Let us know if you find any problems. We will revise the appendix Web page according to your error report.
In this article, we presented a brief analysis of Asian researchers at the CHI conference. Asia is a rapidly growing region in HCI research, and the results indicate that there is room for more growth. We hope many Asian researchers contribute to the CHI 2015 conference. Likewise, we expect that even more Asian researchers will actively participate the next time the CHI conference is held in Asia.
As noted, we used a quick-and-dirty method for the analysis. Please be advised this analysis may contain errors, so we caution use of these results outside this discussion.
Thanks to Christoph Bartneck for his help.
Daisuke Sakamoto is a project lecturer at the University of Tokyo. His research interests include human-computer interaction, human-robot interaction, user experience, and interaction design.
Copyright held by author. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.