Computers and various communication devices are becoming an integral part of daily life for many Chinese. The number of PC, Internet, and mobile phone users has grown significantly in China in the past 10 years. In contrast to this explosive increase, human-computer interaction (HCI) research and practice in China, the goal of which is to bring the real power of computing devices to end users, is in the early stages of growth. This article will briefly summarize research on and practice of HCI in China in the areas of interface technology, multimodal user interface, and human performance for HCI and usability. Future opportunities for human-computer interaction research and practice will also be discussed.
Research on and practice of human-computer interaction in China has a relatively short history, compared with its history in the United States and Europe. People in China only recently began to recognize the significant contribution of HCI research to the spectacular success of the whole PC industry and the Internet. I expect HCI to make a major contribution to the future development of next-generation information technology in China, as it has elsewhere.
China is becoming the biggest market for PCs, laptops, and mobile phones. About 10 million PCs are shipped every year. Although China is by far the largest market in Asia and already ranks number two in the world, we still can imagine the huge growth potential given the country’s 1.3 billion people. Part-way into 2002, the number of mobile phone users was 190 million, which is an increase of about 45 million since the previous year. It was predicted that this number would increase an additional 10 million, to a total of 200 million by the end of 2002.
The number of Chinese Internet users has also been increasing rapidly. The most recent report on the development of the Internet in China, prepared by CNNIC (China Internet Network Information Center, 2002) shows that the number of Internet users has grown to 45.8 million, up from about half a million in 1997. The number of computers having access to the Internet is about 16.1 million, and we can expect additional significant increases in the future. HCI research in China is also growing, although it has not kept pace with the tremendous increase in the use of computing devices.
HCI research is usually interdisciplinary and is diverse in nature. Following are four areas of HCI research in China: interface technology, multimodal user interfaces, human performance for HCI, and usability.
1. Interface Technology
In the last 10 years, Chinese handwriting and speech recognition, text-to-speech (TTS), and Chinese input methods using the QWERTY keyboard have been perceived as attractive user interface (UI) technologies. Much significant progress has been made in these areas. However, it is also evident that, although Chinese input is not an absolute obstacle to the use of personal computers, Chinese handwriting and speech recognition have not proved to be the easiest input technology for Chinese.
Handwriting and Speech Recognition
Chinese handwriting recognition is certainly a successful area in both research and applications. The Institute of Automation, part of the Chinese Academy of Science (CAS), developed the first Chinese handwriting recognition system in 1985. Through some 20 years of research and development, a current state-of-the-art system from HanWang (a company started by researchers from CAS) can recognize more than 27,000 Chinese characters with an accuracy of 99.8 percent for noncursive writing at a speed of 12 Chinese characters per second. The company also provides an embedded version for palm-size devices. It was estimated that there are more than three million users using handwriting recognition systems in China. Although impressive, this number is very small compared with the number of PCs in China.
Continuous Chinese speech recognition is another important research area in China. Microsoft Research Asia and IBM have played a critical role in this area. IBM’s Via-Voice was the only voice-recognition system for PC software in China until Microsoft Research Asia developed its own system for a Chinese version of Office XP. After 15 years of research, the Institute of Automation announced its first voice-recognition system targeted mainly for voice portal, information service, and embedded systems.
Microsoft Research Asia and a lab at the University of Science and Technology of China have developed new TTS systems. One of the interesting features of these systems is mixed Chinese and English capability. The wide use of mobile computing devices in China creates a great opportunity for both telephony speech recognition systems and TTS technology for speech dialogue systems.
Chinese Input Systems
The biggest challenge when people first attempted to use computers to process Chinese more than 20 years ago was to find an optimal coding system to represent thousands of Chinese characters. Today, people use the standard QWERTY keyboard to input Chinese characters instead of having a special keyboard. Each Chinese character must be mapped to a series of English letters on a standard QWERTY keyboard. Unique combinations of letters determine each Chinese character to be input. More than a thousand variants of this basic input system were invented in the last 20 years and fewer than 10 are still used. Two fundamentally different approaches are to code Chinese characters by pronunciation or stroke structure. The pronunciation-based (PinYing) system is widely used, and the stroke-structure-based system is usually aimed at professional typists. Using these systems, a champion typist can type more than 120 Chinese characters per minute.
Because Chinese input is no longer an obstacle for Chinese information processing to the degree it used to be, people are raising their expectations about its ease of learning and use. Unfortunately, only a few usability studies on Chinese input system have been prepared on and reported for these thousands of input systems.
Chinese input systems for mobile devices are becoming another urgent research area. Unlike input systems for the PC, mainstream input methods for mobile devices were not originally developed in China. Because of the popularity of short message service (SMS)about 30 billion short messages in the first six months of 2002applying easy and rapid input methods using small buttons is still an interesting area for research and development.
Natural User Interface
Besides the traditional handwriting and speech recognition technology, enabling technology for next-generation natural user interfaces is also a critical part of overall interface technology research. Microsoft Research Asia has conducted broad research on pen computing and digital ink. Some of the work on digital ink by Microsoft Research Asia has already been incorporated into the Microsoft Tablet PC to support free-form note taking. Human interface-based vision technologies, such as gesture recognition, face recognition, lip reading, and sign language, are also being explored in China.
The wide use of mobile computing devices in China creates a great opportunity for both telephony speech recognition systems and TTS technology for speech dialogue systems.
Virtual reality was considered one of the new HCI paradigms for the next-generation user interface. Research in this area has focused on 3-D interaction in a virtual environment, information visualization, and system tools to build distributed virtual environments. Research on wearable computers, ubiquitous computing, and computer-supported cooperative work (CSCW) are also active research areas in China.
2. Multimodal User Interfaces
Developing multimodal user interfaces attracts many researchers in China. Research in this area has tried to integrate various interface technologies to define a new paradigm of user interface for different applications. UI design tools and formal descriptions for multimodal user interfaces are also being developed and explored.
"Multimodal User Interface and Its Application" was a key project on HCI funded by the National Science Foundation of China (NSFC) in 1995. It was the first and, at the time, the biggest HCI project in China. This interdisciplinary project involved both computer scientists and psychologists and had the goal of understanding the paradigm, interaction design, implementation, evaluation, and applications of multimodal user interfaces. One indirect but critical impact of this project was that the importance of HCI was well recognized and the value of including both technological and cognitive scientists was understood. As recognition of the importance of HCI has grown, additional HCI research projects have been funded. In 1996, the first International Conference on Multimodal Interfaces was held in Beijing and became a regular conference until it merged with the Perceptual User Interface Conference in 2002.
3. Human Performance for HCI
Ergonomics and human factors research has a long tradition of studying human performance in human-machine systems. Human factors research in China can be traced back more than 40 years. It was, therefore, quite natural to investigate human performance in human-computer systems when HCI was first discussed in China more than 15 years ago. This research attempted to understand the effects of computer hardware and software design on human performance, such as design of the visual display terminal, menu interface design, Web navigation, and effective browsing. Subjective evaluation, task completion time, physiological measurement, eye movement patterns and other measurements were used in these studies. Some experiments were also performed to understand the underlying processing of Chinese typing.
The importance of usability is now being recognized in China.
Unlike previous areas we discussed, very little serious work has been carried out about usability because of the low awareness of usability in China. When I first helped to do usability tests for Motorola and Symantec about five years ago, most companies in China did not recognize usability as a valuable aspect of products. Implementing functionality was the only focus of the product development at that time.
The importance of usability is being recognized today. Microsoft Research Asia (MSR) established its first usability lab in China more than two years ago. At about the same time, Siemens also opened its first user study lab in China. Whereas the usability lab at MSR is mainly for research purposes, the lab at Siemens provides usability services for the products of both Siemens and other companies. More usability labs in both local industry labs and universities have been established since then.
HCI Practice and Funding
To better understand the state of HCI research, it is necessary to understand the research infrastructure for HCI in China, which is different from that in the United States and much of Europe. One key difference is that HCI research in China is largely done in universities and by the Chinese Academy of Science, not in local corporate labs. This has been changing recently as foreign corporations like Microsoft, Intel, and IBM have opened research labs in China.
The National Laboratory of Human Factors at Hangzhou University (now Zhejiang University) started interdisciplinary HCI research on human performance, user interface design guidelines, and multimodal user interfaces as early as in 1992. This research finally evolved into a joint project with Beijing University and the Institute of Software of the CAS, funded by the NSFC. The lab is currently the only HCI lab hosted by a psychology department in China.
The Human-Computer Interaction and Intelligence Engineering Laboratory at the CAS Institute of Software developed from a software engineering background. Founded in 1999, it focuses on multimodal integration, cognitive models and distributed cognition, context-aware interface, pen interface, and sketch understanding. The Advanced Human-Computer Communication Technology Laboratory is another research institute at CAS that conducts research on multimedia and intelligent human-computer interaction technology, including biometrics, face detection, and gesture recognition.
The Institute of Human-Computer Interaction and Media Integration at Tsinghua University focuses on distributed multimedia and communications, human-computer speech interaction, and human-face recognition and synthesis. Gavriel Salvendy, who currently chairs the Industrial Engineering Department at Tsinghua, has been building up a human factors program there since October 2001. Usability and HCI are two major research interests for the new program. The department is also launching a new cooperative plan with the recently created usability center of Legend Research Institute, Legend Group Limited. Legend is the largest PC manufacturer in China.
Long-term research on human-computer interaction has been listed as an objective for various national research projects. Funding for human-computer interaction is provided mainly by the National Science Foundation of China, National 863 high-tech project, and National 973 fundamental research project. The National Science Foundation of China has wider coverage of the research areas in human-computer interaction.
As mentioned in the previous section, China recently attracted many foreign companies to open research and development centers because of its huge talent pool and market potential. Research is now being conducted in China by Microsoft, IBM, Motorola, Intel, Siemens, and Nokia. Following is a brief overview of some work going on at these organizations.
- Microsoft Research Asia’s research on next-generation user interfaces is one of the four main research directions at Microsoft Research Asia, which was established in Beijing in 1999. Its research covers natural user interfaces for pen computing, digital ink, human-computer interaction models, usability, speech recognition and interface, TTS, and natural-language processing.
- IBM China Research Center focuses on research in alternative Chinese input methods for mobile phone or other small devices and Chinese handwriting recognition for both PCs and embedded systems.
- Motorola China Research Center (MCRC), established in Shanghai in 2000, conducts human-interface research to support Motorola’s mainstream products in the future China market. Its research topics cover human-machine interface technologies, such as embedded Chinese speech recognition, speech generation, Chinese natural-language processing, and Chinese character recognition and multimodal communication over China’s third-generation (3G) wireless systems.
- Intel opened the Intel China Research Center as an information technology research and development center in Beijing in 1998. The center focuses on applied research in Internet, speech, and intelligent human-computer interaction, including speech signal processing and recognition, speech synthesis, handwriting and natural-language processing.
- Siemens and Nokia also have labs to focus on usability test for their products in China marketplace. These foreign corporate research and development labs have been influencing HCI research and practice in China.
Besides research at universities and industry labs, HCI-related standardization is another driving force for funding HCI practice in China. The China National Institute of Standardization is a government organization responsible for establishing a standards system similar to that of the International Standards Organization (ISO) that includes the standards for human-computer interaction. As part of its recent work, the institute is working on a national standard on human-centered design processes for interactive systems, which is the equivalent of ISO 13407. We anticipate that additional similar standards will be adopted in the future.
Challenges and Opportunities
Rapid growth in the application of computers and use of mobile devices provides more space for HCI research and practice in China. The HCI community in China is facing both challenges and opportunities in the future.
HCI Professional Community
The research community of HCI professionals is still very small in China. There is, as yet, no national HCI professional organization to organize professional activities and no annual conference to exchange research findings and share the vision. As the importance of HCI is becoming more widely recognized, we anticipate having a large and strong HCI professional community in China. We hope that ACM SIGCHI can help with the development of this community.
Impact on Product Development
User interface design is still not an integral part of software development in China. Research at universities has no direct impact on software and product development yet. The UIs of all major office productivity software packages developed in China are still quite similar to those of Microsoft Office. The publisher of a leading local office suite touted, as one of its key features, that UI for the suite is 100 percent identical to that of Microsoft Office. With more research advances in HCI, it is inevitable that state-of-the-art research results will be incorporated into new products.
Education and Training Program
University education is essential for capitalizing on the enormous potential of HCI in the future. Currently almost no interdisciplinary education programs for HCI exist in China and no HCI textbook is available for university students. HCI is still not a required course in most top computer science departments in China. Beijing University has been the first top computer science department to teach a human-computer interaction course for undergraduate students. Most of the top 10 universities do at least offer an HCI course for their graduate program. Further development of advanced training programs for HCI practitioners is also very necessary.
People are becoming increasingly interested in not only the "computer" side of HCI but also in the "human" side of HCI, including usability.
Job Opportunities for Usability Engineers
Today, almost no job positions for usability engineers in the software industry exist in China, except in some foreign companies. Unlike many American and European companies, leading software companies in China usually do not have a professional usability test group. Providing more job opportunities for usability engineers will not only help improve the quality of software and products, but will also facilitate the growth of HCI in general in China.
Short- and Long-Term HCI Research
Although applied HCI research in China is increasing, much less fundamental or basic research is being conducted in China than in United States and Europe. Applied empirical studies on existing HCI systems do not help to solve the HCI problems specific to China. A balanced vision of both short- and long-term HCI research is critical in order for us to advance in solving known problems and identify promising research areas.
China is becoming the largest marketplace for mobile communication and computing devices. This is taking HCI research to a new level and pushing HCI beyond the desktop and into our pockets, couches, and streets. People are becoming increasingly interested in not only the "computer" side of HCI but also in the "human" side of HCI, including usability. Given the number of users, China must do more state-of-the-art research, in order to significantly affect future products and begin to play a leading role in some areas of HCI.
I am highly optimistic about the future of HCI in China. In the last three months of 2002, I had the opportunity to participate personally in a number of events that seem to support this optimism:
- The APCHI 2002 conference was held in Beijing for the first time. It was a very successful conference, at which I was honored to deliver the keynote speech.
- I reviewed a national standard for Human-Centered Design Process for Interactive Systems. Additional standards relevant to HCI are being drafted.
- A special session on HCI research in China at HCI International 2003 is being organized, at which I have been invited to give a presentation.
- The writing of this article for interactions about HCI research in China.
I believe these are not isolated events. I feel that great opportunities for HCI research and practice are coming, and I am excited about the opportunity to personally play a role in this bright future.
Dr. Jian Wang is Research Manager of Multi-modal User
Interface group at Microsoft Research Asia. His research
interests have been in the areas of ink and pen computing,
usability, multi-modal user interface, virtual reality and human
Before joining Microsoft in 1999, Dr. Jian Wang was a position of professor and Chair of the Psychology Department as well as the director of the State Lab of Human Factors at Zhejiang University . He received his Ph.D. (1990) in engineering psychology from Department of Psychology, Hanghzhou University.
©2003 ACM 1072-5220/03/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2003 ACM, Inc.