Jennifer Pearson, Simon Robinson
In the past few years, there has been a surge of research focused around the so-called developing world. Much of this work is currently conducted by researchers trained in the West, moving from more privileged environments to a completely new place, in an entirely different contextcommonly known as research in the wild. It's easy, when undergoing such a shift, and despite having read much of the literature, to have many preconceived notions about the type of environment you're going to encounter in this new situation.
Our work has recently followed this same path. As early-career researchers, we have been part of several large projects, and although they have differed significantly in their focus, each, in our view, has tended to follow the same pattern. That is, the overarching aim is to find a gap in existing knowledge; design novel, useful technology to fit the context; evaluate via user studies; and then publish.
A year ago we started working on a project called Scaling the Rural Enterprise (StRE), a U.K.-India collaboration focusing on designing technology and policy developments to help people in rural areas (in both the U.K. and India) scale up their businesses . Our particular focus is mobile interaction design, and we have been developing various technologies in collaboration with researchers on the Indian side of the project, over several extended visits.
StRE has differed greatly from previous projects we have worked on. It's been a big transition, and we've had to significantly adapt the way we usually work to fit the new context. In this forum, we want to share some of our experiences and the key things we've learned so far. These are by no means definitive, but we hope they will help other researchers in similar situations in the future.
There are many obvious challenges for researchers in a new environment, particularly if this involves working in an unfamiliar, underdeveloped part of the world. As we are well aware, many of the issues surrounding research in developing contexts have been widely discussed previously (e.g., [2,3,4]). The most prominent examples in our experience are that we rarely share a common language with local people, and that textual literacy is often low. Other common issues that come with working in developing regions include low disposable income, intermittent power supply, little technology exposure, and low Internet connectivity (both mobile and fixed line).
In addition to these common points, there are many other challenges that are not so immediately apparent. India is strange and bewildering to us! This is a cliché, of course, and it is important to avoid the dangers of being seduced by the strangeness of new and different places . But it is also hugely important to appreciate the cultural and behavioral differences of people from environments so drastically different from our own. It's easy to underestimate how much time and effort it takes to begin living and working in such unfamiliar surroundings.
Rather than the obvious issues, then, here we will focus on our own experience conducting research in this different environment. Through living and working in India, we've had to radically alter our research practices (and day-to-day lifestyle!). We are used to developing high-end prototypes for use by technology-savvy, literate, and comparatively wealthy users. In this new environment, where we cannot rely on these assumptions, our usual approach is bound to fail.
We are traditionally taught to build something based on gaps in knowledge, problems with current systems, or feedback from probes; however, all of these approaches are predicated on people's knowledge of existing infrastructure, or their ability to use a particular technology. In new devices, common interaction paradigms, such as swipe to scroll or pinch to zoom, are often accepted as standard and put into interfaces on the assumption that people will intuitively understand their usage. After all, we all have experience with existing models, so an incremental change that uses standard functions for another purpose is relatively easy to understand. However, this is often not the case in the situations we have been encountering in India. We can't just design something based on our, or even the users', perception of "easy to use" and expect it to be fit for purpose .
Many of the people we've worked with have experience with only low-end feature phones, and often use just a single feature of the device (e.g., calling, via memorization of numbers rather than using the device's phonebook). As a result, they would be lost trying to use some of the state-of-the-art devices or applications available today. As an example, think back to your first experience of using a touchscreen, and the learning curve required to get used to tapping gently rather than pressing physical buttons. This, coupled with the additional complexity of functions we might see as standard, means that completely new designs would quickly be discarded, repurposed, or used only for their basic functionality .
In addition to the problems with user experience, there are further challenges when evaluating. We are used to carefully planning and running user studies, with clear measures derived from qualitative questioning of participants. However, this strategy is not a viable option in our current work, as people we have worked with are highly respectful and reluctant to give poor feedback. In fact, we've found it almost impossible to get negative feedbackor even constructive criticismfrom users. This is not (as we would like to believe) because our systems are perfect. Far from it! Rather, it is due to the underlying cultural respect community members have for people who are perceived to be educated or senior to themselves, or who are there to "help" them.
Other cultural differences add to the challenges. For example, there is very little urgency in India, as people are so pleasantly laid-back. Bureaucracy is also an issue; filling in forms is a national pastime, which adds to the time required. Transport, navigation, and infrastructure are inconsistent, all of which contribute to making things more challenging than we are used to.
All these points could be read to suggest that HCI for developing regions is a near impossible, and perhaps even futile, task for newcomers. This is a naive and defeatist attitude to take. While there are indeed challenges, they are not by any means insurmountable. There are also several things that make development in these regions extremely worthwhile, offering huge opportunities for genuine, perhaps even life-changing, benefits.
In our particular research area, there are many positive aspects. For example, there is already a large infrastructure of low- and middle-end mobile phones, mobile charging centers, and cell coverage within India as a whole , which makes telephone-based services relatively easy to deploy. There is also a large sense of community and an overwhelming eagerness and willingness to help in the design and implementation of new ideas, particularly if there is potential for future improvement. People we have worked with in rural communities are willing to use early versions (even if they are less than satisfactory prototypes) to better inform later ones. Furthermore, not only are people eager to help, but they usually also have lots of time with which to do sotime isn't money quite as much as we're used to. Also, in contrast to the so-called developed world, people in the communities we've worked with are extremely respectful, and genuinely interested in what we have to say, making it a really enjoyable place to conduct research.
Here are some of the ways in which we have adapted our research methods to living and working in India.
Technology. In contrast to working in the U.K., where the most up-to-date and state-of-the-art devices are quickly adopted by users, the communities we have worked with differ significantly. People have neither the money nor the experience to use high-end mobile devices. Cheap, low-end, dumb, or feature phones (e.g., J2ME/Symbian) are commonplace . Even if people do not own a handset personally, devices are often shared between friends and family members . As a result, we have targeted our work to be usable on the low-end devices that people are already using. For example, we have worked closely with the team behind the Spoken Web , which is a telephone-based information service already in use in several rural Indian communities. We've designed several new interaction methods for this IVR system, but have ensured that they can always be used on existing devices, with no loss of functionality.
Another factor in our evolved design process is related to the cost of usage. For instance, both airtime (for calling and SMS) and mobile data plans (for Internet access) are often prohibitively expensive for users in India . As such, our designs aim to minimize the users' day-to-day running costs of the systems. For example, we've focused on using telephone-based services, which are cheaper than using a data connection, and come with the added benefit that coverage is more widespread.
One of the many design lessons we've had from working in India is the need to test our prototypes in minor increments with small groups of people, and to demonstrate and explain developments. Patience is vital. Due to the lack of technology penetration, teaching communities to use any new system, regardless of its benefits or complexity, can be hugely time consuming and challenging. For example, we know that many people have mobile phones, so assuming that they will be able to dial a number and use the keypad seems natural. This is by no means the case!
One approach that is particularly beneficial is to utilize the help of local "experts," who are generally highly respected within the community. While these experts are often no more proficient in using a particular technology than anyone else in the community, they are renowned locally for their expertise in, for example, farming, or healthcare, and are trusted by others to pass on any newly learned information in a manner that they can understand.
Local researchers. In-situ and user-centered design are well-established paradigms within the HCI community, but, as has been shown, these methods don't quite go far enough [3,4]. We would not be able to work on this project without the close relationship we have had with local researchers (or human access points ). This relationship goes beyond simply translation, relying on local researchers' broader experiences of work in this context and their immersion and trusted relationships with rural underdeveloped communities.
But translation issues remain. Many regions in India have a large number of different language dialects within very small geographical areas. This spread of languages can at times make designing, and consistently evaluating, far more difficult. It can be very hard to translate concepts, particularly with two separate language conversions, plus a discussion to help both sides fully understand both the question and the answer. We've found it beneficial to work primarily in groups, where both demonstrations and questions need less clarification and explanation. In addition, participants who have questions can help each other understand, and individuals reluctant to ask when alone may be more confident. Simple, one-response questions help to avoid them getting lost in translation, and planning in advance for as many answer eventualities as possible helps to avoid misunderstandings during a study. For example, we've tended to design multiple versions of the same question beforehand, and specifically address possible ambiguous responses.
Evaluation. Participatory design is a well-known and trusted method for designing successful systems. However, in our experience this approach can be problematic when people find it difficult to criticize things. This problem can be magnified hugely when people not only don't know what they want, but also don't know the capabilities of the technology being used . It is important to know people's needs, but not necessarily focus on their design input at all times. With this in mind, we have tended to conduct qualitative interviews via local researchers to explore people's core needs, and ensure that our systems are tested throughout each of their design phases. Instead of working with end users directly, we have been working with people who grew up, know, work, and live in this context but are also well educated in computer science and its limitations (that is, local researchers, rather than local experts).
To cope with reluctance to criticize and the tendency to give wholly positive feedback regardless of merit, we have tweaked and refined our approach to qualitative feedback. Rather than asking people to rate a system or to give feedback on one design (which usually results in high scores all around, regardless of actual suitability), we have switched to making several similar but comparable systems and asking people which one is preferred.
Like many people making a first foray into research in developing regions, we have learned why the common patterns of designing, building, and user testing are rarely appropriate. Indeed, it would be naive to think that designing from our own models would succeed in regions where the challenges facing end users are completely different . But reading and gathering advice from other people gets us only so far. So, is it worth it for people who are not from a "developing" country, such as India, to go there to conduct research? Yes, absolutely. From a personal point of view, in the process we've extended our experience and hugely expanded our worldview. From a design angle, we've been able to improve our work with insights from other often-unexpected perspectives, and as a result we're now coming up with better designs.
To conclude: Assume nothing! Our ideas of simplicity and usability can be impossibly complex for people with different experiences to understand. Patience is imperative: It's crucial to demonstrate use, explain concepts, and train users for each small prototype iteration. Designs will be quickly discarded or repurposed if they are not usable , Working with people, particularly local experts and local researchers, to understand problems and limitations is the best way forward.
Mainly, it is hard to plan, and despite best efforts, things will inevitably change along the way. Go with the flow, and let the designs evolve.
8. Telecom Regulatory Authority of India. The Indian telecom services performance indicators: October-December 2011; http://goo.gl/g8xSI
Jennifer Pearson is a researcher in the FIT Lab (fitlab.eu) at Swansea University. Her research interests include active reading improving digital books, and mobile interaction development for developing regions.
Simon Robinson is a researcher in the FIT Lab (fitlab.eu) at Swansea University. His research interests include eyes-off interfaces, sensor-based mobile devices, and providing high-end interactions on low-end devices.
©2013 ACM 1072-5220/13/03 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.