Cultural and personal impact

XV.6 November + December 2008
Page: 61
Digital Citation

UNDER DEVELOPMENTAudiophoto narratives for semi-literate communities

David Frohlich, Matt Jones

back to top 

It is widely assumed that the Internet is a global information resource. This is not true. For many people in the poorest parts of the world, the Internet is both technically and psychologically inaccessible through lack of infrastructure, money, and the requisite forms of textual and computer literacy. The StoryBank project has been tackling some of these issues by using the fast-growing infrastructure of mobile telephony to support an alternative form of information sharing in pictures and sound.

Situated in the Indian village of Budikote and inspired by developments in audiophotography and mobile imaging [1, 2], we have been exploring the possibility of semiliterate communities using the camera phone as a new kind of pen and paper for creating and sharing audio-visual stories. The system design has been described in a recent conference paper [3], and we are currently preparing a full write-up of the trial results. Here we want to promote the simple story format arrived at in the research, and point to some of the interaction design challenges of supporting it in this context.

The mobile is undoubtedly a transformative technology for development work. Networking and power-management innovations and large-scale investment mean that even very remote rural locations are getting connected. But a word of caution: One cannot necessarily deploy inbuilt phone interfaces and applications for populations that do not have our exposure to computing or the levels of textual literacy we assume.

Hence, three non-textual applications were written for the Nokia N80 camera phone: StoryCreator, StoryPlayer, and StorySender. This was a considerable challenge, since all existing mobiles employ a menu-based style of interaction with textual labels. In contrast, we used culturally sensitive icons developed with our village population to control simple multimedia and file-handling functions. StoryCreator was used to author short audio-photo narratives, comprising a storyboard of up to six still images synchronised to a voice-over of up to two minutes long (see Figure 3). Users are led through a story-creation process to fill media slots in a template, either image first or sound first. Once the media elements in each stream are recorded, users are prompted to synchronise the streams by replaying the sound clip and tabbing through the images at the time they want them to appear. The only editing supported is to review and delete media elements or their synchronisation.

Despite the creative limitations of this design and a very slow response time on some of the actions, rural Indian users were able to use it in a one-month trial to record a variety of story content with minimal training. One hundred and thirty-seven stories were recorded by 79 people, using 10 phones, on topics ranging from agriculture and health to education, self-help groups, and entertainment. The average number of images used was 4.5, with a mean voice-over length of 66 seconds. A typical story is shown in Figure 1, with the local Kannada language voice-over translated and transcribed below the picture to which it relates. A young boy describes the challenges of rearing cows in a short agricultural story lasting 1 minute 50 seconds; this plays back full-screen like a PowerPoint slideshow with spoken narration. A range of creative effects were demonstrated across the corpus, including the use of song during activities, the unfolding of procedures, the sequencing of landmarks in route directions, the illustration of different products, benefits or problems with individual images, and the demonstration of yoga positions.

In some ways, this format can be seen as a simpler (non-textual) form of multimedia message than those currently in the marketplace or proposed as future extensions [4]. However, we prefer to see the audio-photo narrative as kind of stand-alone digital story [5], capable of interpretation by a wider audience than an MMS and providing a different structure and aesthetic than a mobile video clip. This is because it builds on a more accessible practice of talking and pointing to things and can be shared more widely and cheaply by uploading to a community repository (or "StoryBank"). Hence, in the project we avoided the cost and bandwidth limitations of MMS by supporting Bluetooth P2P sharing between phones and Bluetooth/cable connectivity to a digital library repository in the village ICT Centre.

Stories could be played through a direct manipulation interface to a changing story collage presented on a touch screen display [3]. This was a second design challenge because stories had to be accessible without recourse to the usual text annotation and search facilities found in other multimedia repositories such as YouTube. So we made use of a combination of other techniques, including ambient recommendations, unique story numbers, and filtering by topic and phone icons. In ambient mode, the display presented about 10 initial index photographs corresponding to 10 stories, any one of which could be played full screen with a touch. These photos grew and shrank in size while drifting around the screen and were systematically replaced by others from the total set. When playing full screen, each story displayed a unique number that could be memorized or noted for direct re-entry later on via an onscreen keyboard. The collage itself could also be filtered by pressing combinations of phone and topic icons down each side of the screen. Phone icons referred to one of 10 phones on which stories were recorded, while topic icons referred to one of nine topics by which stories were classified at the end of the creation process. These were developed collaboratively with villagers and reflected their "best guess" as to what information would be most useful to share on the system. Topics include student, entertainment, farming, health, legal, news, Panchayat (local council), self-help groups, and education.

The potential value of audio-photo narratives for information sharing in this context is suggested by an analysis of trial story content and preference. Stories were recorded across the full range of designated topics, confirming the initial categorization. No one category of story was significantly favored over others because people tended to watch stories related to their own interest or profession, leading to a spread of preferences. However, a small number of functionally similar forms of story dominated the corpus, indicating particular value to the community. These included advertisements for local produce and handicrafts, farming and business problems or processes (such as that shown in Figure 1), and community news or advice.

While the first two types of story are related to economic "development" issues, the third is not, relating more to a form of personal and cultural expression. Such content included mythical tales, moral stories, festival recordings, advice to children and the community, and the occasional song such as that translated in Figure 2. This was recorded by a teacher in a children's dance practice session.

In many ways the content of these stories was similar to that of news items on a community radio station run in Budikote village called Namma Dwanhi ("Our Voices"). In contrast to the Internet, Namma Dwanhi is a popular and effective way of sharing information in the area [6]. However, in contrast to radio broadcasts, the audio-photo narratives of the StoryBank system are shorter, illustrated, and easier to create by a broader section of the community, including children. They are also accessible at any time from the community display and open to new forms of mobile circulation and distribution between people and places.

From a development perspective we have begun to see this medium as an extension and complement to community radio, rather than as a new form of Internet access as we expected. A future challenge is to bring these two perspectives together by reintroducing wide area or even global communications into our architecture and considering how spoken narrative content can transfer outside the language speaking area in which it was developed. We believe this will involve the kind of mobile and situated device ecology used in StoryBank, with new connections to paper-based information such as booklets, magazines, and posters. The mobile phone is critical to this ecology because it forms the bridge between large distributed information repositories and local people, places, and things. It can also serve as a new kind of multimedia pen and paper as we have shown.

Because of price sensitivity and the community orientation of life in developing communities, phones and other technologies will continue to be shared resources rather than personal ones for some time to come. So another challenge for Western designers is to shift from a user-centered design approach to what we have called a "community-centered design" approach, involving different elements of a community in the design of shared technology for community benefit. Looking over the shoulders of the crowd of villagers using the StoryBank repository for the first time, we began to see the great potential of this approach for the rural Indian context in which we were located. In a culture founded on extended family and community living, this very public interface, with its noise and color, seemed to be an altogether more fitting form of Internet than we usually think of in the West, and a perhaps a lesson in how far we still have to go in making it more accessible and culturally appropriate in other parts of the world.

back to top  Acknowledgements

This work was supported by a grant from the Engineering and Physical Research Council in the UK. Thanks to our NGO partners Voices and Myrada for facilitating our contact with the people of Budikote village, and to colleagues on the StoryBank project, including Eran Edirisinge, Dhamike Wickramanayaka, Ram Bhat, Maxine Frank, Dorothy Rachovides, Will Harwood, Mounia Lalmas, Roger Tucker, Paul Palmer, Arthur Williams, and Kiriaki Riga.

back to top  References

1. Frohlich D.M. Audiophotography: Bringing photos to life with sounds. Springer Verlag, 2004.

2. Sarvas R., M. Viikari M., J. Pesonen, and H. Nevanlinna "MobShare: Controlled and immediate sharing of mobile images." Proceedings of Multimedia 2004, 724–731. New York: ACM Press, 2004.

3. Jones M, W. Harwood, D. Bainbridge, G. Buchanan, D. M. Frohlich, D. Rachovides, M. Frank, and M. Lalmas. "Narrowcast yourself: Designing for community storytelling in a rural Indian context." Working Paper. DIS 2008, Capetown, South Africa, 2008.

4. Jokela T., J. T. Lehikoinen, and H. Korhonen. "Mobile multimedia presentation editor: Enabling creation of audio-visual stories on mobile devices." Proceedings of CHI 2008, 63–72. New York: ACM Press, 2008.

5. Lambert J. Digital Storytelling. Capturing lives: Creating communities. Digital Diner Press, 2003.

6. Pringle I. and S. Subramanian, eds. "Experiences in ICT Innovations for Poverty Reduction." UNESCO Report, 2004.

back to top  Authors

David Frohlich is the director of Digital World Research Centre and a professor of interaction design at the University of Surrey. He worked at HP Labs for 14 years before joining Digital World in January 2005 to establish a new research agenda on user-centred innovation for the consumer market. As part of this work, he was principal investigator on the StoryBank project. David also teaches a new M. Sc. module on interaction design for the department of computing.

Matt Jones is a reader in computer science at Swansea University where he is helping to set up the Future Interaction Technology Lab. He has worked on mobile interaction issues for the past 13 years and has published a large number of articles in this area including Mobile Interaction Design (Wiley & Sons, 2006) with Gary Marsden. He has had many interactions and collaborations with leading industry partners and is currently a visiting fellow at Nokia Research, Finland.

back to top  Footnotes


back to top  Figures

F1Figure 1. Example farming story of up to six photos with the corresponding voice-over translated and transcribed (#1268).

F2Figure 2. A song sung during a children's dance practice (#1229).

F3Figure 3. Cameraphone interface to story-creation application

back to top 

©2008 ACM  1072-5220/08/1100  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.

Post Comment

No Comments Found