Blog@IX

XXV.3 May-June 2018
Page: 6
Digital Citation

My users taught me to read with my ears


Authors:
Mario Romero

back to top 

This blog post started as a response to the column What Are You Reading? in Interactions. While there are a number of books I wanted to discuss, the topic made me reflect on how I read. My reading mechanisms have evolved from working with blind people. I continue to read novels mostly with my eyes—but for research papers, I use my ears.

For many years now, I have mainly read thesis drafts. In my role as supervisor and thesis examiner, I read approximately 30 master’s theses and five Ph.D. theses annually. I know what you may be thinking—yes, I actually read and review them. Furthermore, as a paper reviewer, I read approximately six papers every year. At an average of 10,000 words per master’s thesis, 30,000 per Ph.D. thesis, and 7,000 per paper, that is about half a million words, or a paperback novel of roughly 1,700 pages.

Now, mind you, I am a slow reader, even when the text is clear. Most of the theses I read are drafts that typically require several rounds of careful reading and editing. If the text is clear, I can read about 250 words per minute (wpm). At that rate, it takes me about 70 hours to do one pass through my annual review material.

My goal here is to share my experience in switching from reading with my eyes to reading with my ears, and from editing with my fingers to editing with my voice. This switch has meant not just a significant boost to my throughput, but also an improvement in my focus and comprehension; it has allowed me to work in many more contexts. Paradoxically, perhaps the greatest benefit for me has been the ability to stay focused on the reading material while performing other physical activities, such as walking home from work. This blog post outlines the methods that I have developed over the years and weighs their pros and cons.

I begin with a short story. A few years ago, I ran a stop sign at the Georgia Tech campus—a policeman stopped me and wrote me a summons. In court, I had the option to pay a $100 fine or perform community service. As a graduate student, I chose service. I signed up to volunteer at the Center for the Visually Impaired in Atlanta. During my week of service, I observed two blind teenagers in class not paying attention to their teacher. Rather, they were having fun with a mysterious device. It had a shoulder strap, and it rested on the girl’s hip. She pressed combinations of six buttons and ran her index finger across a stripe at the bottom. She giggled and shared the experience with her friend, who also giggled as he ran his fingers across the device. I was bedazzled. What was producing this strong emotional response?

I discovered they were using a portable braille notetaker, which includes a six-key chorded keyboard and a refreshable braille display, the stripe they were reading with their fingertips. After that encounter, my colleagues and I started researching how blind people enter and read text on mobile devices. We discovered that the device the teens were using was called a BrailleNote. It cost a hefty $6,000.

We also determined that smartphones were, ironically, a much cheaper alternative. Although smartphones do not have braille displays, they can use voice synthesis to read out screen content. The user places a finger on the screen and the reader voices the target, an efficient method of reading out information. Unfortunately, inputting text is not as simple. While many users speak to their phones using speech-recognition technology, they cannot do so under many conditions, particularly in public environments where privacy concerns and ambient noise render the task impractical. In our research, we developed a two-hand method for entering text using braille code called BrailleTouch [1]. Apple’s iOS has since incorporated a similar method of text input.

What we did not expect to discover was the extraordinary ability of people with experience using screen readers to listen to text at tremendous speeds. We observed participants using screen readers at speeds so high, we ourselves could not comprehend anything whatsoever. Participants reassured us they understood, and that it was a skill they’d acquired through simple practice—nothing superhuman about it.

That realization sent me down the path of using my hearing for reading. I started slow, first learning to enable the accessibility apps on my iPhone. Then I learned to control the speed. I started at about 50 percent, or 250 wpm. I could understand at that rate, which happened to be my eye-reading rate. I downloaded PDF versions of my students’ theses into my phone and tablet and read using VoiceOver [2]. Unfortunately it did not work as well as I expected. The reader did not flow on its own, stopping over links, paragraphs, and pages. I had to push it along by swiping.

After some research, I discovered an app called vBookz PDF Voice Reader. I uploaded my students’ theses in PDF format and started to speed things along. A few months later, I was reading at 400 wpm. The app also shows a visual marker of where you are on the text, so you can follow it with your eyes. About a year in, I was reading at full speed, 500 wpm with full comprehension. More important, I no longer had to follow with my eyes. I could, for example, walk during reading and see where I was going and remain fully focused on the text. Today there are myriad text-to-speech reading apps and I encourage you to explore them to find the one that best suits you.

ins01.gif A typical portable braille writer and reader, the Freedom Scientific PACmate, which we employed in our user studies. Note the buttons on top allow for chorded input, and the pins at the bottom push up and down for a dynamic “refreshable” braille text output.
ins02.gif A participant in our study using BrailleTouch.

This newfound ability has meant a dramatic change in my reviewing practices. Yet it is not the whole picture. There are more practices I have explored, and there are important limitations as well. I started dictating feedback on my phone. I stopped typing and started using Siri to provide comments and corrections. The upside is that I can continue to stay focused on both reading and writing while remaining physically active. The downside is that I need to verbally state the context of the feedback, as in “paragraph 3, sentence 2.” Nevertheless, I find it liberating and efficient.

What are these other important limitations? First, the text-to-speech reader voices everything: page numbers, URLs, footnotes, image names, citations. It even states “bullet point” for the dot initiating bullet points. Most of these landmarks are distracting, and I now know to disregard them when reading with my eyes. Second, tables and graphs are a nightmare to read with my ears. I have to stop the reader and use my eyes when I reach text with a nonlinear structure. Third, figures are legible only as far as their meta-description allows it, and even in that case it is better to use my eyes. Fourth, and last, I have to use headphones. Otherwise, the echo from the device’s speakers coupled with ambient noise renders high-speed reading impossible for me.

Despite these limitations, which are current research topics in eyes-free reading, I find myself fortunate enough to have recruited participants who had the generosity and sense of pride to share their expertise. I am a better research supervisor for that. Yet, for all the boost in speed and comprehension I get from ear reading, I still enjoy reading novels with my eyes and listening to audiobooks read by human actors at normal speed. My mind’s voice and that of other humans remains my preferred method for pleasure reading. Perhaps in a few years, voice synthesizers will become so human that they pass this version of the Turing test.

back to top  References

1. Southern, C., Clawson, J., Frey, B., Abowd, G., and Romero, M. An evaluation of BrailleTouch: Mobile touchscreen text entry for the visually impaired. Proc. of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 2012, 317–326.

2. https://help.apple.com/voiceover/info/guide/10.12/

back to top  Author

Mario Romero studies immersive visualization environments and how people use them. marior@kth.se

back to top 

Copyright held by author

The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.

Post Comment


@Mark Albin (2012 06 30)

This is a very interesting article about social bots, thanks for sharing.

@Aman Anderson (2012 07 18)

This is great
“So what’s the center of a design? In one sense, it is the designer’s nuanced understanding of the problem or opportunity at hand. The focus of design is problem solving, not self-expression.” - Uday Gajendar, Interaction Designer

@Bill Killam (2012 07 31)

This is a long overdue article.  And I couldn’t agree with it more.  I’m current working on yet another Federal RFP that is asking for us to do work using short cut methods that are likely make it harder to get them quality results, and we can probably propose a cheaper and more data rich approach if they didn’t specify how we had to do the job.  Sad.

@Demosthenes Leonard Zelig (2012 08 12)

Great Article, it is funny to notice that such huge corporations do not even bother to do a market research before releasing products on a new market. However, I guess we are still learning from our mistakes.

@karla.arosemenea@gmail.com (2012 10 24)

Hi everyone, In the Technological University of Panama there is also a movement. There is a 2 years MS in IT with a specializtation in HCI. We are also trying to include HCI as part of our main curricula. This year we started a research with a company interested on incorporating usability in their development. We expect to receive a Fulbright Scholar next year in this area…

Regards,

Karla Arosemena
Professor

@John Michael Sheehan (2012 11 06)

There are thousands of blogs that requires comments on them. What is the intention of blog comments? Sent From Blackberry.

@Junia Anacleto (2012 11 07)

A very shallow and naive view of a much more rich and complex context.
I am still waiting for a fair position paper to be presented.

@Rick Norton (2012 11 17)

Excellent article raising significant issues that are largely overlooked.  The prospect that the collapse of sustainability for a growth/consumption related societal model is inevitable, is a topic I have often wondered about, given the nature of capitalism as we know it today.  Even the “Great Recession” of current times gives me pause to wonder just how long we can keep this economic engine going before we have to face the reality that we are all going to have to learn to “live with less”.  (A quantitative assessment, not necessarily qualitative.)

Keep up the good work.  Hopefully, you will raise awareness of these topics.

@Noah McNeely (2012 11 27)

Very nice article, that raises meaningful questions.  I actually think that the idea of sustainable products and sustainable product development is a bit of a myth.  All products consume energy and other resources in one form or another during their production, use, or re-use.  The key, ultimately is to balance resource consumption with resource production, but we will always need to be producing new resources.  See my blog post on the subject at ( http://productinnovationblog.blogspot.com/2012/11/are-there-sustainable-materials_7159.html )

@ed.h.chi@gmail.com (2012 11 30)

The quote in the article mis-contextualize James Landay ‘s essay. James actually is actively working to break down those stereotypes, but you can’t do that without understanding what the deep problems are.

James’ blog post on this is at
http://dubfuture.blogspot.com/2011/12/china-will-overtake-us-in.html

@Lee Crane (2012 12 03)

This is a topic that is thought provoking and important.  The message explores how humans can escape and survive the world they have jumbled.  So many of the theories and ideas are basic.  Our future may look a lot like the distant past.  And indeed we may be happier for it.

@ 4996484 (2012 12 19)

this is a great article David and Silvia!  I’‘m so excited that you guys wrote this up and are showing everyone the complexities in this space. I hope Interactions features more of this kind of research on China.  Although I agree w/ @landay’s assessment of China’s creativity problem - but he’s working with a very different population than you guys. I think you research is absolutely on point - creative folks are going to hacker spaces like Xinchejian, they aren’t ending up in institutions like Tsinghua!  I explain more here:  http://www.88-bar.com/2012/12/where-are-all-the-creative-chinese-people-hanging-out-in-hacker-spaces-apparently/

@Joe (2013 01 04)

I think that if you study the Elliot Wave Theory it can answer your questions.

@Rafeeque (2013 01 06)

good one

@zhai (2013 01 17)

Enjoyed reading this article. I finally got why Harold wants to call it “the Fitts law”. If enough people write it that way I would never have to correct another submission making the embarrassing mistake of ‘Fitt’s law”.

I did not completely get the following remark though:

        “The Accot and Zhai paper about the Fitts Law [3] has a clever title that illustrates
        the rules on letters, “More than dotting the i’s…”—a bad pun on eyes.”

I came up with the title, but the word “eyes” never came to my mind. We meant that the point-and-click style of UI is like dotting the i’s everywhere—- placing a click on constrained targets as the fundamental action in interaction. Why not using ” Crossing the t’s ”  as an alternative action?  Indeed, we presented models of a new style of UI, which systematically reveals when crossing is superior to clicking,  hence the subtitle of the paper “Foundations for crossing-based interfaces.”

Shumin Zhai

@Mohamadou M. Amar (2013 03 22)

I am a Doctoral student in I/O Psychology with Touro UW and need to access your articles.

@Mohamadou Amar (2013 03 22)

Need access for Doctoral Research

@William Hudson (2013 04 09)

Gilbert overlooks the important issue that the ‘big boys’ largely do not appreciate the need for design all and the problems that real people have with technology. I admit that we’ve had a hard time selling UCD but I am not persuaded by the arguments here to abandon it. Perhaps have a look at my article on a similar subject - User Requirements for the 21st Century - where I take a more pragmatic view of trying to address real users’ needs in the development process. http://bit.ly/agile-ucd

@ 0343665 (2013 04 29)

Fantastic text. I came here by searching for people that quote the Standford study on multitasking. The introduction is fantastic as it builds up an argument that attention has some features that do not change over time.

@Simon Taylor (2013 04 30)

not wanting to do anything so grandiose as building a (technology for) a world parliament, I have in essence been working on the same problems and facing the seven challenges with a project called ‘company.’ [https://gust.com/c/littleelephantltd]

In 2011, working with senior software developers - gratis - although neither the ethical undertaking nor the promise of sweat equity were enough to keep them involved - I established the technical feasibility of ‘company.’
h
In 2012, turning from the ‘voluntary’ ‘principled’ participation model - because the attractions of real paying jobs had lost me my team - I received financial support from the New Zealand government. This part-funded an Intellectual Property Position Review - which government considered a pre-requisite - as commercial due diligence - to investing in an initial build, or beta. The IPPR recommended I do proceed… However, government offers only part-funding and without a team - either technical or commercial - there has been little to no investor interest.

As things stand at present, I have the tools and schematics for a beta build of something which would fit the sort of use imagined here. If you have any interest in helping, please contact me.

Best,
Simon Taylor