ACM Interactions

Perspectives on handwriting and digital ink in schools.

Going through customers to reach users is a challenge as old as HCI. When computers cost a fortune, acquisition decisions weren’t made by hands-on users. Those responsible believed that they knew what users needed. They were often wrong. Making life worse for designers, the marketers who spoke with customers felt they knew best what users needed and often blocked access to customers and users. Developers were two unreliable jumps from use.

Enterprise settings today haven’t changed much. Marketing and acquisition remain overconfident about their understanding of user needs. However, users now have more options. They can request inexpensive software or customize what is provided. Employees who experience decent consumer software are bolder about communicating their needs. If not listened to, they bring in their own tools for some tasks.

The consumer market has one fewer hurdle—customers are the users. A product organization still deals with distributors who may be overly optimistic about their knowledge of consumers, but this self-corrects—a product sells or it doesn’t. A useful product ignored by distributors may fail, but a poor consumer product isn’t forced on users, as often happens in enterprises. 

Although people, including me, have long praised efforts to get direct feedback from hands-on potential “end-users,” many teams settle for A/B testing or less. In the critical endeavor of educating the world’s billion school-age children, an unusually clear illustration of the challenge has appeared. Resistance to an advantageous change has different sources, including cost, but even where cost is not an issue, a chasm separates customers and users, delaying what seems desirable and probably inevitable.

“The pen is mightier than the keyboard.”

This is the title of an elegant 2014 paper published in Psychological Science by Pamela Mueller of Princeton and Daniel Oppenheimer of UCLA [1]. Three experiments compared the effects of taking lecture notes with a pen or a keyboard. In the first, subsequent memory for factual information was equal, but students taking notes by hand had significantly more recall of conceptual information. Keyboard users had taken more notes, typing large chunks of verbatim lecture text. Those writing by hand couldn’t keep up, so they summarized; this processing appears to aid recollection.

In the second experiment, students were told that verbatim text is not as useful as summaries. Despite this guidance, the results were the same: lower performance and more verbatim text for keyboard users. In the third experiment, students could study their notes prior to testing: Although the typists had more notes, they again did worse.

Sharon Oviatt, author of The Design of Future Educational Interfaces (Routledge, 2013), conducted a wide range of rigorous comparisons in educational settings. She looked at pen and paper, the use of styluses with a tablet, and handwriting with a normal pen and special paper that enables digital recording. The results were dramatic. Students with pen interfaces did significantly better in hypothesis-generation and inferencing tasks. They solved problems better when they had a pen to diagram or to jot down thoughts.

Oviatt and others have observed that digital pens as a keyboard supplement help good students, but somewhat unexpectedly they can dramatically “level the playing field.” Students who think visually, who compensate for shorter memory spans by quickly jotting down notes, or who benefit from rapid trial-and-error, engage more and perform better. How this technology, with others, can save students from “falling through the cracks” could fill another essay.

Mueller and Oviatt discussed their studies in these keynote presentations at WIPTTE 2015, well worth 90 minutes. The authors primarily make a case for handwriting over keyboards for a range of tasks. More surprising at first glance, Oviatt found digital pens outperforming pen and paper. In a Venn diagram task, digital pens led to more sketches and more correct diagrams than pen and paper. Digital ink supports rapid trial and error due to the ease of erasing. Page size, page format (blank, lined, grid), ink colors, and line thicknesses are easily varied, engaging students and supporting task activities. Handwriting recognition enables students to search written and typed notes together. Digital notes are readily shared with collaborators and teachers.

Students can’t use keyboards to write complex algebraic equations (or even practice long division). A keyboard and mouse aren’t great for drawing the layers of a leaf or light going through lenses, placing geographic landmarks on a map, creating detailed historical timelines, or drawing illustrations for a story. SBAC annual state assessments, it was announced, will require digital pens in 2017.

Who would resist including a digital pen with computers for students, the key users of education?

Customers

Last week, a journalist friend mentioned that although he didn’t use a digital pen, his daughter borrows his tablet and uses its pen all the time. So did my daughter before she got her own tablet, which she insisted have a good pen.

I’m never without a traditional pen. I take notes, mark up printed drafts, make sketches, and compile weekly grocery-shopping lists. The journalist takes interview notes with a pencil, which holds up better when paper gets wet. We rarely use our digital pens. I use a digital signature for letters of recommendation, that’s about it.

Why would children but not their parents use digital pens? Well, few adults write as many equations as the average child, but it may be more relevant that unlike students, we rarely share handwritten work with others. We were taught that it’s unacceptable or unprofessional to turn in handwritten work—essays are to be typed up, illustrations recreated with a graphics package. Meeting minutes that are taken by hand are retyped before distribution. A whiteboard might be photographed after a meeting, but the notes are then typed. Colleagues who comment on my drafts may initially write on a paper printout, but they then typically type it into comment fields—additional effort for them, easier to read but more difficult for me to contextualize than ink-in-place would be. We consider handwriting second-class and let it deteriorate. “I can’t even read my own handwriting half the time anymore,” said a colleague.

The customers—superintendents or administrators making the purchasing decisions—don’t use a digital pen. Digital ink enthusiasts tell them that they would be more efficient if they did, but these customers are successful professionals, happy with how they work and not planning to drop everything to buy a new device and learn to use a digital pen. “Are you saying I’m inefficient?” Such exhortations can be counterproductive.

Instead, remind such customers that their needs differ from their users’ needs. K-12 is different from most professions: Handwriting is part of the final product for both students and teachers. Students don’t retype class notes, which include equations and sketches. They don’t resort to professional graphics programs. Teachers mark papers by hand; it is more personal and more efficient. Lecturing to an adult audience, I can count on them to follow my slides, but teachers guide student attention by underlining, circling, and drawing arrows.

These customers are unaware that they don’t fully appreciate the world of the users: teachers and students. A superintendent thinks, “Digital pens are a frill, an expense, they’ll get lost or broken. Students should improve their keyboard skills, which is better professional training anyway.” But students can’t type electron dot diagrams or feudal hierarchy structures.

The future seems clear, but these customers are not always wrong about the present. When a student uses a computer once a week or in one class a day, digital ink has less value. Most class notes will be on paper in a binder or folder, so digital notes will be dispersed unless printed. Students have no personal responsibility for the pen. This changes when a student carries a tablet everywhere: to classes, home, on field trips, to work on the bus to an athletic competition. Two years ago I described forces that were aligning behind 1:1 device:student deployments, which are now spreading in public schools. Several new low-cost tablets come with good digital pens. Prices will drop further if pens come to be considered essential.

Who will win?

Steve Jobs railed against digital pens, but he also opposed color displays until he embraced them. Apple is now patenting digital ink technology, feeding rumors about a better pen than the capacitive finger-on-a-stick iPad stylus. Google education evangelists described handwriting as obsolete, but recently Google announced enhanced handwriting recognition. Microsoft stopped most digital ink work when it embraced touch, but is now strongly committed to improving it.

An adverse trend: A comfortable digital pen is too wide to garage in ultra-thin tablets. On the other hand, vendors may realize that 80% of the world’s population does not use the Roman alphabet and finds keyboard writing very inconvenient. In China, tablets are available with digital pens of higher resolution than can be purchased elsewhere. Cursive writing and calligraphy may not return to fashion, but digital pens are likely to. 1975–2025 may become known as “the typing era,” a strange interlude forced on us by technology limitations.

Acceptance may be slow. 1:1 deployments will not be the norm for a few more years. Aging customers who speak on behalf of middle-aged teachers and young students rarely sit through classes and may not learn new tricks. In an era of tight budgets, many don’t grasp the implications of the downward trajectory in infrastructure and technology costs and the upward trajectory in pedagogical approaches that can take advantage of technology that, at last, has the capability and versatility to help.

The greatest challenge

Students have absorbed the message: Professionalism requires typing. Long essays must be typed so overworked teachers can read them more quickly. A job résumé should look good. No one points out that for rapid exchanges, handwriting is often faster and more effective—and the world is moving to brief, targeted communication.

Oviatt asked students whether they would prefer keyboarding or writing for an exam. They choose the keyboard, even when they get significantly better outcomes with a pen! The customer has gained mind control over the users. Education needs a reeducation program.

In concluding, let’s pull back to see this education example in the larger framework of the conflict between customers and users. Overall, much is improving. Users have more control over purchases and customization. Users have more access to information to guide their choices. They have more ways to express dissatisfaction. Customers, too, have paths to greater understanding of the users on whose behalf they act. They do not always succeed in finding those paths, as we have seen, so vigilance is to be maintained.

Endnote

1. This play on "the pen is mightier than the sword” was independently used almost a decade ago by computer graphics pioneer Andy van Dam, for talks lauding the potential of digital pens or styluses.

Jonathan Grudin