Just as most good research papers start with a problem statement, many of my columns start with a gripe masquerading as an observation.
Today's is ROI, aka return on investment.
I have no gripe with the idea of ROI in itself. Rather, I'm bothered by the fact that some people seem to think considering ROI is outside their scope of thinking, that it is solely the purview of business strategists and analysts. Or, worse, that consideration of ROI is rather vulgar and will squeeze the living creativity out of any research discussion.
Certainly, I agree that an overly singular focus on any topic is likely to make for limited creativity. I also know quite a few people whose rigid focus on ROI makes them tedious collaborators—and poor dinner companions. But I do think the thought experiment of considering the ROI of our work is a useful, creative exercise in telling the story of contribution and impact within the world of technology design and development. As HCI scholars and practitioners, sharing with others the value that HCI as a discipline can and does bring to business is a very interesting contemplation.
So, in this short column, I want to shift the conversation a little and try to convince some naysayers that learning about the ROI potential of your ideas—modeling what is gained for all the effort you have put in—is worthwhile. More importantly, I'd argue that it is worth your time to engage very directly in conversations about ROI. I invite you to think about what is defined and measured as "return," about how we as HCI scholars can understand and narrate "investment," about definitions of success/failure and attribution of credit/blame for those successes/failures, and, crucially, about the role of scope and timeframe in thinking about ROI.
First, let's be critically reflective and address the meaning or meanings of ROI.
ROI typically stands for return on investment. Being a hybrid creature who sits between academia and industry, I have experienced a lack of clarity on what constitutes an investment. I have also met people who seem to mistake the R in ROI to be revenue rather than return. Many UX blog posts on the cost payoffs of usability work link directly to revenue. A great example is the meme that has been around awhile that every $1 invested in UX research and design yields a $2 to $100 return. That's certainly a good story, but it is meaningless. Let's think, though, about new measures and proxy measures that can be artfully, creatively linked back to cash measures if needed. The links are real, not fictional. But they are often not storied. Whereas revenue on investment and simple models of return on investment may seem dull, unimaginative, or misguided, proxy measures and models of impact in terms of effort spent can be interesting, and possibly even fun. Think of the spontaneous expressions of delight or the furrowed brow of concentration you see on gamers' faces when they are truly immersed in a game, for example. Or the sense of achievement someone has when they accomplish a complex task thanks to a tool you developed. These are measures of success—palpable returns on your investment of time and energy. They may seem to be hard to link to business metrics, but that makes for a great challenge. The somewhat nebulous "net promoter score" took a while to be accepted as a measure of business success but it is now firmly rooted in industry discourse .
Similarly, let's think about the word investment. Any attribution model is hard. This is more the case for HCI, writ large, than it is for usability, writ small. If you work primarily on product usability, tracking the impact of an idea you had a day, a week, or maybe even years ago may seem perfectly doable. However, if you work on more ambiguous projects that deal with the innovation end of the spectrum of HCI practice, such tracking is harder. You may not get attribution for the impact of your invested time and energy, in part because it is so hard to tease apart what factors led to success. But don't just give up—this is a great opportunity for building the investment-to-return connections and sharing the stories of those connections clearly.
HCI is not solely about interface or interaction design. And not all of HCI is about changing behavior. Much of HCI is about changing thinking, shifting mindsets, and introducing new ways of thinking. It is also about shifting ways of measuring and evaluating. An opportunity for HCI storytelling emerges when you partner with other people whose role it is to design, develop, and launch a technology experience or service and discuss what metrics they use to demonstrate their impact. Talk to experts in engineering, marketing, data science, and business analytics. Learn from them but also think about how tools you design and develop can transform their practice. Easy examples include innovations that reduce support costs and those that increase user trust, and therefore loyalty and advocacy. Innovations like these show how HCI principles can make teams function more effectively, make training courses more fun, reduce stress, and increase happiness—all relevant and within the purview of HCI's potential for storytelling around a "return on investment." To reiterate: HCI scholars and practitioners design information as well as technologies and sociotechnical systems—and that information can include what is measured and how it is measured, and the mechanisms for interpretation.
Finally, it is important to shift the conversations around ROI to be clear on the scope and timeframe for return. Often, when the term ROI is used, the discussion of "what aspects and in what time frame" does not take place. Business is often characterized as short-term-focused and too tactical, with little regard for the longer term or larger scope of impact. The bigger impact of HCI in the long term has been discussed before, but we have not, to date, been as good as we could be in charting and sharing those stories. I am excited to see Jonathan Grudin's article, now a book (From Tool to Partner: The Evolution of Human-Computer Interaction), that details some of the lasting threads in human-computer interaction and computer-supported cooperative work, and I remember reading Richard Thomas's Long Term HCI years ago. These are good examples of taking a long-term, historically grounded view and offering insights about the value of our work over time.
In closing, I am much in agreement with Ben Shneiderman and his co-authors in their "Grand Challenges for HCI Researchers" article from Interactions (September–October 2016). My comments here are perhaps a meta-comment to that, though, which is that we also need to think about our "business" model and value as HCI researchers and practitioners. I invite us to tell the stories of our impact in ways that translate into and have the potential to transform success measures for the industries within which we work. We are trained to tell stories and to share our ideas in various formats, from two-column conference papers to short and long talks, to blog posts and lectures, to video demos and improvisation. This is another genre. Let's start storying our value and reconfiguring the meaning of ROI.
1. From Wikipedia: "Net Promoter or Net Promoter Score (NPS) is a management tool that can be used to gauge the loyalty of a firm's customer relationships. It serves as an alternative to traditional customer satisfaction research and claims to be correlated with revenue growth." https://en.wikipedia.org/wiki/Net_Promoter
Originally from the U.K., Elizabeth Churchill has been leading corporate research at top U.S. companies for the past 18 years. Her research interests include social media, distributed collaboration, mediated communication, and ubiquitous and embedded computing applications. firstname.lastname@example.org
Copyright held by author
The Digital Library is published by the Association for Computing Machinery. Copyright © 2017 ACM, Inc.