ACM Interactions

Authors:

---

Over the years, this column has explored how user-centered design (UCD) fits into the design and development process in the business world. In the context of traditional practices, UCD is often viewed as an added cost, one that has at most long-range benefits. Of course, when a conflict is perceived between what is needed to maximize long-range value and what is needed to successfully meet urgent near-term challenges, the latter usually wins out. Therefore, practitioners of UCD tend to search for examples of how the UCD approach can provide a model for leadership in design and development that integrates the short- and long-term perspectives. This article explores the role that the "demo version" (henceforth referred to simply as the "demo") plays in the product lifecycle, its impact on design, and how UCD can help manage this impact.

UCD practitioners talk both about how UCD can help make the design process more efficient and less costly in the near term and about how it helps ensure the long-range success of products (for example, by avoiding the significant rework needed when an entire conceptual and logical design has to be re-engineered late in the process). To achieve this, we have to understand the points of contact in product development among the perspectives of the business, of design, and of implementation, in order to find their common ground and the "win-win" opportunities.

A recent series of experiences has suggested that one main point of contact is the product demo and its impact on design and development. If this impact is not well managed, the demo may well have a powerful and distorting impact on the design process. One of us (Siegel) was intrigued by how the UCD approach can help manage the potentially negative influence of the demo on the design of the product or system. The other (Rouchka, whose company is profiled in the second case study) realized that UCD can help meet the challenge of keeping the demo current with the evolution of the application or product, which often is a significant executive concern. Thus, the demo may be a point of contact where urgent business concerns and the concerns of good design can be brought together in a way that can gain acceptance of UCD at the highest levels. This of course will not happen by default, but only with effective leadership.

Demo as Vaporware

About a year and a half ago, a client asked one of us (Siegel) to help with interface design for a new browser-based application. The company faced a number of immediate challenges, centering on two familiar themes: limited resources and limited time. In this case, limited time brought an interesting twist. The company was just a few weeks away from the annual conference for the profession the application was designed to support. At that conference, at least four or five direct competitors would be showing their software offerings. The client needed to have something of its own to show. Its developers were already working feverishly on a demo, with no input from professional designers, much less from UCD professionals. Their attitude was that they simply needed to get something "up on the screen" as fast as possible. It was a typical case of "vaporware."

The initial reason for calling in a consultant was to get some help for the developers so that more screens could be ready to show. Given the time pressure, and the perceived strategic importance of this conference, the company was not receptive to the idea of taking more time to study its intended users and create a prototype of some conceptual designs. They were, however, open to two compromises that seemed likely to at least help them start thinking more systematically about design and about iterative user evaluation. First, they wanted expert input into the design that the developers were working on. Second, they agreed to try to carry out some user testing at the trade show.

A quick heuristic review of the screens that had already been designed revealed both design decisions that were obvious problems, and other decisions, including basic approaches to the conceptual design and logical structure, that should be tested. It quickly became clear that the team was not open to making changes in their demo. For one thing, even though this was early in the process, they had already adopted some conventions and designed a series of screens based on them. It is always the case that the more people have built, the less happy they are about having to change things that will have a broad impact. But beyond this, the fact that this was a demo lowered their motivation to make changes before the show. Not only did the urgent need for the demo interfere with anyone taking the time to rework screens they already thought of as "finished," but also, because it was "only" a demo, they told themselves they were not wedded to the design. The mantra of the team became, "We know this is not a good design. We are just throwing it together for the show, and then we will fix it."

"Oh, no, you won't!"

This is what you are probably saying to yourself right now. You would be right. The factors that make it more and more costly to change a design the further one goes in implementation also apply to the development of even a nonfunctional demo. By the time you have designed several screens, the work of changing some early assumptions or the basic navigational model can seem overwhelming, and time for this revision is probably not built into the project schedule. You become increasingly invested in the design "solutions" achieved so far and increasingly unable to perceive them as anything other than the most elegant possible solutions.

Beyond this, though, factors specific to the nature of a demo and its role in the business process can also introduce specific obstacles to good design practice. Of course, the need for a demo does not end with one trade show. Thus, the team could persist in telling itself that "it is only a demo" for some time, even as they continued elaborating it and thus building their commitment to it. What is more, the fact that the demonstration is typically used for sales purposes can give it a disproportionate impact on the product's design. As we know, carrying out a demonstration is radically different from conducting a user test. It is not at all surprising when potential customers react positively to even a poorly designed demo user interface (UI). More accurately, in the context of a demonstration, users are not even reacting to the UI so much as they are to the description of features and functionality. Nevertheless, the accumulation of positive user reactions is often taken as a validation for the design, making it harder to convince decision makers of a real need for improvements. An accumulation of positive reactions to the demo can even suggest that it is far too risky to make fundamental changes, since reactions to the existing design seem like the "known quantity." Finally, since the product as represented by the demo is what management and marketing have been selling to investors and potential customers, management sometimes takes the position that they are now obligated to build it this way.

Another factor that can give demos disproportionate impact derives from who their "owners" are. Commonly, these are roles that are at a strategic level in the company, whether in marketing or some other executive function. In many startups and smaller companies, the CEO may be the primary user of the demo, showing it to potential investors or the first generation of potential customers. In contrast, design is at a tactical level. Thus, disproportionate political power often backs the high priority given to the demo, adding to the weight that is attached to perceptions derived from experiences with the demo. Furthermore, these are organizational players who are often not very sensitive to the fine distinctions between the process of conducting a demonstration, which can look like a vehicle for getting feedback, and the methodologies of UCD.

What Happened

Recall that the second compromise the client agreed to was that we could use the demo at the trade show to do informal user testing. It would be encouraging to report that data from these tests acted to counter some of the problematic dynamics described earlier. In reality, this was not the case. Some of this was attributable to the logistical challenges of trying to conduct a usability evaluation at the trade show without adequate preparation or buy-in of all stakeholders. Unfortunately, the testing was treated as a small sideline activity at the conference. Although a small number of conference attendees (selected by the marketing people) were invited to evaluate the interface, a very large number were walked through the demo in the exhibit hall in the traditional marketing style. The stories that were carried home about how much these attendees "liked" the product outweighed the fact that the small number who tested the design using simple task scenarios identified the same problems that the expert heuristic review had turned up in the basic navigational model. Furthermore, expressions of interest by some potential customers, which were met by promises about delivery dates, only served to add even more urgency to the development timetable. True to the prediction ("Oh, no, you won't!"), the company continued to build upon the flawed navigational model, originally "thrown together" for the demo, as it implemented more and more modules of the fully functional software.

There is a need to manage both the impact of the demo on design and impact of design on the demo.

Almost exactly one year later, this company had installed its own software for its executives to use in their own firm. When, in their own work, they encountered the navigational problem that had been identified a year earlier and was supposedly to be corrected right after the trade show, they confronted the product manager, saying that usability should have been made more of a priority before the software was implemented! Ironically, this product manager had been the usability champion all along, but had not had the support to counterbalance all the dynamics described earlier.

Most key elements of this story are probably recognizable. We have certainly seen dynamics like these play themselves out in a variety of contexts. So, a fair summary statement might be that, although the demo is a design prototype, it is often constructed with even less commitment to design quality and early incorporation of user-centered methods than software in general, likely to generate misleading feedback, and championed by people with a powerful strategic corporate role. The potential negative impact that demos can have on the quality of the ultimate design flows from this constellation of factors. Must it always be so?

Demo as Prototype

In a more recent consulting project, the second author (Rouchka) asked the first author to assist with evaluation and redesign of the interface for a new browser-based application for enrolling in medical benefits and coverage, one that allows a new degree of personalization. While learning about the concepts of UCD, she realized that UCD has implications for more effectively managing the relationship between the demo and the design of the actual system. Specifically, UCD might facilitate coordination between the demo and the actual software. Earlier we said that a demo can often tie a company prematurely to a design. But it is also important to recognize that maintaining the demo so that it remains current during product evolution can also be a challenge. Because of their firsthand involvement with the demo and their reliance on it for key business processes like raising capital and selling, maintaining the integrity of the demo is often a key concern of executives, who thus may be receptive to anything that facilitates it. Perhaps it is fairest to say that the challenges associated with demos operate in both directions. There is a need to manage the impact of the demo on design and the impact of design on the demo. This provides an opportunity to build a bridge with corporate executives and give them an added stake in UCD. Here may lie a potential advantage. The case of Vivius will help to illustrate this.

Demo and Design at Vivius, Inc.

Vivius has developed a concept and software that allow users to select and sign up for medical benefits in a new way. Users can select a personalized panel of medical providers, for many specialties, based on information about the providers' practices and costs. They can base their choices on the referral patterns of a key trusted provider, or they can completely customize a selection of providers regardless of referrals. They can also choose different co-pay levels. What is especially unusual in the U.S. healthcare market is that this product allows different members of the same family to choose both different panels of providers and different co-payment levels. Although the range of options offered to enrollees will depend on decisions made by employers and by the underwriting insurers, the fundamental concept is to give medical benefit enrollees an unprecedented degree of individual control over their choice of medical providers and their out-of-pocket costs, including both co-payments, and total premiums.

As can be said about many startup companies, Vivius began as an idea and a demo. In the beginning, the purpose of the demo was to paint a picture of what the system would look like to potential investors and partners. If a picture is worth a thousand words, in this case it was worth several million dollars.

Once Vivius obtained venture capital, the focus of development shifted from the demo toward building the actual site. As is typical with all startups, Vivius faced the dilemma of making rapid progress and spending its venture capital wisely. Moreover, the fundamental business model was, in the beginning, likely to be a moving target. This engendered the idea that the initial design was not necessarily where it would end up. Partly for this reason, Vivius decided to build version 1.0 without dedicated UI or usability resources. Although the interface was to be a critical piece of the final system for many reasons, Vivius did not think those reasons were applicable at that early stage. Because they expected changes in the business model as Vivius further defined itself in the market, they thought that development would essentially be done twice. Thus, management's perspective was that it did not also make sense to design the entire interface twice. It was therefore decided to defer a focus on the interface until later in the product definition and development process. Clearly, this is a reflection of the waterfall methodology, not the process advocated by proponents of user-centered design. Good interface design is the result of a sequence of UCD methodologies applied at each stage of the product's evolution, from the earliest definition of the concept, rather than something to be tacked on after functionality or the database architecture are defined. However, the waterfall methodology is probably both a more common practice and an understandable response to the particular pressures that startups face.

While Vivius's development team began work on the actual system, the executives and others were still using the original demo to sell the product in the marketplace. This arrangement worked fine until developers began to turn out actual system screens. Once the executives saw the new screens, they immediately wanted the demo updated. This made perfect sense because the purpose of the demo had evolved. Whereas initially they used the original demo to attract investors, they now wanted to be able to show the market what the system was going to be, not what it was last week or last month.

The original demo had been built on a technology that Vivius moved away from; therefore, it did not make sense to continue to update it. Consequently, the company began to look for the best way to create and, more important, efficiently maintain a demo. From the sales department's perspective, the demo should be as close as possible to the real thing. It took several months before they settled on simply using HTML screen shots. This involved pulling the HTML code from each page of the application, stripping the real functionality, and putting in a few strategically placed hyperlinks for navigation. When a screen changed, which happened frequently, a new screen shot would be pulled and the demo updated. This solution also helped preserve development resources, partly because maintenance of the demo could be outsourced.

Although this was a useful approach, a problem remained. The demo could not be updated until development had built the feature. Each time a new set of screens was released, a high sense of urgency (if not emergency) arose about updating the demo To make things worse, the bigger the change, the sooner the updated demo was needed, so that the executives were not demonstrating obsolete functionality. Of course, these were the changes the company took the longest to develop.

By this time the business model had stabilized and Vivius was on the point of signing a contract for its first customer. Now, the need to focus on the UI design and usability became more obvious to some within the company. A connection became apparent, and it seemed that UCD might provide an answer to the demo problems. Specifically, a different way of generating and using demos might be a natural side effect of usability work. The prototypes generated in the early stages of a UCD process could in effect provide the materials for the demos. Furthermore, since a UCD approach focuses on the appropriate level of design for different stages of development, it could help ensure that the screen designs serving as demos reflected the appropriate level of design decision for that stage of development. For example, early in the product definition process, it is appropriate to focus on designs of high-level screens that reflect the current conceptual design, and these are the screens that UCD would lead the team to prototype first. Coordinating the demo and development around the iterative process of working through stages of design could greatly simplify the challenge of maintaining the demo. In fact, this shift could allow the demo to be driven by design rather than conversely.

Vivius gained some experience with some aspects of this approach during a redesign project on a portion of the site—the portion that was typically the focus of demos. The screens were designed and made into prototypes before any development work began. This allowed Vivius to put together a demo before development built the site instead of after. The executives could hardly wait to get out and show the new screens, and this time they didn't have to wait. In hindsight, it became apparent to Vivius that it would have been cost-effective to place a higher priority on the UI and take a UCD approach from the beginning. From the design perspective, and from the UCD perspective, this may sound obvious. What was not so obvious was that a UCD approach might actually have supported the sales effort in ways that would have immediately addressed a prime concern of the executives. This is the twist that suggests a new opportunity to build support for UCD. Of course, this change in thinking about demos would also help reduce their potential negative organizational influence on the design process.

Unanswered Question

A more fundamental change would be for the demo process itself to be transformed into a component of UCD, or at least for some elements of UCD to be incorporated into some of the common types of product demonstrations. If companies felt free to acknowledge that what they were showing was a nonfunctional prototype, then we could begin to identify the UCD methodologies that would fit in the contexts where demos usually take place, versus where they are just impractical or inappropriate. We are not talking just about the common use of demos as a stimulus for discussion and customer feedback. Although even this is a start, the problem is that because demonstrations tend to focus on showcasing features and functionality, they tend to yield more and more requests for added features without careful weighing of the implications and trade-offs for design and usability.

As the experience at the trade show discussed earlier revealed, it is not easy to "graft" a little usability test onto what is fundamentally a marketing activity. Thus, we would also have to figure out what it would take to get all parties who interact around demos to adopt this paradigm shift. We also need to ask when this new approach to using the demo as a component of UCD would in fact fulfill the business purpose of demos. We should also ask if any sales advantages could be derived from creating and using demos in a new way. It seems reasonable that companies that follow this different approach should be able to derive some competitive advantage from the opportunity to showcase their commitment to UCD, especially when they are competing with companies whose impressive-looking demos are sometimes (or often) vaporware.

We have data and anecdotal evidence that users are favorably impressed by efforts to seek their input, including participation in usability tests, but we need to find out to what extent this also applies to venture capitalists. Interestingly, we may soon gain some insight into this, because, at Vivius, the board of directors, who are also the investors, is now expressing interest in serving as participants in a usability test of the new design.

Authors

David Siegel
Dray & Associates, Inc.
2007 Kenwood Parkway
Minneapolis, MN 55405, USA
+1-612-377-1980
fax: +1-612-377-0363
[email protected]

Tracy Rouchka
Vivins, Inc.
5775 Mayzata Blvd.
#300
St. Louis Park, MN
55416-1235
[email protected]

Business Column Editors

Susan Dray & David A. Siegel
Dray & Associates, Inc.
2007 Kenwood Parkway
Minneapolis, MN 55405, USA
+1-612-377-1980
fax: +1-612-377-0363
[email protected]
[email protected]