IX.4 July 2002
Page: 25
Digital Citation

Demo-driven design or design-driven demos: vaporware, demos, and prototypes


Over the years, this column has explored how user-centered
  design (UCD) fits into the design and development process in
  the business world. In the context of traditional practices,
  UCD is often viewed as an added cost, one that has at most
  long-range benefits. Of course, when a conflict is perceived
  between what is needed to maximize long-range value and what
  is needed to successfully meet urgent near-term challenges,
  the latter usually wins out. Therefore, practitioners of UCD
  tend to search for examples of how the UCD approach can
  provide a model for leadership in design and development that
  integrates the short- and long-term perspectives. This
  article explores the role that the "demo version"
  (henceforth referred to simply as the "demo") plays
  in the product lifecycle, its impact on design, and how UCD
  can help manage this impact.


UCD practitioners talk both about how UCD can help make
  the design process more efficient and less costly in the near
  term and about how it helps ensure the long-range success of
  products (for example, by avoiding the significant rework
  needed when an entire conceptual and logical design has to be
  re-engineered late in the process). To achieve this, we have
  to understand the points of contact in product development
  among the perspectives of the business, of design, and of
  implementation, in order to find their common ground and the
  "win-win" opportunities.


A recent series of experiences has suggested that one main
  point of contact is the product demo and its impact on design
  and development. If this impact is not well managed, the demo
  may well have a powerful and distorting impact on the design
  process. One of us (Siegel) was intrigued by how the UCD
  approach can help manage the potentially negative influence
  of the demo on the design of the product or system. The other
  (Rouchka, whose company is profiled in the second case study)
  realized that UCD can help meet the challenge of keeping the
  demo current with the evolution of the application or
  product, which often is a significant executive concern.
  Thus, the demo may be a point of contact where urgent
  business concerns and the concerns of good design can be
  brought together in a way that can gain acceptance of UCD at
  the highest levels. This of course will not happen by
  default, but only with effective leadership.

Demo as Vaporware


About a year and a half ago, a client asked one of us
  (Siegel) to help with interface design for a new
  browser-based application. The company faced a number of
  immediate challenges, centering on two familiar themes:
  limited resources and limited time. In this case, limited
  time brought an interesting twist. The company was just a few
  weeks away from the annual conference for the profession the
  application was designed to support. At that conference, at
  least four or five direct competitors would be showing their
  software offerings. The client needed to have something of
  its own to show. Its developers were already working
  feverishly on a demo, with no input from professional
  designers, much less from UCD professionals. Their attitude
  was that they simply needed to get something "up on the
  screen" as fast as possible. It was a typical case of


The initial reason for calling in a consultant was to get
  some help for the developers so that more screens could be
  ready to show. Given the time pressure, and the perceived
  strategic importance of this conference, the company was not
  receptive to the idea of taking more time to study its
  intended users and create a prototype of some conceptual
  designs. They were, however, open to two compromises that
  seemed likely to at least help them start thinking more
  systematically about design and about iterative user
  evaluation. First, they wanted expert input into the design
  that the developers were working on. Second, they agreed to
  try to carry out some user testing at the trade show.


A quick heuristic review of the screens that had already
  been designed revealed both design decisions that were
  obvious problems, and other decisions, including basic
  approaches to the conceptual design and logical structure,
  that should be tested. It quickly became clear that the team
  was not open to making changes in their demo. For one thing,
  even though this was early in the process, they had already
  adopted some conventions and designed a series of screens
  based on them. It is always the case that the more people
  have built, the less happy they are about having to change
  things that will have a broad impact. But beyond this, the
  fact that this was a demo lowered their motivation to make
  changes before the show. Not only did the urgent need for the
  demo interfere with anyone taking the time to rework screens
  they already thought of as "finished," but also,
  because it was "only" a demo, they told themselves
  they were not wedded to the design. The mantra of the team
  became, "We know this is not a good design. We are just
  throwing it together for the show, and then we will fix


"Oh, no, you won’t!"


This is what you are probably saying to yourself right
  now. You would be right. The factors that make it more and
  more costly to change a design the further one goes in
  implementation also apply to the development of even a
  nonfunctional demo. By the time you have designed several
  screens, the work of changing some early assumptions or the
  basic navigational model can seem overwhelming, and time for
  this revision is probably not built into the project
  schedule. You become increasingly invested in the design
  "solutions" achieved so far and increasingly unable
  to perceive them as anything other than the most elegant
  possible solutions.


Beyond this, though, factors specific to the nature of a
  demo and its role in the business process can also introduce
  specific obstacles to good design practice. Of course, the
  need for a demo does not end with one trade show. Thus, the
  team could persist in telling itself that "it is only a
  demo" for some time, even as they continued elaborating
  it and thus building their commitment to it. What is more,
  the fact that the demonstration is typically used for sales
  purposes can give it a disproportionate impact on the
  product’s design. As we know, carrying out a demonstration is
  radically different from conducting a user test. It is not at
  all surprising when potential customers react positively to
  even a poorly designed demo user interface (UI). More
  accurately, in the context of a demonstration, users are not
  even reacting to the UI so much as they are to the
  description of features and functionality. Nevertheless, the
  accumulation of positive user reactions is often taken as a
  validation for the design, making it harder to convince
  decision makers of a real need for improvements. An
  accumulation of positive reactions to the demo can even
  suggest that it is far too risky to make fundamental changes,
  since reactions to the existing design seem like the
  "known quantity." Finally, since the product as
  represented by the demo is what management and marketing have
  been selling to investors and potential customers, management
  sometimes takes the position that they are now obligated to
  build it this way.


Another factor that can give demos disproportionate impact
  derives from who their "owners" are. Commonly,
  these are roles that are at a strategic level in the company,
  whether in marketing or some other executive function. In
  many startups and smaller companies, the CEO may be the
  primary user of the demo, showing it to potential investors
  or the first generation of potential customers. In contrast,
  design is at a tactical level. Thus, disproportionate
  political power often backs the high priority given to the
  demo, adding to the weight that is attached to perceptions
  derived from experiences with the demo. Furthermore, these
  are organizational players who are often not very sensitive
  to the fine distinctions between the process of conducting a
  demonstration, which can look like a vehicle for getting
  feedback, and the methodologies of UCD.

What Happened


Recall that the second compromise the client agreed to was
  that we could use the demo at the trade show to do informal
  user testing. It would be encouraging to report that data
  from these tests acted to counter some of the problematic
  dynamics described earlier. In reality, this was not the
  case. Some of this was attributable to the logistical
  challenges of trying to conduct a usability evaluation at the
  trade show without adequate preparation or buy-in of all
  stakeholders. Unfortunately, the testing was treated as a
  small sideline activity at the conference. Although a small
  number of conference attendees (selected by the marketing
  people) were invited to evaluate the interface, a very large
  number were walked through the demo in the exhibit hall in
  the traditional marketing style. The stories that were
  carried home about how much these attendees "liked"
  the product outweighed the fact that the small number who
  tested the design using simple task scenarios identified the
  same problems that the expert heuristic review had turned up
  in the basic navigational model. Furthermore, expressions of
  interest by some potential customers, which were met by
  promises about delivery dates, only served to add even more
  urgency to the development timetable. True to the prediction
  ("Oh, no, you won’t!"), the company continued to
  build upon the flawed navigational model, originally
  "thrown together" for the demo, as it implemented
  more and more modules of the fully functional software.

There is a need to manage both the
  impact of the demo on design and impact of design on the


Almost exactly one year later, this company had installed
  its own software for its executives to use in their own firm.
  When, in their own work, they encountered the navigational
  problem that had been identified a year earlier and was
  supposedly to be corrected right after the trade show, they
  confronted the product manager, saying that usability should
  have been made more of a priority before the software was
  implemented! Ironically, this product manager had been the
  usability champion all along, but had not had the support to
  counterbalance all the dynamics described earlier.


Most key elements of this story are probably recognizable.
  We have certainly seen dynamics like these play themselves
  out in a variety of contexts. So, a fair summary statement
  might be that, although the demo is a design prototype, it is
  often constructed with even less commitment to design quality
  and early incorporation of user-centered methods than
  software in general, likely to generate misleading feedback,
  and championed by people with a powerful strategic corporate
  role. The potential negative impact that demos can have on
  the quality of the ultimate design flows from this
  constellation of factors. Must it always be so?

Demo as Prototype


In a more recent consulting project, the second author
  (Rouchka) asked the first author to assist with evaluation
  and redesign of the interface for a new browser-based
  application for enrolling in medical benefits and coverage,
  one that allows a new degree of personalization. While
  learning about the concepts of UCD, she realized that UCD has
  implications for more effectively managing the relationship
  between the demo and the design of the actual system.
  Specifically, UCD might facilitate coordination between the
  demo and the actual software. Earlier we said that a demo can
  often tie a company prematurely to a design. But it is also
  important to recognize that maintaining the demo so that it
  remains current during product evolution can also be a
  challenge. Because of their firsthand involvement with the
  demo and their reliance on it for key business processes like
  raising capital and selling, maintaining the integrity of the
  demo is often a key concern of executives, who thus may be
  receptive to anything that facilitates it. Perhaps it is
  fairest to say that the challenges associated with demos
  operate in both directions. There is a need to manage the
  impact of the demo on design and the impact of design on the
  demo. This provides an opportunity to build a bridge with
  corporate executives and give them an added stake in UCD.
  Here may lie a potential advantage. The case of Vivius will
  help to illustrate this.


Demo and Design at Vivius, Inc.


Vivius has developed a concept and software that allow
  users to select and sign up for medical benefits in a new
  way. Users can select a personalized panel of medical
  providers, for many specialties, based on information about
  the providers’ practices and costs. They can base their
  choices on the referral patterns of a key trusted provider,
  or they can completely customize a selection of providers
  regardless of referrals. They can also choose different
  co-pay levels. What is especially unusual in the U.S.
  healthcare market is that this product allows different
  members of the same family to choose both different panels of
  providers and different co-payment levels. Although the range
  of options offered to enrollees will depend on decisions made
  by employers and by the underwriting insurers, the
  fundamental concept is to give medical benefit enrollees an
  unprecedented degree of individual control over their choice
  of medical providers and their out-of-pocket costs, including
  both co-payments, and total premiums.


As can be said about many startup companies, Vivius began
  as an idea and a demo. In the beginning, the purpose of the
  demo was to paint a picture of what the system would look
  like to potential investors and partners. If a picture is
  worth a thousand words, in this case it was worth several
  million dollars.


Once Vivius obtained venture capital, the focus of
  development shifted from the demo toward building the actual
  site. As is typical with all startups, Vivius faced the
  dilemma of making rapid progress and spending its venture
  capital wisely. Moreover, the fundamental business model was,
  in the beginning, likely to be a moving target. This
  engendered the idea that the initial design was not
  necessarily where it would end up. Partly for this reason,
  Vivius decided to build version 1.0 without dedicated UI or
  usability resources. Although the interface was to be a
  critical piece of the final system for many reasons, Vivius
  did not think those reasons were applicable at that early
  stage. Because they expected changes in the business model as
  Vivius further defined itself in the market, they thought
  that development would essentially be done twice. Thus,
  management’s perspective was that it did not also make sense
  to design the entire interface twice. It was therefore
  decided to defer a focus on the interface until later in the
  product definition and development process. Clearly, this is
  a reflection of the waterfall methodology, not the process
  advocated by proponents of user-centered design. Good
  interface design is the result of a sequence of UCD
  methodologies applied at each stage of the product’s
  evolution, from the earliest definition of the concept,
  rather than something to be tacked on after functionality or
  the database architecture are defined. However, the waterfall
  methodology is probably both a more common practice and an
  understandable response to the particular pressures that
  startups face.


While Vivius’s development team began work on the actual
  system, the executives and others were still using the
  original demo to sell the product in the marketplace. This
  arrangement worked fine until developers began to turn out
  actual system screens. Once the executives saw the new
  screens, they immediately wanted the demo updated. This made
  perfect sense because the purpose of the demo had evolved.
  Whereas initially they used the original demo to attract
  investors, they now wanted to be able to show the market what
  the system was going to be, not what it was last week or last


The original demo had been built on a technology that
  Vivius moved away from; therefore, it did not make sense to
  continue to update it. Consequently, the company began to
  look for the best way to create and, more important,
  efficiently maintain a demo. From the sales department’s
  perspective, the demo should be as close as possible to the
  real thing. It took several months before they settled on
  simply using HTML screen shots. This involved pulling the
  HTML code from each page of the application, stripping the
  real functionality, and putting in a few strategically placed
  hyperlinks for navigation. When a screen changed, which
  happened frequently, a new screen shot would be pulled and
  the demo updated. This solution also helped preserve
  development resources, partly because maintenance of the demo
  could be outsourced.


Although this was a useful approach, a problem remained.
  The demo could not be updated until development had built the
  feature. Each time a new set of screens was released, a high
  sense of urgency (if not emergency) arose about updating the
  demo To make things worse, the bigger the change, the sooner
  the updated demo was needed, so that the executives were not
  demonstrating obsolete functionality. Of course, these were
  the changes the company took the longest to develop.


By this time the business model had stabilized and Vivius
  was on the point of signing a contract for its first
  customer. Now, the need to focus on the UI design and
  usability became more obvious to some within the company. A
  connection became apparent, and it seemed that UCD might
  provide an answer to the demo problems. Specifically, a
  different way of generating and using demos might be a
  natural side effect of usability work. The prototypes
  generated in the early stages of a UCD process could in
  effect provide the materials for the demos. Furthermore,
  since a UCD approach focuses on the appropriate level of
  design for different stages of development, it could help
  ensure that the screen designs serving as demos reflected the
  appropriate level of design decision for that stage of
  development. For example, early in the product definition
  process, it is appropriate to focus on designs of high-level
  screens that reflect the current conceptual design, and these
  are the screens that UCD would lead the team to prototype
  first. Coordinating the demo and development around the
  iterative process of working through stages of design could
  greatly simplify the challenge of maintaining the demo. In
  fact, this shift could allow the demo to be driven by design
  rather than conversely.


Vivius gained some experience with some aspects of this
  approach during a redesign project on a portion of the
  site—the portion that was typically the focus of demos.
  The screens were designed and made into prototypes before any
  development work began. This allowed Vivius to put together a
  demo before development built the site instead of after. The
  executives could hardly wait to get out and show the new
  screens, and this time they didn’t have to wait. In
  hindsight, it became apparent to Vivius that it would have
  been cost-effective to place a higher priority on the UI and
  take a UCD approach from the beginning. From the design
  perspective, and from the UCD perspective, this may sound
  obvious. What was not so obvious was that a UCD approach
  might actually have supported the sales effort in ways that
  would have immediately addressed a prime concern of the
  executives. This is the twist that suggests a new opportunity
  to build support for UCD. Of course, this change in thinking
  about demos would also help reduce their potential negative
  organizational influence on the design process.


Unanswered Question


A more fundamental change would be for the demo process
  itself to be transformed into a component of UCD, or at least
  for some elements of UCD to be incorporated into some of the
  common types of product demonstrations. If companies felt
  free to acknowledge that what they were showing was a
  nonfunctional prototype, then we could begin to identify the
  UCD methodologies that would fit in the contexts where demos
  usually take place, versus where they are just impractical or
  inappropriate. We are not talking just about the common use
  of demos as a stimulus for discussion and customer feedback.
  Although even this is a start, the problem is that because
  demonstrations tend to focus on showcasing features and
  functionality, they tend to yield more and more requests for
  added features without careful weighing of the implications
  and trade-offs for design and usability.


As the experience at the trade show discussed earlier
  revealed, it is not easy to "graft" a little
  usability test onto what is fundamentally a marketing
  activity. Thus, we would also have to figure out what it
  would take to get all parties who interact around demos to
  adopt this paradigm shift. We also need to ask when this new
  approach to using the demo as a component of UCD would in
  fact fulfill the business purpose of demos. We should also
  ask if any sales advantages could be derived from creating
  and using demos in a new way. It seems reasonable that
  companies that follow this different approach should be able
  to derive some competitive advantage from the opportunity to
  showcase their commitment to UCD, especially when they are
  competing with companies whose impressive-looking demos are
  sometimes (or often) vaporware.


We have data and anecdotal evidence that users are
  favorably impressed by efforts to seek their input, including
  participation in usability tests, but we need to find out to
  what extent this also applies to venture capitalists.
  Interestingly, we may soon gain some insight into this,
  because, at Vivius, the board of directors, who are also the
  investors, is now expressing interest in serving as
  participants in a usability test of the new design.



David Siegel

  Dray & Associates, Inc.

  2007 Kenwood Parkway

  Minneapolis, MN 55405, USA


  fax: +1-612-377-0363


Tracy Rouchka

  Vivins, Inc.

  5775 Mayzata Blvd.


  St. Louis Park, MN



Business Column Editors


Susan Dray & David A. Siegel

  Dray & Associates, Inc.

  2007 Kenwood Parkway

  Minneapolis, MN 55405, USA


  fax: +1-612-377-0363

Post Comment

No Comments Found