CHI 99 featured the first live interview sessions to be conducted at a CHI conference. Tapped for these interview sessions were some of the most original thinkers in human-computer interaction (HCI). Clement Mok and Jakob Nielsen were asked to address the Web and Web design limits to HCI. Bill Buxton and Clifford Nass tackled human limits to HCI. Wayne Gray and Bill Gaver compared their perspectives on methodological limits to HCI. And Don Norman and Janice Rohn squared off to address organizational limits to HCI. The interviewer was the Interviews Chair for CHI 99-Richard Anderson, whose work is all about setting the stage for avoiding or overcoming limits to HCI and who has conducted several interview sessions in recent years on the stage of the BayCHI program.
Two of those CHI 99 sessions were published in the January-February 2000 issue of interactions. A third is presented here. The fourth was, unfortunately, not recorded.
The session published in this issue focused on organizational limits to HCI and featured two of the best experts.
As you’ll see, I didn’t need to do much to get the two of them talking.
Richard Anderson: I am seeing an increasing number of job postings these days for user interface designers, information designers, usability people, user research specialists, and the like. Is this a good sign?
Don Norman: No. A mantra I will repeat over and over again today is that a company that is proud of its usability labs is a company in trouble. It’s like being proud of your beta tests. Basically, if you’re finding problems in your usability labs, it’s all over…it’s too late. And any company that’s proud of its usability labs and touts them is a company that doesn’t understand what it’s about. Microsoft is proud of its 18 months of beta tests. Yuck! Any product that requires 18 months of beta tests is in trouble.
Janice Rohn: Wait a second now, how can more job postings in our area be a negative? This is a great sign for our field, that there is more and more recognition, that there is a need for this type of work.
Don Norman: Nah, it’s like more and more advertisements for ditch diggers. If that’s what you want to be, if you want to be the low status, low on the totem pole person in your company, then yeah, rejoice in the fact that you are hiring user testers. User testing is not where the action is. The action is with those people who decide what product to build in the first place. That isn’t the user tester community, but it should be the CHI community. You know, I’m sick and tired of hearing the CHI community complain that they’re never listened to. Yeah, it’s great we’re hiring all these people who are never listened to, right? Well, the reason they’re never listened to is their own fault. (audience laughter and applause)
Janice Rohn: We could go off in that direction….
Don Norman: And we will.
Janice Rohn: And we will, definitely. But, when was the last time you saw a lot of high-level executives actually making the day-to-day decisions of how products look? It’s very rare that that actually happens. They’re usually busy working on a lot of other things. People working on the productson the product teamshave a huge impact on the product. They can make a big difference on how it actually comes out. We hear all the time that the high-level plans for how products should look and what they should end up like are not at all how they actually end up, because the development teamthe people working on the productshave gone off in their own direction. These are the peoplethe people being sought by all the job postings out therewho can be making an effect on how products actually end up.
Don Norman: Yeah, but the real important decisions are, indeed, made at the top. Those are the people who will decide what direction you’re moving in, what the time frame is, what the budget will be, where the emphasis is… and we need more people from the CHI community to be those executives…to be making those decisions which will eventually empower this profession. Look, this field should not be about usability; that’s a silly thing. This field should be about empowering users, and that decision is made at the executive level.
Janice Rohn: And indeed, there are a lot of decisions made at the executive level. But just because you’re an executive… just because you have a title, doesn’t mean that you are a key influencer in the company. It doesn’t mean that you have respect and can make a big difference.
Don Norman: So, I want CHI people to be the people with influence who do have the respect. What’s the matter with that?
Janice Rohn: But you can make a big difference, no matter where you are in the company, if you make a strong contribution and you know what you’re talking about.
Don Norman: No. You can’t make a contribution if you’re called in after the product is finished and your job is to test it in the lab with one-way glass mirrors (in a very artificial setting) and you’re under huge time pressures and money pressures. If you find any problems, people will hate you, because you’re going to set them behind schedule.
Janice Rohn: Right, and oftentimes that is the first thing that happens when you are working with a new product teamyou come in too late in the development cycle. But what you do is educate them about the importance of getting in earlier in the cycle. And the next time aroundand sure enough, we see this over and over againthey are back earlier in the process. You educate them; it’s a process of working with each of the teams and making them understand.
Don Norman: And then the company reorganizes, and there are new teams. And guess what? You’re back in the trenches again, and you have to reeducate. That’s no way to live your life.
Janice Rohn: But the great thing about a reorganization is that instead of having six people who know the importance of usability in one team, you’ve got people in different teams who then bring their new teams back to yours.
Don Norman: Look, why do you think marketing dominates so much?
Janice Rohn: Marketing doesn’t always dominate so much.
Don Norman: Why do you think they have so much of a role determining the products? It’s not because they’re brighter, and it’s not because they have any more truths. It’s because they know how to play the game better. What I suggest is that it’s time we learn how to play the game.
Janice Rohn: The advantage that marketing has is that it’s a well-known concept. Executives have typically gone to business school where marketing is well-known. One of the problems that we have is that we don’t have usability engineering and HCI in most curricula for business and engineering.
Don Norman: And why not?
Janice Rohn: This is something that we’ve got to change. But it’s a relatively new field.
Don Norman: One reason we don’t is that we don’t talk the language of business.
Janice Rohn: That’s very true.
Don Norman: The company does not care about usability, nor should it. Companies are not in the business of making usable products; they’re in the business of making money. So what we have to do is learn how to understand what motivates a company and speak that language. We shouldn’t say, "Hey, you’ve got to make your stuff more usable." We should say, "Hey, I can teach you how to make more profit."
Janice Rohn: Exactly.
Don Norman: We can do that by cutting out your service lines and your returns.
Janice Rohn: One of the things we don’t do enough of is customer research on our own organizationunderstanding the terminology it uses, understanding the value systems…. We’re as guilty as any other field of having our own set of terminology.
Don Norman: It’s not just terminology.
Janice Rohn: It’s not just terminology, but it’s understanding values and understanding how to translate them.
Don Norman: We argue that we should understand our users, right? Well, our users are our management. So the first thing we should do is study our management and understand them, and make our pitch in terms management understands.
Janice Rohn: Exactly, and that’s why you need to translate values. You need to learn how to look at costs. You need to learn how to look at total cost of ownership… the life cycle cost. Oftentimes, the only thing that’s measured is the original development cost and the schedule. What’s not looked at are the life cycle costs. If you can put in measures of the life-cycle costs, then you could be…
Don Norman: No, that won’t work. You don’t understand how people get promoted.
Janice Rohn: That’s already worked. How can you say that doesn’t work?
Don Norman: Life-cycle costs don’t get me promoted. See, I get promoted if my division has a good P&L [profit and loss] sheet this quarter. So, it’s my job to increase the profits this quarter. If you tell me about life-cycle costs… Ha! The life cycle is a year or two years away. I’ll leave that to the person who follows me. It doesn’t impact my promotion; therefore, I’m going to cut my costs in order to get promoted.
Janice Rohn: That is a problem today, because oftentimes the support organization is in a different division and has a separate P&L. So you are goaled on these short-term metrics rather than on the total life cycle.
Don Norman: That’s right. So if I spend more time and money to make a better productand therefore decrease support costswhat’s going to happen is that I’ll be passed over for promotion next time, and those support people will actually be promoted. So, in fact, that’s not the route to success.
Janice Rohn: However, if you do produce a product that gets good usability reviews, and [it] has a lot of people promoting it and saying it’s a much better product because it’s more usable, that has more of a near-term effect.
Don Norman: Nonsense. The product reviews don’t do anything.
Janice Rohn: No, that’s not nonsense; they do have an impact.
Don Norman: Take a look at the industrial design community. Almost all of the products that receive the prizes in industrial design are failures in the marketplace. And believe me, what matters in the company is the monetary success.
Janice Rohn: Since we both worked at Apple, we know that sometimes the coolest design is not the most usable. So it’s not a surprise that it would fail in the marketplace if it’s a cool design but not usable and doesn’t meet the requirements of the customers.
Don Norman: Actually, that’s wrong. Often, what really did succeed in the marketplace was the coolest design. If the customers thought it was cool, it didn’t matter whether you could use it or not.
Janice Rohn: It depends on what kind of product it is. Is it a consumer product that people are making more aesthetic choices on, or is it a product that’s for doctors for whom aesthetics isn’t as much of an issue?
Richard Anderson: I wonder if I could ask a question. (audience laughter)
Don Norman: Excuse me, who are you? (more audience laughter)
Richard Anderson: Janice, you’ve argued that user research needs to play a role, not only in product design, but also in setting the direction of businessin setting its marketing position. Talk about the role of early (up-front) user research. What organizations are actually open to this?
Janice Rohn: The organizations that we’ve had repeated experience with are definitely open to it. As I was mentioning before, typically people come to us late in the process, and we will work with them as long as something can be changed at that point. At that point, we can make only the superficial changes, because it’s too late. But, in order to make the fundamental changes, we really need to get involved early. As we work with them, we point out the things that would be a lot more beneficial to do earlier in the process.
It is important, also, to unhinge the customer research from the development cycle. We’re in a period where we’re seeing increasingly shorter development times. Teams are under a lot of pressure to get their products out in very short time framesweeks or a few months rather than many months or years. So, we need to unhinge the research from the development cycle. Do good research all along and have that feed into the product as things progress without holding up the product waiting for the results of that research.
"[Computer] companies are starting to recognize that they need to change. But I don’t believe they can do so… because they are run by teenagers who love technology for the sake of technology."
Don Norman: I’m still concerned that you’re sticking yourself late in the process. You said, "Yeah, we’ll get earlier and earlier, and so we’re going to be there early, and we can tell them what to consider." No. You want to be the person who says what the product ought to be in the first place.
Janice Rohn: In some cases, we have done that. We have set visions, and we have created products based on those visions.
Don Norman: Enduring products?
Janice Rohn: Products that evolved into other products and as enduring as any product is in this world.
Don Norman: What we should do is pair up with the marketing team. Because marketing thinks they own the customer, and we think we own the customer. The answer is that we both do, because we have different aspects of the customer. What marketing does is go and talk to customers, usually in focus groups and questionnaires, and they end up with this list of features that customers want. Then marketing has this wonderful, skillful way of lyingand it’s a professional skill, so you learn this in business schoolin which you start off with some fake assumptions about how big the market is. Then you talk about how much money each one spends. Then you say, "If only we had six percent of that market, why we’d have three billion in sales next year." And then you go and do the projections, and by the time you’re finished, it really looks impressive.
We never do that. We could be doing the same thing. We should point out what our products would be. We should say, "Hey, we’ve done some field studies, and here’s actually where the costs are…here’s what the people need. And if we build this, here’s the schedule, here’s the spreadsheet, here are the projections…" and whoomp…you know, 2.73 million dollars profit the first year and 72 percent return on your investment, et cetera, et cetera. We can make up numbers just as well as they can. (audience laughter)
Janice Rohn: And there are some people in the usability field who do, and they are pretty well-known. But anyhow… (audience laughter)
Don Norman: But you won’t be in the usability field if you are doing thisthat’s the point. If you call yourself a usability person, then you are this resource that gets called on to dig the ditches and do the user tests. You want to be up there helping drive the company and move its products. That’s when you have influence, and that’s when you can call upon the other people to do their jobs.
Janice Rohn: For one thing, it’s an overgeneralization to say that marketing always has influence. That isn’t true. In a lot of companies, they don’t have the strongest influence; they do in some, they don’t in others.
Don Norman: Yeah, but when they don’t, it’s worse. It’s the technologists who have the influence…like at Sun. (audience laughter)
Janice Rohn: Well, the point is to align yourself with whoever has the influence. That’s the smart thing to do.
Don Norman: No. The point is to make better products for the customers.
Janice Rohn: Right, and the way to achieve that is aligning yourself with whoever has influence.
Don Norman: No. The way you do that is by getting the influence yourself.
Janice Rohn: Well, you have influence by influencing others, right? It’s a very good thing to say we should be in alignment with marketing. But one of the problems is that it’s often very difficult to do. For one thing, there is often a lack of cooperation or a lack of marketing people who are available. Secondly, in organizations where engineering is influential, engineering doesn’t always trust what marketing is saying, and they often won’t go with that. It’s not uncommon for marketing to give very high-level requirements, which then could map into any one of a host of design solutions. It’s really up to the HCI people working with engineering to figure out exactly what to do. That’s where HCI people have a huge amount of influence.
Don Norman: Yeah, but you’re also talking about a company, you know, like Sunwhich is technology driven and makes products that are too complicated for mortalswhere marketing and engineering are continually fighting. That’s not a good environment to start with.
Janice Rohn: Well, I’ve never seen an organization where there haven’t been disagreements between marketing and engineering.
Don Norman: But healthy disagreements are healthy. Because all of design actually is a series of trade-offs, where you do have disagreements. The critical thing to realize is that even though you are disagreeing, you’re trying to build the best possible product.
It’s not whether engineering is right, or designers are right, or user experience is right, or marketing is right. It’s, so what’s best for the product? One of the problems with our discipline is that we, too, think that we are right, and we keep complaining about others who ruin our products. We have to make compromises, too. And our methods are the wrong methods.
Janice Rohn: Exactly, and the point that you’re making is that Sun is making the right products for its customers, but its customers often are not the end users. For that category of products, things like availability and reliability are important. But there are components of usability that map into availability and reliability. Certainly, if a user error brings down a server, that’s a problem.
So we always have to ask, what are the business goals? What type of product is it? And, how important is usability in that sense?
Don Norman: That’s what I would do if I were at Sun. I would start looking at the factors of the product that are really important to the customer set. Yeah, they want 7x24 reliability, and they want to do hot swapping, and they want to make sure that nothing ever goes down. So you want to make sure that it’s easy to do the hot swapping and easy to bring things up.
Janice Rohn: And that’s exactly what we do.
Richard Anderson: How do you align with marketing…with the decision makers? How do you achieve that alignment?
Janice Rohn: By understanding their goals…understanding the pressures on them. Figure out what is it that they need. If you want to work with marketing, talk about it, not in threatening terms like, "We’re going to be determining what happens with the products," but, "How can we help you do customer visits? How can we help you with field studies? How can we help you?" Offering it as a help, rather then a threat, is one way to do it.
Don Norman: I have found that it’s not that hard to work with marketing. Indeed, you have to be offering yourself as a service. Marketing is about selling the product, and they know what the customers care about regarding buying it. What we’re about is the using of the product, which is different; in fact, it sometimes doesn’t impact the sales of a product. Together, we can be complementary.
Hey, I agree with you, Janice!
Janice Rohn: Wow!
Richard Anderson: Don, earlier you used the term "user experience." What role is played by the words used to label the work or the focus of the work?
Don Norman: I don’t like the term "usability," and I don’t want to be called an "HCI expert." I believe that what’s really important to the people who use our products is much more than whether I can use something, whether I can actually click on the right icon, whether I can call up the right command… What’s important is the entire experience, from when I first hear about the product to purchasing it, to opening the box, to getting it running, to getting service, to maintaining it, to upgrading it. Everything matters: industrial design, graphics design, instructional design, all the usability, the behavioral design… so, I coined the term "user experience" some time ago to try to capture all these aspects. However, the problem is that it requires about six different disciplines to cover itdisciplines that often aren’t comfortable with each other. For example, behavioral designers are very different from graphics designers; they are trained differently, and they don’t always work well together.
Janice Rohn: One of the problems that I see is that "user experience" is sometimes used synonymously with "usability." User experience for a raw technology that’s being pushed out with a thin veneer of an interface for lead adopters does not need the "usability." Have you seen any problems with the use of this term?
Don Norman: One problem is that the term "user experience" has become very popular. Everybody uses it; therefore, they use it blindly. One of the real failures at Apple was when everybody had a User Experience division, though they didn’t understand the concept.
But, you just hit on something that is the theme of a lot of my recent work: early adopters actually don’t need us very much. In the life cycle of a technology, there are different stages. In the early technology-driven cycle, we’re wasting our time, because the adopters are people who care about the technology. They want more, faster, and better technology; they don’t give a damn about usability or industrial design. It’s a waste of our effort to try to produce it.
Janice Rohn: Right. You can actually hurt yourself, because you could be releasing that raw technology, finding out what the lead adopters do with the technology, and then using that information to help cross the chasm to addressing the people who don’t live for technologythe people who only want to achieve their goals regardless of whether it happens to be through the use of technology.
Don Norman: Yeah. Companies like Sun ignored their users in the early life-cycle stages, and I think they were correct to do so. But when you go towards the late adopters, when you go to a mature business, then the world changes. The computer is 50 years old now, so it is mature. And companies are starting to recognize that they need to change. But I don’t believe they can do so. I personally think the computer companies will never get it, because they are run by teenagers who love technology for the sake of technology. The only companies that really understand consumers include Procter & Gamble and Disneyand in the computer business, America Online and Intuit. That’s about all of them.
Janice Rohn: But we’re seeing a lot of merging among those two kinds of organizations. There are going to be a lot of new companies formed out of such combinations.
Don Norman: Well, wait until the culture clashes, such as between Netscape and America Online, become visible. Wow!
Janice Rohn: I’d like to say more about the terminology we use. This is something that hinders us quite a bit. Because we use a lot of different terms to describe ourselves, we really confuse people outside of our field (and oftentimes people inside our field). It would be nice if we could come up with standardized terms and better definitions. So often we’re talking about things and thinking we’re talking about the same things, using terms like "heuristic evaluation" or "contextual inquiry." But everyone has his or her own definition. We really need to standardize our terms.
Richard Anderson: Do the right wordsthe words that should be usedvary by organization?
Don Norman: I don’t know how important the words are, since the words get distorted anyway. When Jakob and I started our company, we didn’t know what to call ourselves. We ended up calling ourselves, "Executive Management Consultants" for the lack of anything better. And what we mostly end up doing is just thatmost of my work is serving on the advisory boards of companies. I spend most of my time working on the business plan, not on the designs, because if you don’t have a good business planif you don’t know what you’re building and for whomthen forget it.
Janice Rohn: I think words are important if you’re trying to convey information and you’re trying to influence people and leverage off of what they already know. If you use the terminology of their field, and if you’re mapping your terminology to theirs, it makes a big difference.
Don Norman: I agree. And that’s why I don’t like the term "usability." People interpret "usability" as "those service people that we’ll call upon when we’re at the end of our cycle to make sure that we don’t have any real flaws," and "Hey, we understand customers, because we have a usability lab. Come, I’ll show it to you." The real purpose of usability labs, as far as I can tell, is PRso you can take visitors through and show them the one-way glass mirrors.
Janice Rohn: Actually, another big benefit of usability labs is that….
Don Norman: Ah, you see, you agree, "and another benefit is…."
Janice Rohn: Well, it is good for customers to understand that the company is doing something about usability…
Don Norman: ...too late in the cycle.
Janice Rohn: No, because we run participatory design sessions in our usability labs. The usability lab is just a physical entity in which you can do all kinds of things.
Don Norman: But you shouldn’t be studying your customers in the lab. You should be studying them in their workplace.
Janice Rohn: We study them in their workplace and we study them in our lab. There are important reasons to bring them in.
But one of the other advantages of a lab is that it serves as a tangible icon of a concept that many people aren’t aware of. We’ve had literally hundreds of people who have seen the labs and then said, "Oh, we should be doing this. We don’t know much about what this means, but we’ll go talk to these people, because there’s this lab." Research that I’ve done with Stephanie Rosenbaum and Judy Humburg reveals that usability labs are powerful for serving as this icon. Also, a lot of people are greatly influenced by direct experience…direct observation of people interacting with their prototypes of their products. Having the ability to have the teams come in and watch it live is really powerful.
"Microsoft pays too much attention to its customers. The result is incomprehensibility."
Don Norman: What I’m afraid of is that you now have this very visible icon of usability, so the company can relax and say, "Hey, we understand it; we have usability labs," though they completely miss the point of how to do human-centered design.
Janice Rohn: No one relaxes if they’re getting beat up about the usability of their products.
Don Norman: But they don’t get beat up.
Look at Microsoft, the world’s biggest company. We often complain about the usability of their products. I just spent three days at Woods Hole, Massachusetts, in a high-level conference with a few high-level Microsoft people who sat there cursing their machines the whole time. (audience laughter) It’s incredible. They have lots of usability labs; in fact a lot of people in the audience are probably Microsoft usability specialists. But Microsoft doesn’t have a clue about how to do good design.
The problem is that Microsoft pays too much attention to its customers. Every feature in a Microsoft product was demanded by a customer. Microsoft is the IRS of the computer industry! Every single thing in those 27 feet of tax regulations the IRS puts outevery one of those was put in for a reason, a sensible reasonsomebody needed that regulation. And the result is incomprehensibility. And that’s what happens when you have user testing facilities and take them seriously.
Janice Rohn: No, no. The problem is that we’re not looking at a free market in this case. If it were a free marketif Microsoft didn’t have a virtual monopoly, then there would be more selective pressures…there would be alternatives. There aren’t the usual full market forces at work. This is an unusual case in that one product owns 90 percent of the market.
Don Norman: But what I’m pointing out is that this is a company that actually does listen to its customers. They really do listen and pay attention. They are concerned about the bug reports and concerned about the negative feedback. But the response is a market-driven responseit’s to add yet more features, more and more and more and more….
Richard Anderson: Will feature bloat end?
Don Norman: I think feature bloat will end when this product ends. The solution to feature bloat from Microsoft’s point of view is "intelligent agents" that walk you through things.
Feature bloat will end, because the PC will end, but it’s going to take 10 years. It will be replaced by invisible computers and by products for which the form factor will matter. We have neglected the form factor. We use one machine for everything, though the physical form could make a dramatic difference in usability. We can produce all the sounds of the musical world with a digital computer. But we make differently shaped musical instruments even when they’re digital, because the piano keyboard is different than the drum pad, which is different than a violin, which is different than…et cetera. That’s how our equipment will change. We’ll have navigational systems in our cars that will be built in and look appropriate. And we’ll have games, and we’ll have cooking devices. We’ll have things that are built inthings we won’t notice as being computers, because the form factor will dominate. Then the feature bloat will go away…until we have several competing markets. Then the market will say, "Gee, I can add three more things that you can’t do." Feature wars are true of all devices: automobiles, running shoes, refrigerators…
Janice Rohn: Eventually, feature bloat is challenged. When something gets to be too complicated, something else comes into the market that’s simple. The Palm Pilot is an example of this; it’s filled this need for a simple interface. The problem with the Palm Pilot will be that it will start getting feature bloat. Then something else will appear that addresses the need for simplicity.
Don Norman: I’ve been studying toasters. I was wondering whether the computer industry would be like the toaster industry if computer devices become simple appliances. Most computer executives tell me, "Oh, we don’t want to be like the toaster industry. There’s no excitement and no money in it. We’ll be selling 20-million-dollar toasters that all look the same, the profit margin will be really low, and we’ll be fighting for shelf space. Is that your dream for the computer market?"
Well, what do you do if you’re making toasters? Indeed, most of the toasters are the same and are about the same price. But there are a few innovative companies. So one thing you can start to do is think about features, and you invent the toaster ovenhey, something that really works differently. Or, you can invent a microwave toaster. Or, you can invent a frozen foods toaster where the container is just the right size to take the frozen food package. Today you can buy $250 toastersdesigner toasters; they’re just like regular toasters, but they’re very pretty. And you can buy special-purpose toasters; I just saw one for sale, again for $250, which has all sorts of controls and features so that you can re-toast breadsee, warming it up the second time is harder than toasting the first time. Those are the things you do if you’re in the appliance industry, and maybe that’s what our industry will someday turn intore-toasting bread.
Richard Anderson:Jakob Nielsen has claimed that 80 percent of Fortune 500 companies will fail, because they won’t make the transition from the visible world to the virtual world.
Janice Rohn: I think he is totally wrong on that.
Don Norman: Well, Jakob also says that 67 percent of statistics are made up! (audience laughter)
Janice Rohn: Fortune 500 companies got there because they are good at adapting, and they are good at making changes and looking at the market, and having the money and resources to see trends and then act on them and oftentimes overpower the smaller companies. So, I think he’s completely wrong on that point.
Don Norman: Very few of today’s Fortune 500 companies were there 50 years ago. The life span isn’t as long as you might think, and things are working more quickly today. Jakob is pointing out that we are in a very rapid transformation of how marketing is done. Marketing is being done today, more and more, through information-rich sourcesthrough the Internet. Notice, by the way, that Jakob never talks about usability; what he does say is that if you have a Web site that infuriates your users and doesn’t provide what they need, they’ll just click somewhere else. That’s the death of a thousand mouse clicks. That’s the way you sell your product. The Web makes it easy, because with the Web, for the first time you can get statistics about how well your users are responding in real-time.
Janice Rohn: Right, but the usability of your e-commerce site is only a fraction of what makes your company succeed or fail. There are quite a number of companies that have been and will be Fortune 500 companies for a long time.
Don Norman: And there are a whole bunch of companies that are Fortune 500 companies today that won’t make it. And there are a lot of companies that don’t exist today that will soon be those big companies, because they will be giving customers what they need, and they will be doing so through different media.
Janice Rohn: Do you think that 80 percent won’t survive?
Don Norman: I don’t know the number, but I think it will be appreciable.
Richard Anderson: There have been several attempts at rating organizations on scales representing the extent to which they attend to user-centered design or usability. With Kate Ehrlich a few years ago, Janice, you described four stages that organizations go through regarding their acceptance of user-centered design. Tell us about that.
Janice Rohn: There are, indeed, various organizational scales out there for rating maturity levels towards usability. Kate and I developed ours several years ago. One of the interesting things is that for larger companies, there’s no single organizational scale rating that you can say applies to the entire company. What you find is that each group is at a different level, and there is quite a range.
To move forward on this, I’ve been working on research that looks at the things that we can do to be most effective and whether there are any organizational attributes that correlate with those things. At this point in the real world, all you can do is have perceptions of what are the most effective things to do. Thus far, we’ve divided the activities into two different categories, one of which we’re calling "organizational approaches"things like high-level founder support, grass-roots efforts, and leveraging related initiatives. We call the second category "usability methods," which are things like lab usability testing and participatory design. We’re asking usability and HCI professionals to rate the effectiveness of these things at influencing their organizationnot influencing product design, but influencing the organization. We’re also getting the demographics of their organizations so that we can see if there are any organizational correlates with the effectiveness ratings and the attributes. We hope to come up with some guidelines based on the things that have been very effective in the past for different types of organizations.
Don Norman: That is all very nice, but there’s a long history of rating organizations by effectiveness, by the Baldridge Quality Award, by this, and by that, and they don’t correlate well with the success of the company. In the end, that isn’t important for the company. What’s important for the company is its profitability and how well it’s meeting its current and future needs of its customers. That isn’t measured by these statistics. That’s why I think it’s really important to concentrate, first of all, on the business plan of the company and to try to understand where the business is today, what the driving issues are in today’s business, where the business will be tomorrow, and what has to be done today to get there. From that, you derive these other measures.
Companies have been burnt. The Baldridge Award for Quality is a well-known example. Everyone was striving hard to get it in the early days, but according to the statistics, the companies that got the quality award have often failed, collapsed.
Janice Rohn: Right, and they poured so much into training and getting the award.
Don Norman: I don’t want the usability maturity ratings to end up in the same way.
Janice Rohn: I don’t see that happening, since one of the important aspects of this is that this is being done in the context of the business goals and the health of the business. The discussions that we’ve had regarding this over the years have always been in the context of "for the health of the business." That’s the number one goal.
Don Norman: But, is it accepted by the top executives? Does it make it to the annual report?
Janice Rohn: In some companies, yes.
Don Norman: Like?
Janice Rohn: Well, Microsoft lists it in their annual report.
Don Norman: That’s right, Microsoft usability…hmm…that shows the irrelevance, doesn’t it?
Janice Rohn: How so? I don’t see that it shows irrelevance. Earlier you said that it’s not that Microsoft hasn’t worked on usability and hasn’t made improvements in usability.
Don Norman: But their usability people are not driving the company. The usability people at Microsoft are not the leaders of Microsoft, and until that happens, we’re going to have some problems.
Janice Rohn: There are some very high-ranking usability people in Microsoft, and Microsoft does pay a lot of attention.
Don Norman: It’s the wrong attention. Look at their products. Look at how much money they spend.
We just got an Outlook server at my company. What do we have to do? Though we are pretty sophisticated in computers, we had to hire a $150-an-hour technician to install our server. And so far, we have paid him 2 or 3 thousand dollars, and it doesn’t work very well yet, but they are working on it. Microsoft has started all these wonderful industries of technicians and support staff and classes. You gotta have Microsoft certified technicians to work on it, you know. Big business for Microsoft! It’s a horrible state of the world.
Janice Rohn: It is a big business. But what you’re really getting at is the fact that there are problemshuge problemswith doing product development.
Don Norman: I’m saying these are crappy products from a usability point of view. If it takes a full-time technician who’s taken six Microsoft courses just to install one product on their serverthat’s crazy.
Janice Rohn: And what I’m saying is that it’s not for lack of good usability people in companies. I’m saying that it’s oftentimes for lack of good project management and an understanding of priorities, resources, and trade-offs. The problem is that there are all these orthogonal priorities.
Don Norman: But I’m saying that you’re describing the problem exactly. We’ve got lots of usability people, but they’re in the wrong places in the company. The correct places are the places you just mentioned.
Janice Rohn: I don’t think that the correct deduction here is that they’re in the wrong places in the company. Yes, if you want to put a usability person in charge (who happens to be a great project manager)...
"If [usability people] want to be successful, they ought to take business administration classes."
Don Norman: ...yes, I do.
Janice Rohn: Then that’s a great thing.
Don Norman: I want the usability people to be the project managers.
Janice Rohn: Then that’s a great thing. And oftentimes we see that usability people become ad hoc project managers, because they do often have a better sense for being the glue among all the various groups.
Don Norman: But why aren’t they real project managers?
Richard Anderson:What is your answer to that question, Don?
Don Norman: Because they don’t have the training. I gave a keynote address at the Human Factors and Ergonomics Society conference in which I scolded them for the same things I’m scolding this audience about, saying, "I’m tired of hearing you complain, I want you to do something about it. Look at the engineering profession. How come they got to be managers?" A guy came up to me afterward and said, "Well, you know the engineers got to be managers because they took business administration classes." Well yes, that’s the point. How many of you [in the audience] have taken business administration classes? If you want to be successful, you ought to take business administration classes. That’s how you get your way up there; that’s how you become the product managers, and eventually the managers of the teams, and eventually the vice president of your product lineand that’s when you can start putting the right structures in place. Janice described the right structures, but we don’t have our people in those structures.
Janice Rohn: I totally agree that if that’s an HCI person’s goal, then taking those business classes is the right thing to do. There are other people who are very happy with being great designers, and taking business classes is not necessarily the right thing for them.
Don Norman: They’re very happy at being great designers and they’re always complaining that nobody’s listening to them.
Janice Rohn: Great designers can be more influential by understanding the technology and being able to implement things in that technology. Then they will not be fooled by the developers when the developers say, "No, we can’t do that."
Don Norman: But they’ll always be a second-class citizen. They’ll always be called on as a resource.
Richard Anderson:Elaborate, Don. Recently via the utest mailing list, you talked about the distinction between being a resource and being a peer. You’re referring to that here. What’s the difference?
Don Norman: An extreme oversimplification that a friend of mine made is that there are two kinds of people in organizationsthere are peers and there are resources. Resources are like usability consultantswe go out and we hire them. We’ll hire a consultant or we’ll have a little section that does usability and think of it as a service organization. We call upon them when we need them to do their thing, and then we go off and do the important stuff. That’s very different than peers, where a peer is somebody I talk to and discuss my problems with, and who helps to decide upon the course of action. As you get higher and higher in the organization, this becomes more of an issue. The executive staff talks to the executive staff, and they have beneath them all this organization, which are their resources that they deploy. But the big decisions are being made among peers. And it’s really important to advance in the world to be thought of as peers. But our consultants aren’tour consultants are mainly resources. We bring in the resources when we need them and get rid of them when we don’t need them anymore. Our usability labs are resources that we call in when we think we have some trouble with usability. We go and spend a few hours and we worry about the budget. But we don’t take it seriously, we only take peers seriously. And you only get to be peers if you speak the right language and if you’re making a contribution at the level the company cares about…which is profitability. In the academic world it’s the same waythe usability people are second-rate citizens. That is, in the science disciplines like psychology or computer science, the practical side is given second-rate attention. So, if you want to make a major contribution in academics, you too have to learn to become peers. It’s possible, but it’s not the way it is today.
Janice Rohn: It is possible. In fact, we have had examples at Sun where usability people were brought in for strategic deals to help determine whether Sun should buy a company, whether Sun should license a product or things like that. So, it does happen in some cases today.
Don Norman: That’s good.
Richard Anderson:Do we understand what user-centered or human-centered design is?
Janice Rohn:Well, we certainly don’t all have a common definition of what it is. Again, that’s one of the problems with our field…we don’t have a common idea about the different steps and activities to do and exactly what they mean. Also, we rarely get the opportunity to implement a full user-centered design cycle. So, a lot of us don’t have all that much practice with it. So one of the problems we’re seeing is that occasionally a group will come in at the very beginning and say, "Yes, you can do that full cycle," and we’ll look around at each other and say, "Wow, this is so unusual. Let’s seewhat do we do?"
Don Norman: Actually, we are better off than that. We probably fight among ourselves about what we mean by user-centered design or human-centered design. But if you step up a bit and look from a higher altitude at what we do, we’re actually in quite a bit of agreement. We all really share much the same kind of structure. We know that you have to have a multidisciplinary team. We know you have to work closely with engineering and marketing and manufacturing and sales. We know that you have to be there from the very beginning. And we sort of know the steps that have to be done. There are a number of different schemes about (and a number of different books and a number of different methods), but if you look at them, they’re really very, very similar.
Janice Rohn: There is a lot of overlap and there are known segments of things to do like customer research and design and evaluation.
Richard Anderson: You both have said that we need more multidisciplinary teams. How do you get those multiple disciplines to collaborate?
Don Norman: Painfully. You get an anthropologist in the same room as an experimental psychologist, in the same room as a graphics artist, in the same room as an industrial designer, and in the same room as a hard-core engineerWow! At Apple, it really did work well after a few cycleswhen people got to respect each other’s discipline and to realize that each person was an expert in their own particular discipline. It did work well. It has to do with the personalities of the people as much anything else. Sometimes it works really badly, and that’s not because of the disciplinesthat’s because of the people.
Janice Rohn: Right, I think that face-to-face time and working together have a big impact. We’ve had a lot of success with participatory design methods where we bring the usability folks and the development team and the customers together in a room, and have other members of the team watching on the other side of a one-way mirror but being involved also. Teams have come back to us saying what a big impact that’s made on them and how much they’ve saved in the development cycle because of that, because they short-circuited all the running around and were able to address these things right from the start. And they’ve gotten to know people from the other disciplines and have been able to set up relationships that are ongoing.
Don Norman: I’m a big fan of having the people on one team. I don’t care whether it’s a matrix organization or whether it’s cross-disciplinary functionally. You have to be working together. The HCI person and the engineer and the marketing person have to have desks near each other. At 3:00 a.m. when the engineer gets some bright idea, the HCI person is then right there to help and to do quick tests and do quick designing. What doesn’t work is when you have walls…when you have the usability section and the industrial design section and so forthso each division passes it on to the other people and then gets it back. There’s not enough communication that waythere’s no bonding, and people end up fighting as opposed to collaborating.
Richard Anderson: Janice, you’ve played a large role in the development of the Usability Professionals Association. Why is UPA distinct from SIGCHI?
Janice Rohn: One of the benefits of having UPA is in having very practical, hands-on, immediately applicable information so people can learn how to do things and learn from examples in other companies, and then immediately take that back to their company and apply it. There’s a heavy emphasis on the practitioner and also a heavy emphasis on the usability evaluation and customer research side of the user-centered design cycle. Most people who are involved with UPA certainly enjoy the different aspects that the different conferences offer. The advantage of both CHI and UPA is that you can immerse yourself in learning about what’s happening…what’s the latest thing, both in the more formal settings of the sessions and the informal settings of networking. One of the problems with both organizations is that they tend to be a little bit insular; oftentimes, we are preaching to the choir. I think we need to get out more to other conferences and other fields in business, to CIO forums, to software development and engineering, to industrial design getting the message out there, doing more presentations at these other conferences so that we don’t have as many problems within our organizations so that people understand, "Okay, this is how this fits into what I’m doing and this is how I can apply it back at my company."
Don Norman: But it’s not just about giving presentations to other organizations, it’s also about learning from them.
Janice Rohn: Exactly.
Don Norman: You know, industry’s perception of CHI is that it’s too academic; I hear that complaint a lot. It has a heavy academic influence even among those who are in industry, because the biggest industrial component of CHI is research industry. Usually, the research division of a company is more similar to academic institutions than to the product divisions of the same company.
Janice Rohn: Right, and that’s one of the reasons that UPA came about. There was a need for being able to address things like how you do participant recruiting and how you do a particular type of field study, and being able to take that information and directly apply it. A lot of times we were seeing people submitting perfectly great informational practitioner papers and presentations to CHI and getting them turned down, and then seeing a lot of emphasis on the academic side. That’s one of the needs that UPA fulfills.
Don Norman: CHI might want to review its acceptance rules and have different panels and different review committees for different kinds of papers, because most papers come from academics or the research side of industry. A lot of the practitioners don’t have time to write papers. When I am in industry full-time, I just don’t have any time to sit back and reflect and write about what happened. I’m on to the next thing. It’s not the secrecy; when I’m all finished and I can talk about it, I’m on the next project. And on top of that, most people in industry don’t know the academic rules of writing papers, so they get rejected by CHI over and over and over again. I’ve been on CHI committees. I’ve been on the review panels. I’ve seen them get rejectedI’ve probably rejected them myself. We need different standards and perhaps we need a different track within CHI. (audience applause)
Richard Anderson: During an interview session earlier at this conference, I asked Clement Mok and Jakob Nielsen what role humility should play in the world of design.
Don Norman: Those are interesting people to ask! (audience laughter)
Janice Rohn: And, by the way, Jakob quoted "Jakob’s Law" during that answer!
Richard Anderson: What role should humility play in changing an organizationits process…its structure?
Janice Rohn: One of the roles it should play is in knowing that we don’t know a lot of things and that we’ve got to learn quite a bit to be more influential. It’s not just, "We have the right answerHCI is the most important thing, and we need to evangelize that and let the rest of the world know that it is the critical thing." That’s wrong. We really need to learn about the business goals and the needs of all the groups in the different functional areas that we work with, and learn how we can better support and leverage what we know is good to do and, at the same time, help them with what they need to do.
Don Norman: That’s a good answer and I agree with that. This is a funny business, because the world I inhabit is filled with incredibly strong egos, and you don’t survive unless you also have a strong ego. The whole computer business is actually dominated by the egos of the players. It’s really important that Larry Ellison really intensely dislikes and is jealous of Bill Gates; that makes a big difference in the way the industry plays out. You’re not going to get up there unless you actually have a strong ego and you stand up for what you believe and make the point.
But the other point, which is really what Janice is saying, is that we have to have humility about the knowledge of the field. Although you can fight for what you want, I’ve always believed it was important to fight honestly and to admit when we don’t know. Our field…no field should dominate. We’re all making trade-offs, so we all are playing the game equally. Sometimes engineering will have to weaken its casesometimes usability willsometimes marketing. I’m involved in a discussion on the CHI-consultants mailing list, and I’ve just been accused of being very arrogant and pompous in the discussionand I probably was. In this group, people ask what I think are intelligent and sensible questions for which we have no data. And that’s all right; they’re good questions, and we don’t happen to have the data…we don’t happen to know the real answers. But this hasn’t stopped people from pontificating upon what the correct design solution ought to be. So I’ve come in a couple of times trying to say we need data that you can’t just give an answer out… you can’t make it up as if we knew. And some people respond, "Well, if you think we need data, why don’t you provide it?" No, no, my point is that if you don’t know, you should say you don’t know. And if you don’t know and you say you don’t know, then you can say, "Since we don’t know and since we don’t have time to collect the data, here’s what we should do…I recommend the following…" At least you’re stating very plainly that this is a guess. It’s an educated guessit’s an intelligent guessit’s based on as much as we know, but it’s a guess, and it could very well be wrong. I haven’t seen that kind of humility. In my pompous, arrogant way, I’ve been asking for more humility.
Janice Rohn: By saying you don’t know, I think you build up your credibility. One of the problems with our field is that oftentimes people will give the impression that they have the design answer and then when it turns out to be not the right answer, then it’s, "Oh, okay, well, we need to change it." If you say up front that this is an idea and we’re going to evaluate this to see if this is the right way to go, then you’re setting yourself up to have a lot more credibility.
Don Norman: You know, engineers are wrong all the time and it’s no big deal: "Oh yeah, that didn’t work, so let me try something else." We ought to learn to be wrong more. It’s okay to be wrong. One of the problems with our discipline is that our methods are too precise and take too long and are not general enough. We need methods that are faster and that are approximate and that are wrong sometimes. If it’s an intelligent thing, it’s not going to be wrong too much. Humility comes into this again. If you know you’re making quick decisions then you also know that you’re going to be wrong. So don’t pretend to be absolutely correct.
Janice Rohn: Right, and it’s important to be explicit about the parameters and the constraints on your decision.
Don Norman: But you have to be cocky and arrogant and sure of yourself, because you don’t want the engineering team to think you’re making it up. Because they’ll say, "Well, hell, I can make it up just as well as you can." No, no, we are making it up from a very important base of knowledge. So, my guesses are actually pretty informed and close, even if they’re wrong. And the engineer’s guesses are ill informed. There’s a difference.
Janice Rohn: Right.
Richard Anderson: Don, who has influenced you most? Who are the people you most admire?
Don Norman: Georg von Bekesy. Most of you have probably never heard of him, but he was actually a really important influence in my early career. He worked in the basement of Memorial Hall at Harvard University and studied the earall he did was study the ear. He did a good job at studying the ear; he got a Nobel Prize for studying the ear. In his side life, he collected art, especially prehistoric and Colombian art, and was a world authority on that, too. What he always pointed out was that the most important thing is to ask the right question. That really has guided my life from then on. I’ve always thought that the most important thing that I can do is try to understand what the correct question is. I never solve the problem that I’m asked to solve; I always ask why is that a problem, and then ask, then why is that a problem? I really think that trying to find the correct problem…to ask the correct question, is the most important contribution.
Richard Anderson: Janice, same question.
Janice Rohn: I would have to say my mother, because she raised me to believe that I was capable of anything and there weren’t any limits on what I could do. I think it is especially important for little girls to have that imparted on them…that they can excel in math and science and whatever they want to do. So, I think that she had the biggest influence on me.
Most people have had someinfluence on me. As with Don, the most important impact has come from people who have influenced how I think about things and how I question thingsmore giving me the tools to approach life than giving particular content.
Richard Anderson: Don, you have worked within organizations like Apple and HP as an employee, and from the outside of organizations as a consultant. Which makes it easier for you to be effective in influencing an organization?
Don Norman: Working on your own is incredibly liberatingfree, finally, of the tyranny of big companies. It’s amazing. HP was really a nice company and I was treated nicely, but as far as I can tell, I had zero impact within that organization. Guess who some of our biggest clients are todayHP and Sun. HP and Sun have hired Jakob and me. It’s really quite amazing. You actually have more impact coming in from outside than you do from inside. It’s strange, but true, and probably most in the audience have experienced that.
Janice Rohn: I would argue that you have more impact coming from inside than outside, because there are a large number of HCI people within Sun. The number of HCI people at Sun has been growing very rapidly over the years and they have made a huge impact internal to Sun, much more than any external person.
"Marketing…should be our biggest ally, because we all have the customers’ desires at heart." (The audience was then invited to pose a few questions.) Audience member: I wonder if Don could actually answer the question that Richard asked first, because I think Don answered a different question. Maybe it was an ap