Kraig Finstad, Wei Xu, Shibani Kapoor, Sri Canakapalli, John Gladding
End users of enterprise software are in a tough spot. While traditional desktop software and websites are increasingly designed with their needs in mind, unique challenges arise with enterprise software. Whether working with a specialized accounting system or a general-use intranet portal, users find themselves confronted with systems that are complex, difficult to learn and use, and dissatisfying.
There are usually sound reasons for an information technology department to implement enterprise software, as opposed to building customized specialty applications. Enterprise software is attractive since total cost of ownership (TCO) calculations support a financial advantage. As Daniel Rosenberg stated, “In the enterprise software market, if the customers of your product are not successful you will go out of business…TCO is related to the real value the product provides .” Additionally, the integration possible with a comprehensive one-supplier solution helps streamline business processes. For example, the purchasing system’s database is accessible to the accounting system, and sales can be linked to inventory. Ideally, the end user is empowered and becomes more efficient and productive, but in the real world it’s not so easy.
The power and high-level efficiency of enterprise software often bring complexity and a lack of flexibility. One of the primary challenges in the world of purchase-and-install off-the-shelf (OTS) solutions is that end users are further removed from the creators of the software than ever before. Instead of user-centered design (UCD) professionals advocating on the part of the users to internal project-development teams, the challenge becomes advocacy to a completely separate entity in the supplier. For many, it’s a completely new approach that requires careful consideration. This article discusses the problems with OTS solutions and proposes ways to successfully implement them.
Case Study: Bypassing User Research
Intel had a homegrown application for the distribution of training material, and significant usability work went into the development process. The tool did not have major usability issues, but in 2005 Intel wanted to upgrade the mainframe back-end system. An OTS solution was selected to replace the homegrown tool. After the supplier was chosen, usability resources were requested. The project usability expert conducted a usability study on the unmodified (“vanilla”) solution. Overall there were a lot of usability issues, and addressing them proved difficult. For instance, the enterprise could not revise some confusing labels without modification. The usability expert conducted a risk assessment for project management, which documented all the risks in terms of level (high, medium, low). The usability expert also proposed modifications in order to address some significant user experience (UX) issues such as the search function. The supplier was willing to do some quick enhancements. The supplier and the usability expert also discussed detailed user interface design. Intel project management eventually approved the modification approach.
The program, however, was cancelled due to a budget cut. Many key members left the program, and all work ceased. One year later, the program was resumed, and OTS applications were released without any follow-up UX work. The new release did not factor in previous usability work, and significant negative feedback from the enterprise users rolled in.
Companies that have been around for a while have homegrown applications and tools running their various information and business needs. Often, these systems overlap in scope, since different groups in the company, unaware of each other’s work, develop them. Thus, multiple applications that offer similar solutions may be found across the corporation. While specific groups using such solutions may be content with what they are getting, the company at large suffers in efficiency due to duplication of systems. In order to eliminate such redundancy in the corporate environment, companies may want to integrate the applications into a single enterprisewide capability by implementing OTS solutions.
While OTS solutions contribute to eliminating or reducing duplicate applications and tools, other issues come into play: those that relate to the users, the enterprise, and the supplier (the OTS solution provider). However, the issues are often interconnected, and it’s important to view them as such (see the accompanying figure).
Enterprise Issues. OTS solutions are developed to fit many business needs and are designed to fit the most general business case. By definition, OTS are standard one-size-fits-all solutions that cannot be customized. However, each enterprise has specific business needs and a specific company culture, with unique characteristics such as the vernacular of people in the workplace. Suppliers of OTS solutions often fail to take these things into account. Thus, while OTS solutions are intended for enterprise-wide application, there is always the risk of not providing certain capabilities that the company requires. On the other hand, OTS solutions may also introduce certain capabilities that the company doesn’t need, leading to unnecessary cost and maintenance. As a result, new and modified business processes can emerge, ones that may not mesh with existing business processes. And there is always a transition period for a new solution. Due to the generally long implementation and integration period of an OTS solution, the technology may become obsoleteadopting new technologies to make the solution more efficient and effective may require too much time. At the end of the project cycle, testing opportunities often focus on the system design and performance, leaving no room to improve or enhance the end-user input on usability and UX (if such input exists at all). End-user needs are at best addressed through mitigation tactics such as training, help systems, and communication.
One of the primary challenges of off-the-shelf solutions is that end users are further removed from the creators of the software ... Instead of user-centered design professionals advocating on the part of the users to internal project- development teams, the challenge becomes advocacy to the supplier.
User Issues. Suppliers often develop OTS solutions for the general user without considering the characteristics of individual users, their roles, their culture, geographical differences, or their work environments. Similarly, when enterprises select a supplier, they do not always take their users’ needs into account. Companies fail to study the UX gaps in the current process and workarounds that users may have adopted. As a result, they do not understand what works well in the as-is process, what users are accustomed to, and where they have difficulties. Other user characteristics also go ignored, such as how users access the tools in their own work environments and their cultural differences (particularly in a global company).
Moreover, enterprises seldom develop a strong partnership with the supplier to address the specific needs of their users and solve UX issues. There is no collaboration with the enterprise and supplier to develop mitigation strategies that would make it easy for users to adopt the OTS solution while transitioning from the original system.
Supplier Issues. There are only a handful of large suppliers dominating the market in OTS enterprise solutions. With such minimal competition, the enterprises have fewer options and could end up choosing a solution that may not be a good fit for the company as a whole. Supplier responsiveness to enterprise requests may not be quick and flexible enough, as the suppliers who serve multiple clients are concerned with time to market and tend to focus on overall costs. Only recently has providing a better UX become a competitive differentiator.
User Experience Design. The sheer amount of variables, issues, and mitigations in these situations warrant a comprehensive approach. The user experience design (UXD) approach incorporates not only usability but also program management, training, and transition change management (TCM), among others. For a more thorough treatment of these themes, see Andrew Sweany and Marla Gomez’s 2007 paper on UXD and approaches to issue mitigation .
Case Study: Internal User Experience Design
Phase 1 of an Intel employee self-service application project was kicked off in 2003. This phase was part of a program to upgrade a large enterprise back-end database system. The system supplier was entering the web-based, front-end application area and had just released its first generation web-based solutions in the business domain. The supplier was offering a free web-based, front-end package as part of the whole system upgrade package. After the first phase was released in fall 2004, distinctly negative feedback came in from the end users.
- Supplier-side issues. Since the web-based package was the first-generation, front-end solution built by the supplier, the supplier had not done enough usability work on the design. The product was delivered with a lot of usability issues and inflexible business processes and configuration capabilities.
- Enterprise-side issues. Due to the cost-sensitive environment at the time, and a prior vanilla backend system upgrade, the program adopted a rigorous no-modification approach for the front-end solutions as well.
Due to the negative feedback, a Phase 2 program was kicked off in early 2005. Intel IT’s usability group was engaged to support this effort. Post-release analysis of Phase 1 clearly indicated that although there were many usability issues, the usability tool issues accounted for only a small proportion (11 percent) of the total issues identified. The overall identified issues were distributed across all aspects, including system, tool, configuration, performance, or user help. Obviously, if we fixed just some of these issues, we still wouldn’t be able to significantly enhance the overall UX. Furthermore, the traditional user centered design (UCD) approach, which focuses mainly on tool usability, had difficulties in meeting these challenges. As a result, the usability expert proposed a UXD approach to Phase 2, which was successfully released in the third quarter of 2006. The overall UX improvement for one major module was ranked ninth best among 102 U.S. corporations in the 2007 Top Employer Web Benchmark by Potentialpark Communications.
A complete UXD approach consists of the following:
- Form a UX team. The UX team includes representatives from different functional teams across quality assurance, business process, TCM, training/ online help, and user support. The usability expert can serve as facilitator of the UXD process and maintain a partnership with other teams by working with these representatives on the UX team. Each of those UX team members owns the planning and execution of the UX component corresponding to their functional area.
- Include the usability expert with program management. In order to keep track of the UXD progress and increase visibility of UXD work, the usability expert should be a member of program management. This differs from traditional UCD, wherein a usability expert is typically embedded within the program as a member of a sub-team.
- Define a UX scorecard and the tracking process. The UX scorecard should define not only success criteria for tool usability design, but also other aspects of UX. A tracking process needs to be defined across the lifecycle of a program, which will enable program management to closely monitor the progress of UXD and take any necessary actions. Besides, the UX scorecard and tracking process also increase the overall awareness of a UXD culture within the program.
- Follow a UX data-driven approach for optimizing business processes. Gather real end-user data; for example, through iterative usability testing, as business processes evolve. For instance, a vanilla solution may require three sub-business processes (legal process for approval) in addition to the existing business process. Testing the impact of these changes can help determine the right trade-off between the UX and business processes.
- Strategically collaborate with the supplier. Leverage usability test data to convince the supplier to fix high-priority usability issues. This is especially relevant in the case of OTS solutions. Getting recommendations embedded into purchased solutions avoids many of the costs and inefficiencies associated with enterprise software.
Case Study: Supplier Influence
One of Intel’s early forays into enterprise software involved a web-based purchasing system for general use. Preliminary usability testing demonstrated considerable difficulty for users, high error rates, and low satisfaction ratings. Several iterations of the system were developed and put through usability testing, and distinct improvements emerged in the Intel-customized version. At the user-interface level, as much data entry as possible was consolidated into a main screen, and controls and labels were altered to match user expectations. At the user-interaction level, warnings were put in place to prevent users from mistakenly losing their data, for instance by navigating away from the system. An even more advanced version of the customized system took into account the issues most important to users (according to tech-support calls and usability testing). Contextual help links were placed directly on the screen at the most problematic areas as a supplement to the global help from the supplier. As a result, the number of tech-support calls on those issues dropped substantially.
Unfortunately, Intel has had to move away from this internal customization model. Mitigations to UX issues often take the form of issue-focused training, help, and transition efforts. In the long term, a mutually beneficial supplier/customer UX relationship was developed that can provide improvements directly to the end user. The supplier receives valuable usability research data, recommendations, and rare insights into their system’s end users. The customer company ultimately benefits from improved systems that accommodate its end users’ needs.
Specifically, the supplier’s system has made the procurement process more straightforward, complete with a single-page approach, clearer controls, and a more efficient screen workflow. A warning dialog has been added to prevent accidental data loss, an improvement that other supplier systems have adopted. Prototypes of future systems show a more context-aware approach to system help, so the user doesn’t have to seek help as a separate (and disruptive) task.
The comprehensive UXD approach provides a solid understanding of enterprise-software end-user needs beyond traditional UCD. In the cases where internal teams handle system development, the end-user and development-feedback loop is entirely self-contained, and improvements may be rapid but expensive. All that has really shifted in the OTS UX approach is an additional part of the loop: the supplier of the solution. This is a slower path to successful influence, but it is ultimately the most cost-effective method. There is lower TCO in terms of upgrading and maintenance, and also an increase in end-user productivity that comes with more usable systems. Moreover, there is the advantage of not only reuse, where the customer no longer needs to remodify a system due to included improvements, but also of “pre-use,” where UX improvements spread beyond the original system to other supplier offerings for future purchase. In the long run, the interaction between external development and internal research and implementation means a productive and efficient experience for end users.
As UX professionals, we must add this process to our toolbox. With more and more companies engaging at this level, the cross-sharing of information in communities and interest groups becomes possible. Even though the larger sphere of influence is different, the fundamentals and end results are the same. If we continue working to understand our users and business, then pass that information on to external suppliers, OTS enterprise software can be powerful, flexible, and easy to use.
All of the authors are human factors engineers in the information technology department at Intel Corporation. The HFE department consists of 16 human factors engineers under the direction of Linda Wooding.
Kraig Finstad holds a Ph.D. in experimental psychology; his interests include problem solving, research methodology, and usability metrics.
Wei Xu holds a M.S. in computer science and a Ph.D. in experimental psychology. His research interests include human computer interaction, cognitive engineering, and aviation human factors. Wei has published numerous academic papers in these areas.
Shibani Kapoor holds a M.S. in information specializing in human computer interaction. Her area of research interest is in the usability variances of different cultures and designing evaluation methodologies that take them into account.
Sri Canakapalli holds a Ph.D. in both business management and human factors psychology. He is particularly interested in enriching the experience of humans when they interact with things. He has three patents pending relating to human computer interaction.
John Gladding holds a B.A. in communication studies. His background is in web and interface design, where he has designed and implemented dozens of sites inside and outside of Intel. He also owns and manages a Sacramento-based portal. The Tomato Pages Network, http://www.tomatopages.com.
Sidebar: Key Takeaways
- Develop a partnership with enterprises and adopt an enterprise-participate design approach (for example, conduct field studies in the enterprises’ workplace).
- Adopt the UXD approach; select target end users for usability studies.
- Set up a user group that includes enterprise members for regular communication of new design and feedback gathering from enterprises.
- Influence their design.
- Consider UX when selecting a supplier to foster suppliers’ UX culture.
- Build an in-house UX team or leverage external consulting.
- Provide post-release feedback to supplier on time and influence improvements of future releases.
- Set up an industry consortium and develop industry standards to foster an industrywide UX culture.
©2009 ACM 1072-5220/09/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2009 ACM, Inc.