XVIII.6 November + December 2011
Page: 60
Digital Citation

Tragic errors

Ben Shneiderman

back to top 

The Obama administration's healthcare reform legislation and its $19 billion stimulus support for Electronic Health Records (EHRs) are beginning to have a transformative effect on U.S. healthcare. An important goal will be to use EHRs to reduce the estimated 98,000 deaths and millions of injuries per year from medical errors [1].

Many medical errors are never reported or even recognized; only a small number are analyzed by internal hospital or clinic review boards. Peer review boards typically keep their analyses private, and only a small number of cases per year receive public discussion [2]. This dramatically limits the ability of healthcare managers and designers to learn from errors, make adjustments that improve performance, or develop more effective workflows. Improved and public reporting would also enable patients to make more informed personal healthcare decisions. High-quality data about the sources of medical errors would support evidence-based consensus on national policies and guidance for local strategies that should lead to better outcomes and lower costs.

An important step toward better reporting and tracking of medical errors will be improved EHR user-interface designs that offer healthcare providers shorter learning times, faster performance, and lower interface error rates. A second step should be agreements on user interface consistency (similar formats for common medical data values such as blood pressure, consistent placement of these common values on the screen, guidelines for choice of colors and management of alerts, etc.) and data interoperability among the 100-plus developers of EHRs. Such guidelines for consistency and data sharing, common in other industries, would allow healthcare professionals who work at more than one location to do their jobs more efficiently and safely.

Sadly a long history of EHR developers seeking competitive advantages with few motivations to support common guidelines for user interfaces and data interoperability has left the U.S. with a patchwork of systems that hampers healthcare, frustrates health professionals, and may harm patients. In the U.K., Microsoft's extensive Common User Interface guidelines are being adapted for use by the National Health Service. In other countries a unified system has been customary practice from the beginning.

Pressuring U.S. private corporations to prepare and publish their user interface guidelines would encourage discussion among usability practitioners, academics, medical professionals, and patient advocacy groups. Public availability of user interface guidelines has been standard practice at Apple, Microsoft, and other leading corporations, as well as at government agencies such as the FAA, NASA, the Department of Defense, and the FDA. An especially well-designed and informative document is the National Cancer Institute's Research-based Web Design and Usability Guidelines that gives prescriptive guidance based on more than 300 empirical studies [3]. Besides these examples of guidelines documents for prominent consumer user interfaces and government contracts, there are also corporate guidelines for many products, such as bank machines, automobiles, and scientific instruments.

While corporations understandably have concerns about inappropriate government-mandated standards that are poorly conceived, vaguely written, too limiting of innovation, or too slow to be updated, participants in industry-government partnerships have produced valuable technology advice for many industries. These guidelines and standards have reduced errors, shortened learning times, improved human performance, and sped development while reducing manufacturing and usage costs. The benefits to all were obvious when the automobile industry standardized the position of brake and gas pedals and the height of bumpers.

Other positive examples include current work by the Federal Election Commission on voting machines, the Department of Energy on programmable thermostats, and the Food and Drug Administration on medical devices. Beyond user interfaces, the success of government-industry partnerships is visible in the work of the National Highway Transportation Agency on cars, Federal Highway Authority on highways, Federal Trade Commission on banking privacy statements, and the Consumer Products Safety Commission's work on hundreds of products. These efforts may be imperfect and sometimes controversial, but the public benefits are widely accepted. When public safety is involved, government collaboration and coordination has shown large payoffs.

back to top  How Can Industry and Government Work Together?

Key EHR vendors report ambitious internal programs of usability testing and responsiveness to customer requests, but their license agreements limit external review and public discussion. They also oppose government certification of EHRs, claiming it would limit innovation and somehow undermine patient safety. Government certification of new passenger aircraft has long been accepted, but EHR providers claim their user interface designs should be reviewed only by potential customers, allowing the marketplace to select the best. While some large medical providers can perform competent reviews, many small clinics and primary care providers do not have the capacity to perform adequate reviews, so they are swayed by marketing, pricing, or compatibility with certain hospitals, clinics, or labs.

Guidelines for user interfaces and public reporting of usability testing for errors would lay the foundation for even bolder changes in the way data is handled, EHRs are integrated into practice, and health information systems are evaluated.

A promising direction is the effort by the National Institute of Standards and Technology [4], which was presented to diverse stakeholders at a June 2011 meeting [5]. The proposal for an EHR Usability Evaluation Protocol focuses on reducing certain medical errors of commission and omission:

  • Wrong patient or treatment actions: Actions with potentially harmful consequences are performed for one patient that were intended for another patient because patient identifiers were not easily selectable or clearly displayed (commission). A patient is not informed of the need for treatment because patient identifiers were not easily selectable or clearly displayed (omission).
  • Wrong medication event: A patient receives the wrong medication, dose, or route because of confusing display formats for patient identifiers or medications (commission).
  • Delay in treatment event: A patient receives a significant delay in the provision of critical care activities due to poor design decisions—for example, not providing an appropriate alert for an abnormal lab test result [6] (omission).
  • Unintended or improper care event: A patient receives unintended care actions due to actions taken to test software, train users, or demonstrate software to potential customers (commission).
  • Sequence error (a subclass of errors of commission): A provider performs some task, or step in a task, out of sequence. For example, a patient with a fever may have a blood culture followed by intravenous antibiotics. If antibiotics are given prior to the blood culture, the sensitivity of the blood culture decreases dramatically. EHRs that support providers in the order of events are more likely to reduce sequence errors.

Other kinds of errors, such as incorrect data entry or missing data, could enrich the list. The usability testing protocols would be developed jointly with industry representatives and testing would be conducted by the companies. Each report would be presented in a Common Industry Format that would enable potential users and others to compare the results across products [7].

Guidelines for user interfaces and public reporting of usability testing for errors would lay the foundation for even bolder changes in the way data is handled, EHRs are integrated into practice, and health information systems are evaluated. While the Open Government Initiative [8] has led to increased transparency of government activities, open EHR design has yet to gain momentum. Protecting individual privacy remains vital, but more open reporting on EHR usage could lead to continuously improving care and lower costs.

back to top  Independent Oversight

Independent oversight panels, convened by healthcare providers, professional organizations, and government agencies, would specify improvements in information technologies and processes for software developers, hospitals, and labs. Then they would assess the impact, measuring progress and developing best practices that could be shared across the health information technology industry [9].

Independent oversight is a well-established practice in most industries, from aviation to construction to accounting services. The FAA closely monitors airlines, pilots, and air-traffic controllers, leading to an admirable safety record. Local zoning commissions review building proposals, inspect new buildings, and track failures. Professional societies, Consumer Reports, Underwriters Laboratory, and consumer-oriented groups evaluate performance of many products and services.

Independent oversight can be done in advance of activities, such as prospective reviews for building construction plans or state approval of new universities' degree programs. Oversight can be continuous, such as in drug manufacturing or food processing facilities. The most common form of oversight is retrospective, such as corporate annual audits or aviation safety annual reviews. Retrospective reviews when major accidents occur also galvanize Congressional involvement, journalistic investigations, and public interest.

Of course, the degree of independence matters. Organizations can elicit oversight from internal boards, friendly colleagues, or sympathetic outsiders, but getting more effective independent oversight requires knowledgeable professionals who declare conflicts of interest. Independent oversight groups must have the power to get accurate and complete answers and to publish their findings so they have an impact. Internal reports that are little more than suggestions rarely produce substantive change.

Impediments to openness include the traditional protectiveness of some physicians for their relationships with their patients, fear of malpractice suits, and reluctance to admit errors. However, there is a growing awareness that openness might lead to more careful work with fewer errors. Genuine apologies for errors might reduce malpractice suits and more data-driven discussions among professionals could yield improved standard practices. A 2011 report from the University of Michigan Health System describes startling outcomes for their practice of "disclosure with offer" in which patients receive an apology for errors and an offer of compensation [10]. In the five years since implementation, the average monthly claims decreased from about 7 to 4.5 per 100,000 patient encounters, while the average rate of lawsuits fell from 2.13 to 0.75, and the time to resolution was much shorter. Total costs for liability, patient compensation, and lawyers also dropped significantly. It was a win for accurate data collection, formal apologies, and admirable openness.

Independent oversight strategies are being tried at the level of professional medical boards with data-reporting programs such as the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP). However, the data is only available to the participating organizations, undermining the capacity for public comparison. Federal reporting, such as in the National Healthcare Quality Report, shows the benefits of electronic data collecting systems, but public data reporting at a local level to support effective interventions and personal decision-making would increase their value.

back to top  Changing the Culture of Health IT

Changing the culture of health information technology to one of openness will not be easy, but accurate, complete, and public reports about EHR-related problems and healthcare errors could substantially increase patient safety, while reducing costs. Unchecked industry opportunism can undermine consumer interests and patient safety. I respect the industry professionals I have met and believe they want to build effective and safe products, but they are sometimes overly fearful of worst-case scenarios.

As industries mature, the respected leaders come to realize the benefits of common guidelines for design, quality control to weed out the inferior products that give everyone a bad name, interoperability to allow a competitive market, common reporting formats that allow comparison among products, and open reporting of failures that promote continuous improvement. These goals are attainable while allowing companies to develop patentable/copyrightable improvements, customize for different branches of medicine and different hospitals, and increase patient safety.

back to top  Acknowledgments

I appreciate the comments on drafts from Zach Hettinger, Harry Hochheiser, Jeff Johnson, Bonnie Kaplan, Lana Lowry, Avi Parush, Dean Sittig, Mark S. Smith, and Art Swanson.

back to top  References

1. Institute of Medicine, Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. National Academy Press, Washington, D.C., 2000.

2. Agency for Healthcare Research and Quality. Morbidity & Mortality Rounds on the Web; http://www.webmm.ahrq.gov/

3. National Cancer Institute. Research-based Web Design and Usability Guidelines. Dept. of Health & Human Services, National Institutes of Health (updated edition 2006); http://www.usability.gov/guidelines/

4. National Institute of Standards and Technology-NIST (2007). Common industry specification for usability requirements NISTIR 7432; http://zing.ncsl.nist.gov/iusr/documents/CISU-RIR7432.pdf

5. National Institute of Standards and Technology. A Community-Building Workshop: Measuring, Evaluating and Improving the Usability of Electronic Health Records. (June 7, 2011); http://www.nist.gov/healthcare/usability/usability-technical-workshop.cfm

6. Singh, H., Thomas, E.J., Sittig, D.F., Wilson, L., Espadas, D., Khan, M.M., and Petersen, L.A. Notification of abnormal lab test results in an electronic medical record: Do any safety concerns remain? Amer. Journal of Medicine 123, 3 (Mar. 2010), 238–244.

7. Schumacher, R. and Lowry, S. Customized common industry format template for electronic health record usability testing. NISTiR 7742 (Nov. 2010); http://www.nist.gov/customcf/get_pdf.cfm?pub_id=907312

8. Open Government Initiative; http://www.whitehouse.gov/open

9. Sittig, D.F. and Classen, D.C. Safe Electronic Health Record use requires a comprehensive monitoring and evaluation framework. J. Amer. Medical Assn. 303, 5 (2010), 450–451.

10. Kachalia, A., Kaufman, S.R., Boothman, R., Anderson, S., Welch, K., Saint, S., and Rogers, M.A.M. Liability claims and costs before and after implementation of a medical error disclosure program. Annals of Internal Medicine 153 (2010), 213–221.

back to top  Author

Ben Shneiderman is a professor in the Department of Computer Science, the founding director of the Human-Computer Interaction Laboratory, and a member of the Institute for Advanced Computer Studies at the University of Maryland at College Park.

back to top 

©2011 ACM  1072-5220/11/11  $10.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.

Post Comment

No Comments Found