Alexander DeWitt, Jasna Kuljis
Until relatively recently, software security was of little concern to computer users. However, the media coverage of severe security breaches has made even relatively computer-illiterate users aware of possible dangers from malicious attacks and misuse to both their systems and sensitive personal information. There are many software packages available to combat these security threats. However, getting the general public to effectively use these toolsor to even consider doing sopresents many challenges, and the usability of such tools is a major factor.
Given the above trend, it seems appropriate to suggest that security software should not be let loose on unsuspecting users without passing some form of usability test. In order to investigate what such a test might involve in practice, we conducted a usability study of Polaris , a promising alpha release for Windows XP used to protect against viruses altering files. Our particular concern was the vulnerability of computer systems to threats coming from the Web and email attachments. Even though Polaris was designed with usability in mind by researchers at Hewlett-Packard, our study established that users found using some of its features problematic and that they would knowingly compromise computer security rather than struggle with the software.
Usability engineering principles are not commonly applied, or at least fully integrated, into the design of security products. This may be because developers are either experts in security or in HCI, but not both, and collaboration between the two groups of experts has been lacking to date. However, this all may be about to change as the newly emerging field of HCI-SEC (HCI and Security) begins to make an impact on the development community (see e.g. [1, 3, 4]). In 2002, Yee  introduced some guidelines for aligning security and usability. One of the most notable of the principles from these guidelines is the Principle of Least Authority (POLA) .
POLA describes the principle of only allowing a program access to resources it absolutely needs to run. This is contrary to the way MS Windows and UNIX systems traditionally operate. For example, a text document opened in MS Windows will give the text-editor program full privileges to traverse the entire file system and make changes to it, thus allowing a virus or Trojan to take control of the program and leverage its power to alter or delete any files. One way to prevent this is to allow the user an opportunity to designate abilities he wants the program to have. For example, if the user wants to edit a file, he uses a file-open dialogue to designate edit capabilities to the program. Principles such as POLA are just one possible step toward aligning security and usability. If we could find out how comfortable and convenient users found software that had been developed following these principles, this might prove to be important for advancing the field of HCI-SEC.
We report a usability study carried out with the alpha release of Polaris (Principle of Least Authority for Real Internet Security) , which is based on the POLA principle. Polaris was primarily developed to make MS Windows safer from viruses and malicious code. Polaris was specifically designed to be highly usable as well as secure. However, Polaris is a retrofit onto MS Windows, designed to allow the latter's millions of users to continue using their established programs. Hewlett-Packard, the creators of Polaris, agreed to a collaborative study with us, the first such formal usability study to examine Polaris. Our findings will help to inform the beta release of Polaris and might contribute to research in the HCI-SEC field on improving the synergy of security and usability.
How Polaris protects applications. Using Polaris, an application can be "polarized," a process that creates a tame version of that application known as a "pet" that is immune to viruses. This "pet" resides in a segregated disk area, which prevents it from accessing files it does not need in order to run. The polarized application can be distinguished from the original application by a label in its title bar with its given "pet" name. Polaris does not protect files from any malicious programs open in the same pet; therefore users can and should create multiple pets for a single application in order to segregate trusted and un-trusted file opening.
The usability study. Our usability study assessed whether users who are not security experts can adequately protect their files from malicious attacks. Three different categories of usability metrics were used to assess the software, and these were: effectiveness, efficiency, and satisfaction. These metrics are based on the definition of usability as given in ISO standard 9241-11. The metrics used for effectiveness were the number of references to documentation, the length of time referring to documentation, learnability (retention and intuitiveness), and the number of errors encountered. Efficiency was measured by the time taken and the number of mouse clicks required to complete each task. Finally, questionnaires and interviews were used to judge user satisfaction.
How the study was carried out. We used a laboratory test in which users were asked to perform tasks that included the use of security. Rather than being the focus of the task, the security features were presented as a secondary task of the primary task that the user was asked to complete, in much the same way as in real life, where we assume users are more concerned with getting their work done than with configuring security.
We used ten participants in the study, each of whom undertook the prescribed set of tasks independently under laboratory conditions. The participants sat alone in front of a PC, and their actions were observed through a one-way mirror and on a monitor mirroring their screen. The participants were asked to complete the following tasks with the help of the documentation: Identify polarized applications; polarize and depolarize applications; customize MS Outlook to use Polaris to safely try email attachments; safely run applications from an unknown source from the Internet, and decide when it is appropriate to create multiple pets. The user was advised by the documentation to use one pet for trusted files and another separately created pet for untrusted files.
After the tasks were completed, the participants were given a short questionnaire and were asked several questions in a short, semistructured interview. A repeat of the study after a period of one weekthis time without the ability to refer to the documentationwas used to investigate the learnability of the software.
What we learned. Some participants had trouble distinguishing between polarized and normal applications. This was an interface issue, but it caused some participants to incorrectly believe they were being protected by a polarized application later in the study. Few participants were able to complete the task that required them to customize MS Outlook. During this task, users consulted the documentation on average 15 times and for about 20 seconds on each occasion, and frequently found themselves presented with unhelpful error messages by Polaris, which they learned to quickly dismiss.
When asked to safely run applications they had downloaded from Web sites, four of the ten participants opened the application without using Polaris, thus risking their PC's security. Two of these explained in the interview that they knew the dangers of doing so but forgot to polarize the application. One participant said they could not tell if they were being protected by Polaris (assuming it would be automatic), and one person said they expected a context-sensitive menu allowing them to quickly choose a "try safely" option. A further three participants, hastily wanting to try the application, first opened it unsafely, and then subsequently opened it using Polaris. But by then the damage would already have been done, had the application been malicious.
To test how the participants made decisions using multiple pets, we first asked them to use a secure Internet banking session, to then close their browser, and to then visit Web sites engineered to appear untrustworthy (the interviews later confirmed that participants did indeed judge these as being risky). It was hoped that for maximum security the users would perform these actions using two distinct "pets." However, none of the ten participants put our hope into effect. Throughout the study only a single pet was used to open all of the Web sites. In two cases, no pet was used at all. In these two cases, the participants opened the Web pages in unpolarized applications because they had opened the wrong shortcut on the desktop and failed to notice that they were not using a pet. Of the participants who used a single pet, three knew the purpose of using multiple pets but chose not to use them in order to complete the tasks faster and, as a result, compromised security. Three further participants knew it was possible to create multiple pets but did not understand why this should be done. The remaining four participants did not know it was possible to create multiple pets, even though at the start of the study all of the participants spent an average of 12 minutes reading the documentation file, in which the notion of multiple pets and the reasons for using them are explicitly explained. If this feature is to be used, it clearly needs to be even more explicit to users. However logical on paper, it also turns out to be unrealistic to expect users to continually make risk-assessment decisions every time a new Web page is opened.
The learnability test, conducted one week later, showed that users who previously had trouble identifying a polarized application had realized their previous mistakes and performed better at this task. However, participants still had a lot of trouble with customizing Outlook and still did not use multiple pets when they were really necessary.
The interview results and behaviors observed in this study show that even users who are not security experts are quite aware of the risks of using the Internet and can readily judge the trustworthiness of Web sites and emails based on their experience, familiarity with the domain or sender, and presence of security features. Nevertheless many of them were unable to make appropriate security decisions, resulting in security compromises. This was sometimes due to a lack of understanding about the way the software works, sometimes because the software did not provide explicit enough prompts. Sometimes, however, the participants knowingly took unsafe workarounds. In these latter situations, they were often too careless and did not stop to consider their PC's security, and showed apathy toward file protection.
However, a limitation of this study is that the participants were aware that they were in experimental conditions, and as such were under no real risk. A repeat of the study in which participants could be induced to have a high motivation for protecting the files as if they were their own would help address this issue.
Results from previous studies in HCI-SEC  support our conclusions that our participants did not know when or how to make security-related decisions and often took unsafe shortcuts to get tasks done faster or easier. In addition, the study corroborated the notion that users are hindered by, and quickly dismiss, confusing error messages.
In conclusion, our interpretation of our study results leads us to concur with a previous published recommendation [1, 3, 7], which is that security and usability are more likely to be better aligned when developed together from the outset, rather than as retrofitting or as an afterthought.
2. Dourish , P. and D. Redmiles. An approach to usable security based on event monitoring and visualization. in 2002 Workshop on new security paradigms. 2002. Virginia Beach, Virginia: ACM Press, 75-81.
3. Flechais, I., A.M. Sasse, and S.M.V. Hailes. Bringing security home: a process for developing secure and usable systems. in Workshop on new security paradigms. 2003. Ascona, Switzerland: ACM Press, 49-57.
4. Garfinkel, S.L. and R.C. Miller. Johnny 2: A user test of key continuity management with S/MIME and Outlook Express. in Symposium on usable privacy and security. 2005. Pittsburgh, Pennsylvania, USA: ACM Press, 13-24.
Alexander J. DeWitt
About the Authors:
Alex DeWitt is a PhD candidate at Brunel University, West London, UK. He gained his degree in computer science also at Brunel, and his research interests include all aspects of usability and email security technologies.
Jasna Kuljis is a reader at Brunel University, West London, UK. Her current research is in human-computer interfaces. She is mostly interested in the design of graphical user interfaces and in the development of new paradigms that would further enhance the usability of interactive computer systems. Dr. Kuljis is the director of the VIVID Research Centre at Brunel University.
©2006 ACM 1072-5220/06/0500 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.