XXII.3 May - June 2015
Page: 60
Digital Citation

The role of tool support in public policies and accessibility

Fabio Paternò, Antonio Schiavone

Since the late 1990s, several countries have begun to enact laws addressing computer-system accessibility for technology funded or provided by the government. One of the first countries was the U.S., where in 1998 Congress approved and the president signed into law Section 508 of the Rehabilitation Act, with the goals of eliminating barriers in information technology, making new opportunities available to people with disabilities, and encouraging the development of relevant technologies.

Two years later, the European Council also began to address the issue, in 2000 approving the eEurope political initiative, whose aim was to support and promote the creation of a society based on knowledge, open and accessible to all—especially to all European citizens with disabilities. In the following years, several action plans (such as eEurope 2002, eEurope 2005, and eAccessibility) were made operational, with the aim of ensuring that all citizens have access to ICT services, removing the technical, legal, and other barriers that some people encounter when using them.



As a result of these initiatives, several European countries have begun to enact national laws on the accessibility of information systems (to name a few: Germany – BITV 2002, Italy – Stanca Act 2004, France – Loi Handicap 2005, Spain – Reglamento sobre las condiciones básicas para el acceso de las personas con discapacidad a lastecnologías 2007). The common purpose of these laws was to require that IT-based services provided by public administrations are presented in such a way as to enable people with disabilities to access the information and take full advantage of the opportunities offered.

The European Union’s efforts in promoting accessibility continued even after the end of the eEurope initiative with the launch of a new program named Digital Agenda for Europe (DAE). This active program is one of the seven flagship initiatives of the Europe 2020 Strategy.

In the same period of time, other countries enacted accessibility laws (to name a few: Canada – Common Look and Feel 2000, Japan – JISX8341 2004, Brazil – e-MAG 2007). The existence of this set of laws shows that Web accessibility is an issue of global concern. Despite these legislative endeavors, many public organizations’ websites are still poorly designed and have accessibility barriers. In 2011, Goodwin et al. published an accessibility analysis of national government portals and ministry websites of several U.N. member states [1]. This study revealed accessibility issues on government websites around the world. In particular, the analysis showed that non-accessible websites are more common in countries with poorly developed economies and low per capita GNI, and where only a small percentage of the population has access to the Internet.

Introducing Automated Accessibility Tools

Many accessibility experts agree that accessibility validation is a process that cannot be fully automated. The W3C itself, in a preparatory document for the development of its WCAG 2.0 guidelines [2], discussed accessibility criteria that are “machine testable,” compared with others that are “reliably human testable,” thus suggesting that not all accessibility audits were automatable. Moreover, it is easy to imagine how difficult it is for an automated tool to assess guidelines such as “text requires reading ability more advanced than the lower secondary education level after removal of proper names and titles, supplemental content” (WCAG 2.0 – Guideline 3.1.5 “Reading Level”).

Despite these inherent limitations, automated tools still play an important role in ensuring the accessibility of websites. Indeed, following the adoption of accessibility laws, various government organizations started paying more attention to accessibility guidelines, but Web accessibility requires constant monitoring of many details across many pages. Thus, in order to simplify the monitoring, analysis, detection, and correction of website accessibility problems, several automatic and semi-automatic tools were developed. A typical category of software tools used for this purpose is the so-called accessibility validators, software programs or online services that help determine if a website meets accessibility criteria, which are usually encoded in “accessibility guidelines.”

The first accessibility validator, Bobby (named for British policemen), was developed in the mid-1990s by the Center for Applied Special Technology (CAST). In general, the first generation of validators had a number of limitations: First, most of these tools allowed validating at any one time only a single webpage, or in rare cases, a single website. In the real world it is more useful to evaluate the accessibility of collections of websites grouped by topic or territory and to monitor the evolution of their accessibility over time, providing a high-level view of whether progress is being made. A number of other issues exist in the use of automated accessibility tools:

  • Expandability and upgradeability. Newer technical guidelines get released, and while there is one international standard (WCAG, currently 2.0), some countries make modifications to it. For the developers of accessibility validators, extending the set of guidelines supported by their tools or upgrading the existing ones can be a major undertaking: In general, it may be necessary to rewrite a significant portion of the validator’s source code, with notable costs in terms of time and resources.
  • Alignment with the latest technology. New versions of languages for describing webpages’ layout and structure have been released, introducing new features such as semantic tags. Mobile browsing has become increasingly widespread, affecting the technical design and development of websites, which nowadays must be able to be viewed on a variety of devices and different screen resolutions. Websites are becoming more and more dynamic and include interactive content. In this ever-changing panorama of technologies, the first generation of validators often appears to be unable to effectively validate websites made with the most modern technologies.
  • Limited effectiveness of the reports. Automated tools provide reports that are sometimes difficult to interpret since they show so many details; it is difficult to identify the main problems and how to address them. One aspect that must be taken into account is that such reports can be accessed by people with different roles (developers, designers, public officers) who need different levels of information for improving the site.

Tools Designed for Governmental Use

In recent years, a new generation of accessibility validators has been developed to overcome the limitations outlined here. For example, AMA [3], an application promoted by the University of Bologna (Italy) as part of the Vamolà project, gathers the accessibility status of large collections of Italian government websites according to different aspects. This tool has been used by the public administration of the Emilia-Romagna region to monitor the accessibility of 380 websites of public institutions that reside in this region (nine provinces, 330 municipalities, and 40 other public organizations). The data collected has been used to elaborate the annual report about the usage of Web technologies in Emilia-Romagna [4].

For the developers of accessibility validators, extending the set of guidelines supported by their tools or upgrading the existing ones can be a major undertaking.

In terms of expandability and upgradeability, some researchers have developed accessibility validators that separate the validation engine from the guidelines-specification phase. With this approach, the guidelines are codified through specifically designed formalization languages and specified in external files; updating or expanding the available guidelines involves simply recoding existing guidelines or codifying new ones. An example of this approach is EvalAccess, an accessibility validation tool developed by the University of the Basque Country (Spain) that uses a guidelines language formalization named Unified Guidelines Language (UGL) [5] to express the requirements of several guidelines sets.


The issue of alignment with the latest technology is particularly important, but also the most difficult to solve, for at least two reasons. The first difficulty lies in the fact that for each site, there may be versions specific to a given type of device (e.g., mobile and desktop versions) or a single version of the site that adapts its user interface according to the resolution of the device (responsive layout). In both cases, an accessibility validator must be able to analyze in a single run the set of all possible user interfaces for that page on all possible devices and screen resolutions. The second and greatest difficulty is determined by the fact that modern websites are dynamic, meaning they utilize technologies (e.g., Ajax scripts) that can change both the webpage’s user interface and content automatically or in response to user interaction. Furthermore, it is increasingly common to come across websites that combine these two latter problems. For example, it is easy to find dynamic websites with responsive design. Nevertheless, researchers from around the world are proposing technological solutions to overcome these difficulties. One example is MAUVE [6] (Figure 1), proposed by the CNR of Pisa (Italy), which is a validation environment that can recover and validate versions of a webpage specific to certain categories of devices, and through the use of some browser plugins, capture and then validate the actual version of a dynamic page in a given time.

Automated Validators in Public Policies

While automated validators are important tools for ensuring compliance with accessibility laws, the laws themselves, as well as associated regulations, often do not make explicit reference to such tools or describe how the tools should be utilized in implementing public policies (for instance, the Italian “Implementation Regulations for Law 4/2004” associated with the Stanca Act does not mention automated tools, but the document [7] associated with the Stanca Act does). Laws and regulations typically mention evaluation by experts and usability testing by people with disabilities as appropriate methodologies for validation. Specific guidance is rarely mentioned. Examples of organizations that have successfully adopted automated testing include some government agencies in Sweden, where results of automated accessibility testing are publicly posted monthly [8], and the U.S. Census Bureau, where 90 percent of webpages receive a monthly review from an automated accessibility inspection (although results are not posted publicly) [9]. Guidance and best practices could be very helpful to guide the appropriate use of automated accessibility validators in implementing public policies.

In our opinion, there are several reasons why public policies should go a step further and provide clear and well-documented advice for adopting automatic tools in supporting accessibility validation, taking into account recent experiences and research in this field:

  • Although the consensus is that automatic validation has limits, even validation performed by human beings has critical issues related to human limits. For instance, experts are certainly able to perform more accurate analyses, but they also require a long time to validate sites containing a large number of pages. Automated tools can help experts analyze large numbers of webpages, detecting the most common and frequent accessibility errors, thus reducing their workload and allowing them to focus on the most sensitive aspects of the accessibility assessment.
  • Usability testing helps uncover problems related to an individual’s specific disability. Automatic validators are able to evaluate accessibility problems related to multiple disabilities, at least for those types of checks that are automatable.
  • Public policies generally give little guidance on methods to follow to ensure that the accessibility of a site is maintained over time. As it is quite unrealistic to assume that a public administration will continually make use of an expert to perform usability testing, it is possible that validators can be used as a system of “warning lights,” able to perform periodic validation of a site’s accessibility (either triggered manually or via timed automated systems) in order to monitor the site’s accessibility over time. In this regard, it is interesting to think about how often a website should be validated. There is no single answer to this question—it depends on the characteristics of the site. A site that has already been made accessible and whose structure, user interface, and content have not changed over time does not need to be validated weekly. On the other hand, a site that is updated daily or several times a day should be validated with greater frequency. In the real world, these two extremes rarely represent a concrete case. The most common case is a site where the structure and user interface are modified over several years. Regarding the content, usually a subset of the site’s pages are updated infrequently (e.g., the contact page), while others have frequently updated content (e.g., the news page). The best strategy is therefore to validate different sections of the site at different frequencies, calculated on the basis of update frequency. In general, the frequency of validation should be equal to, or slightly lower than, the update frequency.


It has been years since the appearance of the first automated accessibility validators, but their adoption has not been fully addressed in public policies. Even if they do not provide a complete analysis of accessibility and can sometimes provide confusing results, accessibility validators can significantly reduce the time and effort to evaluate websites, making the validation process more efficient, consistent, and reliable.

Public policy stakeholders should consider the progress in this field of research for at least three reasons. First, according to several national laws on Web accessibility, public officials are obliged to ensure accessibility in online communications. Further, pushed by the recent economic crisis, many public authorities are moving their services to the Web to speed up processes, reduce costs, and optimize document management. It is important that these e-government services be accessible. Finally, for ethical reasons, public administrations have the duty to support all citizens, giving everyone the same quality of services. Automated accessibility testing tools can play an important role in making public websites more accessible.


1. Goodwin, M., Susar, D., Nietzio, A., Snaprud, M., and Jensen, C.S. Global Web accessibility analysis of national government portals and ministry Web sites. Journal of Information Technology & Politics 8, 1 (2011), 41–67.

2. W3C’s “Requirements for WCAG 2.0 Checklists, Techniques, and Test Files”;

3. Mirri, S., Muratori, L.A. and Salomoni, P. Monitoring accessibility: large scale evaluations at a Geo political level. Proc. of the 13th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, 2011, 163–170.

4. Benchmarking della società dell’informazione in Emilia Romagna, 2013 (available only in Italian);

5. Arrue, M., Vigo, M., and Abascal, J. Including heterogeneous Web accessibility guidelines in the development process. Engineering Interactive Systems Lecture Notes in Computer Science 4940 (2008), 620–637.

6. Paternò, F. and Schiavone, A.G. An extensible environment for guideline-based accessibility evaluation of dynamic Web-applications. Universal Access in the Information Society 14, 1 (Mar. 2015), 111–132.

7. Guidelines containing the technical requirements, the definitions of the different accessibility levels and the technical methodologies for the testing of Web site accessibility;

8. Gulliksen, J., vonAxelson, H., Persson, H. and Göransson, H. Accessibility and public policy in Sweden. Interactions 17, 3 (2010), 26–29.

9. Lazar, J. and Olalere, A. Investigation of best practices for maintaining Section 508 compliance in U.S. federal Web sites. Proc. of the 2011 Human Computer Interaction International Conference. 2011, 498–506.


Fabio Paternò is research director at C.N.R.-ISTI in Pisa, where he founded and leads the Laboratory on Human Interfaces in Information Systems. His main research interests are in user interface software and technologies in ubiquitous environments for improving accessibility, usability, and user experience.

Antonio Giovanni Schiavone is a graduate fellow at the Human Interfaces in Information Systems Laboratory of the National Research Council. His research is focused on the analysis of the various issues related to Web usability and Web accessibility, in particular in relation to the latest Web technologies.


F1Figure 1. The MAUVE homepage.

©2015 ACM  1072-5220/15/05  $15.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.

Post Comment

No Comments Found