You are on page 1of 24

INTCOM 1143

Interacting with Computers 12 (1999) 119142 www.elsevier.com/locate/intcom

User interface guidelines and standards: progress, issues, and prospects


P. Reed a,*, K. Holdaway b, S. Isensee c, E. Buie d, J. Fox e, J. Williams f, A. Lund g
Lucent Technologies, 7282 Silverhorn Drive, Evergreen, CO 80439, USA b UIS Consulting Inc., 3112 Fox Hollow, Round Rock, TX 78681, USA c BMC Software, 10415 Morado Circle, Austin, TX 78759, USA d Computer Sciences Corporation, 15245 Shady Grove Rd., Rockville, MD 20850, USA e Pacic Northwest National Laboratory, 901 D Street, SW, STE 900, Washington, DC 20024, USA f Bellcore, 6 Corporate Place, PYA 1M186, Piscataway, NJ 08854, USA g US WEST Advanced Technologies, 4001 Discovery Drive, Suite 340, Boulder, CO 80303, USA
a

Abstract This article reviews progress in the development of standards and guidelines for humancomputer interaction, including those developed within international and US standards bodies. Guidance for incorporating software ergonomics standards and guidelines into software design and development processes is discussed. Several different techniques that have been dened for assessing the conformance of a product to guidelines are reviewed. In addition, the strategies employed by formally approved standards developed in ISO and ANSI for determining conformance are discussed. Finally, we discuss the prospects and challenges for software ergonomics standards and guidelines that must be addressed as the pace of technological change continues to accelerate. 1999 Elsevier Science B.V. All rights reserved.
Keywords: Software ergonomics standards; User interface design; User interface guidelines; Software usability standards; Software ergonomics in design and development; Usability conformance assessment; Tools for working with guidelines

1. Introduction Since the mid-1980s, formal standards and published guidelines for software user interfaces and HumanComputer Interaction (HCI) have increased in importance as the
* Corresponding author. Tel.: 1-303-670-6955; fax: 1-303-670-6955. E-mail addresses: psreed@lucent.com (P. Reed), erb37a@prodigy.com (K. Holdaway), scott_isensee@ bmc.com (S. Isensee), ebuie@csc.com (E. Buie), jeff.fox@pnl.gov (J. Fox), jwillia2@notes.cc.bellcore.com (J. Williams), alund@acm.org (A. Lund) 0953-5438/99/$ - see front matter PII: S0953-543 8(99)00008-9 1999 Elsevier Science B.V. All rights reserved.

120

P. Reed et al. / Interacting with Computers 12 (1999) 119142

Fig. 1. First documented use of the shall statement to create a mandatory standard.

use of computers has become pervasive in work settings and other environments. Substantial benets for both end-users and employers can result from the use of software user interface standards and guidelines including: increased productivity, reduced mental and physical stress, reduced training expense, improved usersystem interoperability across applications, and improved overall product quality and aesthetics. The purpose of this paper is to: (1) provide an overview of standards and guidelines for humancomputer interaction, including the International Organization for Standardization (ISO) and American National Standards Institute (ANSI) standards development activities in the area of software ergonomics, (2) discuss how software ergonomics standards and guidelines may be incorporated into software design and development processes, (3) review the techniques that have been dened for assessing the conformance of a product to guidelines and how the formally approved standards developed in ISO and ANSI address conformance, and (4) examine the prospects and challenges for software ergonomics standards and guidelines that must be addressed as the pace of technological change continues to accelerate. There are a wide range of resources available to system developers seeking guidance regarding the design of software user interfaces. Numerous books, e.g. Nielsen [1], principles, e.g. Mayhew [2], style guides e.g. Apple Computer [3], standards e.g. ISO [4], and guidelines, e.g. Brown [5] have been published in the extant literature. Depending on the requirements for a particular project, some or all of these sources may be consulted. The least formal of these sources are books (often containing principles) and guidelines, which usually do not undergo any systematic consensus-building process, and which usually provide general guidance that may be applied to most user interfaces. Style guides are generally published by software producers (or consortium of software producers) that wish to dene a specic look and feel, to a fairly high degree of detail, for applications developed on a dened hardware/OS/middleware platform. Style guides are formal to the extent that there is usually a systematic review process conducted with representatives from consortia members, application developers, and other directly concerned interests, but there is no formal, due process dened for achieving consensus among all potentiallyaffected interests (e.g. end-users). Standards such as those in ISO and ANSI are distinguished by documented standards development procedures that have been designed to ensure that all companies and

P. Reed et al. / Interacting with Computers 12 (1999) 119142

121

individuals that are materially- or directly-effected by a particular standard will have an opportunity to represent their interest and participate in the standards development process. These procedures require consensus-building processes to be utilized before any standard is approved. Each individual design guidance statement in a formal standard is either a requirement or a recommendation. A requirement must include the word shall its statement providing design guidance, and anyone who claims conformance with the overall standard must conform with each and every requirement (i.e. shall statement) contained in the standard. In contrast, a recommendation must use the word should in its statement of design guidance, and a software product does NOT have to conform with each and every recommendation in order to claim conformance with the overall standard. As shown in Fig. 1, humans have been challenged to comply with standards, with varying degrees of success, for many centuries!

2. History and organization of ergonomics standards and committees As recently as a dozen years ago there were few if any standards to be found which could be directly applied to the software design of user interfaces. There were some guidelines available, such as The Mitre Corporation Guidelines on Designing User Interface Software published in 19831986, but no national or international efforts by formal standards bodies were underway. In the 19851986 timeframe however, several events occurred which became the catalyst to the formation of groups of interested professionals who felt the time was right to consider whether or not work should begin on standards for software user interfaces. Some of these events were: The rapid development and delivery of different graphical user interfaces for personal computers. While these new GUIs promoted ease of use and increased productivity, they brought inconsistencies, incompatibilities, and questions concerning the sound scientic basis for some of the techniques used; The emergence of standards for the hardware or physical aspects of the user interfaces such as VDT character legibility, screen refresh rates, or keyboard characteristics, but no attempts at standards for the cognitive aspects of the interface. The software issues seemed to be overlooked; Widespread concerns over health and safety issues relative to emissions from VDTs caused others to be equally concerned about mental stress resulting from poorly designed software. While the concerns over emissions have largely gone away, the concerns about mental workload and mental stress have remained. These and other similar events prompted human factors and user interface design professionals, working through the auspices of recognized standards organizations, to create subgroups within the standards organizations that would specically address the software ergonomic issues. Two of these groups, one a US national group and the other an international group (described below), are active today and are producing practical standards which can guide developers in developing human centered user interfaces. Today there are approved or draft standards for many aspects of software user interfaces. For example; how should menus be constructed, how and when should color be

122

P. Reed et al. / Interacting with Computers 12 (1999) 119142

used, what factors should be considered when using multi-media in the interface? Standards on these and many other issues dealing with the design of user interfaces are available or under development within formal standards organizations throughout the world. 2.1. International software ergonomics standards International Standards Organization (ISO), Technical Committee 159 (TC159-Ergonomics), Sub Committee 4 (SC4-Ergonomics of Human System Interaction), Working Group 5 (WG5-Software Ergonomics and Human Computer Dialogs). WG5 rst met in 1985 to decide what could or should be done in developing software user interface standards. Other Working Groups within the parent organization SC4 were busy developing standards for hardware user interface issues such as front of screen characteristics or keyboard design, but no one was working on software concerns. So, WG5 became an ofcial ISO group chartered to develop ergonomic software standards for the use of VDTs doing ofce work. Over the next several meetings they outlined 8 additional software parts (see Appendix A) to be added to the 9 hardware parts already dened and underway within the overall SC4 standard ISO 9241Ergonomic Requirements for Ofce Work with Visual Display Terminals (VDTs). WG5 is comprised of professionals from government, academia, industry, and user groups representing approximately 12 different countries. The standards are developed within the committee using consensus as the method for agreement on proposed material. Once the committee believes the draft standard is ready for public review, it is sent to the member bodies of ISO where it is reviewed, voted, and commented upon three separate times. After each vote and comment period, the draft is returned to WG5 where they modify the document to accommodate as many of the comments as can be agreed upon. Finally the standard is approved as an ISO international standard and is available for use by anyone. Although ISO standards are intended to be used on a voluntary basis, sometimes they are cited in legislation or in a procurement process. When this happens the standards cease to be voluntary and instead become a mandatory condition of doing business. A summary of the titles and content of the parts of the standard being developed by WG5 can be found in Appendices A and B. Information about the availability of ISO standards can be found at www.iso.ch or www.ansi.org or in the source column of Appendix A, B and C. Also, Smith [6] provides an excellent treatment of ISO and ANSI ergonomic standards for computer products. 2.2. U.S. national software ergonomics standards The Human Factors and Ergonomics Society (HFES) Human Computer Interaction (HCI) committee was formed in 1985 by a group of interested User Interface (UI) professionals who were concerned with the lack of empirical and other reliable evidence supporting proposed software user interface standards. Because of the concentration of user interface design expertise in the US, it was believed that the US should have some mechanism for contributing to the ISO work. This group initially consisted of approximately 15 people from government, academia, and industry, all members of the HFES. They banded together in the common interest of promoting UI standards based upon

P. Reed et al. / Interacting with Computers 12 (1999) 119142

123

scientic research and empirical evidence rather than abstract principles and ad hoc current implementations. For the rst several years the focus was on reviewing the ISO documents and providing technical reports to ISO to aid them in their developing standards. These technical reports subsequently became the base material for several parts of the 9241 standard being developed by ISO WG5. After years of contributing to the ISO activity, the HFES HCI committee felt the work by ISO needed to be extended in several areas and updated to include more recent technologies and research. In addition, they felt that a US standard was needed to represent the United States perspective since the US is a major world wide developer of user interface software. The HFES-200 committee was accepted as an accredited Standards Developing Organization (SDO) by the American National Standards Institute (ANSI). This standard has been given the title HFES 200Ergonomic Requirements for Software User Interfaces. In 1994, the Human Factors and Ergonomics Society (HFES) obtained approval from the American National Standards Institute (ANSI) to develop a national standard that addresses Ergonomic Requirements for Software User Interfaces. HFES-200 is the title designating this project as well as the committee (formerly known as the HFES HCI Committee) formed to develop the standard. Both ISO and the HFES also have committees working on standards which address hardware user interface topics (HFES100see Appendix C). HFES 200 will adapt parts 1217 of the 9241 ISO standard by updating them to reect current research, correcting any information considered to be in error, or modifying recommendations that may have a uniquely U.S. requirement. In addition, the standard will contain new material which was felt to be lacking in the ISO standard. It will add major sections which deal with: use of color, computer accessibility, and voice input/output. In its formative years, the HCI committee decided that UI recommendations must be based upon a critical minimum level of sound evidence before they could be included in their standard. A three level test of evidence was developed which is applied to each proposed recommendation. Is there documented scientic research? Is there documented empirical evidence? Is there general agreement among the committee members and in documented the state of the practice? One or more of these criteria must be afrmed before a recommendation can be accepted. The HCI committee works (like ISO) on a consensus method of seeking agreement on the content of the draft standard. The committee will release the proposed national standard for public review through a canvass process. This ensures that interested and materially affected individuals can vote, comment and suggest changes to the document. The HCI committee will review the votes and comments and attempt to accept as many of the suggestions as can be agreed upon. If the negative votes were in the majority, a second draft will be prepared and second canvass will be conducted. This process continues until general agreement is secured. It is expected the standard will be ready for the rst canvass public review around late 1999. For a listing of the content of HFES 200 see Appendices A and B. Information about the availability of HFES standards can be found at www.hfes.org or in the source column of Appendices B and C.

124

P. Reed et al. / Interacting with Computers 12 (1999) 119142

3. Using HCI standards in product/system development Whether a projects software life cycle process is waterfall, spiral, or something else altogether, it will accommodate and benet from the application of HCI standards. This section provides some guidance for using standards during the software design and development process. Additional guidance on humancomputer interaction design in product development can be found in Karat [7] and Hix and Hartson [8]. Standards use involves eight steps: Dene project objectives Manage expectations Select standard(s) to use Tailor the standard(s) to your project Determine compliance approach Apply the tailored standard in designing the interface Assess compliance of interface design Conduct usability testing.

In addition, some specic success strategies can help projects get the most out of using standards. 3.1. Dene project objectives The rst step is to identify what the project aims to achieve in using HCI standards. Here are some possibilities: Provide a familiar look and feel. Standardizing the look and feel of software allows users to transfer the skills they learn on one piece of software to another. Training costs are minimized. Provide consistency. Standardization may occur in varying scopes. Examples include the various components of an application, applications that will be used together, and an operating system and all the software that runs on it. The more broadly a standard can be applied, the greater the benets. Use human factors ndings. Standards take advantage of the large body of human factors research and accepted practice. The authors of the standards assimilate and interpret the research, turning it into guidelines (best practice) for designers to follow. Streamline development. Standards make many design decisions routine. This frees designers to spend time on decisions that are more difcult or critical. Evaluate usability. Standards provide one basis for judging the usability of products. All else being equal, a product that meets an HCI standard should be more usable than one that does not. Comply with requirements. Standards compliance for the software may be required by the buyer (e.g., in many US government contracts) or by law (e.g., in Europe). 3.2. Manage expectations The second step is to manage project expectations, as many organizations may have

P. Reed et al. / Interacting with Computers 12 (1999) 119142

125

inated ideas of what standards can do. First, be aware of some general limitations of standards: No guarantees: Following an HCI standard will not guarantee that a product or system is usable. Standards cannot address sufciently the information structure and content, often the hard part of HCI design. Evolution: User interface knowledge and technology are continually changing and improving, and so are standards. Although many of their guidelines are based on consistent human characteristics and are enduring, others need to be replaced by more current advice when new research and practices suggest better techniques. There are times when a designer should try innovative solutions before the standards have caught up. (See Section 5 for a more in depth discussion of this issue). Generality: Standards provide guidance to meet a broad range of circumstances, not to help design for specic intended users and their tasks. It is usually benecial to tailor the general guidelines into task- or system-specic design rules. Leading edge? Standards provide guidance that is stable and well accepted. They are not intended to be leading edge. State-of-the-art? Long development times can put standards somewhat out of date before they are released. It is not unusual for an HCI standard to spend several years or more in development. Strong? The consensus process involved in standards development ensures that guidance is well thought out and meets the needs of many parties. However, many proposed guidelines are controversial and are eliminated, made optional (should rather than shall), or qualied. Platform-specic? HCI standards are designed to be platform independent. For example, ISO 9241 applies to all GUIs and character based interfaces. Platform independence ensures that standards are nonproprietary and can be applied widely; however, the guidance must be general and can be difcult to interpret. Universal? Design situations vary and a given recommendation is unlikely to be optimum in all cases. Applying a standard requires the designers judgment.

3.3. Select standard(s) to use Which standards should be followed, and whether they are required or optional, depend on project application domain and location, and on project requirements. First, determine whether you are looking to start with high-level principles, more specic guidelines, or organizational conventions. Principles specify high-level goals a user interface should meet (e.g. minimize memory load). Guidelines specify techniques that can be used to accomplish these goals and that have been shown to have appropriate usability characteristics (e.g., provide a keyboard accelerator for every important menu item). Standards sometimes provide principles (e.g., ISO 9241 Part 10) and usually provide guidelines. In contrast, conventions specify which of a set of acceptable design techniques or elements an organization is going to adopt and how it is going to apply them to its

126 P. Reed et al. / Interacting with Computers 12 (1999) 119142 Fig. 2. Factors concerning the users, their tasks and environment, and the hardware and software implementation drive the selection and tailoring of guidlines for a project.

P. Reed et al. / Interacting with Computers 12 (1999) 119142

127

product(s) (e.g., use the letter T as the accelerator for the Templates menu item). Organizations typically develop their own design conventions. Second, determine which type(s) of standard applies to your project: International (e.g., ISO, ITU) standards are applicable worldwide. Regional (e.g., CEN) standards are applicable within a given region such as Europe. National standards (e.g., ANSI, DIN, AFNOR, SIS, BSI, CSA) are applicable only within a given country. Military or government standards (e.g., DoD, MIL, NASA, ESA) are applicable to products and systems being developed for specic military or government organizations. Platform (e.g., Mac OS, CUA, Windows, Motif) style guides are applicable to all applications developed for a given operating environment. Independent (e.g., books by individual authors, company-specic) standards are either very general or are applicable to the domain for which they are written. 3.4. Tailor the standard to your project Tailoring is the process of creating a detailed, project-specic HCI specication from the set of design guidelines in the selected standard(s). First, select the guidelines that apply to your project. If, for example, the project team selects ISO 9241 and the user interface is going to include menus but not direct manipulation, Part 14 will directly apply and Part 16 will be irrelevant. Within Part 14, the project team should identify the specic shalls and shoulds that apply to your users, their tasks and environment(s), and relevant implementation constraints. Fig. 2 shows the factors that you should consider when determining which guidelines apply to your HCI. See Section 4 for detailed information on determining applicability. The other factor that will drive guidelines selection is compliance. You may need to follow the shalls but not the shoulds; you may have to follow both shoulds and shalls; or you may not be required to comply with any specic guidelines in the standard. Next, translate the selected guidelines into specic design rules for your project. Review the selected guidelines in the context of the users, their tasks and environment, your implementation constraints, and the overall requirements for the product or system. Use prototyping as necessary to rene and validate the design rules. 3.5. Determine compliance approach Standards compliance is most effectively applied during product design and implementation. If the product or system is not evaluated for compliance until the end of the development cycle, changes will be difcult to make. Compliance with some guidelines may be difcult to determine by examining the interface. They may require knowledge of the users, the designers intent, and the development and testing procedures. Examine the tailored standard to identify the best compliance approach for each guideline. For each guideline in the tailored standard, determine the following:

128

P. Reed et al. / Interacting with Computers 12 (1999) 119142

Level of compliance required: What is the level of compliance that is desired or required for your project? As mentioned earlier, you may have to follow just the shalls, or both shoulds and shalls; or compliance may be at your discretion. Method of assessment appropriate for that guideline: Is the guideline best suited for compliance assessment by content analysis, documented evidence, observation, analytical evaluation, or empirical evaluation? The degree of compliance documentation also varies. Detailed information about compliance with each guideline and the method used to judge compliance may be required; a letter stating that the product is compliant may be required; or no statement may be required. As the standard is tailored, develop and maintain a checklist to help in assessing compliance. If a checklist was provided with the standard, start with a copy of that and rene it, or create a checklist for the selected standard(s). (See Section 4 for detailed information on evaluating compliance). 3.6. Apply the tailored standard in designing the interface The application of standards to HCI design is most effective when it is part of an overall usability engineering process for the product or system. Standards may guide this process as well as the characteristics of the interface itself. For specic information, see ISO 13407, Human Centered Design of Interactive Systems. First, ensure that everyone who will be involved in making HCI design decisions is familiar with the tailored standard and the checklist. As you make, modify, and implement HCI design decisions, consult your tailored standard and your checklist to ensure that your design follows the standard. Issues will probably arise during implementation that will necessitate revising the tailored standard. Hardware and/or software congurations may change; functions may need to be added or removed. User interactions (e.g., messages about the status of the system or the software) will need to be designed that could not have been foreseen or designed in advance. A process for revising the tailored standard should be established, in which all parties affected by a change are informed about it and (where appropriate) are involved in determining design decisions. 3.7. Assess compliance of interface design As stated earlier, compliance with some guidelines may not be assessable from an examination of the interface, but may require knowledge of the users, the designers intent, and the development and testing procedures. (See Section 4 for detailed information on assessing compliance). 3.8. Conduct usability testing Standards can promote usability but by themselves cannot guarantee it. Even with the assistance of organizations (such as test houses) or tools (such as checklists or software products) that can verify or even certify standards compliance, you still cannot be certain

P. Reed et al. / Interacting with Computers 12 (1999) 119142

129

that your product or system is adequately usable until you have subjected it to usability testing. After assessing standards compliance, conduct usability testing to evaluate how well the design, prototype, or built product/system supports the effectiveness, efciency, and satisfaction of the intended user population as they perform their tasks in the intended context of use. Have a representative sample of users perform realistic tasks, and collect two types of information: Qualitative: The problems and sources of confusion that the users have in doing the assigned tasks, and an identication (if possible) of HCI design changes that could alleviate the problems. Quantitative: Measures of the effectiveness (completeness and accuracy) and efciency (resources expended with respect to completeness and accuracy) of the user tasks performed, and of the users satisfaction in using the product or system to perform these tasks. Use this information to identify and prioritize the areas of the HCI that need further change, and to drive the design of the necessary changes. For more information on usability and usability evaluation, see ISO 9241, Part 11. 3.9. Success strategies Here are some strategies for success when applying standards: Follow the guidelines in the standard throughout product design and development; dont wait until the end to think about standards Make sure all appropriate members of the design team are aware of the standard. It seldom works well to depend on a standards czar to achieve compliance. Use tools whenever possible. For example, some GUI builders take care of setting up the menu bar, adding keyboard accelerators, etc. Do not follow standards blindly. Conduct usability testing to verify that the guidelines are correct for your situation and that the interface is usable. 4. Compliance and conformance evaluation techniques A number of methods have been dened in the eld of software ergonomics for determining the applicability of a standard or guideline and for determining whether the requirement or recommendation has been met. The particular choice of a method depends on various project drivers and constraints. Established methods (extracted from the human factors literature on evaluation techniques) include: Content analysis This refers to the analysis of any documents which may describe the general and specic properties of user interface and the system. Such documents may include design documents containing system and user requirements, manuals, user guides, etc. Content analysis is used to determine whether a standard or guideline is applicable on the basis of its relevancy to the particular system under development or being evaluated.

130

P. Reed et al. / Interacting with Computers 12 (1999) 119142

Documented evidence This refers to any relevant documented information about the task requirements or characteristics, ow of work, user skills, user aptitudes, existing user conventions or biases, test data from the design of similar systems, etc. Such information may be used to determine whether a given recommendation is applicable as well as whether a standard or guideline has been met due to previously document supporting evidence as to the appropriateness of a particular solution. Observation This consists of the examination or inspection of a user interface element for the presence of a particular observable property (e.g., applicability is based on the observation that a source document is used for input or a recommendation has been met because it is observed that instructions are provided on the form). Observations can be made by anyone who has the necessary skill to systematically check the interface and determine if it has the particular properties associated with the applicability of, or compliance with, a given conditional standard or recommendation. Due to their obvious nature, such observations readily can be conrmed by another person. Analytical evaluation Refers to Informed judgments concerning the properties of a user interface by a relevant expert (i.e., of those properties). This method is typically used for the evaluation of properties which can be judged only in the context of other information or knowledge. In addition, analytical evaluation may be appropriate when the system exists only in terms of design documents, user populations are not available for empirical evaluation, or time and resources are constrained. Analytical evaluation can be used to determine whether a particular recommendation is applicable, e.g., to determine if a temporary save function would be appropriate to the users task. In addition, analytical evaluation can be used to determine if a particular standard or recommendation has been met, e.g., analytical evaluation might be used to determine whether a form has distinctive labels. In the above case, distinctive is the judgmental aspect. Analytical evaluation can be performed by any suitably qualied person who has the necessary skill and experience to judge the relevant property of the interface. Where these properties concern the application of ergonomic principles, the expert should possess appropriate skills in software ergonomics. If the properties concern the work environment, system characteristic, or other aspects of the design, the judge needs to be an expert in the particular relevant domain. It is important to note that analytical evaluation can verify the tenability of a design, but cannot validate the design. Validation can be accomplished only by using empirical evaluation. Empirical evaluation Is the application of test procedures using representative end users to determine the applicability of a standard or recommendation. This method is most appropriate when a prototype or the actual system is available and potential or actual user population representatives are available. Many kinds of test procedures could be used, but in each case the test subjects must be representative of the end user population and be of sufcient number that the results can be generalized to the user population as a whole. For example, empirical evaluation to determine the applicability of whether error feedback should be provided as soon as the user completes a eld in a form could be done by having typical users enter and correct eld values under both the condition of providing the feedback immediately after the eld is completed and providing the feedback prior to transmitting the completed form. The task performance of end users using a software application could be analyzed to determine

P. Reed et al. / Interacting with Computers 12 (1999) 119142

131

whether the various standards or recommendations have been met. Such tests could be performed both during the development process (e.g. by prototyping) and after the design and implementation of the system (e.g. by system evaluation techniques) and could be based on both objective and subjective user data. Although empirical evaluations typically are used to determine whether standards/guidelines have been met by comparing the test results against specic software ergonomic standards or recommendations, it also may be necessary to evaluate test results in terms of effectiveness. That is, the dialogue supports the user in his/her task in a manner which leads to improved performance, results in a difcult task being performed with less difculty, or enables the user to accomplish a task that he/she would not have been able to otherwise). It should be noted that empirical evaluation should be conducted by individuals possessing appropriate skills in testing methodology and evaluation techniques. The above methods for determining conformance can be applied to any evaluation of software ergonomic standards or guidelines. Although shalls are given more importance than shoulds in most standards, to optimize the quality of the user interface it still is important to evaluate whether the applicable shoulds have been met. In addition, some software procuring organizations may require that all of the recommendations as well as the requirements be met. 4.1. Evolution of the conditional approach for specifying software ergonomic standards/guidelines The conditional approach to stating ergonomic guidelines for software applications was initially developed by the Human Factors Society Human Computer Interaction (HCI) Standards Committee (see Williams [8]). During the process of dening guidelines relevant to user interface design from various literature sources, the HCI Standards Committee noted that a major problem with most proposed guidelines was that the applicability of a particular guideline depended on a number of variables. The most common variables were: the type of user (e.g. expert vs. novice), the task activity being accomplished, the system conguration being used, and the environment. As a result, the HCI Standards Committee began to use an ifthen structure to state the conditions relevant to the applicability of a guideline where ever sufcient evidence allowed. Since a number of the initial drafts of the dialogue parts of ISO 9241 were submitted by the HCI Standards Committee as a US contribution, this approach carried over into the software parts of ISO 9241. The conditional approach is also used to determine whether or not a major section/ clause of a standard is applicable. For example, if the application does not using pointing devices, recommendations concerning the use of pointing devices would not apply. The following provides some examples of conditional statements extracted from some of the ISO 9241 software parts: User characteristics: If casual or intermittent users may enter data on the form, instructions should be provided on the display (or easily accessible through a help facility). If users are experienced and/or need fast access to specic menu options, one or both of the following methods should be applied:

132

P. Reed et al. / Interacting with Computers 12 (1999) 119142

Task characteristics: If the task requires error management by the user, the dialogue should provide a means (information and/or function) of enabling the user to continue. If information concerning unavailable options is not required for the task or other supporting activities (e.g., training), only options available to the user. System capabilities: If a system or application has modes (i.e. a specic user action has different results depending on the state of the system), users should be able to discriminate the current mode. If the system response to option execution will be delayed (more than three seconds after initiation), an indication should be provided to the user of the system. Interface design characteristics: If options are placed in columns, options and groups of options should be visually distinct from one another and should be arranged to minimize search time. If brief error messages are displayed, users should be able to request more detailed on-line information or should be referred to additional off-line information. 4.2. Traditional view of conformance to a standard and problems for software Most standards, particularly hardware standards, specify attributes that can be objectively measured or directly observed in some way (e.g., voltage, dimensions, pressure). Conformance to such standards can be demonstrated by measuring the product or device to determine whether it complies with the requirements stated in the standard. In addition, these standards can establish those attributes that are required (shall statements) from those that are desirable, but not required (should statements). Unfortunately, in the domain of software user interface standards, it is rarely possible to unambiguously state what is required and what is desirable. The conditional nature of many software ergonomic standards statements requires the applicability of a particular statement to depend on a number of variables. Due to these often-complex dependencies, most software ergonomic experts are unwilling to designate such statements as a requirement because of the difculty of precisely determining the dependencies. An additional concern is that developers will only pay attention to requirements and ignore recommendations. It is also important to note that organizations that utilize a consensus process for developing standards (e.g., ISO and ANSI) have a much more difcult time obtaining agreement on shalls than do organizations which follow a less open review process (e.g., ITU, government agencies). 4.3. A pragmatic approach to conformance with software user interface guidelines Utilizing the conditional approach and the evaluation methods described above, a two step process for determining conformance to software ergonomic standards/guidelines was developed based on extensive work in ISO. The rst step of this process is to determine the applicability of the standard or guideline and the second step is to assess

P. Reed et al. / Interacting with Computers 12 (1999) 119142

133

Fig. 3. ISO 9241 conformance evaluation process ow chart.

134

P. Reed et al. / Interacting with Computers 12 (1999) 119142

whether the requirement/recommendation has been met by the developer, i.e. compliance (see Fig. 3 for a ow diagram of the process). Applicability can be determined both as part of the early design process when the designer is specifying elements of the interface, and later by both the designer and evaluators as part of any heuristic evaluation of the design. It is strongly recommended that the evaluation be made using the most detailed method which is appropriate to the requirement or recommendations. For example, it would not be necessary to do usability testing (empirical evaluation) to determine that menu options keyboard equivalents (underlined character) are shown when it can be readily observed. The intent of the pragmatic approach is to minimize the time and effort required of an organization to determine whether they comply with a standard or guideline. The pragmatic approach to conformance has been incorporated in the software parts of ISO 9241 and is planned to be utilized in the HFES 200 standard as well. 4.4. Conformance to the ISO 9241 software standard All of the dialogue parts of ISO 9241 now use a common approach to conformance as was agreed by the national bodies on Part 14. This approach includes the two step process (Applicability and Adherence) as a sample procedure, and allows users of the standard to use other processes. It should be noted that the term adherence was agreed because compliance was thought to be too related to requirements and would not apply to recommendations. The only actual statement of conformance in 9241 Part 12 through 17 is: If a product is claimed to have met the applicable recommendations in ISO 9241(14), the procedure used in establishing requirements for, developing, and/or evaluating (menu dialogues) shall be specied. The level of specication of the procedure is a matter of negotiation between the involved particles. The two step process recommends that the user of the standard rst determine which of the recommendations are applicable and then determine whether those recommendations that are deemed applicable have been met. It should be noted that the approach described in ISO 9241 suggests applying the method most relevant and cost effective to determining the applicability and adherence for a particular recommendations. For example, if the recommendation specied an attribute that could be directly observed (e.g. left justication of a menu option list), the method indicated as the most relevant would be observation. Thus the sample procedure is intended to be as cost effective as possible. Several other means for determining conformance also have been devised. Some of the test houses in Europe (e.g. TUV Rheinland) have been evaluating products for their conformance to the software as well as the hardware parts of ISO 9241. The actual method used for these evaluations is not known since the test houses have not revealed the exact procedure used. A somewhat different approach has been used by Lloyds Register who has developed a procedure to assess the application of the ISO 9241 software parts as part of the overall software quality assessment procedure. Another approach to evaluation of the software ergonomic parts is the 9241 Evaluator developed in Germany (Oppermann, R and Reiterer, H. [9]). In their paper, Oppermann and Reiterer provide an overview of different evaluation techniques and describe the 9241 Evaluator as an expert-based method for conformance testing of the 9241 standard. While an important step towards the development of a performance support system for standards evaluation, the 9241

P. Reed et al. / Interacting with Computers 12 (1999) 119142

135

Evaluator is intended to be used only by experts after the system has been developed (at least to a prototype stage). However, the ISO 9241 software parts state that the intended users include designers as well as evaluators and that the standards should be used during all stages of system design and development. All of the approaches to conformance described above satisfy the conformance clause stated in the software parts of ISO 9241. The HFES 200 Committees current direction is to use the same basic approach to conformance as stated in ISO 9241. However, it is important to note that many organizations (particularly in the United States) are stating the same recommendations as can be found in the ISO and emerging ANSI software ergonomic standards as requirements (i.e., shall statements). For example, many of the statements found in the recently released Federal Aviation Administration The Human Factors Design Guide section on human computer interfaces contain shalls. 4.5. Utilizing the pragmatic conformance approach for evaluating other software ergonomic standards or guidelines As noted above, the two step process and the methods used to determine applicability and adherence can be used to evaluate many types of standards and guidelines. The process is particularly relevant where standards or guidelines have been qualied in terms of the conditions in which they apply (e.g., conditional recommendations as described above). Currently many style guides require users to complete a check list indicating both which style guide requirements and recommendations apply and which have been met. Although most style guide requirements and recommendations are look and feel type statements which usually can be evaluated for compliance by observation, many recommendations concern design strategies which could be evaluated by means of many of the other applicability and adherence methods described above. The pragmatic conformance approach, particularly when a checklist is incorporated as is used in Annex A of the 9241 software parts, also provides an audit trail which can be used to satisfy ISO 9000 and ISO 9001 requirements. 5. Future challenges and the evolution of HCI standards Two issues commonly identied in discussions of standards are: (1) Standards are not useful., and (2) Standards stie creativity. A fundamental issue underlying each of these statements is the belief that technology is changing so rapidly and standards take so long to create that they either do not address the current set of technologies in which design is taking place or they should not address the technologies. The argument that they should not address the current environment is based on the assumption that the behavior that we need to measure in order to provide empirical justication for standards is only now emerging for these technologies. Standardizing prematurely may result in standardization of the wrong things. Further, it would follow that creating standards that apply to future technologies should be impossible. 5.1. Moores law, the Internet, and HCI standards There is a great deal of face validity to these arguments. Many humancomputer

136

P. Reed et al. / Interacting with Computers 12 (1999) 119142

interface designers have moved from programming with punch cards and paper tape to designing animations, collaborative video conferencing, and virtual reality interfaces, within the time it takes a child to complete their schooling. The explosive growth of the Internet and novel technologies based on the Internet is well known. A recent conference marking the rst 50 years of computing invited pioneers and seminal thinkers in the computing eld to use the last 50 years as a foundation for predicting the next 50. Bell and Gray [10] pointed out that with processor speed, storage capacity, and transmission rates doubling every 18 months consistent with Moores Law, the clear trend is the emergence of essentially zero-cost, specialized, fully networked computers embedded in virtually everything around us, and attached to our bodies. Bell and Gray state The only limits will be our ability to interface computers with the physical world-that is, to design the interface between cyberspace and physical space. It would appear that virtually any interface that can be conceived will be possible to build. Nathan Myhrvold [11], at the same conference, pointed out that software is also following Moores Law. He stated So we are talking 20 years hence about computers being able to do in 30 seconds what one of todays computers at a comparable price would take a year to do. But 40 years hence we are talking about a computer doing in 30 seconds what one of todays computers would take a million years to do. So what will we do with all that power? Well, the answer is software. The increasing pace of software evolution will drive corresponding changes in HCI design. Those looking at a standards process that attempts to integrate years of design guidance about what works and does not work in user interfaces with a given technology may be justiably skeptical, especially given that it may take years for the standard to be approved once it is dened. We already know that in the time it has taken to approve standards addressing personal computer technology, we have moved into designing for virtual reality. As we move forward, we will be designing the next generation of interfaces without the benet of standards utilizing behavioral research focused on human interaction with next-generation technology. Fortunately for designers, human capabilities and limitations in perception and performance do not evolve as rapidly as modern computing technology! Practically speaking, therefore, can standards be written in an environment driven by Moores Law? Can standards be written that will be useful for designers working with technologies emerging for the Internet, computers appearing in novel forms such as telephones and televisions, and technologies with which we have had even less experience? Should they be written? We believe they can and should be written. For useful standards to be written for the next generation of interfaces, however, we will need to continue to improve the process of writing and maintaining the standards and the sophistication of our taxonomy of standards will need to increase. 5.2. Lessons we have learned In the last few years, several important lessons have been learned that will shape effective standards activities in the future. First, a key assumption that we reached in developing the standards for ANSI and the work for ISO is the assumption of humility. For standards to accommodate rapidly changing circumstances, they need to build on what should be done more than what shall be done. In other words, they need to provide

P. Reed et al. / Interacting with Computers 12 (1999) 119142

137

guidance and exibility, such that the nal arbiter of a design is the user. Users adapt effectively to the situation in which they nd themselves. The combination of the design elements and the structure of the design as it supports the users task can have as much inuence on user satisfaction and performance as individual aspects of the design that have been designed according to a standard. Applying the guidelines may even require tradeoffs between guidelines, such that a given guideline must violated in order to apply a guideline that is more important in the circumstance. The conditional approach to specifying design guidance must be extended to provide new guidelines that better address the context of application will be needed, and we will need to continue to write guidelines that can be adapted to and provide guidance in changing circumstances. A second lesson that will shape standards in the future is that it is important to continue to improve our ability to dene standards that are useful and yet that are as technology independent as possible. Fundamentally, human beings do not change as rapidly as the technologies they are using. At a very basic level, our needs remain the same. The human perceptual system works the same now as it did 15 years ago. The properties of the cognitive system remain the same. This, I believe, is the key to standards in the future. If we can dene standards in terms of the reliable phenomena of human behavior, and eventually in terms of any theoretical consensus, we are more likely to dene standards that are useful when applied to emerging technologies. This implies that a more tightly integrated dialogue between academia and industry will be needed, with research directions being set at least in part as a result of direction from those writing or requiring standards. A third lesson is that the best standards emerge from data rather than politics. While we began the standards effort attempting to build standards largely from well controlled laboratory studies, over time we have found that data and conclusions emerging from practice are also important in shaping a useful standard. Harnessing the wealth of data that is automatically generated in the current environment of design mutation and evolution on the Internet and in the world of consumer products would be expected to further inform the standards process. It would be reasonable to predict that a day of activity on the Internet results in more people being exposed to more design alternatives than all the labs where testing is currently taking place could produce in a year. 5.3. Improving our approaches to user interface guidelines and standards Standards efforts, like usability work in general, may need to harness the trend of trying a design and rapidly xing it in response to feedback from the eld in order to respond more quickly to emerging technologies. Indeed, one approach to increasing the ability of standards to reect emerging technologies is to explore new ways of developing standards, new processes. Perhaps the solution is in nding new ways to involve the stakeholders in the development process, nding ways to access emerging data being collected in otherwise inaccessible corporate labs, and shortening cycles of approval. The challenge, of course, is that national and International standards involve consensus building. It is not clear that we have identied the processes yet that meet the needs of technical standards development as well as the political and cultural realities of standards approval. Another approach for developing useful standards in the future is to dene standards

138

P. Reed et al. / Interacting with Computers 12 (1999) 119142

that are virtually technology independent. These standards would dene how task properties should inuence design primitives, and their interactions. An example of this would be a set of standards that recommends list length for the task of selecting a category from a list presented auditorially, when the contents and names of the categories are not known ahead of time. The approach that would be most effective to these standards would be the integration of behavioral science ndings and theory focusing on the capabilities and limitations of human beings, and applied research identifying the fundamental elements of tasks as shaped by the properties of the user. The next level of standards would be standards applying to combinations of these primitives associated with generations of technologies. An example of this kind of standard would be standards that describe the design of a form that supports pointing and clicking, or the category of window currently typically used in Macintosh and Windows95 applications. This is the level of standard that is most typically dened today. These kinds of standards currently are dened based on research using the technologies for which the standards are being written (which means the standards appear well after they are needed by designers), and based on the years of experience and logical thinking of those writing the standards. In the future, this level of standard could initially be shaped in part by applying the higher level standards to relevant properties of the specic technology (coupled with that same experience and logical thinking that is currently being applied). The resulting standards could be validated and extended (where the higher levels do not sufciently address the design options) through usability studies of specic implementations. With a sound base of technology-independent standards, it should be easier to produce exible, useful standards for emerging technologies in a timely way. This structure depends on a richer understanding of human behavior in applied settings than we currently have, and it has the potential to provide a mechanism for useful design guidance to precede technology-dependent data collection. A process governor will have to be added in order to restrain standards bodies from spinning too far from the empirical base underlying the two levels. While lower level standards should be consistent with higher level standards, they would also be expected to be inherently technology dependent. These standards would include standards that attempt to address consistency of applications built for a given operating system and standards that attempt to dene unique brands within such environments. An area that is missing from even this view of standards, however, is the relationship between design for usability and emotion. Increasingly it is being recognized that even a successful business application needs to be attractive and often fun. Applications built using emerging multi-media technologies increasingly explicitly engage the total person. When the aesthetic elements of the design result in an application being learned more completely and used more often, these elements should be included in the denition of usability and should be instantiated in standards. Macroergonomic considerations of when applications are adopted and product usability as it interacts with brand identity also suggest that the denition of usability will be broadened in the future. To the extent that standards serve as an instantiation of accumulated knowledge about how to create usable design, the standards will also address a wider range of design considerations. Extending our knowledge of the relationship between aesthetic elements, design that impacts enjoyment, the creative design process itself and performance, and design as it

P. Reed et al. / Interacting with Computers 12 (1999) 119142

139

relates to the diffusion of technology is needed. Combined with the standards approaches that we have just dened, these information sources will enrich the body of available design guidance, and produce a set of standards that are technology independent and that can be effectively applied to technologies that will emerge in the future.

Appendix A. Process standards


Process standards (HCI/GUI design, evaluation, and verication) Standard Title Contents 9241 Part 11: guidance on usability Source

ISO 9241 (Part 11) Ergonomic requirements for ofce work with visual display terminals MIL-STD-1801

ISO/CD/13407

International Standardization Organization, 1 Rue de Varembe Case Postale 5b, CH-1211, Geneva 20, Switzerland; Tel.: 41-22-734-1240; URL: www.iso.ch Usercomputer (Various requirements for Wright-Patterson AFB ATTN: interface verication) ASD/ENES Wright-Patterson AFB, OH 45433 International Standardization Assessing the benets of Human-centred Organization, 1 Rue de Varembe design for human-centred design. Case Postale 5b, CH-1211, Geneva interactive systems Interactive system development processes and 20, Switzerland; Tel.: 41-22-734-1240; human-centred design. Planning the human-centred URL: www.iso.ch design process. Humancentred design activities. Managing and documenting human-centred design.

140

Appendix B. Software user interface standards Software standards and guidelines (HCI/GUI appearance and behavior) Standard ISO 9241 (Parts 10-17) Title Ergonomic requirements for ofce work with visual display terminals Contents Part 10: Dialogue principles. Part 11: Guidance on usability. Part 12: Presentation of information. Part 13: User guidance. Part 14: Menu dialogues. Part 15: Command dialogues. Part 16: Direct manipulation dialogues. Part 17: Form lling dialogues. Same as ISO 9241, Parts 10-17, plus color accessibility for people with disabilities voice input/output Source International Standardization Organization, 1 Rue de Varembe Case Postale 5b, CH-1211, Geneva 20, Switzerland; Tel.: 41-22-734-1240; URL: www.iso.ch

P. Reed et al. / Interacting with Computers 12 (1999) 119142

HFES-200 ANSI Project (under development; canvass draft due in late 1998)

Software user interfaces

ESD-TR-86-278

Guidelines for designing user interface software

MIL-STD-1472D (Section 5.15)

P1201.2 This failed two ballots and was withdrawn. It is here because other documents may reference it.

Human engineering design criteria for military systems, equipment and facilities Graphical user interface drivability

Data entry, data display, sequence control, user guidance, data protection, data transmission Data entry, data display, interactive control, feedback, prompts, defaults, error management Keyboards, pointing devices, menus, controls, windows, user guidance, common user actions

Human Factors and Ergonomics Society, PO Box 1369, Santa Monica, CA 90406, USA; Tel.: 1-310-394-1811; URL: www.hfes.org Electronic systems division: Air Force systems Command, Hanscom AFB MA 01731 Commander, US Army missile command. ATTN: AMSMI-EDS Redstone Arsenal AL 35898 IEEE Standards Ofce, 445 Hoes Lane, PO Box 1331, Piscataway, NJ 08855, USA; Tel.: 1-800-678-4333; fax: 1-908-562-1571

Appendix C. Hardware user interface standards Hardware standards and guidelines P. Reed et al. / Interacting with Computers 12 (1999) 119142 Standard ANSI/HFES-100-1988 (currently under revision) Title Human factors engineering of visual display, terminal workstations Contents Working environment visual display design, workstation design, keyboard design, measurement techniques Source Human Factors and Ergonomics Society, PO Box 1369, Santa Monica, CA 90406; Tel.: 1-310-394-1811; Fax: 1-310-394-2410; URL: www.hfes.org International Standardization Organization, 1 Rue de Varembe Case Postale 5b, CH-1211, Geneva 20, Switzerland; Tel.: 41-22-734-1240; URL: www.iso.ch

ISO 9241, Parts 3-9

Ergonomic requirements for ofce work with visual display terminals

MIL-STD-1472D (selected parts)

NASA-STD-3000 (selected parts)

Human engineering design criteria for military systems, equipment and facilities Mansystems integration standard

Visual display requirements, keyboard requirements, workplace requirements, environmental requirements, display requirements with reections, requirements for displayed colours, requirements for non-keyboard input devices (various)

Commander US Army missile command, ATTN: AMSMI-EDS Redstone Arsenal AL 35898 Johnson Space Center, ATTN: SP34 NASA Houston TX 77058

(various)

141

142

P. Reed et al. / Interacting with Computers 12 (1999) 119142

References
[1] J. Nielsen, Usability Engineering, Academic Press, New York, 1993. [2] D. Mayhew, Principles and Guidelines in Software User Interface Design, Prentice-Hall, Englewood Cliffs, NJ, 1992. [3] Apple Computer, Macintosh Human Interface Guidelines, Addison Wesley, Cambridge, MA, 1995. [4] ISO 9241-14: 1997 (E), Ergonomic requirements for ofce work with visual display terminals (VDTs): Menu dialogues, International Organization for Standardization, Geneva, Switzerland, 1997. [5] C.M. Brown, Human-Computer Interface Design Guidelines, 1988. [6] W. Smith, ISO and ANSI Ergonomic Standards for Computer Products, Prentice-Hall, Upper Saddle River, NJ, 1996. [7] J. Karat (Ed.), Taking Software Design Seriously: Practical Techniques for HumanComputer Interaction Design Academic Press, New York, 1991. [8] J.R. Williams, in: G. Salvendy, M. Smith (Eds.), Guidelines for dialogue design, in Designing and Using HumanComputer Interfaces and Knowledge Based Systems, Elsevier Science, Amsterdam, 1989. [9] R. Oppermann, H. Reiterer, Software evaluation using the 9241 Evaluator, Behaviour and Information Technology 16 (4/5) (1997) 232245. [10] G. Bell, J.N. Gray, The revolution yet to happen, in: P.J. Denning, R.M. Metcalfe (Eds.), Beyond Calculation: The Next Fifty Years of Computing, Copernicus, New York, 1997, p. 3. [11] N. Myhrvold, The future of software, the software industry, and possibly, the features of Windows 47, Speech given at ACM97, The Next 50 Years of Computing, March 3, 1997. Viewable at www.vxtreme. com/live/acm97/archived/nathanmyhrvold/nmyhrvold.html.

You might also like