White Paper: Advanced Testing Techniques for User Documents

Abstract
The credibility of any organization depends on the user documents that accompany its products. There is always a challenge in creating good products and good user documents to ensure a market niche. A product undergoes many quality checks before its release. There are many documents that do not provide information to the user at times of need. Either the information is missing in the document or the user spends valuable time and effort in searching the information due to missing table of contents or bad index. All these problems can be avoided if the user document too undergoes rigorous quality tests. The various methods of testing user documents are elaborated in this paper.

Introduction
A user document provides information of the product to its end users. For example, installation guide, design specifications, maintenance guide and operation guide. Today, it is mandatory for all manufacturers including software to provide user manuals or user documents to their customers. User documents are of two types: Hardcopy documents are commonly available for electrical accessories and electronic gadgets. Online Help which, is available for most of the software applications.

Providing a good and effective user document is a challenge for many organizations. Due to stiff competition, organizations are investing a lot of time, resources and capital in developing good user documents for their products. To check the effectiveness of user documents, many organizations resort to testing the user documents, also known as usability, prior to product launch. The theory being that the documents should be able to serve the purpose for which it was developed.

Testing User Document or Usability

Documents differ from one another in terms of scope, audience, terminologies and content. For an effective document it is important to test user documents also known as usability. Usability is defined as the extent to which a product can be used by a user, to achieve specified goals, with effectiveness, efficiency and satisfaction in a specified content of user (ISO 9241-11). According to the United States Department of Health and Human Services, usability measures the: Ease of learning: Whether a first- time user of the application can accomplish basic tasks Efficiency of use: Whether an experienced user can accomplish tasks easily Memorability: Whether a user can subsequently use the application without referring to the help function or the product manual. Error frequency and severity: The kind of errors generated and how the user is able to resolve the errors, also known as troubleshooting Subjective satisfaction: Opinions of the user about the application

The advantages of providing good user documents are: Less dependency on the Technical Support staff Cost saving for the organization Increases the credibility of the organization Satisfies the customer Enhances customer base

Types of Usability
Usability can be achieved in two ways:

The Writers Block

www.twb.in

Page 1 of 4

White Paper: Advanced Testing Techniques for User Documents

Usability Evaluation: Evaluation includes considerable amount of judgment on the part of the evaluators. This is because the evaluators prepare the final test report. It is important to conduct usability as soon as the draft document is ready. Surveys and questionnaires: In this kind of evaluation, the evaluators select a group of users or participants for the test. The participants come from varied backgrounds. The participants then work with the product and the evaluator conducts an interview. The evaluators ask questions that are open ended, thereby encouraging the test users to provide input about their experience with the product. While conducting a survey, the evaluator should not use technical jargons, as it would confuse the participants. The evaluator should also adopt a neutral stand while answering any queries from the participants. The questions should be similar to real life situations. The advantage of this technique is that it provides detailed information. As the participants are from varied backgrounds, their inputs will provide valuable information on the usability of the product. The same technique could be adopted for checking the usability for Web sites. The disadvantage of this technique is that it cannot be used for conducting remote usability evaluations. Observational evaluations: This evaluation is based on observing the participants while working with the product. Each action performed by the participant should be observed. For example, their reactions while using the index to gather information. There are two methods of observational evaluation: Obtrusive Observation: In this form of evaluation, the evaluator has interactions with the participant. The evaluator asks open-ended questions. For example, what do you think when you see this? The key behind open-ended questions is that the participant is prompted to provide details. At the end of the evaluation process, the evaluator collects valuable information about the product. Unobtrusive Observation: In this form of evaluation, the evaluator does not interact with the participant. The evaluator makes observations of the participants actions and notes down questions raised by the participant during the evaluation process. It is important to inform the participants prior to the start of the process that there will be no interaction during the whole process. This is because human mind has a tendency to help at times of need. Irrespective of the type of evaluation process adopted, after the process, it is advisable for the evaluator and the participants to have a discussion. Evaluator must encourage the participants to ask questions or talk about their experiences while performing a usability test. Heuristic evaluation: This is a popular evaluation technique. This involves finding usability issues in a user interface by having a small number of evaluators. The evaluators examine the user interface and judge its compliance with usability principles or heuristics. Heuristics can be applied to Web sites or Web applications such as online Help. It is advantageous to have more than one evaluator for heuristic evaluation to enhance the possibilities of finding out usability issues. Usability Testing: In this type of evaluation, the participants actually work with the product. It is generally advisable to have a complete user document prior to the stage as the effectiveness of both can be verified. Cognitive evaluation: Cognitive means the act or process of knowing. For an effective user document, it is important to conduct a cognitive walkthrough to identify areas in the user document that could pose a problem to the user. There are situations wherein a user refers to a user document to find relevant information and either it takes longer time than required to find the information or the information is not present in the user document. Cognitive evaluations should be conducted at the development phase of the Document Life Cycle (DLC) as it is easier to incorporate changes then rather than at the end. An important point to remember is that the user document should be user friendly and task oriented. Keeping in mind the task likely to be preformed and the sequence of tasks, Technical Writers is able to include information that would help the users. Writing a user document in a language that users speak helps in creating an effective user document.
www.twb.in

The Writers Block

Page 2 of 4

White Paper: Advanced Testing Techniques for User Documents

While performing cognitive evaluations for Web sites or Web application, it is advisable to include people with disability as participants. Observing disable participants interact with the Web site or Web application will help to design the Web site or application effectively. For example, too many hyperlinks can be avoided by incorporating mouse rollover instructions. Expert reviews: The effectiveness of a document depends on its review. An effective user document should go through review with the aim of usability. There are different types of review such as: Technical review: The document is reviewed alongside the application. This is done to check whether the procedure of using the application is the same as documented. Examples cited in the document are also validated. The document is also checked for the correct use of technical terminologies. Peer review: In a peer review, a colleague reviews the document against the application for its working and the documented instructions. Quality review: In a quality review, the document is reviewed for consistency, presentation, readability, and copyediting. The document is checked for navigation by virtue of the index, table of contents, cross-references and hyperlinks. User review: In this review, a participant works with the application. The participant should be instructed to refer the document to accomplish the tasks. This will help the author to understand if there are any instructions that are missing in the documentation that might affect its usability. The author also gets an insight into whether the user is able to navigate easily through the document. Publication review: In this review the user document is reviewed to ensure that necessary changes have been incorporated. Acceptance Testing: After the completion of a user document, customers perform a check of this document against the application. Customers check for the appropriate use of screen terminologies, instructions to accomplish a task, ease of navigation within the document, index entries and the language. To avoid failure of a document, it is important to have exhaustive checks of the document at the development and review phase.

When to Conduct Usability Tests

Usability tests should be conducted throughout the DLC. Before commencing work on a new document, the documentation team should identify any problems with the previous version of the document. For example, a new user manual is required for an existing product with few enhancements. Such an activity will help repeating the same mistake. During the development phase of the DLC, it is advisable to have usability tests conducted on a draft version of the document. This can be conducted by asking colleagues to review the document against the application. This will ensure that the document has the necessary information in a user-friendly language. Further more it also ensures that problems faced in the previous version of the document have been addressed. The final usability testing can be conducted after all the preliminary tests with a larger group of audience.

Cost for Usability Tests

The cost for usability tests depends on the nature of the test, number of tests and the number of participants involved. Time is a major factor for usability tests and accounts for: Planning the test Recruiting the participants from either the same organization or from a recruitment agency Observing the participants while conducting the tests Analyzing the results Discussing the results with team and developers Incorporating the results

Cost for conducting a usability test includes: The Writers Block

www.twb.in

Page 3 of 4

White Paper: Advanced Testing Techniques for User Documents

Renting a space to conduct the test Renting video cameras, projectors, computers and so on for the test Transporting the equipments for the test to a location Printing the questionnaires and other related documents Gifts for the participants Food and beverage costs for the participants and the team

Though usability testing may be a cumbersome process involving a lot of preparation and planning, nevertheless the credibility of your organization is directly proportional to the usability testing.

References:
Lewis Clayton and Rieman John, Task Centered User Interface Design. Accessed on 03-Oct-06 from www.hcibib.org Kaner Cem and Pels David, User Documentation and Customer Care. Kanter Laurie,1994. Techniques for Managing a Usability Test. Transactions on Professional Communication, Volume 37 (3), pages 1-9. Morgan Kate, How to find out if your document really works. Accessed on 05-Oct-2006 from www.lawfoundation.net.au Online resources www.usability.gov. Date accessed 05-Oct-2006 www.w3.org Date accessed on 09-Oct-2006

TWB Contacts
TWB is a leading technical documentation development and design company. Should you want more information from the TWB repository of information on techniques in technical documentation please contact:

The Writers Block

www.twb.in

Page 4 of 4