Usability and the Web: An Overview

by George Murray and Tania Costanzo

Network Notes #61
ISSN 1201-4338
Information Technology Services
National Library of Canada

August 1999

1.0 Introduction

An organization's Web site is a gateway to its information, products and

services. As such, it should ideally be a reflection of the needs of the
clients it serves. Unfortunately, Web site design and development is
often driven by technology or by organizational structure or business
objectives, rather than by user needs. In recent years however, Web site
owners and developers have gradually begun to acknowledge and
address the issue of usability.

The key to Web site usability is ensuring that the site is both useful and
usable for the intended audience.

This paper will review usability issues as they apply to the Web, and
provide an overview of different usability techniques that can be used in
Web design and development.

2.0 Usability and the Web

The discussion as to what constitutes a " usable " Web interface is

ongoing. To a certain degree usability depends upon the purpose and
target audience of a particular site. However, there is general agreement
that a usable Web interface is one that is accessible, appealing,
consistent, clear, simple, navigable and forgiving of user blunders.

Web users are notoriously impatient and fickle - if they are frustrated on
a site they will quickly go somewhere else and likely not return. For
commercial sites this is critical. A site that is confusing or difficult to use
may result in the loss of clients or reduced revenues due to unsuccessful
transactions. For informational sites, lack of attention to usability
principles may result in users being unable to find the information they

1 of 11
Usability and the Web: An Overview

need. In recent Web usability studies, users were able to find the correct
answers to test questions only 42 percent of the time. 1

Usability engineering for the Web grew out of the software development
discipline of Human Computer Interaction (HCI). However, the Web is
different from software, and the nature of the Web poses new challenges
to designers and developers who are trying to incorporate usability into
their sites.

Due to the global nature of the Web and the wide ranging
demographics of people accessing the Internet, a target audience
can be difficult to define.
Diversity in end user configurations (hardware, software, browsers,
connectivity and bandwidth) means that users may have wildly
different experiences of the same site.
Inflated user expectations of Internet technology can be difficult to
satisfy.
The rapidly changing nature of the Web results in short
development schedules, making it difficult to incorporate
user-centered design techniques.
Unlike a software package, the user has not made an investment in
a particular site, and other options are easily available and
accessible.

3.0 Focus on the User

Before employing any usability techniques, the first challenge is to define

the Web site's target audience. Whether the site is at the beginning
stage of development, at the user testing stage, or due for a redesign, it
is critical for the designers to have as much knowledge and
understanding as possible of the site's users. The second challenge is
then to find actual representative users to participate in design or testing
activities.

3.1 Audience Definition

Identifying a Web site's target audience is generally very straightforward.

A software company's target audience is developers; a sales company
has a marketing profile of their ideal customer; an academic library's
target audience is made up of faculty and students. However, simply
identifying a site's audience does not provide the information about user
needs, characteristics and abilities that is needed to create an audience
definition.

There are a number of methods of collecting information for audience

definition, including focus groups, individual interviews, demographic
research and collection of feedback from a pre-existing site.

2 of 11
Usability and the Web: An Overview

One of the most common ways used to collect information for Web site
audience definition is a user survey. On-line surveys can be posted on a
pre-existing site, e-mailed directly to known users, or posted to
newsgroups and mailing lists.

An audience definition survey may collect any or all of the following

information:

user profile (demographic information, job or recreational

preferences),
surfing profile (how do they use the Web),
site usage (likes, dislikes, task requirements), and
level of technology (hardware, browser type, connection speed).

The survey method is relatively quick and inexpensive but the drawback
to this method is that it may not result in a representative sample. For
example, the respondents' may only be those dissatisfied with a current
site or sophisticated enough to use an on-line form. For this reason,
information gathered in an audience definition survey should ideally be
supplemented by other sources.

3.2 Finding Representative Users

There are a number of different approaches to finding representative

users to participate in site design or testing activities. If the focus is on
performing small, " quick and dirty " tests to provide some fast results,
acquaintances or colleagues may be suitable. For more formal,
extensive or complex activities, the following sources may be used:

customer lists (including marketing mailing lists or records of

sales),
related organizations or associations,
e-mail discussion lists or newsgroups,
conferences and events,
temp agencies or focus group companies.

4.0 User-Centered Design

The key to building a useful and usable Web site is to involve the user in
the development process from the beginning. In recent software
development, user-centered design has generally led to improvements in
software interfaces and a higher likelihood that the software will actually
deliver what the user wants and needs. Computer users are now
expecting that Web sites will likewise be easy to use.

4.1 Focus Groups

The traditional focus group consists of a small group of users who

3 of 11
Usability and the Web: An Overview

discuss issues and concerns, guided by a moderator. The discussion is

recorded, sometimes on videotape, and then analyzed. Focus groups
can also be conducted electronically, via groupware or informally through
e-mail or pre-existing newsgroups.

Focus groups are primarily a market research technique. In Web site

design, the best use of a focus group is to determine users wants,
expectations or perceived needs.

Focus groups are not a good method for assessing interface usability
because they are a forum for opinions only and don't show how the user
would actually interact with the site.

4.2 Card Sorting

Card sorting exercises involve the user in determining site structure and
content organization.

Individual users are provided with index cards containing content

descriptions and are asked to organize the content into rational
groupings then provide labels or descriptions for their groups. Users may
also be given blank cards to create content categories that they feel are
missing. The site designers then analyze the different groupings for
consistency and develop an overall site structure.

4.3 Category Identification, Category Description and Category

Labeling

Category identification, category description and category labeling are

follow-up activities to card sorting. These activities help to ensure that the
content categories, labels and descriptions that have been developed
meet user expectations, and alert the designer to changes that may be
required.

Category identification involves presenting users with the content

groups and labels, and asking them to indicate which label they would
expect to " click on " to find each content group.

Category description involves presenting a user group with the content

labels and asking them to describe the content they would expect to find
behind a particular label.

Category labeling involves presenting users with several labels for

some sample content and asking them to choose the most appropriate
label.

4.4 Wire-Frame Validation

In wire-frame validation, the data gathered from the card-sorting and

4 of 11
Usability and the Web: An Overview

follow-up exercises is used to create one or more simple structural

models or " wire-frames " of the site. A wire-frame model consists of
simple page mock-ups that show the site structure and location of
content. Wire-frame models can then be tested by users to determine
whether the site structure makes sense from a user perspective.

Because no design elements are involved, wire-frames can be quickly

assembled and tested. This allows for iterative development, i.e.,
wire-frames can be revised and re-tested a number of times until the site
structure is finalized. 2

4.5 Greek Pages

" Greek pages " can be used to provide insight into whether a particular
Web page design and layout will be effective in helping users to navigate
the site and find content.

On a Greek page all the text has been replaced with nonsense. Users
are then asked to see if they can identify specific content elements of the
page, such as the page title, main content items or navigation elements
by relying solely on the page layout and graphic design.

5.0 Usability Inspections

Usability inspection methods are based on having evaluators inspect a

user interface to identify usability problems in the design.

Some inspection methods also rate the severity of usability problems as

well as evaluating the overall usability of an entire system. The various
methods can be distinguished in two ways:

the evaluation criteria used by the inspectors, and

the process they use to reach their judgments.

Depending on the method, an interface may be inspected by one

evaluator at a time, or by a group working together. Evaluators are often
usability specialists, but can also be knowledgeable developers or
users.

Usability inspection methods have a number of advantages:

Because these methods are based on inspection rather than

actual testing, they don't require a working system. They can be
done on prototypes, even paper.
They can be performed early in the development cycle, and
repeated easily at different stages of development.
They can be integrated easily into many different development
methodologies.
They are relatively cheap to use, and don't require special

5 of 11
Usability and the Web: An Overview

equipment or labs.
An inspection by a usability specialist can be a simple way to
transfer usability knowledge to a larger group.
Expertise is not required to conduct a simple inspection. An
effective, basic inspection can be done with as little as one hour of
training.
They provide quick, concrete results.

In the world of Web site design, the most popular usability inspection
method is heuristic evaluation. 3

5.1 Heuristic Evaluation

Heuristic evaluation involves having a small group of evaluators (usually

three to five) examine an interface and compare it against a set of
recognized usability principles called heuristics.

Depending on the circumstances, the site designers and evaluators may

choose to use a pre-existing set of heuristics, or may develop a
customized set of principles to address specific site issues. 4 A search
of the Web will turn up literally hundreds of guidelines, standards, and
usability criteria that can be used to develop a list of general principles
for evaluation purposes. 5

In a heuristic evaluation the evaluators work independently, identifying

usability problems and matching each problem to the usability principle
that it violates. In some cases, scenarios (list of steps the user would
take) are used to assist evaluators to understand realistic tasks. When
the evaluators are finished all the results are merged, and the problems
are rated according to severity.

The drawback to heuristic evaluation is that it does not provide design

solutions, it simply identifies usability problems. Also, it does not
address the positive aspects of the design. However, the general
usability principles can be used as guidelines for fixes or re-designs and
a debriefing or brainstorming session at the end of the process may also
help to generate design ideas.

6.0 Usability (User) Testing

Preconceptions that user testing is expensive, difficult and

time-consuming prevent many Web designers from doing any testing at
all. However, there is general consensus on the Web that doing any user
testing, however minimal, is better than no testing at all, and that useful
results can be obtained in a short amount of time with a relatively small
number of users.

While large, complex corporate sites may use elaborate user testing

6 of 11
Usability and the Web: An Overview

methods such as usability labs, videotaping, eye-tracking etc., it is also

easy to do quick, cheap and simple tests that will give valuable results.

Basic user testing involves having real users performing real tasks
while explaining their decisions by thinking out loud. Test users should
be representative of the site's target audience and the test tasks should
represent the way users will actually be using the site. Having users give
a running commentary as they are proceeding through the test tasks
provides valuable insights into the problems that they may encounter.

Jakob Nielsen estimates that an average usability study using traditional

methodology takes 39 hours total, including planning and final reporting.
6
Simple tests can be conducted in a couple of hours.

Because users must be able to perform tasks, user testing requires that
a working system of some sort be in place, whether a prototype of a new
site, or a pre-existing site that is being tested for the purpose of a
redesign.

Some additional steps can be taken to maximize the effectiveness of

user testing:

If possible, it may be advantageous to test at the user's site.

Observing the user operating the site using their own equipment in
their own environment will give a truer picture of the user's
experience of the site.
Testing is usually carried out on a one-to-one basis to allow the
facilitator to closely observe the user's behaviour. A second
facilitator may be useful for recording purposes.
The inclusion of a second test subject has also been found to
produce useful test results. The dialogue between the two subjects
encourages thinking aloud. This phenomenon, known as
co-discovery learning, is similar to brain storming.
The facilitator or tester must be able to be objective. For this
reason, it is often more effective to hire an outside facilitator.

User testing does not generally produce statistically significant results,

nor does it need to in order to be useful. The key to interpreting the
results of user testing is to look for general trends and behaviour
patterns that indicate problems with the usability of the site.

7.0 User Feedback

Once a site has been posted live to the Web, it is useful to continue to
collect user feedback to monitor ongoing user satisfaction with the site.
The most common electronic methods of collecting user feedback are
electronic mail and on-line questionnaires. It is important to remember
though, that these methods have limitations:

7 of 11
Usability and the Web: An Overview

Users must have the ability to submit their response, e.g., an e-mail
form might require that the user's e-mail address be configured
within their browser or that their Javascript be enabled.
Users must be willing to respond. Because the respondents are
self-selected, correspondence will, as a rule, be very positive or
very negative. There will be very few responses from individuals
whose attitude toward the site ranges from mildly negative to mildly
positive.
Users must have the appropriate software configurations, e.g.,
technologies that require a certain type of browser or plug-in will
limit responses.
The location of the feedback mechanism within the site will also
affect the response rate.
A user who is responding to an on-site feedback mechanism will
not have the same level of commitment and attention as someone
who is participating in a user test. As a result, feedback received
through such mechanisms is often too generic to be useful.

Given the limitations of these feedback mechanisms, they are most

effectively used as general indications of user satisfaction. Before
proceeding with a major site redesign, other usability techniques, such
as user testing, should also be employed.

8.0 Usage (Log File) Analysis

A log is a file to which a server's transactions are logged. It allows a Web

administrator to track file use and server traffic. A log file includes such
information as: the date of the transaction, the name of the computer
requesting the file and the name and size of the file transferred. Logs
may also include information about the user's browser and previous sites
visited. 7

Log file analysis is commonly used to track site usage. Increasingly, site
administrators will also use the data to draw conclusions about the
popularity and/or usability of certain pages or areas of the site.

However, while there are some valid uses of log files, the data can also
be highly misleading. For example, a substantial portion of logged
transactions is the result of the Web robots or spiders that probe sites for
pages to index, usually for search engines. Even when non-human
visitors are eliminated, effective log file analysis requires context. To
understand user behaviour it is important to know the sequence of
actions taken by the user, and whether the information they found was of
value to them. Neither is well documented by log files.

9.0 Conclusion

8 of 11
Usability and the Web: An Overview

Given the unique nature and staggering rate of growth of the Web, it is
clear that usability is an issue that is becoming increasingly important for
Web site owners and designers to address.

To a certain extent, Web site users expect instant gratification.

Successful Web sites then, whether they be commercial or informational,
will increasingly be those that meet the usability criteria of being clear,
appealing and easy to use.

To meet these objectives, there are many usability methodologies

available that can be used quickly, cheaply and easily to provide fast and
useful information to improve Web site usability. For best results it is
recommended that a variety of methodologies be used and that Web
site development and design be treated as an iterative and ongoing
process.

10.0 Resources

Fleming, Jennifer. " User Testing: How to Find out What Users Want. "
June 15, 1998.
<http://ahret.com/guides/design/199806/0615jef.html>

Fuccella, Jeanette and Jack Pizzolato. " Creating Web Site Designs
Based on User Expectations and Feedback. " Internetworking ITG
newsletter. June 1998.
<www.sandia.gov/itg/newsletter/june98/web_design.html>

Fuller, Rodney and Johannes J. de Graaff. " Measuring User Motivation

from Server Log Files. " October, 1996.
<www.microsoft.com/usability/webconf/fuller/fuller.htm>

Instone, Keith. " Conducting Your First User Test. " Webreview.com.
May, 1997.
<http://webreview.com/pub/97/05/30/usability/index.html>

Instone, Keith. " Site Usability Evaluation. " Webreview.com. October,

1997.
<http://webreview.com/pub/97/10/10/usability/index.html>

Kalin, Sari. " Mazed and Confused. " CIO WebBusiness Magazine. April
1, 1999.
<www.cio.com/archive/webbusiness/040199_use.html>

Kirakowski, Jurek. " Questionnaires in Usability Engineering: A List of

Frequently Asked Questions (2nd Ed.) ".
<www.ucc.ie/hfrg/resources/qfaq1.html>

Levi, Michael D. and Frederick G. Conrad. " Usability Testing of World

9 of 11
Usability and the Web: An Overview

Wide Web Sites. " May 7, 1998.

<http://stats.bls.gov/ore/htm_papers/st960150.htm>

Marketwave. " Web Mining: Going Beyond Web Traffic Analysis. " 1999.
<www.marketwave.com/press/whitepaper.htm>

Nielsen, Jakob. " Cost of User Testing a Website. " Alertbox. May 3,
1998.
<www.useit.com/alertbox/980503.html>

Nielsen, Jakob. " Guerrilla HCI: Using Discount Usability Engineering to

Penetrate the Intimidation Barrier. " 1994.
<http://useit.com/papers/guerrilla_hci.html>

Nielsen, Jakob. " Heuristic Evaluation. "

<www.useit.com/papers/heuristic/>

Nielsen, Jakob. " Technology Transfer of Heuristic Evaluation and

Usability Inspection. " November 27, 1995.
<www.useit.com/papers/heuristic/learning_inspection.html>

Nielsen, Jakob. " Testing Whether Web Page Templates Are Helpful. "
Alertbox. May 17, 1998.
<www.useit.com/alertbox/980517.html>

Nielsen, Jakob. " Top Ten Mistakes of Web Management. " Alertbox.
June 15, 1997.
<www.useit.com/alertbox/9706b.htm>

Nielsen, Jakob. " The Use and Misuse of Focus Groups. " 1997.
<www.useit.com/papers/focusgroups.html>

Nielsen, Jakob. " Users First: Survey Your Users. " DevHead. February
18, 1999.
<www.zdnet.com/devhead/stories/articles/0,4413,2211547,00.html>

Nielsen, Jakob. " User Testing. " DevHead. October 5, 1998.

<http://zdnet.com/devhead1stories/articles/0,4413,2137666,00.html>

Nielsen, Jakob. " What Is Usability? " DevHead. September 29, 1998.
<www.zdnet.com/devhead/stories/articles/0,4413,2137671,00.html>

Spool, Jared. " Why On-Site Searching Stinks. " 1997.

<http://world.std.com/~uieweb/searchar.htm>

Yu, Jack J., Prasad V. Prabhu and Wayne C. Neale. " A User-Centred
Approach to Designing a New Top-Level Structure for a Large and
Diverse Corporate Web Site. " June 5, 1998.
<www.research.att.com/conf/hfweb/proceedings/yu/index.html>

10 of 11
Usability and the Web: An Overview

__________
Notes

1
Spool, Jared. " Why On-Site Searching Stinks. " June, 1997.
http://world.std.com/~uieweb/searchar.htm

2
The techniques of card-sorting, category identification, description and
labeling, and wire-frame validation are components of IBM Corporation's
Web development methodology and are described in detail in the article
Creating Web Site Designs Based on User Expectations and
Feedback, by Jeanette Fuccella and Jack Pizzolato, June 1998.
www.sandia.gov/itg/newsletter/june98/web_design.html

3
For a brief overview of other usability inspection methods, see Jacob
Nielsen's Summary of Usability Inspection Methods at
www.useit.com/papers/heuristic/inspection_summary.html. Constantine
and Lockwood's Web Usability Inspections slide show from the User
Interface '99 conference also provides a good general summary
www.foruse.com/Presentations/WebInspectUI99/index.htm.

4
The original software development heuristics developed by Jakob
Nielsen (with annotations for the Web by Keith Instone) are available at
http://webreview.com/wr/pub/97/10/10/usability/sidebar.html.

5
An example of a simple set of general Web usability principles is
available at www.foruse.com/Presentations/WebInspectUI99/sld017.htm.

6
See Jakob Nielsen's Alertbox for May 3, 1998, Cost of Testing a Web
Site www.useit.com/alertbox/980503.html

7
For a more detailed discussion of log file analysis, see: Haigh, Susan
and Janette Megarity. " Measuring Web Site Usage: Log File Analysis. "
Network Backgrounder #57. September 11, 1998.
www.collectionscanada.ca/9/1/p1-256-e.html

Created: 2000-11-24 T I
Updated: 2001-02-27

11 of 11