A Usability Analysis of WorldCat Discovery

Beta Interface for J. Murrey Atkins Library

Submitted by:
Tayler Green
Kate Calabro
Rabee Khan
ENGL 4181, Writing User Documents
24 February, 2016

Submitted to:
Dr. Gregory Wickliff, Assoc. Professor of English
Rachael Winterling, Usability Coordinator

Executive Summary

For our ENGL 4181 Writing User Documents class, we were tasked with analyzing the usability of the
WorldCat Discovery beta interface. The J. Murrey Atkins Library at UNC Charlotte plans to implement
WorldCat Discovery in place of WorldCat Local. To prepare for this transition, our class developed usability
tests were in hopes of challenging problems from the old interface and reveal additional problems with the
Discovery interface. We chose 5 users in an attempt to represent a diverse range of undergraduate students.
They included users: male/female, native students, international students, and freshman to senior students.
The usability tests were performed using a laptop, a printed task list, an observation by a
group member either recording or taking notes. The users had very few problems performing tasks, and all participants finished in under 10 minutes. Since our users ranged from novice to advanced experience with the previous interface WorldCat Local, tasks on the Discovery interface were of an equal difficulty for all participants. Both the problems encountered and
comments received were similar. The most notable difficulties were broken into two main groups:
1.

Layout and Design

Inability to locate the sort by function

Inability to find the due date of library materials
Distractions from the sidebar location of other
relevant searches

Layout and design complaints stemmed from the size of text, which interfered with finding certain features, such as the sort by and due date functions. One user complained that the sidebar was very distracting. This led our group to consider the option to have a collapsible sidebar and/or user-personalized site.
2.

Content, Language and the User

General user apprehension to use site, test features

Lack of user knowledge of key library search terms
Inability to locate articles with full content

Usability testing uncovered as much about the interface as it did the J. Murrey Atkins Library users.
Two out of the five users had no experience with the original WorldCat Local site, and out of those two
users, one claimed she, ...never tried to use it because it looked scary. From this group feedback, we
concluded that a tutorial would be a necessary addition to the new WorldCat Discovery.
ii.

Table of contents

Executive Summary
ii
Introduction
1
Methodology
2
Test Results
4

1. Layout and Design Problems
4

2. Content, Language, and the User Problems 5
Findings and Recommendations
7
References
8
List of Figures
Figure 1.1: User Demographics
Figure 1.2: Usability Test Task List
Figure 1.3: User Post-Test Questions
Figure 2.1: WorldCat Local versus WorldCat Discovery
Figure 2.2: Distracting Sidebar
Figure 2.3: Example A. and B. Search Functions and Text Size
Figure 2.4: Example C. Refining the Search
Figure 3.1: User Quotes by Task
Figure 4.1: Suggested WorldCat Discovery Search Visual
Figure 4.2: Suggested WorldCat Discovery Short-Term Changes

2
2
3
4
5
5
6
6
7
7

iii.

A Usability Analysis of
WorldCat Discovery Beta
Interface for
J. Murrey Atkins Library
Introduction
Since WorldCat Local was introduced to the UNC Charlotte in 2013, various usability tests have
been done to identify, make recommendations, and solve problems associated with the interface.
J. Murrey Atkins Library will introduce the latest interface from Online Computer Library Center
(OCLC), WorldCat Discovery. The primary purpose of beta version usability test were to find inconsistencies and areas of user frustration.
J. Murrey Atkins library got in touch with Dr. Wickliff in the English department to help conduct
usability tests. Each group was assigned the task of creating usability tests for the beta site, along
with finding participants to take the tests. Before starting our project, our class met with the usability coordinator, Rachael Winterling, to ensure an understanding of the usability testing and basic
library interface system in place. Our team, which consisted of three people, devised a task list for
usability testing that explored various popular features of the interface. The group then recruited a
variety of participants for this test to meet the wide demographic of undergraduate users at UNCC.
This ensured equal representation and diverse results for the usability tests.
The scope of this report will address the two main issues with the WorldCat Discovery beta interface, being 1) Layout and Design problems, and 2) Content, Language and User problems.
The usability tests will relay information to librarians at UNCC, as well as designers at OCLC to
improve the new WorldCat Discovery interface.

methodology
In an attempt to replicate the wide variety of J. Murrey Atkins Library users, our group recruited
participants from different majors, levels of study, genders, and levels of expertise within the library
interface. Each group member asked people from their social circles, which included friends, roommates, etc.
Before starting the test, participants were asked a series of pre-test questions to create a demographic
of library interface users, in Figure 1.1 below:

Figure 1.1: User Demographics

To perform the usability test, the group provided users with a task sheet. When the facilitator began
recording, the test was administered to the users and took about 10-15 minutes depending on the
user. The task list is below in Figure 1.2:

Figure 1.2: Usability Test Task List

For each test, the group provided participants with a laptop, cleared of history and cookies. Two of the
five usability tests were conducted using laptops installed with Open Broadcaster Software (OBS),
which captured both audio and video. The other usability tests were supplemented with a note taker
to record observations on body language, and signs of frustration to identify where and how the user
had problems with the tasks.
Facilitators and note-takers encouraged the users to think out loud through their execution of the
tasks, and reminded users that the test was on the library interface, and not of the user, him or herself.
The group did not provide assistance to the users when they had questions about the site, and only
spoke if a user took a long time or expressed great frustration with a task.
Following the test, users were asked a series of post-test questions. These questions helped to give a
better picture of what users thought, giving them opportunity to provide additional feedback. Posttest questions are below in Figure 1.3:

Figure 1.3: User Post-Test Questions

Our findings were presented with tables, screenshots, and other visual aids from data gathered by
note-taking and reviewing two user test videos.

Test Results
Four out of five users were to complete the tasks entirely, but all users encountered difficulties. It was
noted that problems were the same among all users. The group was able to divide the main problems
into two categories. We also propose their respective solutions.
1.) Layout and Design Problems

Figure 2.1: WorldCat Local versus WorldCat Discovery

The initial search engine for WorldCat

Discovery was found by one user to be,
"Not very user friendly, like the old one
[World Cat Local]." In comparison with
the WorldCat Local layout, WorldCat Discovery lacks the tabs to sort one's search by
materials. When asked what changes could
be made to the interface, one user suggested the WorldCat Discovery search mimic
the old WorldCat Local layout, in terms of
having the tabs present to sort the search.
The old and new search engines areto the
left, in Figure 2.1:

The inability to locate library features, such as the "sort by" or "my list" function stemmed from the
small size of text. WorldCat could implement text size changes to accomodate users' visual needs, or
potenttially offer user preference option to customize the size of text on the site.
There could be a potential bias of the usability test results regarding layout and design, because two
out of the five users had never used the library interface, whereas the other three had experience
with the WorldCat Local layout. However, if users prefer the usability of the WorldCat Local design,
which allowed users to sort their search immediately, adding the tabs to the search would increase the
effeciency of library searches.
One user who admitted she never used the library site because of having no idea how to use it. This
suggests that the layout of WorldCat to a novice user may not appear to be user friendly. The WorldCat Discovery search offers no visible help or tutorial function on either the initial search page or
the search results page. We propose the addition of a tutorial link to the home page to assist users in
getting started.
A common complaint our group received that the sidebar for relevant searches, which appears after
clicking on a search result, is distracting. The sidebar, which shows other relevant searches to the left
side of the screen, was deemed overwhelming [outlined in red in Figure 2.2 below], or in the users
words, This is so distracting. Is it for what I clicked on, or what?
4

Figure 2.2: Distracting sidebar

The user then realized that to

find the information required of
the tasks, she would use the drop
down bars on the right instead.
The interface lacks the ability
to collapse the side search bar.
Perhaps, like the right side of
the page, where the information
is not revealed without being
clicked on, the relevant searches
on the left side of the page could be a drop-down option. Otherwise, the width of the left sidebar with
relevant searches should be able to be collapsed or minimized to a degree, which would not distract
from the source that the user clicked on.
2.) Content, Language, and the User Problems

Figure 2.3: Example A. and B. Search Functions and Text Size

When users were asked to add the first search result to the WorldCat Discovery feature My List,
we had five out of five users complain that the My List function was difficult to find due to the size
of text. In turn, users did not know the function of either of the two icons in Example B., or that the
second button was the Add to My List function. Most users seemed hesitant to click on the buttons
without knowing their function, where one stated, "I don't want to click on this--what does it do? I
don't want to mess something up."
5

Figure 2.4: Example C. Refining the Search

The tasks regarding changing the formats of the search (e.g. sorting for an audiobook or locating
only peer-reviewed articles) revealed much about the librarys users and user-friendliness of the
interface. When users were asked to sort articles by peer-reviewed results only, one user expressed
her frustration with formats available to her, as, I dont even know what the difference between a
peer-reviewed article and normal article is.
With all five users complaining about the size of text when searching for the function, and two users
stating that they did not know the function of the My List button, it is clear to our group that the
lack of explanation on of the sites features poses a problem.
Our users were encouraged to think out loud when navigating the beta interface. Some of our users
became distraught, while more of them were unsure if they found the right feature we were looking
for. Here are some of the direct quotes from users:

Figure 2.5: User Quotes by Task

As mentioned above majority of problems that users encountered were similar throughout all tests.
The most common problems included:
Users having trouble opening articles on external links
Users were not always able to find the due dates for material that could be checked out of the library
Inability to collapse the search bar,
Inability to locate the sort feature to sort by newest to oldest
Text size was too small
6

Findings and Recommendations

The findings from these five user tests helped us to realize that some areas of the new system still
need improvement. Even though the majority of our users completed the user tests with only minor
concerns and problems, all of the users had suggestions for improvement, which created a productive,
collaborative dialogue for how the user experience could be improved in future updates of the WorldCat Discovery interface.
Taking both usability tests and conversations from the users into consideration, the group has produced five suggestions for change, and has a drawn a mock-interface with those suggested changes.
For the short-term changes, the group felt that the most prevalent issue with the interface and the easiest fix would be to increase the font size of the interface. By doing so, the problems with finding the
due date of library materials and the drop down bars to sort searches would be fixed, in turn.
To appease user preference and thoughts on user-friendliness of the site, and at the suggestion of users
tested, the group proposes using the WorldCat Local search layout of the initial search. The Local
search interface provided tabs that would specify the search down to the format desired (i.e. articles,
books, etc.). The Discovery search requires the user to do a general search, then find the appropriate
check mark on the sidebar to narrow results. The ability to narrow the searches down on the first
search would save the user time and effort.

Figure 4.1: Suggested WorldCat Discovery Search Visual

Because users did not understand the language and terms of a library search, the final quick-fix is
to provide definitions of the unclear terms and functions on the site. This could be accomplished by
highlighting the terms, and as users hover their mouse over the term or function, a short definition or
explanation could appear. In our example below, if the user were to hover over a peer-reviewed article
to refine their results, the definition of the term would appear, or if they did not know what the MyList function was for, the user could hover their mouse of the button and receive a result without
having to click away to another page.
7

Figure 4.2: Suggested WorldCat Discovery Short-Term Changes

As for long term changes, the group would like to see the design of the interface adaptable to user
preference, so that if a user finds the left sidebar distracting, they can collapse or minimize the
width of the column. With the idea of user preference and accessibility in mind, the user should
also be to adjust the size of text to suit their individual needs.
The group would also like to see a tutorial service provided for users because both novice and advanced users of the site may not know or be aware of the functions that make their library search
easier. By providing a tutorial, perhaps either by video or PDF, the users would at least have the
option to receive help navigating and understanding the site. This function would be useful to firstyear students, English Language Learners, non-traditional students, and international students, all
of whom may be new to searching on WorldCat and using academic language to refine their results.

References
Usability Test Participants:
User 1, UNC Charlotte, 2/10/2016
User 2, UNC Charlotte, 2/15/2016
User 3, UNC Charlotte, 2/06/2016
User 4, UNC Charlotte, 2/06/2016
User 5, UNC Charlotte, 2/12/2016
Supporting Documents
Greg Wickliff, Rachael Winterling, Questions from ENGL 4181/5181 Students: Responses
from Rachael Winterling," 2/01/2016
Musings About Librarianship, "Published Findings about the Discovery Interface," 2/08/2016
UX Testing WorldCat Discovery, 2/08/2016
Google Analytics, "WorldCat Location", 2/15/2016