USABILITY STUDY

COMPARING UNIVERSITY WEBSITES

BY: DYLAN LAMBERTON

 Table of Contents
INTRODUCTION ............................................................................................................................................ 0
METHODS ..................................................................................................................................................... 1
Tasks ...................................................................................................................................................... 1
Test Environment .................................................................................................................................. 1
Evaluation Criteria................................................................................................................................. 2
RESULTS ........................................................................................................................................................ 2
Task 1 Access Prospective Students page to apply for college....................................................... 2
Task 2 Find information on campus safety ........................................................................................ 4
Task 3 Find the current Colleges football roster ............................................................................... 6
Task 4 Navigate to LGTBQ E-mail information ................................................................................... 7
Task 5 Access greek life sign up application....................................................................................... 8
Task 6 Find the academic calendar and see when finals week begins .............................................. 9
Task 7 Schedule a campus tour with the university......................................................................... 10
CONCLUSION .............................................................................................................................................. 12
BIBLIOGRAPHY............................................................................................................................................ 13

INTRODUCTION
This white paper describes the results from a usability test of the University of Idaho and University of
Utah websites. The usability test compared a series of ten tasks that were measured by five component
standards that define usability. All tasks were graded by myself over the course of two days in the same
location, although typical users of these websites could vary in age and technical ability depending on
the task they are performing. For a website to have strong usability it had to score highly in efficiency,
effectiveness, error tolerance, user friendliness, and engagement for the user.
In order for universities to continue attracting new prospective students they have to have a way for
their college to stick out above all the rest. The students of today require streamlined ways to compare
colleges with one another through certain metrics that are important to them. Whether these metrics
include the average graduation rate, retention rate, campus life, athletics, financial aid, etc. depends
entirely on the person. Another important factor is the returning alumni who may want to schedule a
visit to their alma mater, donate back to the university, or attend an athletic event. Thus colleges need
to have a functional and usable website that caters to many different needs to draw future, present, and
past students in and keep them engaged with the university. There are always ways to improve the
usability of a particular product, websites are no exception.
Bias for the University of Idaho was initially thought to be a concern at the beginning of the test since I
have attended this college for two years now. To negate this bias, tasks were selected that were
considered commonplace and crucial for any website to have easily available. A point variance was also
introduced to help negate bias, which is defined in the Conclusion (Page 15) of this white paper.

METHODS
TASKS
A total of ten (10) tasks were performed to test the usability of the University of Idaho and
University of Utah websites. A task is defined in this white paper a set point of information or
data input box to reach through navigation of a website by mouse and keyboard alone. All tasks
began at the homepage for each website. The tasks performed include, in no particular order of
importance:
1) Access Admissions Page
2) Find information on campus safety
3) Find the colleges football roster
4) Navigate to the LGBTQ e-mail information link
5) Access Greek life recruitment sign up
6) Find the academic calendar
7) Schedule a campus tour
8) Find a picture of the student recreation center
9) Locate financial aid information
10) Find a research article from a current student

TEST ENVIRONMENT
The usability test was conducted on October 7th and October 8th 2017 in the University of Idaho
Library at 9:00 AM and lasted until about 10:30 AM both days. This time was selected to limit
external distractions since the test was taken in a public setting. The University of Idahos test
was conducted first followed by University of Utahs test the next day to avoid memorizing
similar navigational routes that could occur between the websites. The test was taken on a
Lenovo ThinkCenter desktop computer with a mouse and keyboard provided by the library.
EVALUATION CRITERIA
To measure the usability of each of the websites a five component standard that defines
usability was used. The components included efficiency, effectiveness, error tolerance, user
friendliness, and engagement for the user. Each component was given a criteria statement that
could qualitatively or quantitatively represent the component it is associated with. These
criteria statements are underlined in yellow in the Likert Scale below in Figure 1.

FIGURE 1 - LIKERT SCALE USED TO MEASURE USABILITY THROUGH GRADING FIVE SEPARATE COMPONENTS

RESULTS
TASK 1 ACCESS PROSPECTIVE STUDENTS PAGE TO APPLY FOR COLLEGE
Task 1 began at each of the websites homepages (Figures 2 and 3).

FIGURE 2 UNIV. OF UTAH HOMEPAGE FIGURE 3 UNIV. OF IDAHO HOMEPAGE

Both paths to the admissions page were accessible through ribbons on the top of the webpage
(Figure 4). This similarity made sense because attracting students is one of the primary concerns
for universities and must be easy to navigate to on the page.
 FIGURE 4 UNIV. OF UTAHS ADMISSION RIBBON (LEFT) AND UNIV. OF IDAHOS PROSPECTIVE STUDENT RIBBON (RIGHT)
After choosing the appropriate link on the ribbon the websites direct the user to the application
process for each university. The University of Idaho had a quick survey that would automatically
place the user into the appropriate application form online, where it then directs users to
applying for financial aid afterwards (Figure 5). The University of Utah required the user to read
through a list of application forms and choose the form appropriate for them (Figure 6). The
University of Utah clearly showed where to send high school transcripts, ACT or SAT scores, and
status tracking in a clearer manner than Idahos did.

FIGURE 5 U OF I SURVEY FOR CORRECT APPLICATION FIGURE 6 U OF U SEPARATION OF APPLICATIONS SELECTED

MANUALLY
 The ranking of each of the two websites for this task can be found on the following page in
Figure 7, accompanied with notes taken during the procedure. In all, the University of Utah
website required more clicks, time, reading, and scrolling to apply for college. Idaho had a
streamlined approach that did most of the work for you, and was incredibly user friendly for
those not keen on navigating webpages. I would recommend that the University of Utah make
changes to this approach in order to create a more usable website.

FIGURE 7 - LIKERT SCALE RESULTS FOR TASK 1

TASK 2 FIND INFORMATION ON CAMPUS SAFETY

Task 2 began at each of the websites homepages (reference Figures 2 and 3 from Task 1). It was here
that I saw the University of Utahs shortcut to campus safety which displayed all police patrols, contact
numbers, security help desk
information, crime rates etc. in one
simple click (Figure 8). The University of
Idaho made it easy to get to campus
safety information but showed less
campus security and police information.
U of I also had multiple drop down
menus in this section with general lab
safety that took more time to thumb
through to get to campus safety, while
U of Us separated them from each
other (Figure 9).

FIGURE 8 - UNIV. OF UTAH'S CAMPUS SAFETY PAGE


FIGURE 9 U OF I SAFETY DROP DOWN MENUS (CONT. ON
WEBSITE)
FIGURE 10 - LIKERT SCALE RESULTS FROM TASK 2
This showed that the U of U knew that its
location in a large city could raise questions
about student safety. The website designers
for their university made getting to this
information far more efficient with
significant or more effective information for
the user.
After using both websites, my
recommendation would be for the
University of Idaho to revamp their safety
information structure so that there is less
reading for the user. I would also like to see
general safety in the classroom/lab to be
separated from campus safety for improved
effectiveness. Figure 10 shows the results of
the usability test.
TASK 3 FIND THE CURRENT COLLEGES FOOTBALL ROSTER
Task 3 began at each of the websites homepages (reference Figures 2 and 3 from Task 1). Both
university websites contained athletic department shortcuts on their homepage, with the
University of Utah proving more efficient with completion of task in 3 clicks as opposed to
Idahos 5. Idaho did however provide more information about the players with a full biography
link and a picture for each team member while Utah did not (Figure 11 and Figure 12).

FIGURE 11 - SNIPPET OF UNIV. OF IDAHO'S FOOTBALL ROSTER MENU

FIGURE 12 - UNIV. OF UTAH'S FOOTBALL ROSTER MENU

The only recommendation I can make for this task is for the U of Us website to include more
information about their players with a headshot of what they look like. Overall, this task was
completed with all five components of usability scoring high marks (Figure 13).

FIGURE 13 - LIKERT SCALE RESULTS FOR TASK 3

TASK 4 NAVIGATE TO LGTBQ E-MAIL INFORMATION
Task 4 began at each of the websites homepages (reference Figures 2 and 3 from Task 1).
Navigation from the main webpage to each universitys LGBTQA office information was very
efficient on my scale (3 clicks). Both websites displayed email and phone contact information to
reach out to the office, however only the University of Utahs email link worked properly. The
University of Idahos email link, when pressed, would open a new error tab that had no
information on what to do next (Figure 14).

FIGURE 14 - LGTBQ LINK CRASH POP UP TAB

Therefore, U of I scores lowly in my Likert Scale rating for error tolerance, engagement, and
learnability. I strongly recommend that the University of Idaho fixes this linking issue to avoid
confusion and frustration for its future users. Likert Scale rankings are listed below in Figure 15.

FIGURE 15 - LIKERT SCALE RATINGS FOR TASK 4

TASK 5 ACCESS GREEK LIFE SIGN UP APPLICATION
Deciding where to live for college is an important investment and needs to be a simple and easy
process to avoid frustration over something users will end up spending money for. Task 5 began
at each of the websites homepages (reference Figures 2 and 3 from Task 1). After navigating the
ribbon and housing links I could not find a path to Greek Recruitment on either of the
webpages. This led me to using the built in search bar on each webpage, typing in Greek Life
Recruitment and Greek Housing. University of Idaho brought me to a series of links that
eventually brought me to their Greek Life page after 5 clicks. The University of Utahs page
showed no results for either search and zero external links or recommendations to sister sites
(Figure 16). Utahs help service also netted no answers or help (Figure 17).

FIGURE 16 UNIV. OF UTAHS SEARCH ENGINE SHOWING NO RESULTS FOR GREEK LIFE

FIGURE 17 UNIV. OF UTAHS QUICK HELP DIRECTORY SHOWING NO HOUSING/GREEK HELP

My recommendations for both websites are to include Greek Housing/Recruitment in the
housing options webpage and after the application process. This will improve the low
effectiveness scores given to both colleges by drastically reducing search times and click
numbers. Univ. of Utahs webpage should also be updated where the search bar will come back
with recommendations based off of keywords and direct users to the quick help page for no
result searches. Doing so would improve U of Us error tolerance and engagement greatly,
overall improving usability. Figure 18 shows the usability results of this task.
FIGURE 18 LIKERT SCALE RESULTS FOR TASK 5

TASK 6 FIND THE ACADEMIC CALENDAR AND SEE WHEN FINALS WEEK BEGINS
Finals week is a stressful time for students. Every student will have to look up on the academic
calendar what day and at what time all of their finals are at. Parents or grandparents often send
surprise care packages to their kids or grandkids and need to know when finals week begins
as well. This stresses the importance of creating intuitive websites that are easy to learn due
different users having different technologic backgrounds. Task 6 began at each of the websites
homepages (reference Figures 2 and 3 from Task 1). Both websites had ribbons leading to the
academic calendar and passed the effective component of usability with the highest rating
possible. The University of Utahs page only had a pdf version of the academic schedule, which
hurts the engagement ranking, but only slightly due to pdfs compatibility with all computers
(Figure 19). The University of Idaho had an online only version of the calendar (Figure 20).

FIGURE 20 UNIV. OF UTAHS ACADEMIC CALENDAR PDF

LINKS
 I have no significant recommendations to make in this section, only that the U of Us webpage
should also have an online only version of the calendar like U of Is webpage. Figure 21 shows
the Likert Scale grades for this task.

FIGURE 21 LIKERT SCALE RATINGS FOR TASK 6

TASK 7 SCHEDULE A CAMPUS TOUR WITH THE UNIVERSITY

A tour would be one of the first things a prospective student will look into when deciding which
colleges to attend. Parents and young students alike will be searching on the colleges website
for campus tour information so the webpage needs to be easy to learn for different users with
different technological backgrounds and error tolerant if mistakes occur. Task 7 began at each of
the websites homepages (reference Figures 2 and 3 from Task 1). Campus tour information for
each website was able to be found using the ribbon link Prospective Students for each
university (Figures 22 and 23).

FIGURE 23 UNIV. OF IDAHOS FUTURE STUDENT RIBBON

FIGURE 22 - UNIV. OF UTAH'S FUTURE STUDENT RIBBON
 Each navigation to this point required less than 5 clicks of the mouse and was therefore strongly
effective and efficient by my metrics. The only problem occurred for the University of Idaho
page when I clicked on the house tour link dates and tried to create a new tab in one effort (ctrl
click). The webpage displayed an error message with no direction on how to solve the problem
(Figure 24). Therefore the U of I webpage receives lower grades in error tolerance and
learnability.

FIGURE 24 - UNIV. OF IDAHO ERROR MESSAGE WHEN CREATING A NEW TAB AND CLICKING ON "TOUR DATES" LINK
My recommendation is for the University of Idaho to rework and reprogram this link to be able
to open a new tab upon ctrl + click command. From what I have found this problem only occurs
for this specific link and causes mild frustration for the user because this link takes the user off
of the U of I webpage. Figure 25 below shows the Likert Scale ratings for this task.

FIGURE 25 - LIKERT SCALE RATINGS FOR TASK 7

CONCLUSION
The University of Idahos and University of Utahs webpages both demonstrated exceptional
usability for most of the tasks that were selected. In terms of the best point scores from the
Likert Scale results the University of Idaho came out just ahead of the University of Utah,
winning 4 out of 10 of the tasks compared to 3 out of 10 for Utah. Meaning that U of Is website
is more usable in comparison by this rating system. There were a total of 3 tasks that ended in a
tie between the websites. For a tie, a task had to have the sum of the ratings from the Likert
Scales for both websites equal each other or be within 2 points. The tie-point-variance was
introduced to account for minor usability differences for some of the tasks, possible user
navigation error, and correct any bias for U of Is product.
Further analysis of the results leads to a more descriptive, professional, and intuitive
comparison between the two websites. In most cases the University of Utahs webpage
displayed more information about what I was trying to get. Whereas the University of Idahos
page seemingly lacked extra information that pertained to what I was searching for, sometimes
giving bare bones information in most instances. A perfect example if this is Task 2 Find
information on campus safety. The only exception to this conclusion is the roster biography and
headshot information that U of I had but was lacking in the U of Us webpage (Task 3). Not
having extra information gave the University of Idahos efficiency and effectiveness scores a
better rating overall (31 total with possible best of 20), where having extra information
impacted U of Us ratings negatively throughout the tasks (36 total with a possible best of 20).
The University of Idaho did have the lowest error tolerance score and the worst error messages
(2 total) with no help menu compared to University of Utahs (1) minor error.
There was one task that each webpage received strongly unsatisfying remarks for. For U of U, it
was Task 5 Access Greek Life recruitment sign-up application. My primary recommendation is
to include Greek Housing or Recruitment options in the application process for prospective
students and to have a Greek Life webpage dedicated to educating students about future
recruitment on the main website. Doing so will increase efficiency and effectiveness by reducing
search time and mouse clicks for users. The remaining recommendations made in the results
section include: advertise student research along with professor research for accessibility, show
financial aid information to students before the application process, include a player profile and
headshot on the athletic rosters page, and to provide a student classification survey for
incoming students that places them in the appropriate application form when applying.
My most significant recommendation for the University of Idahos page is to correct the error
messages that occurred when sending an email to the LGBTQA email link and for creating a new
tab when selecting the academic calendar (ctrl + click on link). The effectiveness and efficiency
scores count for nothing if the website doesnt allow the user to engage with the webpage.
Therefore correcting these linkage issues will make the U of I website more intuitive, creating a
more functional and usable product. The remaining recommendations from the results include:
consolidate drop down menus, include extra information for what is being searched to improve
engagement rating, and have a printable version of the academic calendar without using the
snippet tool in Windows.
 BIBLIOGRAPHY
1
University of Idaho - Offering top-ranked programs in the Northwest. (n.d.). Retrieved October 17,
2017, from http://www.uidaho.edu/
2
The University of Utah. (n.d.). Retrieved October 17, 2017, from http://www.utah.edu/