Introduction

For this white paper, I conducted usability tests on two popular web applications:
Google Gmail and Microsoft Outlook Web App (Outlook). These applications are
primarily used to draft and read emails. They also provide the functionality to manage
emails once they are already sent or received, such as allowing a user to choose multiple
messages and place them into a folder for later reference. Being free, popular
applications, Gmail and Outlook have a broad user base: the products are intended for a
general users with basic computer know-how.
This white paper specifically aims to be useful for a reader of Technology Today, a
general user seeking information to help them compare the services. It will explore the
drafting, management, and organization of email in order to offer a big picture view of
what a typical user can expect to find when using the services to complete common
tasks. The paper includes a recommendation section to provide a more nuanced view of
the test results.

Methodology
For this project, one usability test was completed at a time: one for Gmail and one for
Outlook. The tests were conducted 30 minutes apart on the afternoon of Thursday,
October 9, 2014 in the University of Idaho library. I used a desk on the quiet floor of the
library to simulate a normal email environment and to be relatively free of distractions.
The tasks were carried out on my personal computer: a 13 Apple MacBook Air using the
OS X operating system and the Google Chrome web browser. Each test included three
main tasks: one to determine what the email drafting environment was like, one to
manage multiple files, and one to organize emails. I chose these tasks because I believe
they comprise the general functions people would like to be informed of when learning
about different email platforms. The tasks and associated subtasks are outlined below:
Task 1: Explore Basic Email Functionality - This task covers the most common
basic use of an email application: actually writing an email.
A. Open New Email to Be Drafted
B. Enter Address into Sender and Blind Carbon Copy (bcc) Boxes
Blind Carbon Copy allows a user to send a copy of an email to a person other than
the recipient - without the recipient knowing. This is useful, for example, in

business applications if a user needed to provide their supervisor with a copy of

an email sent to a client.
C. Add Attachment to Message
I wanted to test attachment functionality because it is common to include files
such as pictures and documents with emails.
D. Save Message As a Draft
I thought saving messages for future use was important. Users can be distracted
when composing an email, and it is important to save and retrieve a draft of an
email if the need arises at a later time.
E. Open Saved Draft from Inbox and Send Message
Task 2: Carry Out a Task Across Multiple Messages - Often it is necessary to
carry out a task across several messages. In an email account where many messages
arrive, it can be frustrating to see 100 unread messages, even if they are newsletters and
spam. By being able to mark these messages as read, users can alleviate this stress and
focus on the most important emails.
A. Find Five Messages over One Month Old and Mark them as Unread
B. Return to Top of Inbox and Mark Unread Messages as Read
Task 3: Use a Folder to Organize Email - Sometimes users can use an email
address for multiple purposes. For example, a student could find it useful separate her
personal emails from her class emails. Folders offer this functionality.
A. Create New Folder, Call it Misc
B. Place Three Emails from the Same Sender into Misc
C. Access Misc from Homepage
Result Comparison - I used a Likert scale to compare the results of each test and
task. This scale covered the spectrum between 1 (strongly disapprove) and 5 (strongly
approve). I also noted data for the time for each task and the amount of mouse clicks
each required.

Results
Task1:ExploreBasicEmailFunctionality

Gmail

Outlook

Easeofuse

5.0

4.0

UserPerception

4.0

3.0

Usefulness

5.0

4.0

Average

4.7

3.7

12

3:38.204

2:47.779

Clicks
TaskTime

Gmail
On opening Gmail, I was greeted with a visible Compose button near the top of the
page. I clicked it and commenced drafting a simple email with myself as the recipient.
Figure 1 demonstrates the sleek, yet useful drafting interface offered by Gmail. Notably,
adding a blind carbon copy took no additional searching on my part since it was part of
the display itself. A simple click of Bcc allowed me to easily add myself as a
not-so-blind blind copy recipient.

figure 1: Gmail makes it easy to bcc

Gmails design also saved me work and worry by displaying unobtrusive messages that
were provided in the course of drafting and sending the email (See Figure 2 below). A
Saved message appeared with each change I made to my draft, negating the need for
to carry out the save message as draft task.

figure 2: Useful messages on Gmail

Once the message was sent, I had the option to view it. Knowing my message was sent
without having to spend the time to click through to any other portion of the site
provided me with peace of mind and a feeling of security.
A drawback I noted of the Gmail drafting function was that the draft appears in a box
that lays on top of the inbox. This subtracted some from my perception because I found
it busy - I had to look at the folders on the side, the emails below, the header on top, and
the draft itself. This seems like a lot to bother with while trying to accomplish something
simple. Busyness aside, Gmail offered an intuitive user experience for email drafting.
Outlook
From the Outlook homepage, I was easily able to click the New Mail button by the top
of the page. This button launched a drafting interface nestled between the folder menu
and the messages in my inbox. The process began smoothly enough and I had no trouble
entering myself as the recipient of the email. Things got tricky, however, once I needed
to add a blind carbon copy. Figure 3 displays the interface I was using. To add the blind
carbon copy, my first instinct was to click the + symbol. This symbol caused me to
navigate away from the drafting interface to another screen altogether.

figure 3: Convoluted message options on Outlook

The other screen included nothing about blind carbon copies, rather it had an address
book created for me for some reason to do who-knows-what with. I navigated back to
the drafting interface after several wasted clicks and seconds. There, I found an ellipsis
that allowed me to add the bcc.
I was disappointed with the draft-saving experience: Outlook saved for me
automatically, but I had to click around to ensure that the save did indeed happen. If a
user was pulled away from an email drafting, they may not expect that their message
was saved or know where to look.
I was not ultimately impressed with my Outlook email drafting experience. This is
supposed to be a simple task, yet I was confused as to what I was having to do on the
other page. A + symbol, after all, does not offer the intuitive functionality one would
assume when seeing something so simple. Other users may also be flummoxed by being
taken away from their draft. However, this is mostly a superficial issue once the learning
curve is breached.

Task2:CarryOutaTaskAcrossMultipleMessages

Gmail

OWA

Easeofuse

2.0

4.0

UserPerception

1.0

5.0

Usefulness

2.0

5.0

Average

1.6

4.7

19

12

3:33.429

1:38.141

Clicks
TaskTime

Gmail
The first part of this task was to find emails over one month old. Because the Gmail
inbox is a single page with a specified number of messages, I had to click an arrow three
times to find the older messages. However, this did not prove hard to figure out. Matters
got confusing once I returned to the first page.
To figure out how to mark all messages read, I found a checkbox near the top of the
inbox. Under the checkbox were different options to check certain emails: read,
unread, and so on. I click the checkbox, I clicked read, I clicked unread, I clicked mark
as read - in short, I clicked everything I could and never was able to mark the messages
read. To complete the task, I simply went back to the third page and clicked the
messages themselves and marked them read the very same way I marked them unread.
This task was a complete failure on Gmail. It made matters worse that it appeared I
could accomplish what I sought to do, but was unable to do so. Clicking on things and
receiving no results makes for a bad experience.

Outlook
To find month-old emails was easy enough with Outlook: the inbox has a continuous
scroll feature that allowed me to scroll right down to some older emails to mark them
unread. I encountered a curious result upon marking the emails unread: 6 conversations
were selected instead of 5. I found out that this happens as a result of Outlook checking
the first email by default. This is a curious default because I am not sure what the
benefit is to have anything checked without user consent. This seems like an easy way
for a user to mistakenly delete their newest email.

figure 5: Outlooks Unread Option

Figure 5 displays a feature I was happy to find when returning to the first page: there,
right at the top, I was allowed to see all my unread emails with a single click. I clicked it,
found them marked, and easily marked them unread.
Outlook impressed me on this task. It can be extremely useful to see unread emails
compiled together and it shows an impressive attention to detail to include this feature
and to make it prominent at the top of the inbox.

Task3:UseaFoldertoOrganizeEmail

Gmail

OWA

Easeofuse

4.0

2.0

UserPerception

4.0

1.0

Usefulness

5.0

2.0

Average

4.3

1.7

12

37

1:54.795

4:13.40

Clicks
TaskTime

Gmail
It was not straightforward to create a folder for messages on Gmail. First of all, folders
do not exist on Gmail per se, but labels do. Once I figured out how to make a label, the
process was quick and efficient. All I needed to do was to create the label, search for the
sender I was looking to add, mark three emails, and label them using a convenient drop
down menu (see Figure 6). To access the label, I just went to the sidebar, located misc,
and clicked through.

figure 6: Labeling in Gmail to create a folder

Though it was easy for me to grasp the label method, it is possible that it would have
taken extra effort for a general user. There is nothing someone would expect, such as a
folder graphic, to guide the process along and comfortably complete the task. Labels
strike me more like they are tags than folders, but they do accomplish the same
function, practically speaking.

Outlook
Creating a folder in Outlook proved difficult. It was easy to locate the emails to place
into the folder using Outlooks convenient search function, but as to what to do with
them afterward was not clear. I quickly began clicking all over the place: options,
arrows, question marks, and ellipses - to no avail. On a whim, I tried right-clicking the
inbox and found the option to create the folder. From there, it was no problem moving
the emails to the folder and accessing it from the folder bar.
The folder creation process on Outlook was frustrating. Here is a web application where
the user is met with icons all over the screen, and yet something as basic as creating a
folder is a mystery. Further, I still cannot figure out what all the other buttons are even
for. This task was underwhelming, to say the least.

Recommendations
When using an online email application, a general user should expect a design that is
simple, efficient, and intuitive. Email apps should be basic and straightforward all but
the most luddite members of the general population. To accomplish the best design, the
apps functionality should be plain to grasp, easy to work with, and free of quixotic
journeys through random blocks of information, clipped menus, and deceptively smooth
icons. Based on the results of this series of usability tests, I recommend Google Gmail
over the Microsoft Outlook Web App. Outlook is by no means a poor product: it offers
users essentially the same features as Gmail. From Outlook, a user can send and receive
emails, manage them, and organize them effectively. However, to accomplish is often
complex and unintuitive. On the whole, this is not true of Gmail. Gmail does more with
less, offers great functionality, and a pleasing interface to use - leaving little to be
desired.