rin Services SI 622: Evaluation of Systems and he at ,C ANN ARBOR.

COM USABILITY rg Ga a TESTING REPORT rim Ga Team Newsies: u, F Danlu Fu, Garima Garg lu an Catherine Le, Caroline Moran D April 2010

,C Le e

ro a

M ine l

an r

 Who is

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

Why Usability Testing?

To understand real users
How they interact with the system e, C L How they feel about the systemne

ro a

M ine l

an r

at ,C Experts know too much rg Ga a rim to fix problem after launch Too expensive Ga u, F nlu Da Adapted From lecture slides

ri he

Usability Testing

rim METHODS Ga u, F nlu Da

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

Participant Selection
Target Participants
Age range: 20 - 30 years old Novice users

Recruiting

5 participants Our personalinetworks, our colleagues and graduate rm Ga students at other schools u, F Incentives: $25 gift cards lu

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

an D

Process
When, Where, Who and What

April 6 and April 7, 2010 ,C Le Ann Arbor.com office conference e in room er1 tester 1 moderator, 1 note-taker and th Ca Laptops with a trial version of Camtasia Studio ,

ro a

M ine l

an r

nlu a

u, F

rim Ga

rg Ga a

Process
During the testing
Introduction Pre-questionnaire 5.5 Tasks Debriefing

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

Process
Tasks

Task 1: Register a new account ,C Le it on Facebook Task 2: Find a specific article and e inshare er area Task 3: Find a crime report tin your h
3a: [if user does not use the Neighborhoods section] Ca rg, the Neighborhoods section Find the crime report in a

ro a

M ine l

an r

G achoose a concert Task 4: Find and rim promotions Task 5: Find some Ga u, F nlu Da

Results and Analysis

Jakob Nielsens (1993) rating scale for usability ro Ca each key problems to rank the severitye, of L finding: ne

M ine l

an r

nlu a

u, F

rim Ga

rg Ga a

at ,C

ri he

Usability Testing

rim KEY FINDINGS Ga u, F nlu Da

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

Finding 1
L e

Neighborhoods Section does not display ro Ca correctly in all browsers e,

M ine l

an r

IE 8.0 browser

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

Firefox 3.6.2 browser on the same computer

Finding 2

Users have difficulty navigating within the ro Ca Events Calendar e,

M ine l

an r

L an event 3 out of 5 users failed the task to e in find er Most users do not notice that there is an actual th calendar Ca rg, Most users did not navigate using the links at the top Ga section a of the Events Calendar rim an article about an event through Users often found Ga the Entertainment section u, F nlu Da

Finding 2
,C Le e ro a

M ine l

an r

nlu a

u, F

rim Ga

rg Ga a

at ,C

rin he

Finding 3

In the Neighborhoods Section, users are ro Ca unclear as to which geographice, region of Ann L Arbor is the most appropriate choice ne

M ine l

an r

i rNeighborhoods tab All users who navigated to the he at paused and expressed confusion about the region ,C rg labels Ga a rim Ga u, F nlu Da

Finding 3

It was equally intuitive for users to seek the ro Ca neighborhood crime information, under the Le News sub-section of the drop-down menu. ne

M ine l

an r

riArbor-wide crime Unfortunately, it shows an Ann he at report rather than the neighborhood-specific crime ,C rg report. Ga a rim Ga u, F nlu Da

Finding 4

Hard to differentiate and find Ann Arbor cityro Ca wide news versus neighborhood-specific e, L news ne

M ine l

an r

ri While searching for neighborhood-specific crime news, he atArbor-wide crime report some users found the Ann ,C rg whereas others found a more neighborhood-specific Ga crime report. a m There is noari to access the neighborhood-specific way ,G crime report from the Crime/News section and there is Futo access the Ann Arbor-wide crime report no lway n u each Neighborhoods section. a Dfrom

Finding 5

Users have difficulty finding the Events ro Ca Calendar e,

M ine l

an r

L section first to Users often went to the Entertainment ine look for local events such ashconcerts er at the right side of the The link is actually located on ,C rg drop-down menu, but not very visible. Ga a rim Ga u, F nlu Da

Usability Testing

rim RECOMMENDATIONS Ga u, F nlu Da

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

Recommendation 1
Run browser compatibility testing ro a

M ine l

an r

Further testing should be done to ensure that all ,C Le consistent across features and are compatible withe n and ri all common browsers and platforms. e

nlu a

u, F

rim Ga

rg Ga a

th Ca ,

Recommendation 2

Increase visibility of the map of Ann o r Arbor Ca area for the Neighborhoods section e,

M ine l

an r

L corresponding A map of Ann Arbor overlaid with the ine the Neighborhoods geographic labels is displayed once er th feedback should be section is reached, but more Ca provided at the onset. , rg Ga or map in the drop-down menu Either a zip code query a would alleviate the confusion users face when trying to rim Ga determine which geographic label link is the most u, for them to click. F relevant nlu Da

Recommendation 3

Include an Events Calendar link in the ro Ca Entertainment section e,

M ine l

an r

L association of There seems to be a strong and ne i intuitive Events (such as concerts) with rthe Entertainment he at section ,C rg One such example being Boston's Ga the Weekly a alternative news source, rim Dig (http://www.weeklydig.com) Ga u, F nlu Da

Recommendation 4

To focus attention, organize eventsro the by Ca week's most highly featured recommendations e,

M ine l

an r

L of events are In LA Weeklys calendar, only a handful ine considered staff picks each week. er th Ca rg, Ga a rim Ga u, F nlu Da

Recommendation 5

Add more cues to describe events at a glance ro

Cabut no legend Icons to the left describe the event type, e, L is given. ine as "Concert, Describe the event type in tags, such er th who are not able to Folk, or the Ark", for those Ca g, determine the eventrtype from titles. Ga a rim Ga u, F nlu Da

M ine l

an r

Recommendation 6

Increase consistency of access to articles ro Ca across site e,

M ine l

an r

Lroute or another, It is equally intuitive to choose one ne ior the Neighborhoods such as choosing the Crime her tab at tab when looking to find reports about neighborhood ,C crime. rg Ga If articles are cross-linked in multiple related suba sections, initial mistakes in information-seeking can be rim Ga there will be a safety net of multiple minimized since u, routes to access the same information. F modes and nlu Da

Usability Testing

rim DISCUSSION Ga u, F nlu Da

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

M ine 5 participants who have never visited ol ar AnnArbor.com ,C Le In real cases new users probably wont use these ine very first visit specific functions in our tasks on their er th During their navigation through news articles, they Ca related sections such as might notice some non-news g, rwhich might further indicate what the Events Calendar, GaAnnArbor.com. else they can doaon rim Further Research Ga u, Recruiting some regular users who never used F functions other than reading the news nlu Da

New Users

an r

Unfamiliarity with the city of oran M Ann Arbor ine l

All of the participants came to Ann Arbor ,C Le within the past year e

ro a

rin familiar with the 2 participants said that they are not he because they are area distribution of Ann Arbor at ,C newcomers here rg Ga a Further Research rim Ga local young participants so as to Recruit several u, the problem of area location for F evaluate nlu Neighborhoods section Da

Task Wording

Many participants clicked Entertainment ro Ca concert instead of the Events Calendareto find , L events ne

M ine l

an r

ri of events which are It might be wise to test othere kinds th menu so as to eliminate not related to any tab on a C top rg, as much as possible. the impact of otheraoptions aG rim Further Research Ga u, tasks such as finding a workshop can be used F Example nlu to replace the concert task Da

Usability Testing

rim CONCLUSION Ga u, F nlu Da

rg Ga a

at ,C

rin he

,C Le e

ro a

M ine l

an r

 Findings:

ro Participants have difficulty finding their neighborhood Ca within Ann Arbor. e, L They also have difficulty with understanding whether ine er they should go to the Entertainment or the Events th Calendar section for concert information. Ca , rgEvents Calendar section didnt Navigation within the Ga a work as users expected. rim news is only in the main News Ann Arbor-wide Ga category, while related neighborhood-specific news is u, F only available in Neighborhood sections. nlu Da

M ine l

an r

 Recommendations:

Unambiguous geographic identification in o ar neighborhood sections. ,C Levisible to users The Events Calendar should be more ine such as and integrated with other sections er Entertainment. th Ca within Events Calendar Better and clearer navigation rg, so as to provide flexible should also be supported Ga paths to desiredaevents. rim IncreasedGa consistency across the general News sections and Neighborhood sections should be considered for u, F future improvements. lu

M ine l

an r

an D

nlu a

,G Fu

,C L Thank you!e ine er Q &th aA ,C rg Ga a rim a

ro a

M ine l

an r