USABILITY TEST PLAN

Usability Test Plan

Version 1.0

Mykala Brown, Sam Boyce, Jeremy Boone,

Jonathan Arnold, Colton Auge, Stephen Andruszkiewicz

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

January 24th 2015

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Executive Summary
This document is a test plan created to gather feedback from participants to aid the
developers of Commerce in ensuring that their game meets the high usability
standards of the consumers.
The facilitators of this test played through the game and created impression and
exploratory questions to provide beneficial usability information for the developers.
The impression questions are designed to find out what the participants think about
the game before they play it. The exploratory questions are designed to get player
feedback about the different mechanics and their usability within the game as well as
the user interface (UI) elements.
A training section has been added as instructions for the facilitator on how to fully
disclose and guide the participant along the test. The procedure and test steps
section following afterwards details the specific procedure in which the facilitator will
test the participant.
A testing methodology section was created as an outline so that the developers know
exactly how each facilitator is running this test and acts as a guideline for said
facilitators. This section also lists the equipment and environment that each
facilitator will be utilizing. The survey that each participant will be taking to
determine their eligibility for the test is also included in this section.
A data collection sheet was created as an easy way for the facilitator to store
gameplay information while the participant is testing the game. This is provided in a
separate excel document.
A flow chart was created to aid the facilitators in creating their impression and
exploratory questions. It also stands as a useful tool to aid the facilitators and
participants in the testing of the game if problems may arrive.
The initial testing was done as a quick bug analysis that the facilitators found in their
initial playthrough of the game. These bugs may help the developers for their next
iteration of their game.Testing Methodology
Each facilitator will select one participant to conduct testing of the usability of the
game Commerce. The participant will be asked a series of questions prior to testing
to ensure they meet the criteria for participation (see Participation Survey below).
Each facilitator has a MacBook Pro with a Windows partition. The participant will use
this computer and partition to run the game for the test. The test may take place in
the facilitators, participants, or a third partys home but will take place in a

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

comfortable, private location. Both during and after testing the participant will be
asked a series of questions, given verbally by the facilitator.

Participants
The participants in this test will play the game Commerce in its current build. They
will attempt to complete the goal of the game and provide feedback on the games
usability and user interface.
Participants will take the following the survey to determine their eligibility for this test
plan:
1.
a.
b.
c.
d.
e.
2.
3.

How many hours per week do you spend playing games?

0
1-5
5-10
10-15
more than 15
What types of games do you enjoy playing?
Are you able to proficiently use a laptop computer running Windows?

Participants can be of any age or gender. However, they must have the following
characteristics:
Familiar with video games, but not a hardcore gamer
Plays games more than 0 hours per week but less than 15
Typically plays casual games, for example (Big Fish Games, 2015):
Puzzle
Casino
Hidden Object
Board/Card
Match 3
Family
Arcade
Able to proficiently use a Windows laptop computer

Training
The facilitator will instruct the participant that they are participating in a test of a
game in development. They will instruct the participant that they should vocalize
their thoughts as they play and inform them that the facilitator may ask questions
during testing and that they should give honest answers/feedback in response to
these questions. The facilitator will also inform the participant that they will be
recorded while playing.

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Participants will be shown the game and instructed to run it. The participants will
then be instructed to complete the tutorial. This will give them the training they need
to interact with the game.

Procedure
When the test begins, the facilitator will ask the participant to find a comfortable
place to play the game. If the test is taking place in the participants home they will
be instructed to play the game in an area they normally play games. The participant
should play until the games objective is complete; for a period of one hour, or until
they no longer feel comfortable playing the game, whichever happens first.
The test will be conducted using the think aloud procedure, wherein the participant
is encouraged to vocalize their thoughts while playing the game. Facilitators will
record the players vocalizations as they play. After completing the training, the
participant will begin playing the game. If the participant is not vocalizing their
thoughts and decisions they may need to be prompted by the facilitator. In this case
the facilitator should ask neutral, open-ended questions (e.g. Why did you go
there?, Why did you sell that item?, Why did you make that decision?). When
the participant begins playing the facilitator should ask questions about their initial
impression of the game. The facilitator should also ask exploratory questions as the
player reaches or completes the relevant portions of the game.
While the participant plays the facilitator will log their decisions and collect data on
their game experience. This should be done out of view of the participant to prevent
biasing their actions.
After the participant is finished playing, the facilitator will ask any relevant
exploratory questions that were not asked during the game play. The player should
also be given a chance to give any open-ended feedback they may have. The
facilitator will record their answers to these questions and any feedback they give.
After the test is complete the facilitator should thank the participant.

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Test Steps
1. Introduction
a. Participant takes Participation Survey
b. Participant selects location for test
2.
a.
b.
c.

Training
Facilitator briefs participant on how the test will be conducted
Participant opens game
Participant completes tutorial

3. Test
a. Participant plays game (until complete, for 1 hour, or until they are no longer
comfortable playing)
b. Facilitator asks impression questions (see Impression Questions) and records answers
c. Facilitator asks exploratory questions (see Exploratory Questions) and records
answers
d. Facilitator records participants vocalizations
e. Facilitator records relevant gameplay data (see Data Collection)
4. Post-test Survey
a. Facilitator prompts participant to give any additional feedback
b. Facilitator records answers
Facilitator thanks participant

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Impression Questions
1. What do you think the story or theme of this game is, based on the main menu?
2. Based on the main menu, what sort of genre do you predict this being?
3. What do you think the final goal of the game might be?
4. What actions do you expect to perform within the game?
5. How complex do you think this game will be?
6. What demographic do you think the developers are trying to reach with this game?

Exploratory Questions
1. What do you think about the guard-hiring feature?
2. How does the world map compare to world maps in other games?
3. Is there something you would change from the shop menu to make it better? If so,
what would it be?
4. How do you feel about the benefit of the tavern rumors?
5. Besides getting robbed, what other challenges could the traveler face?
6. How do you feel about the one day travel time between towns?
7. How do you feel about the upgrade system?
8. How do you feel about the navigation of the map?
9. How do you feel about the thirty day time limit?
10. In your opinion, what is the best strategy when it comes to buying and selling?
11. What do you think about the games overall difficulty?
12. How does the difficulty using this game compare with other games youve played?

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Data Collection

Facilitators will gather the following data about their participants game usage during
their test:
Player turns to $2000
Gameplay time to $2000
Total funds on day 30
No. of instances of travel
No. of Items bought
No. of Items sold
Gameplay time to 30 Days
Average Gold earned per minute
Average Gold earned per turn (day)
Player win/loss

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Initial Testing
During the initial playthrough of the game the facilitators ran across a wide array of
problems such as design issues, bugs, and game crashes. The facilitators marked
down these problems as they came across them in hopes of aiding the developers in
future iterations of the game. The complete list of problems can be found below.
Completing a job at the tavern permanently freezes the completion text In the
barkeepers dialogue box rendering it impossible to read any rumor text.
Boat and Winter Gear only cost 1 gold.
When travelling to a new town there is a chance that the trading menu will not open
locking the player in the map screen permanently.
Game does not end. Still going at 101 days.
No UI to inform the player whether a robbery was successful or defended by guards.
No information about what was robbed after robbery.
Game becomes unresponsive after completing the tutorial.
Goal of the game is never properly explained.
Hitting Escape sends you back to the menu with no warning, causing you to lose all
your progress.
Able to travel to a town even if you cant afford the travel fee.
Map is sometimes unresponsive to mouse movement input.
If a player happens to reach zero gold, the game has no default setting to tell the
player that they lost. The game will keep playing even if you are at zero gold.
The mouse lags making it difficult to navigate the game.
The barkeep does not inform the player the direction of the town they need to deliver
good too. Player must scroll around the entire map pretty much playing I spy trying to
find the town.
Traveling to any town costs the same amount of days no matter if it is across the
town or across land.
Game sometimes crashes after a robbery.
Game ran at a constant 10fps on MacBook Pro Windows partition.

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

Full Sail University - Usability - Hexanalogue

USABILITY TEST PLAN

References
Big Fish Games. (2015, January 1). Retrieved January 22, 2015, from
http://www.bigfishgames.com/download-games/top-pc-games.html

Full Sail University - Usability - Hexanalogue