Professional Documents
Culture Documents
INTRODUCTION
page 3
THE STUDY
page 4
RECOMMENDATIONS
page 8
REFERENCES
page 12
APPENDIX A (Reflection)
page 13
page 14
page 16
page 17
page 21
Introduction
The DePaul CDM website (http://www.cdm.depaul.edu) is part of a larger network of
websites supporting DePaul University. The College of Computing and Digital Media
(CDM) is a college within the University that offers programs of study exclusively within the
realm of the computing sciences, cinematic design and display, and various graphic and
media arts. Team in2it2ion previously performed a content inventory of the existing
website content (Duffield, Kachavarapu, Steinkamp, & Sanderson, 2011) and uncovered
several problematic areas, namely poor visual prominence of ostensibly important
information, unnecessary or non-useful redundancy, and lack of a clear organization
scheme.
As a result of the content inventory, Team in2it2ion undertook a study of the target
audience of the DePaul CDM website to determine their perceptions and categorizations
of the content of the site. The goal of this study was to asses the way that the target
audience views the content in question, and propose a re-organization of the content to
better suit their collectivized mental model. The target audience of the site is fairly broad,
encompassing current and prospective students, professors, administrative staff, and
interested community members (benefactors, parents, local participants, etc). The content
inventory revealed that the DePaul CDM website content needed to be re-organized to
better serve the needs of these users and provide an easier and more efficient interactive
experience for them, and this study sought to address that end result.
To complete this study, Team in2it2ion performed a free list exercise to gather the
full breadth of content that users in the target audience might expect to find on the CDM
website. After the free-listed data was collected and analyzed, 99 of the most relevant
items were retained for inclusion in a card sort. The result of the card sort was a
representation of the way that CDM target audience members view the content they wish
to see on the site, and how they imagine it mentally. Based on the results, it was apparent
that users of the CDM site were comfortable with the sites layout (or at least did not direct
their content organizations in a way that would belie a desire for a different structure), but
that they preferred the content to be organized a bit more granularly. New top-level
categories are recommended for Advising, Facility Information, and Calendar, while a
name change is recommended for the Academics section to Degrees & Programs.
This report will detail the methods used to gather content items and ideas from
members of the DePaul CDM website target audience, as well as how they were
categorized via card-sorting. The findings of the card sort will be outlined and will inform
recommendations for changes to the content organization scheme. All auxiliary
information is contained in the Appendices for reference purposes.
The Study
The components of this study included a content inventory (Duffield, et. al,, 2011), a freelist
exercise, a card sort, and a proposed re-organization of the DePaul CDM website based
on the collective results. Since the inventory had been done previously, the results of the
freelist and card sort are outlined below.
Freelist Exercise
Participants
The freelist exercise required that members of
Team in2it2ion recruit individuals who were
members of the target audience. These
individuals were asked to participate in a freelisting exercise, which was explained as a list of
terms, concepts, and ideas that pertain to
information that a member of the target audience
might expect to find with a visit to the CDM
website. A total of 11 members of the target
audience participated in the freelist exercise.
Method
Each participant was asked to provide a list of
items as it pertained specifically to DePaul CDM
website content, but no other constraints on the
type (or format) of the items were given. These
solicitations, and the receiving of the freelist data,
were all conducted electronically through email,
and the resulting submissions were sorted
together in a spreadsheet for analysis.
The raw freelisting data from each
participant was put into a spreadsheet for
analysis. The initial methodology was to sort
through the freelisting data individually, but it was
then decided to create a spreadsheet to do the
bulk of the analysis, and then refine further after
that if necessary. The freelisting data was broken
into 3 separate sheets, each performing separate
functions to analyze the data. The first sheet was
simply the raw freelisting data organized by each
participant. The second sheet was set up to
convert the raw freelisting data into lowercase
strings in preparation for the next step in the analysis. Converting to lowercase strings
was achieved with the following function: =ArrayFormula(LOWER(Participants!A2:K)).
Once converted to lowercase strings, the third sheet was used to analyze the prepared
data. The sheet consisting of 4 columns, each having a specific function to analyze the
data. The first column in the sheet sorted the unique instances of the freelisting data
alphabetically by the following function: =SORT(unique(transpose(split(ArrayFormula
(concatenate(Lowercase!A2:J&",")),",")))). The second column then tallied the times
that each item was listed by using: =ArrayFormula(if(A2:A<>"",countif(Participants!
A2:K,A2:A),iferror(1/0))). The third and fourth column worked together by creating a
"Working list by frequency" where the third column was the freelisting data ranked from
high to low with respect to frequency by the function: =SORT(A2:A, B2:B, FALSE), and the
fourth column listed the items count using: =SORT(B2:B, 1, FALSE). The fourth column
also consisted of conditional formatting on the cells to indicate items that appeared more
than four times by green, two or three times by yellow, and once by red cell coloring. This
list was used a jumping off point for further refinement such as merging categories that
were semantically identical.
This resulted in a frequency-weighted spreadsheet of the terms from the freelist, in
order of usage by participants (Appendix C-2).
Findings
A total of 11 participants produced 470 total freelist terms, of which 366 were at least
marginally unique. The most popular terms were predictably clustered around admissions,
advising, employment, research, events, and news, among others. Items of close
secondary concern were information about student life, scholarships, academic programs,
and distance learning tools, among others.
The breadth of the freelist data revealed that it would be unwise to assume the
target audience was using the DePaul CDM website primarily for academic programrelated inquiries. Many other areas of interest surfaced such as location, getting around,
connecting with alumni, facilities/lab information, and distance learning options.
Card Sort
Card Content
The above-mentioned freelist was pared down to a final 99 items in a collaborative effort
that included the entire team. The list was put into a Google Doc Spreadsheet and was
interactively edited and commented in real-time, resulting in quick and efficient decisionmaking to eliminate non-obvious duplicates, extraneous items, and to condense subgroupings that were unnecessarily faceted (E.g., course information, course reviews,
course location, and course syllabus would be condensed into a single, more useful
item). The resultant 99 items comprised the content of the cards in the online card sort
(Appendix C-1).
Participants
The card sort required team members to again
recruit target audience members who would be
willing to participate in an online card sort. 12
participants who were members of the CDM
website target audience agreed to perform the
card using the WebSort web application.
Participants were directed to a unique URL for the
card sort study, where they were shown a
Method
The team used a virtual card sorting tool called WebSort to perform the card sorting
exercise. All participants in the card sort were remote and performed the sort using the
online tool, there were no in-person sorts performed. The combined responses from the
users were obtained from WebSort, which provides numerous methods of exporting
results data. The team studied formats of data such as Items x Items, Categories x
Items, Category Summaries, and a Tree Graph. The methods used for analyzing the
results were based on the instructions for the use of the spreadsheet developed by Donna
Spencer (Spencer, 2007).
Based on the instructions and the results of the card sort, the analysis began with
the importation of the results from all sort participants into the Spencer spreadsheet.
When all the raw data was entered, the categories were standardized by A) using the most
frequent raw category labels for standard category labels, and B) condensing categories
with similar names, meanings or card content into single, standardized categories. After
the set of standardized categories was determined (our card sort found 18), all unique
categories from the raw sort data were replaced with their corresponding standardized
category name. After this, all duplicates were removed from all sorters' results, and all
results were combined to create a single list of all standardized categories. Each of these
was then given a code number for use as a key in the rest of the spreadsheet.
The results of the spreadsheet were used mainly to refine and determine categories
for the new content organization. Based upon the category standardization from the raw
sort data, the 18 standard categories provided the basis for the new organizational
structure of the content.
Statistical Analysis
The results of
statistical analysis
showed evidence for
the new content
categories, with
surprisingly strong
negative correlation
between topics related
to Advising and topics
related to Degrees &
Programs. Currently,
advising information is
buried in the section of information for current students, and was shown by the results to
be more desirable as its own section. The other notable statistical result was the rather
nominal correlation of calendaring or event-type information with anything else on the site
except for information about the college/campus. The card sort results showed, however,
that users consistently wanted this information in its own category as well.
Findings
The results of the card sort provided evidence that there are not currently enough
categories in the DePaul CDM website to sufficiently organize the content. The card sort
revealed that the organization would be improved by adding additional categories and
renaming Academics to Degrees & Programs. The recommended additional
categories are Advising, Facility Information, and Calendar. Moving the Advising category
from Current Students to the homepage provides immediate access to Advising content,
which sort participants indicated they would prefer;the card sort demonstrated that most
users wanted to place Advising content in its own category and not within a broader
category, such as Current Students. Adding Facility Information and Calendar categories
also provides immediate access to content that currently is not properly categorized and is
difficult to discover. The card sort revealed that the users wanted categories for
information about facilities (E.g., labs, buildings, resources) and events of note (academic
calendar, special events, student organization events, deadlines, etc). A re-organized site
map can be found in Appendix D.
Recommendations
The current navigation structure of
DePaul CDMs website is a structural
global navigation menu. The website
has two main navigation elements: 1)
the main menu on the left, and 2) the
supplemental menu on the right. The
supplemental menu displays options
for the three different schools of CDM
and acts as a filter for the left menu
by displaying only the information that
pertains to the selected school. The
main global navigation menu is a
seven category hierarchical menu. It
is recommended that the CDM
Home main menu item be removed,
as users expect this functionality by
clicking on the homepage logo.
The results of the card-sort and freelist provided no evidence the
Proposed New Wireframe
navigation structure of the website
should be altered. As a result, the
navigation structure is recommended to remain as is, but the sub-navigation items should
be displayed in a Mega Menu for easier access to the content in the sub-categories.
CDMs website has a large amount of content with many sub-categories. A mega menu
will allow the sub-categories to be more easily discoverable, and making more complex
Duffield, Kachavarapu, Steinkamp, & Sanderson (Team in2it2ion) - 8
navigation choices easier to see at a glance (Tidwell, 2011). Moving the sub-categories to
a more accessible mega menu will allow the complex navigation discoverability to be
moved out of the sitemap footer current in use. The current sitemap footer is useful, but
awkward and out of place in a page-intensive site like the CDM website, since there is a
general lack of whitespace due to long passages of content. This detracts from its
effectiveness, as the user must often scroll through several screens worth of text before
realizing the benefits of discoverability that a sitemap footer offers (Tidwell, 2011). By
moving this functionality above the fold into a mega menu, users will discover additional
navigation more quickly and easily.
A look at the before and after wireframes illustrates the changes proposed to
the main navigation along the left side, with the added categories and removal of the
redundant CDM Home link:
Before Re-organization
After Re-organization
Scenario 2: Parents of a student graduating from the Digital Cinema program wish to get
information about attending graduation. Specifically, father wishes to get directions and
parking information for the DePaul CDM campus. He visits the DePaul CDM main page,
and sees the main menu item for Facility Information. He clicks on the menu item, and
from the resulting mega menu, selects information about Directions and
Parking (theoretical, not pictured):
Scenario 3: A prospective student is interested in applying to the DePaul CDM, and visits
the website for more information. He quickly sees a main menu item specifically for
Prospective Students, and clicks it. The mega menu reveals several points of
information on applying and the application process:
References
Duffield, C., Kachavarapu, S., Steinkamp, J., Sanderson, T. (2011, Spring). HCI 454
Interactive Systems Part 1: DePaul University College of Computing and Digital
Media Website Inventory. HCI 454: DePaul University.
Spencer, D. (2007). Instructions for use: Card sort analysis spreadsheet. Retrieved from
http://www.rosenfeldmedia.com/books/downloads/cardsorting/
cardsort_analysis.pdf.
Spencer, D. (2009). Card sorting: designing usable categories. Rosenfeld Media, LLC:
Brooklyn, NY.
Tidwell, J. (2011). Designing interfaces, second edition. OReilly Media, Inc.: Sebastopol,
CA.
Appendix A (Reflection)
This project began with the expectation that the DePaul CDM website was in need of a
major content overhaul. This was due in large part to perceived difficulties on the part of
those of us in Team in2it2ion when using the site. Our expectation was that those of us in
computing-related disciplines would have an easier time navigating complex information
structures than those who were less apprised of such systems, or otherwise unskilled or
inexperienced using them. Through the content inventory, we discovered that pertinent
and important information was being relegated to non-intuitive and inconspicuous
placements, and we were hopeful that the freelisting and card sorting exercises would tell
us which pieces were, in fact, important enough to move to the fore. In-person interviews
would have been helpful at this stage, potentially coupled with website browsing task
analyses. We did not get a good feel for how people actually use the site today, only what
thoughts they had about the information contained in it. It would have been useful to
observe users browsing the site and record any actual difficulties, rather than relying on
our (admittedly biased) expert user perceptions of where they may lie.
The freelisting exercise was successful as far as it went, but we all suspected that
people were not really thinking freely, and that they tended to write down items that they
remembered viewing or searching for on the site. Our freelist instructions might have
needed to be clearer, or perhaps an explicit instruction to NOT try to retrieve the site
map from memory would have mitigated this. We did have a good selection of items that
were, as suspected, not featured prominently on the current site, and were good
indicators of the re-organization needed. The card sort itself was an illuminating exercise
but again, we all felt that participants might have been creating a site map rather than
truly responding viscerally to where they felt the content truly belonged. If the team had
the exercise to do over again, we would push for more in-person, physical card sorting to
mitigate the mistaken perception on the part of users that they needed to create a site
map. We all felt that the conversation around a physical card sort of this content would
have been quite helpful. The main take-aways from the card sort were the addition of top
level content areas, which was in line with what we suspected initially. We did not
anticipate the actual areas that appeared, but that discovery added to the intrigue and
enjoyment of the exercise for the team. At the end of the day, the team felt that the card
sort was successful, but that it was limited by the inability of users to sub-categorize the
large amount of content, and by the impersonality of the online tools; perhaps for large,
existing content sites the better route is to perform these exercises live and in person, to
steer conversations away from the recitation of rote usage patterns from the users past.
The natural next step would be to perform usability testing on a working prototype
of this new content scheme, but ideally the team would re-run the card sort live first.
There is a nagging suspicion that an in-person card sort of this content would reveal more
data that the online sort did, and ultimately the site would benefit more from such data.
Overall the experiment was enjoyable and interesting, and illustrated the responsibilities of
an interaction design professional well.
faculty
faculty blogs
financial aid
game development
graduation
Graphic Design
GRE
how to apply
Human-Computer Interaction
important links
independent study information
international students
internship information
jobs at CDM
lab hours
majors
library
minors
Master's Degrees
myCDM
Network Technology
new students
news
online learning
online programs
peer advising
peer mentoring
people (directory)
Ph.D. information
prerequisites
Professor bios
program descriptions
prospective students
registration requirements
request information packet
returning students
scholarships
Software Engineering
student handbook
student organizations
study abroad programs
Bookstore
tours
transferring
transportation
tuition
tutoring
Bachelor's Degrees
work study programs
Alphabetized Unique
Instances
Freq.
about us
application
academic center
events
academic policies
news
academics
research
advising
accomplishments
alumni
account information
computer science
acm
courses
activities
directions
address
employment
administration
faculty
administration contact
jobs
administration staff
labs
administrative information
about us
admission
academics
advising
address
advising hours
application information
advisor
calender
advisors
campus connect
alumni
col
alumni information
course catalogue
alumni survey
current student
application
d2l
application information
degrees
appointmenst
directory
appointments
dual degree
around chicago
articles
facilities
assignment
game development
assistanships
graduate
assistantship
graduation
attractions
history
best practices
international
bookmark
location
building addresses
multimedia
building hours
my cdm
building map
mycdm
requirements
calender
scholarships
campus
statistics
campus connect
students
campus facilities
tutor
campus map
academic center
campus maps
academic policies
Freq.
career
career advising
accomplishments
career center
account information
catalog
acm
cdm
activities
cdm building
administration
administration contact
administration staff
cdm clubs
administrative information
admission
advising hours
cdm events
advisor
cdm homepage
advisors
alumni information
cdm jobs
alumni survey
cdm professors
appointmenst
appointments
cdm requirements
around chicago
cdm schools
articles
certificate
assignment
certification
assistanships
chat
assistantship
chicago
attractions
best practices
class schedule
bookmark
class schedules
building addresses
building hours
classes offered
building map
club info
campus
club news
campus facilities
clubs
campus map
co-ops
campus maps
col
career
col info
career advising
career center
collaborate
catalog
cdm
college of interactive
media
cdm building
combines degree
cdm clubs
complain
computer graphics
cdm events
computer science
cdm homepage
Alphabetized Unique
Instances
Freq.
contact
cdm jobs
contact info
cdm professors
contact us
costs
cdm requirements
counselors
cdm schools
course
course catalogue
1
2
certificate
certification
1
1
course description
chat
course online
chicago
course registration
course reviews
1
1
1
1
course schedule
class schedules
course timimgs
course-path
courses
1
3
classes offered
club contact info
1
1
courses offered
club info
courses taught
club news
credits requirements
current class schedule
1
1
clubs
co-ops
1
1
current news
col info
current student
1
1
collaborate
cyber security
d2l
dean info
Alphabetized Unique
Instances
Freq.
education advancement
opportunities
credits requirements
emergency information
employment
current news
event schedule
events
current students
events
cyber security
exam
dean info
existing students
dean's list
deans list
facilities
degree paths
facts
degree planner
faculty
degrees offered
faculty advisor
demographics
faculty blogs
depaul
fall quarter
fee
filmmaking
financial aid
digital cinema
discussion board
forms
document
full time
funding
dynamic
game development
easy
education advancement
opportunities
emergency information
gpa calculator
event schedule
grade
events
dean's list
combines degree
graduate
exam
deans list
complain
graduate information
existing students
degree paths
computer graphics
graduate program
facts
degree planner
graduates
faculty advisor
degrees
faculty blogs
degrees offered
contact
graduating/degree
information
fall quarter
contact info
graduation
demographics
graphics
fee
gym
filmmaking
history
financial aid
honors programs
hotels
forms
hours
full time
funding
depaul
contact us
costs
1
1
counselors
course
1
1
digital cinema
course description
directions
course online
course registration
course reviews
how to apply
course schedule
course timimgs
human computer
interaction
dual degree
course-path
ide
gpa calculator
important information
grade
important links
graduate information
directory
discussion board
document
dynamic
courses offered
easy
courses taught
Alphabetized Unique
Instances
Freq.
independent study
information
graduate program
integrated
graduates
interact
graduating/degree information
intern / co-op
graphics
international
gym
international information
honors programs
international students
hotels
internship
hours
internship information
internship opportunities
how to apply
internships
intranet
ide
is
important information
job oppurtunities
important links
jobs
jobs at cdm
integrated
lab hours
interact
intern / co-op
international information
laboratory
international students
labs
internship
learn
internship information
lectures
internship opportunities
library
internships
intranet
is
link to mycdm
job oppurtunities
links
jobs at cdm
lab hours
loaction
laboratory
local internship
opportunities
learn
Alphabetized Unique
Instances
Freq.
mfa
microsoft
major
minor
major / minor
minor information
major information
minors
majors
multimedia
my cdm
maps
my cdm link
masters
mycdm
masters degree
networking
masters students
new student
mfa
new students
microsoft
news
minor
octodad
minor information
offerings
minors
on campus jobs
my cdm link
online
networking
online classes
new student
online learning
new students
online program
octodad
opinions
offerings
opt/cpt
on campus jobs
organizations
online
part time
online classes
peer advising
online learning
peer mentoring
online program
people (directory)
opinions
phd information
opt/cpt
placements
organizations
post
part time
prerequisites
peer advising
previous faculty
peer mentoring
professional development
people (directory)
professor bio
phd information
placements
lectures
professor bios
library
professor courses
post
professor information
prerequisites
professor rating
previous faculty
link to mycdm
professor reviews
professional development
location
major
links
proffesors
professor bio
major / minor
proffessor
professor bios
major information
program descriptions
professor courses
professor information
majors
loaction
projects
professor rating
maps
prospective student
professor reviews
masters
prospective student
information
proffesors
masters degree
mac lab
1
prospective students
proffessor
masters students
Alphabetized Unique
Instances
Freq.
Alphabetized Unique
Instances
Freq.
prospective students
program descriptions
submit
study strategies
quarter dates
suggestions / questions
submit
question
projects
supplementary material
suggestions / questions
ranking
prospective student
supplementary material
reference material
surroundings
registering
prospective students
syllabus
surroundings
request
prospective students
teachers
syllabus
quarter dates
text books
teachers
requirements
question
time
text books
requirements registration
ranking
tools
time
research
reference material
tours
tools
research info
registering
transferring
tours
request
transportation
transferring
respond
tuition
transportation
returning students
requirements registration
tution fee
tuition
review
research info
tutor
tution fee
room
tutoring
tutoring
rss
respond
tutors
tutors
sandbox
returning students
u-pass
u-pass
scholar
review
undergrad
undergrad
scholarship
room
undergraduate
undergraduate
scholarships
rss
undergraduate information
undergraduate information
school description
sandbox
undergraduate program
undergraduate program
school history
scholar
undergraduates
undergraduates
school info
scholarship
university
university
school map
school description
university ranking
university ranking
school of cinema
school history
upsilon pi epsilon
upsilon pi epsilon
school of computing
school info
virtual map
virtual map
school map
visiting faculty
visiting faculty
services links
school of cinema
watch
watch
simple
school of computing
snippets
services links
social media
simple
software engineering
sound
snippets
what?
what?
spring break
social media
software engineering
statistics
student
sound
student blogs
spring break
whiteboard
whiteboard
student forms
student
student handbook
student blogs
why cdm
why cdm
student forms
wimba/col information
wimba/col information
student organizaitons
student handbook
student work
students
student organizaitons
zoom
zoom
study abroad
student work
study abroad
study strategies
Appendix D (New
Sitemap)