ULS Online Discussion:

Surveys in Libraries

ULS Facilitators:
Julie Watson
Research Librarian
Marywood University
Emily Daly
Head, User Experience Department
Duke University
Rebecca Blakiston
Instructional Services Librarian
University of Arizona

ACTIONABLE SURVEYS
Wendy Begay
Librarian
Tohono Oodham Nation

Actionable Survey - Overview

Process Improvement
What is an Actionable Survey?
Survey Design

Overview of Process Improvement

Develop/Define a question/problem
Review information
Gather your data
Measure your process
Analyze/test your data
Implement your changes
Feedback or summary

Actionable Survey
Emphasis Survey questions
What will we do with this information?
Ex. Likert Scale responses

Feedback Plan and share feedback & summaries

This is what you told us you wanted. This is what were
delivering/changing.

Actionable Survey
Service Example - Action Gap Survey1
What do we do well? List of 10 services

(multiple choice: choose 3 services)

What do we need to improvement?
(multiple choice: choose 3 services)
Which service(s) are important to
respondent? (rank all 10 services)
Cravendo, J., & Sandvig, B. (2003). Survey for Action, Not Satisfaction.
Quality Progress, 36 (3), 63-68.
1

Action Gap Example

Survey Design Tips

Dont try to solve World Peace in a Survey
Keep it simple and focused
Edit out promotional questions
No Silver Bullet, or magic formula for questions
Do ask demographic questions
Ex. Undergraduate or Graduate question: will you change a

service or product based on the response? Will your survey

results summaries be different or specific to separate groups?

If you can, do meet with [Statistics or Sociology]

faculty to discuss and review your

survey/data/methodology
For big changes in services/products/space, check and
consult with other [customer] data sets to see

QUESTIONS?

SURVEYS:
CONCEPTUALIZATION AND
CONSTRUCTION
Dr. Jason Martin
Stetson University
jmartin2@stetson.edu

Bad Surveys

What are some

experiences you had with
bad surveys? What
made them bad?

Survey Blueprint
What do you want to know?
Information needs
Stay on topic
Use a theoretical framework or model
Information Literacy Standards
Customer Satisfaction
Parts of a Service or Topics in a Course

IL Standards
Topic

Questions

Information Needs

Access Information

Evaluate Information

Use Information

Ethically Use Information

Information Literacy

15

Customer Satisfaction
Topic

Questions

Customer Expectations/Perceived
Quality

Perceived Value

Customer Satisfaction

Customer Complaints & Loyalty

Customer Satisfaction

10

Access Services
Interlibrary Loan

Questions

Friendliness

Efficiency

Effectiveness

ILL Service

Types of Survey Questions

Likert Scale
Select One
Open-Ended
Drop Down Menu
Check all that Apply
Continuum of Data

Likert Scale
This webinar is interesting.
Disagree
(1)

Somewhat Neither Somewhat

Disagree Agree Nor
Agree
Disagree
(2)
(3)
(4)

Agree

Poll Question!
Does Neither Agree Nor

Disagree have a value?

(5)

NA

Other Questions
Open Ended
Avoid overuse
Value of the answers is hit or miss
Good catchall at the end
Check All that Apply
Avoid overuse
Primacy effect

Other Questions
Drop Down Menu
Make sure all the possible options are present
Continuum of Data
No overlapping
One year to five years; five years to ten years;

ten years to fifteen years

Less than one year to five years; more than five
years to ten years; more than ten years to
fifteen years

Writing Survey Questions

Poll Question!!

Use clear and concise language.

Do not confuse respondents or

leave words or phrases open to

interpretation.

Writing Survey Questions

Poll Question!!!
Avoid negative questions.
Answer in the negative when

you mean the positive.

Writing Survey Questions

Did you use the library on 17

November 2012?
Too specific
Do not make survey respondents go
find information.
Ask questions they can easily answer.

Survey Layout
Clear Navigational Flow
Avoid Clutter
Avoid Jumping Around from Question Type

to Question Type
Place like question types together
Not necessarily subject type
Place Demographic Information at the End
Somewhat controversial

QUESTIONS?

AN ALTERNATIVE TO
LIBQUAL
Rick Stoddart
Assessment Librarian
Oregon State University
richard.stoddart@oregonstate.edu

Poll Question:
What tool does your library mainly use to gather patron

feedback about library services?

A. A LibQUAL product
B. LibSat survey by Counting Opinions
C. A survey built in-house
D. Other (some other tool or technique)
E. Our library doesnt formally gather patron feedback

Some Characteristics of Surveys

Countin
g
Opinion
s
Ongoing

Home
Grown

Complex

Analysis &
Reports

Reliability?
Validity?

Able to Compare
to LibQual Peers

Limited External
Benchmarking

Not able to
compare to
peers

LibQUA
L
Comprehensive
Snapshot

Full Control of
Questions

Counting Opinions
LibSat
Library Satisfaction Survey
7 or 15 minute survey

InformUs
Form and Survey Tool
LibPas
Data and Reporting Tool

LibSat Survey

SQUIRE Index

Opportunity Index

QUESTIONS?

ADDING A LIBRARY QUESTION

TO CAMPUSWIDE SURVEYS
Lisa R. Horowitz
Assessment Librarian
MIT Libraries

Poll Questions

Pilot Learning Outcome

ACRL STS standard
The information literate student determines the nature and extent
of the information needed.
Pilot outcome
Undergrads will be able to employ information access skills learned

while at MIT in order to identify appropriate sources of information

in their own field and in unfamiliar subject areas.

Process
Office of Institutional Research
COFHE surveys
MIT Surveys:
Pre-freshman survey
Enrolled student survey
Senior Survey
Alumni Survey
Triennial MIT Libraries Survey

Question to include in MIT Surveys

Learning Outcome:
Undergrads will be able to employ information access skills learned
while at MIT in order to identify appropriate sources of information
in their own field and in unfamiliar subject areas
Based on what you know now, how well do you think your

undergraduate experience prepared you to:

Select scholarly or professional information from a variety of

sources.

Results
2010 Senior Skill
Selecting scholarly or professional
information from a variety of sources

Think analytically and logically

Acquire broad knowledge in the arts
and sciences

Synthesize and integrate ideas and

information
Gain in-depth knowledge of a field

Pearson
Pearson
Coeff*
2012 Senior Skill
Coeff*
Selecting scholarly or professional
1 information from a variety of sources
1.00
Make firm decisions and take action
even if some of the facts about the

best choice are not clear

0.587

Conducting scholarly research

0.539
Deliver on all elements of a difficult job
or project you agreed to do within the

accepted time frame

0.531
0.304 Thinking analytically and logically
0.398
Acquiring broad knowledge across a
0.287 number of fields
0.393
An ability to use the techniques, skills,
and modern tools necessary for my

profession
0.392
Synthesizing and integrating ideas and
0.340 information
0.379
In-depth knowledge of a field or
0.388 discipline
0.378

* The most frequently used method to determine the strength of the relationship between two
variables is called the Pearson Correlation Coefficient. The closer the number is to 1, the more
strongly the two variables relate to one another.

QUESTIONS?

Takeaways
Actionable surveys require that you understand what

you intend to do with the data, and that you report back to
the respondents indicating how you acted on it.
Survey blueprints help to identify your information needs
and keep you on topic. Keep your questions and survey
simple.
An advantage of vendor surveys is that they compile
data into meaningful reporting tools, helping to provide
actionable data.
Incorporating a library question into a campuswide
survey enables you to get longitudinal user feedback and
to focus on life-long learning skills.

Thank you for joining us!

ULS blog: http://www.acrl.ala.org/ULS/
ULS Facebook: https://www.facebook.com/acrl.uls
ULS Midwinter events:

http://alamw14.ala.org/sessions/all?keys=uls