Questionnaire Design

A survey is only as good as the questions it asks

Questionnaire Modes

Face-to-face interview

Telephone survey
Self-administered survey (i.e. mail survey)

Types of Questions
The background question used to obtain demographic characteristics of the group being studied, such as age, sex, grade, level of assignment, and so forth. This information is used when you are categorizing your results by various subdivisions such as age or grade. Therefore, these questions should be consistent with your data analysis plan. Multiple choice or closed-end question. It is used to determine feelings or opinions on certain issues by allowing the respondent to choose an answer from a list you have provided . The intensity question (Attitude Measurement) a special form of the multiple-choice question, is used to measure the intensity of the respondents feelings on a subject. These questions provide answers that cover a range of feelings. The free response or open-end question. This type requires respondents to answer the question in their own words .It can be used to gather opinions or to measure the intensity of feelings.

Questionnaire Construction

Keep the language simple. Keep the number of questions to a minimum. Limit each question to one idea or concept. Do not ask leading questions. Allow for all possible answers. Avoid emotional or morally charged questions. Include a few questions that can serve as checks on the accuracy and consistency of the answers as a whole. Organize the pattern of the questions
Place demographic questions at the end of the questionnaire. Have your opening questions arouse interest. Ask easier questions first. To minimize conditioning, have general questions precede specific ones. Group similar questions together. If you must use personal or emotional questions, place them at the end of the questionnaire.

Questionnaire design The Effect of One Word

How important do you think it is to have a hotline telephone number people can call to get help when they have problems with their health plan? How important do you think it is for the GOVERNMENT to set up a hotline telephone number people can call to get help when they have problems with their health plan?

70%

43% 33% 21% 13% 5%

To have a hotline Very Important Somewhat Important

11%

3%
For government to set up a hotline Not too important Not at all important

Kaiser Family Foundation/Harvard University National Survey of Americans Views on Consumer Protections in managed Care, January 1998

Leading Questions
Do

you think the food in the hotel made your sick Did the hotel staff seem unhiegenic to you? Do you agree that hospital staff were close to exhaustion

Questionnaire design Balance in question wording

Whats wrong with these questions?
Do you support the United States taking military action against Iraq? If you heard that Candidate X supported stricter gun control laws, would that make you more likely to vote for her? Did you happen to vote in the presidential election in 2000?

Questionnaire design Balance in question wording

Bad questions, improved:
Do you support or oppose the United States taking military action against Iraq? If you heard that Candidate X supported stricter gun control laws, would that make you more likely to vote for her, less likely to vote for her, or wouldnt it make much difference?

Did you happen to vote in the presidential election in 2000, or not?

Questionnaire design
Open-ended or Closed-ended?
What do you think is the most important problem in health or health care for the government to address? (top 7 answers)
Cost of health care Cost of Rx drugs Coverage for uninsured Senior citizens care Other medical conditions Rx coverage for seniors Cost of insurance

Which of the following do you think is the most important health care issue for the President and Congress to deal with?
Helping seniors pay for prescription medicines Increasing the number of Americans covered by health insurance Making Medicare more financially sound for future generations Lowering the cost of medical malpractice insurance for physicians Helping states with the cost of Medicaid

15% 15% 15% 12% 10% 8% 7%

27%

27%

18%

11%

8%

Kaiser Family Foundation/Harvard School of Public Health Health News Index, April 2003 (conducted April 3-6, 2003)

Questionnaire design
Dont Know Answers offered or volunteered?
Do you favor or oppose the government offering parents money or "vouchers" to send their children to private or religious schools, or public schools outside their district?
Oppose

Do you favor or oppose the government offering parents money or "vouchers" to send their children to private or religious schools, or public schools outside their district, or havent you heard enough about that to have an opinion?
Oppose

36% 54% 42% 31%

Favor Favor

33%
No opinion

4%
No opinion
NPR/Kaiser Family Foundation/Kennedy School of Government Education Survey, June 1999 (conducted June 25 July 19, 1999)

Types of questions

Types of fixed alternative questions

Single dichotomy or dichotomous-alternative
questions Are you currently registered in a course at the University of Lethbridge? Yes____ No____

Respondent chooses one of two alternatives

(yes/no; male/female)

Types of fixed alternative questions

Multi-choice

alternative

Respondent chooses from several alternatives Many types

Multi-choice alternative questions

Determinant

choice

Choose only one from several possible responses

Which faculty are you currently registered in at the University of Lethbridge? Management ___ Education ____ Arts/Science____ Health sciences____ Combined degree____

 Frequency

determination Asks for an answer about frequency of occurrence In a typical week, how often do you purchase chocolate chip cookies? __never __ once __ 2 or more times

 Check

list

Provide multiple answers to a single question Should be mutually exclusive and exhaustive What brands of chocolate chip cookies have you, to the best of your memory, purchased in the past month (check all that apply?) __ Dare __ Chips Ahoy __ Presidents Choice Decadent etc. etc.

Attitude

rating scales

Attitude: An enduring disposition to consistently respond to various aspect of the world, including persons, events and objects

Attitude Scales: Scaling Defined

The term scaling refers to procedures for attempting to determine quantitative measures of subjective and sometimes abstract concepts.

Types of attitude scales

Simple

attitude scales Most basic form respondent responds to a single question Do not allow for fine distinctions or placement on continua
You are at a company party and are feeling nervous, but you are obligated to be there. Do you: __ find someone you know to buddy up with __ take it as an opportunity to meet new people

 Category

scales

More sensitive; provides more information Overall, how satisfied are you with the high speed performance of your Mercedes: __ very satisfied __ somewhat satisfied __ neither satisfied nor dissatisfied __ somewhat dissatisfied __ very dissatisfied If you could choose, how long would each term be? ___26 weeks __ 13 weeks __ 6 weeks ___4 weeks

 Summated

rating scales the Likert scale

Respondents indicate their attitudes by checking how strongly they agree or disagree with statements Chocolate chip cookies are my preferred variety of cookie
Strongly disagree (1) Disagree (2) Uncertain (3) Agree (4) Strongly Agree (5)

Likert Scale
Kids apparels provided by the retailers do not address the safety issues of babies
No, I strongly disagree No, I disagree quite a lot No, I disagree just a little I am not sure about this Yes, I agree just a little Yes, I agree quite a lot Yes, I strongly agree

 Semantic

Differential Rating scale

An attitude measure consisting of a series of seven-point bipolar rating scales allowing response to a concept
Think of your favorite type of cookie. Rate it on each of the following continua: Hard----------------------------------------------------Soft Lots of chips----------------------------------Fewer chips Crispy------------------------------------------------chewy

Example of Semantic Differential Scaling Form

Service is discourteous ___:___:___:___:___:___:___ Service is courteous Location is convenient ___:___:___:___:___:___:___ Location is inconvenient Hours are convenient ___:___:___:___:___:___:___ Hours are inconvenient Loan interest rates are high ___:___:___:___:___:___:___ interest rates are low

Figure 5: Contrasting Profiles of Banks A and B

Service is discourteous ___:___:___:___:___:___:___ Service is courteous Location is convenient ___:___:___:___:___:___:___ Location is inconvenient Hours are convenient ___:___:___:___:___:___:___ Hours are inconvenient

Loan interest rates are high ___:___:___:___:___:___:___interest rates are low

Bank A

Bank B

 Numerical

Rating scale

Similar to a semantic differential except that it uses numbers as response options to identify response positions instead of verbal descriptions
Think of your favorite type of cookie. Rate it on each of the following continua: Hard----------------------------------------------------Soft 8 7 6 5 4 3 2 1

 Constant

Sum Scales

Attributes based on their importance to the person. Respondents are asked to divide a constant sum to indicate the relative importance of attributes Example: Suppose the photocopy budget per professor was $100 per month. How much should be allocated to the following. Divide the $100 according to your preference: ____ photocopying for student needs; ____ photocopying for research needs; ____ photocopying for committee needs.
==== $100 TOTAL

 Stapel

Scales

An attitude measure that places a single adjective in the center of an even-number range of numerical values Example: Research Methodology +3 +2 +1 Exciting -1 -2 -3

Example of a Stapel Scale

Service is courteous Location is convenient Hours are convenient Loan interest rates are high

-5 -4 -3 -2 -1 1 2 3 4 5

 Graphic

Rating Scales

An attitude measure consisting of a graphic continuum that allows respondents to rate an object by choosing any point on the continuum

 Rank-Order

Scales

Scales in which the respondent compares one item with another or a group of items against each other and ranks them.
Example: next slide

Most important skills

Adaptability to change Problem identification Listening skills Written communication Leadership Informal Oral communication Analytical thinking/problem solving Time management Coping with stress/job pressures Interpersonal relations Formal oral presentations

Most important skills

Adaptability to change Problem identification Listening skills Written communication Leadership Informal Oral communication Analytical thinking/problem solving Time management Coping with stress/job pressures Interpersonal relations Formal oral presentations

8 6 1 2 4 3 5 7 11 9 10

9 6 1 4 2 3 5 10 7 8 11

 Paired

Comparison Scales

For each pair of sports car, place a check beside the one you most prefer if you had to choose between the two.
- BMW Z3 - Porsche Boxter - Chevrolet Corvette - Porsche Boxter

- Chevrolet Corvette - BMW Z3

 Sorting

Respondent indicates their attitudes or beliefs by arranging items.

Example: Please sort the following cards with pictures of cookies into the following categories Like Dislike Neither like nor dislike

Examples of Poor Survey Questions

Which of these services would you be interested in? ____ loans ____ mentoring ____ business counselling ____ information referral (This question should have an other category) What beverages do you drink? ____ Milk ____ coke ____ non-cola drink ____ coffee ____ tea ____ juice (This question is too broad. Most of us will have drunk some of these at some time. Is the respondent to check a number of boxes or only one?

Sample Questionnaire
(developed by a woman who was interested in selling southwestern jewelry made by Native Indians) Q1. Have you ever purchased or received southwestern jewelry? Yes ______ No ______ Q2. Have you ever purchased or received southwestern jewelry made by native Indians? Yes ______ No ______ If Yes, what type of jewelry? Necklace____ Ring _____ Bracelet _____ Earnings _____Other _____ Q3. Would you be interested in purchasing the above mentioned jewelry made by native Indians? Yes ______ No ______ Q4. Do you know where to shop for such jewelry? Yes ______ No ______ Q5. When buying jewelry, what do you value the most? On a scale of 1 through 5, list in order according to your preference. One represents your most valued choice. Craftsmanship_____ Cost _____ Uniqueness _____ Other_____