D-1369

Questionnaire Design
Asking Questions with a Purpose
Mary G. Marshall
Program Development & Evaluation
The Texas A&M University System

CONTENTS

Constructing a Questionnaire 3

Kinds of Information 4

Wording the Questions 6

Response Option Guidelines 9

Types of Questions 11
Close-Ended Questions with One Choice Answers 12
Two Option Response
One Best Answer
Rating Scale
Ordered Choice 14
"Other, Please Specify"
Items in a Series 15
Paired Comparisons
Matching
Close-Ended Questions with Multiple Choice Answers 16
Check All That Apply
Lists
Ranking 17
Open-Ended Questions 18
Fill in the Blank

One Question Structured Four Different Ways 19

Formatting the Questionnaire 20

Pretesting the Questionnaire 24

References 26
 CONSTRUCTING A QUESTIONNAIRE
For many evaluation activities, questions will be the main source of information. As a series of
questions in written form, a questionnaire is a tool for getting similar information from people that
permits numerical tabulation and discussion. An alternative method for acquiring information is an
interview, but it is verbal in form.

Writing questions and constructing a questionnaire takes time and attention. The starting point is to
know what kind of evidence is needed to meet the purpose of the study and to know how the
information is to be used.

* Make a list of what you want to know. What do you really want to find out? What do you
want to achieve through the use of the questionnaire?

* From the beginning, think through what you will actually do with each piece of
information. What do you want to be able to say? Do you expect to use frequencies,
counts, percentages, rankings, multivariate analysis?

* Ask a question only when it has a purpose, if it is directly related to the purpose of the
study. A tendency to collect more and more data adds to costs in time and money and
results in information overload. Eliminate all the "nice to know" pieces that aren't
essential. Eliminate ambiguous questions.

* Check to see if the information is already available. Many questions are unnecessary.

* In writing questions, look through the respondent's eyes:

Will the question be seen as reasonable?

Will the question infringe on the respondent's privacy?
Will the respondent be able and willing to tell you what you want to know?

Be selective and realistic. Know what information is needed, why it is wanted, and how you will
use the information.
 KINDS OF INFORMATION
A questionnaire can help you collect information about what people do, what they have, and what
they think, know, feel or want.

Five different types of information may be distinguished. Any one or a combination of these types
of information may be included in a questionnaire.

KNOWLEDGE - what people know; how well they understand something.

These questions ask what people know, are aware of, understand. Choices implied in knowledge
questions include correct/incorrect, accurate/inaccurate, what is accepted as true or factual. For
example:

What is the major cause of accidental deaths among children inside the home?

The most effective weight loss plan includes exercise. TRUE-FALSE

The ideal refrigerator temperature is .

BELIEF - what people think is true; an opinion.

Beliefs are judgments of what people think is true or false, what one thinks exists or does not
exist. Choices implied in belief questions include what did or did not happen. Questions may seek
perceptions of past, present or future reality. For example:

In your opinion, does positive self-esteem among adolescents prevent drug abuse?

Do you think that lower beef prices would increase beef consumption?

ATTITUDE - how people feel about something; a preference.

Such questions ask people to indicate whether they have a positive or a negative feeling about a
subject, what they value. Words typically used in attitude questions include: prefer/not prefer;
desirable/undesirable; favor/oppose; should/should not; satisfactory/unsatisfactory. For example:

Do you favor or oppose controlled calving for your operation?

Do you agree or disagree that eating beef causes heart disease?

BEHAVIOR - what people do Ñ may be a physical/manual or mental behavior.

Questions about behavior ask people what they have done in the past, what they are doing now, or
what they plan to do in the future. For example:

Have you ever attended an Extension program about cotton production?

Do you treat your cotton for bollworms?

How are you currently using the information gained in the food storage workshop?

ATTRIBUTES - what people are; what people have.

Attributes are a person's personal or demographic characteristics such as age, education,

occupation, income. Questions on attributes ask people about who they are rather than what they
do. For example:

Where do you currently live?

How many children do you have?

What percentage of your household income comes from off-farm employment?

To write meaningful questions, be clear about the intended uses and type of information desired.
If questions are vague, the questionnaire may elicit attitudes and beliefs when the intent is to
document actual behavior.

Likewise, questions related to each type of information present different writing problems.
Questions concerning attitudes tend to be more difficult to phrase, given the complexity underlying
most attitudes. Careful attention should be given to the wording of such questions. In contrast,
questions about knowledge, behaviors and attributes tend to be more direct.

The response or information you obtain is only as good as the question. To get the type of
information you want, you must ask the right question!
 WORDING THE QUESTIONS
Wording questions to gain what is wanted and also to be understood by all respondents is a
challenging task. In writing questions, consider three things: 1) the particular people for whom
the questionnaire is being designed, 2) the particular purpose of the questionnaire, and 3) how
questions will be placed in relation to each other in the questionnaire.

Some suggestions appear below.

* Use simple wording. Adapt wording to the vocabulary and reading skills of people
who will be asked for information, but don't talk down to them. Do any words have
double meanings or are any words confusing?

* Be specific. A question about older youth should specify what age or grade is
considered as "older." And in the question "How many times did your 4-H club meet last
year?" the year should be specifiedÑ1997, or the last 12 months, 1997-98, or September
1997 through August 1998.

* Avoid using abbreviations, jargon, or foreign phrases. Will respondents

understand what is meant by such terms as issue-based programming, learning
experiences, IMPAC programs? Or the technical terms that professionals commonly use as
short-cuts.

* Use clear wording. Words such as regularly and occasionally mean different things to
different people. Examples of vague terms include these: majority [more than half of
what?], often [daily, twice weekly, weekly?], governmental [city, county, state, federal?],
older people [how old?].

* Include all necessary information. In some cases, respondents may not know
enough to adequately answer the question. For example: Do you agree or disagree with
the county's new environmental policy? Respondents may not know what the policy is or
whether it is the most recent one. Provide a statement summarizing points of the policy that
distinguishes it.

* Avoid questions that may be too precise. People's lives are usually not so orderly
that they can recall exactly how many times they ate out last year or how many Extension
meetings they attended in 1996. To help respondents formulate an answer, they response
category might provide a range for selection, such as 0 to 5, 6 to 10, 11 to 15, etc.
* Phrase any personal or potentially incriminating questions in less
objectionable ways. Being asked to indicate income level, ethnic background, drug
use, or eating habits may be objectionable to respondents. One method is to ask
respondents to select from among broad categories [income less than $10,000, $10,000 to
$19,999, $20,000 to $29,999, $30,000 and over, etc.] instead of specifying precise
information. Also, a series of questions may be used to soften or overcome the
questionable nature of certain information.

* Avoid questions that are too demanding and time consuming. Examples are:
Rank the following 15 items in order of their importance to you. In 25 words or less, what
is your philosophy of 4-H leadership?

* Use mutually exclusive categories. Make sure that only one answer is possible. In
the example of "How did you hear about the Extension seminar?" the response categories
are: a) from the Extension office, b) at work, c) from a relative, d) from the newspaper, e)
from a friend, f) at an Extension meeting. The respondent may have heard about the
seminar from a friend at work, so that more than one answer is possible on this list.

* Avoid making assumptions. Questions such as "How many children do you have?"
or "Do you prepare beef when you have friends in to eat?" make assumptionsÑthat the
respondent has children, that the respondent has friends in to eat. A set of questions would
be preferred with the first question establishing the situation, followed by the question of
concern. For example: "Do you have children?" YES/NO [If yes, "How many children
do you have?"]

* Avoid double questions. Two questions written together gives no opportunity for
people to respond in favor of one part or the other. In this exampleÑ "Did the poultry
production seminar help you to identify ways to improve the sanitation and increase the
nutrition of your cage bird operation?" Ñit would be better to ask about sanitation and
nutrition separately.

Other double questions may be unduly ambiguous. This exampleÑ "Do you favor
legalization of marijuana for use in private homes but not in public places?" Ñgives
respondents no way to say whether they favor both places, oppose both places, oppose
home but favor public use, or oppose legalization as a concept in general.

* Avoid bias in questions. Such questions influence people to respond in a way that
does not accurately reflect their position. A question can be biased in several ways: 1)
when the question implies that the respondent should be engaged in a particular behavior,
 2) when unequal response categories are given or the responses are loaded in one
direction, 3) when words with strong positive or negative emotional appeal are used, such
as bureaucratic, boss, equality, etc.

Some examples of biased questions are shown here:

1. How would you rate the housing in which you live?

1 SATISFACTORY
2 GOOD
3 EXCELLENT
No negative options are provided.

2. More farmers in Greater County are using Superb than any other variety of wheat. Do you
use Superb?
1 YES
2 NO
This question suggests that the respondent should be using Superb.

3. Do you agree that funding for Extension in your county should be increased?
1 NO
2 YES
This is a leading question. A better question would state:

4. Do you agree or disagree that Extension funding should be increased? (Circle one)
1 STRONGLY AGREE
2 AGREE
3 DISAGREE
4 STRONGLY DISAGREE

* Make the response categories clear and logical. Too often the answers are confusing, not
in local order, or improperly spaced on the page. For example:

POOR LOGIC POOR SPACING BETTER

1 1000 ACRES 1 10 ACRES 1 0 ACRES
2 999-500 ACRES 2 1-9 ACRES 2 1-200 ACRES
3 499-100 ACRES 3 1 0-99 ACRES 3 201-400 ACRES
4 99-10 ACRES 4 100-499 ACRES 4 401-600 ACRES
5 9-1 ACRES 5 500-999 ACRES 5 601-800 ACRES
6 0 ACRES 6 1,000 ACRES 6 801-1000 ACRES
7 OVER 1000 ACRES

* Use complete sentences. Trying to keep questions simple and concise may result in
questions being too cryptic and misunderstood. Check for any incomplete sentences.
 RESPONSE OPTION GUIDELINES
Guidelines to help in answering and in tabulating data have been established by Extension's Data
Center to emphasize consistent response instructions, "positional parameters" and coding
categories.

Positional parameters are the numbers/letters which identify each piece of data for computer entry.
These letters or numbers point to the place where the response can be seen. "Codes" are the values
representing each response on the questionnaire. Instructions for answer are the same for each
type of question:

* FILL IN THE BLANK responses are indicated by a blank line to show the response space,
and the respondent is instructed to fill in the blank.

5. Please list the number of cattle (if any) sold in 1996.

[Write total number on the line.]
A. COWS SOLD
B. CALVES SOLD
C. BULLS SOLD
D. STEERS SOLD

* If ONE ANSWER is wanted for the whole question or one answer for each item within
the question, numbers (with no periods) are used as the codes for each response, and the
respondent is asked to circle one choice.

6. Would you recommend this program information to your friends and neighbors? [Circle one]
1 NO
2 YES
7. To what extent do you use the following sources of information when you do educational work
with the elderly? [Circle one number for each item.]

NOT VERY SOME- VERY

AT ALL LITTLE WHAT MUCH

A. Radio 1 2 3 4
B. Newspaper 1 2 3 4
C. Senior Center 1 2 3 4
D. Professional journals 1 2 3 4
E. Popular magazines 1 2 3 4
F. Department of Health 1 2 3 4
* If MORE THAN ONE ANSWER is sought or likely, brackets or boxes are used for
marking; the respondent is instructed to check all that apply.

8. What types of minerals do you use? (Check all that apply.)

¥ a. SALT
¥ b. SALT/BONEMEAL
¥ c. HIGH CALCIUM, LOW PHOSPHOROUS
¥ d. LOW CALCIUM, HIGH PHOSPHOROUS
¥ e. EQUAL CALCIUM AND PHOSPHOROUS

Using these guidelines will improve data quality by bringing clarity and
consistency to Extension-produced questionnaires and improve the tasks of data
entry and analysis. In this booklet, examples are presented that use the various
types of response options. Each example is coded and the instructions provided
are compatible with [Extension Information Technology] Extension Data Center
guidelines.
 TYPES OF QUESTIONS
Questions are open-ended or close-ended. Some questions have answers which fall along an
implied continuum (rating scales). Others have answers in no particular order (lists). Some
questions have multiple choice (check all that apply) options, and others provide relevant answer
choices but respondents are free to add another answer.

Each type of question has certain advantages and disadvantages.

Close-ended questions have answer options provided and respondents must select either one
answer or multiple answers from what is given. These questions have greater uniformity in
responses but depend on your knowing and including all relevant responses. Responses for close-
ended questions must be exhaustive [i.e. include all possible answers] and also mutually
exclusive in providing for the selecting of a single response [without the choice seeming to
belong to more than one option].

Open-ended questions are those where respondents provide their own answers to the question,
without any previously provided options. These are fill-in-the-blank responses. They allow
respondents to express their own thoughts and comments but are more demanding of both the
respondent and the person doing the analysis.

The various types of questions are further explained on pages 12 to 19, with examples of some
response options and formatting on page 20 to 23.
 CLOSE-ENDED QUESTIONS
WITH ONE CHOICE ANSWERS

Respondents are given a list of answers and asked to circle the choice that they feel is the best.

¥ TWO OPTION RESPONSE Ñ This is the simplest response format. The response options
might include: NO-YES, DISAGREE-AGREE, FALSE-TRUE, OPPOSE-FAVOR. For example:

9. Do you raise stocker cattle?

1 NO
2 YES

Depending on the purpose of the information, this may be the most appropriate format and is often
used as a beginning question in a series on the same topic. However, using a rating scale or a
ranking (when appropriate) offers more information.

¥ ONE BEST ANSWER Ñ These questions can be used to solicit information or to test
knowledge. They are appropriate when all relevant answer choices are known and can be listed.
Respondents are provided with the list of answers and asked to check or circle the choice they feel
is the best. Responses are independent of one another, instead of being gradations along a
continuum. An example follows:

10. What does the work "nutrition" mean to you? [Circle one number]
1 GETTING ENOUGH VITAMINS.
2 THE FOOD I EAT AND HOW MY BODY USES IT.
3 HAVING TO EAT FOODS I DON'T LIKE.
4 HAVING GOOD HEALTH.

¥ RATING SCALE Ñ Often, questionnaire respondents are asked to tell their choice at the
most appropriate point on a scale. Whether you use a scale of three, four, five or more categories
depends on the question and the amount of differentiation that is possible and desired. A greater
spread of numbers allows for greater accuracy in statistical analysis. See next page for examples.
 11. To what extent do you agree or disagree with the new zoning code? [Circle one]
1 STRONGLY DISAGREE
2 MILDLY DISAGREE
3 NEITHER AGREE OR DISAGREE
4 MILDLY AGREE
5 STRONGLY AGREE

12. When purchasing new herd bulls, how important are the following traits in your selecting
process? [Circle one number for each selection trait]

SELECTION TRAIT OF LITTLE HIGHLY

IMPORTANCE IMPORTANT
A. Performance 1 2 3 4 5 6 7
B. Conformation 1 2 3 4 5 6 7
C. Pedigree 1 2 3 4 5 6 7
D. Breed 1 2 3 4 5 6 7

Starting with either positive or negative response options appears to have little effect on response.
neither does it matter whether Yes or No is listed first. But you must be consistent in the order
followed throughout the questionnaire.

For greater differentiation, use a numerical scale running from 0 or 1 to some number [see second
example above]. A five-point option series seems to be most useful for attitude scaling. A four-
point option series appears useful for ratings [e.g. excellent, good, fair, poor). Many people may
relate best to a ten-point scale.
Another decision is whether to provide an even or odd number of response options. An odd
number of categories provides a middle or neutral position for selection. An even number of
categories forces the respondent to take a side. This is appropriate when you want to know in
what direction the people in the middle are leaning.

Some examples of rating response categories are these:

Very Dissatisfied No Help at All Decreased

Somewhat Dissatisfied Slightly Helpful Stayed the Same
Somewhat Satisfied Fairly Helpful Increased
Very Satisfied Very Helpful

Strongly Unfavorable Strongly Disagree Poor

Generally Unfavorable Disagree Somewhat Fair
Uncertain Uncertain Good
Generally Favorable Agree Somewhat Excellent
Strongly Favorable Strongly Agree
¥ ORDERED CHOICE Ñ In this type of question, the responses are usually intended to
measure degree or intensity in an ordered sequence or scale. These questions are particularly
suited for evaluating attitudes or opinions. They are appropriate when the topic is well defined and
the choice represents a gradation along a single dimension. See these examples:

13. Within your 4-H club, describe the extent to which you were included in making important
decisions. (Circle one number)
1 NEVER
2 RARELY
3 SOMETIMES
4 OFTEN

14. How do you feel about this statement: "I wish this community had more recycling centers."
(circle one number)
1 STRONGLY DISAGREE
2 MILDLY DISAGREE
3 NEITHER AGREE NOR DISAGREE
4 MILDLY AGREE
5 STRONGLY AGREE

Responses need to reflect a clear difference and be balanced both positively and negatively. A poor
example is: DECREASED, STAYED THE SAME, INCREASED A LITTLE, INCREASED
SOMEWHAT, INCREASED A LOT, which gives three positive choices but only one negative
option.

It is not necessary to use the same categories for each question, but do it where possible. It is
more important to choose those that are most appropriate to the question being asked.

¥ "OTHER, PLEASE SPECIFY" Ñ Here the respondent is offered a choice of answers plus
an open-ended response category such as "Other (Please specify)." This protects you against
leaving out an important answer choice. It also means that you will have narrative text to analyze,
so think about what you will do with these responses. Too often they are never used because they
cannot be entered neatly into the computer. Examples follow with open-ended choices included:

15. Which of these community recreational facilities to you use the most? (Circle one number)
1 PARKS
2 TENNIS COURTS
3 SWIMMING POOLS
4 OTHER
 16. What do you consider the main responsibility of your county 4-H agent? [Circle one number]
1 WORK WITH PEOPLE WHO REQUEST HELP
2 WORK WITH 4-H MEMBERS
3 WORK WITH VOLUNTEER 4-H LEADERS
4 PLAN AND ORGANIZE COUNTY YOUTH EVENTS
5 ORGANIZE AND EXPAND NEW 4-H CLUBS
6 OTHER (please specify)

¥ ITEMS IN A SERIES Ñ When various questions use the same response category, it is
possible to present the responses in tabular form. Identify the answer choices with a horizontal
bracket which guides respondents to the answer choices, as in the example below.
17. How often do you eat the following meats? (Circle one number for each meat)
LESS THAN 1-3 TIMES 4-6 TIMES
NEVER ONCE/WEEK A WEEK A WEEK DAILY
A. Beef 1 2 3 4 5
B. Lamb 1 2 3 4 5
C. Pork 1 2 3 4 5
D. Poultry 1 2 3 4 5
E. Fish 1 2 3 4 5

¥ PAIRED COMPARISONS Ñ Respondents are asked to compare one item to another,

usually expressed in terms of "either/or" or one item "versus [vs] another.

18. In comparing beef to other meats, which does your family use more often?
[Choose one from each comparison, circling the number on that line.]
1 BEEF OR 2 POULTRY
3 BEEF OR 4 LAMB
5 BEEF OR 6 PORK
7 BEEF OR 8 WILD GAME (VENISON, ETC.)

¥ MATCHING Ñ Respondents are asked to match responses to a list of items.

19. Match each food to the proper food group by putting the correct lower case letter in the
blank. For example: A. c Strawberry
A. ____ WHOLE WHEAT BREAD a. Meat and meat products
B. ____ NECTARINE b. Milk and milk products
C. ____ HAM c. Fruits and vegetables
D. ____ YOGURT d. Breads and cereals
E. ____ PUMPKIN e. Sweets
F. ____ OATMEAL
 CLOSE-ENDED QUESTIONS
WITH MULTIPLE CHOICE ANSWERS

¥ CHECK ALL THAT APPLY Ñ This common response format is actually a series of YES -
NO items. It is a fast and easy way to obtain such in formation which also saves space. There is a
risk, however, that the respondent will not consider each item Ñ so don't make the list too long.

Examples:
20. How do you currently market your cattle? [Check all that apply.]
¥ a. LOCAL AUCTION BARN
¥ b. LOCAL MEAT PACKER
¥ c. TERMINAL MARKET
¥ d. HEDGING
¥ e. DIRECT SALES CONTRACT (SELLING FORWARD)
¥ f. LIVESTOCK DEALER

21. What information would you like covered in the next Extension workshop?
(Check all your choices)

¥ a. COW HEALTH
¥ b. CALF DISEASES
¥ c. INTERNAL PARASITES
¥ d. BRUCELLOSIS
¥ e. GRUBS OR TICKS
¥ f. REPRODUCTIVE DISEASES

¥ LISTS Ñ A list provides a series of answers. Respondents may choose one or more answers,
depending on the instructions.

22. Listed below are some adjectives that might be used to describe a person. Please
indicate for each adjective, whether the adjective does or does not describe you.

DESCRIBES DOES NOT DON'T

ME DESCRIBE ME KNOW
A. Ambitious 1 2 3
B. Happy 1 2 3
C. Idealistic 1 2 3
D. Outgoing 1 2 3
 23. From the list provided, select THREE adjectives which best describe you. (Place the
letter of the alphabet on the lines provided.

1. _______ a. Ambitious
2. _______ b. Happy
3. _______ c. Idealistic
d. Outgoing

¥ RANKING Ñ Rank ordering is a multiple-choice option. Respondents are given various

responses and asked to rank them in order of importance or indicate a "top three." See the
examples below.

24. What would you like to know more about? Select three responses from the list and
rank them in order of 1, 2, 3.
1. _______ a. What to eat to look better
2. _______ b. How food affects me
3. _______ c. Weight control
d. Health foods
e. Physical conditioning through diet
f. Vitamins

25. What would you like to know more about? Select up to four responses from the right-
hand column and rank them in order of first, second, third, fourth choice.

1. _______ first choice a. What to eat to look better

2. _______ second choice b. How food affects me
3. _______ third choice c. Weight control
4. _______ fourth choice d. Health foods
e. Physical conditioning through diet
f. Vitamins
g. Reading labels to find the fat content
h. Saving costs in food buying
 OPEN-ENDED QUESTIONS

FILL IN THE BLANK Ñ An open-ended question is often the easiest way to ask a question,
but the responses are not easy to analyze. Answers are likely to be varied; they must be categories
and summarized. The computer or data enter person can't "magically" tabulate open-ended
questions, so think about how you will analyze the responses.

* Most often used to stimulate free thought, solicit creative suggestions, or recall information
learned [See examples 1 and 2 below.]

* Can be used as a probing question to elicit more detail.

26. What do you think should be done to improve the Family and Consumer Science
program in this county?

27. Name the five basic food groups.

28. Please say how you intend to use the information you gained during the workshop.

29. We are interested in knowing any other comments you might have concerning the 4-H
program and your role as 4-H leader. Please write in the space below any thoughts you'd
like to share with us.

* When asking for a numeric response, include the unit which is to be used.

30. Please list the number of acres, if any, of temporary pasture you planed in 1997.
A. ____ ACRES OF WHEAT
B. ____ ACRES OF OATS
C. ____ ACRES OF RYE GRASS
D. ____ ACRES OF CLOVER
E. ____ ACRES OF SUMMER ANNUALS
F. ____ ACRES OF OTHER (and specify):_______________________________

* Useful when respondents are asked to supply a specific answer and a large array of
responses is possible (example 3 above) or when all possible answers are not known.

* Often used at the end of a questionnaire to ask respondents for any additional comments
they might have (example 4 above).
 ONE QUESTION STRUCTURED
FOUR DIFFERENT WAYS

¥ CLOSE-ENDED WITH UNORDERED RESPONSE CHOICES

Which of these four topics would you like most to see as the primary program emphasis for
next year? (Circle the number of your answer)
1 Effective Parenting
2 Child Development
3 Guidance and Discipline
4 Communications

¥ CLOSE-ENDED WITH ORDERED RESPONSES

How important to you is each of the following possible program emphases? [Circle a
number for each item]

NONE LITTLE SOME MUCH

A. Effective Parenting 1 2 3 4
B. Child Development 1 2 3 4
C. Guidance and Discipline 1 2 3 4
D. Communications 1 2 3 4

¥ PARTIALLY CLOSE-ENDED

What topic do you feel should be the main program emphasis for next year? [Circle
number of your answer]
1 Effective Parenting
2 Child Development
3 Guidance and Discipline
4 Communications
5 Other (please specify)

¥ OPEN-ENDED

What would you like to see as the main program emphasis next year?
 FORMATTING THE QUESTIONNAIRE
As the questions are determined, a series of decisions must be made about the questionnaire
format: its appearance, length, and order of questions. The questionnaire must be pleasing to look
at and easy to complete.

The following guidelines may help in formatting the questionnaire.

¥ Begin with an introduction which includes the questionnaire's purpose, who is conducting
it, to what use the information will go, and confidentiality. In mailed questionnaires,
reinforce points that were made in the cover letter.

¥ Make the first questions nonprovoking and interesting. Make them clearly related and
useful to the topic of the questionnaire. The beginning questions should not be open-ended
or questions with a long list of answer choices.

¥ Put the more important questions at the beginning.

¥ Arrange the order of questions to achieve continuity and a natural flow. Try to keep all
questions on one subject together. Put the more general questions first, followed by a
more specific question. For example, if you want to find out about a person's knowledge
of insurance, start with questions about types of insurance, purpose of the different types,
followed by questions about costs of these various types.

¥ Try to use the same type of question/responses throughout a particular train of thought. It
breaks the attention span to have a multiple choice question following a YES/NO question,
then an open-ended question.

¥ Place demographic questions (age, gender, race/ethnicity, etc.) at the end of the
questionnaire.

¥ Use quality print in an easy-to-read type face. Allow sufficient open space to let the
respondent feel it is not crowded and hard to read.
¥ Always use the same number for the same answer throughout the questionnaire. For
example, if you begin with:
1 NO
2 YES
then don't switch later to:
1 YES
2 NO

¥ Keep the whole question and its answers on the same page. Don't cause respondents to
turn a page in the middle of a question or between the question and its answers.

¥ Be sure that the question is distinguishable from the instructions and the answers. Maybe
put the instructions in boldface or italics. Dillman [1978] suggests using lower case letters
for questions and upper case letters for answers.

¥ Try to arrange questions and answers in a vertical flow. This way, the respondent moves
easily down the page, instead of side to side.

Example: Instead of horizontally:

___ EXCELLENT ___ GOOD ___ FAIR ___ POOR

Format vertically:
1 EXCELLENT
2 GOOD
3 FAIR
4 POOR

¥ Give directions on how to answer. Put directions in parentheses immediately after the
question. It is better to repeat directions too often than not enough. Specific instructions
may include: (Circle the number of your choice.) (Circle only one.) (Check all that
apply.) (Please fill in the blank.) (Enter whole numbers.) (Please do not use decimals or
fractions.)

¥ Pre-code items and response categories as much as possible, to help in data tabulation and
analysis. This will save time and money since data can be entered directly from the
questionnaire without recoding the responses. As much as possible, put the response
blanks in the same location on the page to make tabulation easier.
¥ Use transitional statements to build continuity. Transitional statements are used in three
ways: 1) to signal that a new topic is beginning, 2) to start new pages, and 3) to break up
the monotony of a long series of questions.

Example transitional statements:

Next we would like to ask you several questions about the community
organizations you belong to.

Another important purpose of this survey is to learn how you feel about the work of
service organizations in your community.

Finally, we would like to ask a few questions about you to help us in interpreting
the results.

¥ It may be useful to make sure that the respondent is referring to the same program and
defining it similarly. A validation item [Bennett, 1982] at the beginning of the
questionnaire identifies the program and sets the stage for the questions to follow. It is a
brief summary of the program's activities and people who were involved.

For example:
The Dell County Extension family life program included a variety of activities
during 1997 that focused on teaching money management and budgeting skills to
help families better manage their resources. These activities included lunch and
learn programs, computer budgeting workshops, letter series and short courses.
Consumers from across the county attended these activities on Money Management
Skills.

¥ Some questions may not apply to every respondent. For such screening or filtering
questions, make it clear who is to answer the question and what should be done by those
who aren't supposed to answer. Dillman [1978] makes three suggestions: 1) use arrows
to guide respondents from one question to the next, 2) indent all questions that may be
screened, or 3) use boxes to direct respondents past the question(s) they don't need to
answer.
See these examples of screening/filtering questions:

Q-5 Do you own or rent the home in which you now live?

1 OWN THE HOME > If you OWN, skip from

2 RENT HOME
here to Q-14 on next
If you rent: Q-6 How much is your page.
monthly rent?

1 LESS THAN $100

2 $100 TO $199
3 $200 TO $299
4 $300 TO $399
5 $400 OR MORE

--- OR THIS FORMAT ---

Q-5 Do you own or rent the home in which you now live? [Circle the number of your
answer]

1 OWN HOME 2 RENT HOME

(If you own your home) (If you rent your home)
Q-6a How much is your monthly Q-6b How much is your
house payment (without monthly rent?
property taxes)?

1 LESS THAN $200 1 LESS THAN $200

2 $200 TO $299 2 $200 TO $299
3 $300 TO $399 3 $300 TO $399
4 $400 TO $499 4 $400 TO $499
5 $500 OR MORE 5 $500 OR MORE

Q-7a How much per month do you Q-7b Which of these, if any,
pay for electricity, heat, water is included in your
and garbage collection? monthly rent?
(Circle all that are included)

1 LESS THAN $30 1 ELECTRICITY

2 $30 TO $74 2 GARBAGE
3 $75 TO $124 3 HEAT
4 $125 TO $199 4 WATER
5 $200 OR MORE 5 NONE OF THE ABOVE
 PRETESTING THE QUESTIONNAIRE
Pretesting is an essential part of questionnaire design. Many practitioners feel that if you don't
have the resources to critically test the usability and understandability of the instrument in advance,
then don't do the study.

This means a careful examination of the individual questions and the questionnaire as a whole.
Allow enough time to incorporate any revisions. Unfortunately, too many people consider pilot
testing as a superficial task they can avoid, but find later that the intended respondents did not
understand well enough to deal with most questions.

According to Dillman [1978: 156], any pretest needs to provide evidence about the following
questions:

Is each question measuring what it is intended to measure?

Are all the words understood?

Are questions interpreted similarly by all respondents?

Does each close-ended question have an answer that applies to each respondent?

Does the questionnaire create a positive impression, one that motivates people to answer it?

Are questions answered correctly? (Are some missed? Do some elicit uninterpretable
answers?)

Does any part of the questionnaire suggest bias on the part of the researcher?

Taking the five steps below, several weeks or even months in advance, will greatly increase the
quality of information obtained from a questionnaire.

1 Have colleagues critically review the questionnaire. Ask co-workers to review all
questions to see if wording and instructions are clear Ñ if the questionnaire will
accomplish the study purposes. Consider the reviewers' comments carefully and see how
they can enhance the questionnaire.

2 Select people like your respondents to "pre-test" the questionnaire. Choose people who
represent a crosssection of the population that will be given the questionnaire. After they
answer the questions, ask for comments about their impressions Ñ understanding of the
purpose, questions, ease of response.
3 Simulate the actual data collection procedure as much as possible. Whether a mail survey,
telephone or direct interview, take it through a practice run of all the intended steps. If a
mailed questionnaire, have people answer it without any help and afterward obtain their
suggestions. If an interview, have the interviewer actually conduct the pilot test, either by
phone or face-to-face, as it will be done.

Obtain feedback about the form and content of the questionnaire. Were any questions
misunderstood? Were the directions clear? How long did it take to fill out? Was it too
long or too difficult? Was there enough space to write in the responses? etc.

It is very important also to assess whether the questions produce the data you need to meet the
purpose of the study.

4 Try the tabulation and analysis procedures. Using the responses from people who
pretested the questionnaire, do the data entry tabulations and analyze in the way you would
to prepare your final report, to see if the questionnaire yields data that can be analyzed.

5 Revise and re-test. Reformulate wherever needed and check the new questions with a new
representative group of respondents. When responses are adequately understood and yield
the type(s) of information you are searching for, the design stage is nearing completion.

Now prepare and check the final draft. Go over each question until you and other users are
satisfied that:
Ñ Specific items will contribute to key questions I am trying to answer.
Ñ This makes a good impression for mail or meeting distribution.
Ñ Information obtained from the questionnaire will mean something for my study and
provide the evidence needed.
 REFERENCES

Babbie, Earl R. SURVEY RESEARCH METHODS. Belmont, CA: Wadsworth Publishing

Co., 1973.

Bennett, Claude F. REFLECTIVE APPRAISAL OF PROGRAMS (RAP): An

Approach to Studying Clientele-Perceived Results of Cooperative Extension
Programs. Ithaca, NY: Cornell University Media Services, 1982.

Dillman, Donald A. MAIL AND TELEPHONE SURVEYS: THE TOTAL DESIGN

METHOD. New York: John Wiley and Sons, Inc., 1978.

Fisher, Christy and Jeffrey D. Layman. EDGE; CONSTRUCTING A

QUESTIONNAIRE. Columbus, OH: Ohio Cooperative Extension Service, The Ohio State
University, 1986.

Newsome, Bob W., et al. NORTHEAST AREA EVALUATION PROCESS. Manhattan,

KS: Kansas State Cooperative Extension Service, Kansas State University, n.d.

Rohs, F. Richard. QUESTIONNAIRE CONSTRUCTION. Athens, GA: Georgia

Cooperative Extension Service, 1985.

Sawer, Barbara J. EVALUATING FOR ACCOUNTABILITY. Corvallis, OR: Oregon

State University Extension Service, 1984.

Sudman, Seymour and Norman M. Bradburn. ASKING QUESTIONS, A PRACTICAL

GUIDE TO QUESTIONNAIRE DESIGN. San Francisco: Jossey-Bass Publishers, 1982.

Extension Computer Technology Group. SURVEY DESIGN - GETTING DATA INTO

THE COMPUTER MANUALLY. College Station: Texas Agricultural Extension Service,
n.d.