Business Research Design

Questionnaire Design

Questionnaire
Definition A questionnaire is a set of questions to be asked from respondents It should also have appropriate instructions for interviewer for proper administration Functions Enables data collection Lends structure to the interview Provides standard mean to write down answers Helps in processing collected data

Various steps in questionnaire design

Preliminary Decisions Question content Response format Question wording Question sequence Pre-test, revision & final draft

Preliminary Decisions
Required Information : The information needed keeping in view objectives of the study. Questionnaire translates information needed into a set of specific questions Target population: The characteristics of respondent group has great influence on questionnaire design.(users/nonusers, illiterates/educated) Interviewing Technique: The format & type of questions will be different for personal interviews, FGD,s ,Mailed Questionnaires & telephonic interviews

Effect of Interviewing Method on Questionnaire Design

Department Store Project Mail or personal Questionnaire Please rank order the following department stores in order of your preference to shop at these stores. Begin by picking out the one store that you like most and assign it a number 1. Then find the second most preferred department store and assign it a number 2. Continue this procedure until you have ranked all the stores in order of preference. The least preferred store should be assigned a rank of 10. No two stores should receive the same rank number. Store Rank Order 1. Lord & Taylor 2. Macy's . . 10. Wal-Mart ____________ ____________

____________

Effect of Interviewing Method on Questionnaire Design Telephone Questionnaire

I will read to you the names of some department stores. Please rate them in terms of your preference to shop at these stores. Use a ten point scale, where 1 denotes not so preferred and 10 denotes greatly preferred. Numbers between 1 and 10 reflect intermediate degrees of preference. Again, please remember that the higher the number, the greater the degree of preference. Now, please tell me your preference to shop at .......(READ ONE STORE AT A TIME)
Store Greatly Preferred 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10

Not So Preferred 1. Lord & Taylor 1 2 2. Macy's 1 2 . . . 10. Wal-Mart 1 2

10

Question Content
Once the information needed is specified & type of interview method is decided next step is to determine individual question content Five major questions need to be answered while deciding question content --Utility of question ( Can the question be dropped ?) --Effectiveness of question (Double barreled question) --Respondents ability to answer accurately( Memory Lapse) --Respondents willingness to answer(Sensitive Question) --Effect of external events( rained last week)

Response Format
Unstructured/open ended: Questions that respondents answer in their own words Structured/ closed ended :Questions that prespecify the set of response alternatives & the response format:( multiple Choice, Dichotomous, Scales)

Open Ended vs Closed Ended

Advantages Open Ended Can discover uncommon but intelligent opinions Greater freedom of expression for respondent No bias due to limited response ranges Respondents can qualify their answers Disadvantages Coding is difficult & time consuming Questions require more thought & time on part of respondents Number of questions that can be asked gets reduced It becomes difficult to pool an opinion across the sample leading to possible misinterpretation of response

CloseEnded

Questions are more specific &easy to answer It becomes easier to code, record & analyse the results No difference between articulate & inarticulate respondents Higher response rates Less expensive & time consuming

The answers might not reveal true feelings of the respondents Poor questionnaire design & limited range of options might lead to misleading conclusions Requires pre testing & prior open ended research to ensure relevant choices

Response Format
Undisguised/Direct: Questions that are directly asked to get respondents answer Disguised/ Indirect :Questions that are indirectly asked to tackle sensitive issues or to probe onto respondents psyche or to get rid of biased responses(Projective techniques used mostly for exploratory research)

Choosing Question Structure Unstructured Questions

Unstructured questions are open-ended questions that respondents answer in their own words. What do you like about Surf Detergent?
Do you intend to buy a new car within the next six months? __________________________________

Choosing Question Structure Structured Questions

Structured questions specify the set of response alternatives and the response format. A structured question may be multiple-choice, dichotomous, or a scale.

Choosing Question Structure Multiple-Choice Questions

In multiple-choice questions, the researcher provides a choice of answers and respondents are asked to select one or more of the alternatives given. What do you like about Surf Detergent? ____ Its cleaning power ____ Its price ____ Its fragnance ____ It dissolves easily ____ Its stain removing ability ____ Any other( Pl.Specify)

Choosing Question Structure Multiple-Choice Questions

In multiple-choice questions, the researcher provides a choice of answers and respondents are asked to select one or more of the alternatives given. Do you intend to buy a new car within the next six months? ____ Definitely will not buy ____ Probably will not buy ____ Undecided ____ Probably will buy ____ Definitely will buy ____ Other (please specify)

Choosing Question Structure Dichotomous Questions

A dichotomous question has only two response alternatives: yes or no, agree or disagree, and so on. Often, the two alternatives of interest are supplemented by a neutral alternative, such as no opinion, don't know, both, or none.
Do you intend to buy a new car within the next six months? _____ Yes _____ No _____ Don't know

Choosing Question Structure Scales

Scales were discussed earlier: Do you intend to buy a new car within the next six months?
Definitely will not buy buy 1 Probably will not buy
2

Undecided

Probably will buy

4

Definitely will
5

Undisguised vs Disguised Questions

Completely disguised questions are sometimes needed to get true answers Are you a good parent? Direct OrUndisguised What is your opinion on deficiencies you have observed in how others bring up their children? Indirect or Disguised A qualified analyst can interpret the answers to gauge how good a parent the respondent is If Pepsi has sponsored a study respondents might be biased towards Pepsi. To get unbiased opinions researchers sometimes do not disclose the sponsors

A QUESTIONNAIRE IS ONLY AS GOOD AS THE QUESTIONS IT ASKS

A Good Questionnaire Appears

As easy to compose as a good poem But, it is usually the result of long, painstaking work

Question Wording
Question wording is translation of desired question content & structure into words that can be easily &clearly understood by respondents Guidelines for question wording define the issue-Define clearly the issue being addressed language of communication- Language respondents understand use non technical words-Matching vocabulary level avoid ambiguous words-Should have a single meaning avoid leading questions-Leads respondent to an answer avoid implicit alternatives-Alternative not explicitly given avoid generalizations & estimates-Qs Specific not general Avoid double barrelled Questions-Two or more ideas included in one question

Question Wording
Define the issue: ---Which brand of shampoo do you use? (incorrect) ---Which brand or brands of shampoo have you personally used at home during the last month? In case of more than one brand please list all the brands that apply .( Correct) ---Who = Respondent ---What =What brand ---When = Last week, last time, last month ---Where = At home

Question Wording
Please list all the departments from which you purchased merchandise on your most recent shopping trip to a department store. (Incorrect) In the list that follows, please check all the departments from which you purchased merchandise on your most recent shopping trip to a department store. 1. 2. 3. 4. . . . 16. 17. Women's dresses Men's apparel Children's apparel Cosmetics ____
____ ____ ____

Jewelry Other (please specify) ____

____ (Correct)

Question Wording(Recall)
How many litres of soft drinks did you consume during the last four weeks? (Incorrect) How often do you consume soft drinks in a typical week? (Correct) 1. ___ Less than once a week 2. ___ 1 to 3 times per week 3. ___ 4 to 6 times per week 4. ___ 7 or more times per week

Question Wording

Use ordinary non technical word: ---Do you think the distribution of soft drinks is adequate? (incorrect) ---Do you think soft drinks are easily available when you want them?( Correct) Avoid ambiguous words --In a typical month how often you shop in a department store?
INCORRECT Never------------CORRECT Never-----

Occasionally-----------Sometimes--------Often-------Regularly---------

Less than once-----1or 2 times-------3or 4 times--------more than four times------

Avoid Leading Questions: ---Do you think patriotic Indians should buy imported cars
that would put Indian labour out of work ? Yes---No---Dont know-----(incorrect) --- Do you think the Indians should buy imported cars Yes---No---Dont know-----(correct)

Question Wording

Question Wording
Avoid Leading Questions: ---Do you think that TV has a negative effect on children
?(Incorrect) --- (What are your views on effect of TV on children? (correct)

Avoid implicit alternatives: Do you like to fly when traveling short distances? (incorrect) Do you like to fly when traveling short distances or would you like to drive ? (correct) Avoid generalizations & Estimates What is the annual per capita expenditure on groceries in your household? (Incorrect) What is the monthly expenditure on groceries in your household? How many members are there in your household? (correct)

Question Wording

Avoid Double barreled questions

Sometimes, several questions are needed to obtain the required information in an unambiguous manner. Consider the question,
Do you think Coca-Cola is a tasty and refreshing soft drink? (Incorrect)

Such a question is called a double-barreled question, because two or more questions are combined into one. To obtain the required information, two distinct questions should be asked:
Do you think Coca-Cola is a tasty soft drink? and Do you think Coca-Cola is a refreshing soft drink? (Correct)

Question Wording
Avoid Double barreled Questions: --Do you like fuel efficient cars with comfortable seats? (Incorrect) --Do you like fuel efficient cars ? --Do you like cars with comfortable seats? ( Correct)

 Questions should be properly sequenced for better & effective responses The sequencing tends to drive the interview through a funnel shaped process starting with general questions & progressing to more specific ones The interviewer before moving to sequential steps should give a brief introduction about purpose of the survey

Order of Questions

Order of Questions
Sequencing involves following steps Lead-In Questions Qualifying Questions Warm-Up Questions Specific Questions Demographic Questions

 Lead-In Questions: ---Introductory, non threatening or ice breaking --Crucial in gaining participants confidence --About hot topics of the day with little importance to survey --Q:It is generally said that economic condition in India is by product of political situation.Do you agree? --Ans: Yes/No

Order of Questions

 Qualifying Questions: ---Questions that slowly lead to surveys objective --Questions that evaluate the respondent & qualify him/her for further questioning. -- Survey for estimating market potential for a new fluoride based toothpaste brand could ask qualifying question like Q: What type of toothpaste do you like ? Ans : Fluoride, Herbal , Calcium

Order of Questions

 Warm up questions: --- After a series of lead in & qualifying questions warm up questions are asked --Questions that play on respondents' mind by making them think of certain facts related to survey -- Survey for estimating market potential for a new fluoride based toothpaste brand could ask warmup question like Q:When was the last time you bought toothpaste? Was it fluoride or herbal ?

Order of Questions

 Specific questions: --- After a series of lead in , qualifying & warm up questions specific questions are asked --Questions that are specific to research objectives -- Are asked to participants who show favorable response or are end users of product (Fluoride Toothpaste) --Questions relating to usage pattern, influential factors in using fluoride toothpaste --These questions play a major role in data collection & analysis --After ensuring that enough rapport has been established this section can probe to get insights into sensitive issues

Order of Questions

 Demographic questions: ---These are necessary part of every survey --Set of questions related to age, sex, location, occupation used for classification in analysis --These are kept at the end to avoid respondent's resistance & to prevent his attention from being diverted --

Order of Questions

 Pre Testing: ---It refers to testing questionnaire on a small sample selected on convenience basis ---To identify & eliminate flaws in all aspects of questionnaire from question content to question sequence ---Pre testing is always done by personal interviews so that first hand information about likely problems in data collection can be obtained ---After pre testing the questionnaire is revised by removing flaws --After the revision the questionnaire is ready as a final draft used for actual survey

Pre Testing ,Revision & Final Draft

The Major Decisions in Questionnaire Design

1. What should be asked? 2. How should each question be phrased? 3. In what sequence should the questions be arranged? 4. What questionnaire layout will best serve the research objectives? 5. How should the questionnaire be pretested? Does the questionnaire need to be revised?

1. Do you believe that private citizens have the right to own firearms to defend themselves, their families, and property from violent criminal attack? Yes No Undecided

2. Do you believe that a ban on the private ownership of firearms would be significantly reduce the number of murders and robberies in your community? Yes No Undecided

1a. How many years have you been playing tennis on a regular basis? Number of years: __________ b. What is your level of play? Novice . . . . . . . . . . . . . . . Lower Intermediate . . . . . Upper Intermediate . . . . . -1 -2 -3 Advanced . . . . . . . Expert . . . . . . . . . Teaching Pro . . . . -4 -5 -6

c. In the last 12 months, has your level of play improved, remained the same or decreased? Improved. . . . . . . . . . . . . . Remained the same . . . . . -1 -2 Decreased. . . . . . . -3

2a. Do you belong to a club with tennis facilities?

Yes . . . . . . . No . . . . . . .

-1 -2

b. How many people in your household - including yourself - play tennis? Number who play tennis ___________ 3a. Why do you play tennis? (Please X all that apply.) To have fun . . . . . . . . . . To stay fit. . . . . . . . . . . . To be with friends. . . . . . To improve my game . . . To compete. . . . . . . . . . . To win. . . . . . . . . . . . . . . -1 -2 -3 -4 -5 -6

b. In the past 12 months, have you purchased any tennis instructional books or video tapes? Yes . . . . . . . -1 No . . . . . . . -2

Dear Passenger: Indian Airlines is pleased to have you on board today. To help us provide the best service possible, we need to know more about you and your opinions of our service. If you are over 18 years old, we would appreciate it if you would complete this questionnaire.

Your flight attendant will pick up your completed questionnaire shortly.

Thank you.

1. Please indicate: Flight number ___________ Date_____________ 2a. At the city where you boarded this particular plane, did you make a connection from another flight? Yes, from IA . . . . 1 Yes, from Other Airline . . 2 No . . . . . . . . . . . . . . . . . . 3 b. Did you board this plane at the airport from which it just took off, or were you a through passenger for which that was an intermediate stop? Boarded here . . . . . . . . . . Through passenger. . . . . . 1 2

3. How would you rate the overall service from IA for this flight, all things considered, from your arrival at the airport terminal until now? Excellent Good 1 2 Fair 3 Poor 4

Overall Service . . . . . . . . . .

4. Please rate each of the following with regard to this flight, if applicable. Excellent Good
1 Courtesy and Treatment from the: Skycap at airport . . . . . . . . . . . . . . Airport Ticket Counter Agent . . . . . Boarding Point (Gate) Agent . . . . . Flight Attendants . . . . . . . . . . . . . . Your Meal or Snack. . . . . . . . . . . . . Beverage Service . . . . . . . . . . . . . . Seat Comfort. . . . . . . . . . . . . . . . . . Carry-On Stowage Space. . . . . . . . Cabin Cleanliness . . . . . . . . . . . . . Video/Stereo Entertainment . . . . . . On-Time Departure . . . . . . . . . . . . 2 3

Fair
4

Poor

Questionnaire Design Process

Specify the Information Needed Specify the Type of Interviewing Method Determine the Content of Individual Questions Design the Question to Overcome the Respondents Inability and Unwillingness to Answer Decide the Question Structure Determine the Question Wording Arrange the Questions in Proper Order Identify the Form and Layout Reproduce the Questionnaire Eliminate Bugs by Pre-testing

Questionnaire Design Checklist

Step 1. Specify The Information Needed

Step 2.
Step 3. Step 4.

Type of Interviewing Method

Individual Question Content Overcome Inability and Unwillingness to Answer

Step 5.
Step 6. Step 7.

Choose Question Structure

Choose Question Wording Determine the Order of Questions

Step 8.
Step 9.

Form and Layout

Reproduce the Questionnaire

Step 10. Pretest

Questionnaire Design Checklist

. Step 1. Specify the Information Needed 1. Ensure that the information obtained fully addresses all the components of the problem. Review components of the problem and the approach, particularly the research questions, hypotheses, and specification of information needed. 2. Prepare a set of dummy tables. 3. Have a clear idea of the target population.

Step 2. Type of Interviewing Method 1. Review the type of interviewing method determined based on various considerations

Questionnaire Design Checklist

. Step 3. Individual Question Content

1. Is the question necessary? 2. Are several questions needed instead of one to obtain the required information in an unambiguous manner? 3. Do not use double-barreled questions.

Questionnaire Design Checklist

Step 4. Overcoming Inability and Unwillingness to Answer 1. Is the respondent informed? 2. If respondents are not likely to be informed, filter questions that measure familiarity, product use, and past experience should be asked before questions about the topics themselves. 3. Can the respondent remember? 4. Avoid errors of omission, telescoping, and creation. 5. Questions which do not provide the respondent with cues can underestimate the actual occurrence of an event. 6. Can the respondent articulate?

Questionnaire Design Checklist

Step 4. Overcoming Inability and Unwillingness to Answer 7. 8. 9. Minimize the effort required of the respondents. Is the context in which the questions are asked appropriate? Make the request for information seem legitimate.

10. If the information is sensitive: a. b. c. d. e. f. Place sensitive topics at the end of the questionnaire. Preface the question with a statement that the behavior of interest is common. Ask the question using the third-person technique. Hide the question in a group of other questions which respondents are willing to answer. Provide response categories rather than asking for specific figures. Use randomized techniques, if appropriate.

Questionnaire Design Checklist

Step 5. Choosing Question Structure 1. Open-ended questions are useful in exploratory research and as opening questions. 2. Use structured questions whenever possible. 3. In multiple-choice questions, the response alternatives should include the set of all possible choices and should be mutually exclusive. 4. In a dichotomous question, if a substantial proportion of the respondents can be expected to be neutral, include a neutral alternative. 5. Consider the use of the split ballot technique to reduce order bias in dichotomous and multiple-choice questions.

6. If the response alternatives are numerous, consider using more than one question to reduce the information processing demands on the respondents.

Questionnaire Design Checklist

.
Step 6. Choosing Question Wording 1. 2. 3. 4. 5. 6. 7. 8. Define the issue in terms of who, what, when, where, why, and way (the six Ws). Use ordinary words. Words should match the vocabulary level of the respondents. Avoid ambiguous words: usually, normally, frequently, often, regularly, occasionally, sometimes, etc. Avoid leading questions that clue the respondent to what the answer should be. Avoid implicit alternatives that are not explicitly expressed in the options. Avoid implicit assumptions. Respondent should not have to make generalizations or compute estimates. Use positive and negative statements.

Questionnaire Design Checklist

.
Step 7. Determine the Order of Questions 1. 2. The opening questions should be interesting, simple, and nonthreatening. Qualifying questions should serve as the opening questions.

3.
4. 5. 6. 7. 8.

Basic information should be obtained first, followed by classification, and, finally, identification information.
Difficult, sensitive, or complex questions should be placed late in the sequence. General questions should precede the specific questions. Questions should be asked in a logical order. Branching questions should be designed carefully to cover all possible contingencies. The question being branched should be placed as close as possible to the question causing the branching, and (2) the branching questions should be ordered so that the respondents cannot anticipate what additional information will be required.

Questionnaire Design Checklist

. Step 8. Form and Layout

1. Divide a questionnaire into several parts.

2. Questions in each part should be numbered. 3. The questionnaire should be pre-coded. 4. The questionnaires themselves should be numbered serially.

Questionnaire Design Checklist

. Step 9. Reproduction of the Questionnaire 1. The questionnaire should have a professional appearance. 2. Booklet format should be used for long questionnaires. 3. Each question should be reproduced on a single page (or double-page spread). 4. Vertical response columns should be used. 5. Grids are useful when there are a number of related questions which use the same set of response categories. 6. The tendency to crowd questions to make the questionnaire look shorter should be avoided. 7. Directions or instructions for individual questions should be placed as close to the questions as possible.

Questionnaire Design Checklist

Step 10. Pretesting

1.
2. 3.

Pretesting should be done always.

All aspects of the questionnaire should be tested, including question content, wording, sequence, form and layout, question difficulty, and instructions. The respondents in the pretest should be similar to those who will be included in the actual survey.

4.
5. 6. 7. 8. 9.

Begin the pretest by using personal interviews.

Pretest should also be conducted by mail or telephone if those methods are to be used in the actual survey. A variety of interviewers should be used for pretests. The pretest sample size is small, varying from 15 to 30 respondents for the initial testing. Use protocol analysis and debriefing to identify problems. After each significant revision of the questionnaire, another pretest should be conducted, using a different sample of respondents.

10. The responses obtained from the pretest should be coded and analyzed.