Designing the Questionnaire

Asmika , 05 November 2010

What is a Questionnaire?
A questionnaire is the vehicle used to pose the questions that the researcher wants respondents to answer.

Ch 11

What makes a well designed questionnaire?

Good appearance
easy on the eye

Short and simple

Relevant and logical High response rate Easy data summarisation and analysis

Questionnaire Design
Questionnaire design is a systematic process in which the researcher contemplates various question formats, considers a number of factors characterizing the survey at hand, ultimately words the various questions very carefully, and organizes the questionnaires layout.
Ch 11 4

The Functions of A Questionnaire

Translates Research Objectives into Specific Questions

Standardizes Questions and Response Categories Fosters Cooperation and Keeps Respondents Motivated Serves as Permanent Records of the Research Speeds up Process of Data Analysis Contains Information for Reliability Assessments (e.g. TestRetest, Equivalent Forms)

Content of Questions
Clear focus on research question
Avoid sidetracking Avoid unnecessary information

Demographic information Contact information (if non-anonymised)

Key criteria
Questionnaire relevancy
No unnecessary information is collected and only information needed to solve the problem is obtained. Be specific about your data needs; tie each question to an objective

Questionnaire accuracy
Information is both reliable and valid

Phrasing Questions
Open ended response versus fixed alternative questions

?
Decision criteria: type of research; time; method of delivery; budget; concerns regarding researcher bias

Avoid
Leading questions Overly complex questions Use of jargon Loaded questions (can use a counterbiasing statement) Ambiguity Double barreled questions Making assumptions

The Questionnaire Development Process

Tabulation & Final Report

Decide Data Collection Methods

Question Development

Gather Data Determine Survey Objectives and Constraints Finalize & Duplicate

Question Evaluation

Revised as Needed Pretest

Gain Approval from Client

DEVELOPING QUESTIONS
The Five Shoulds of Question Wording Each Question Should...
Be Focused on a Single Issue or Topic Be Brief Be Interpreted the Same Way by All Respondents Use the Respondents Core Vocabulary

Be a Grammatically Simple Sentence if Possible

DEVELOPING QUESTIONS
The Eleven Should Nots of Question Wording The Question Should Not
1. Assume Criteria That Are Not Obvious 2. Be Beyond the Respondents Ability or Experience 3. Use a Specific Example to Represent a General Case 4. Ask the Respondent to Recall Specifics when Only Generalities Will Be Remembered 5. Require the Respondent to Guess a Generalization

12

DEVELOPING QUESTIONS
The Eleven Should Nots of Question Wording The Question Should Not
6. Ask for Details That Cannot Be Related 7. Use Words That Overstate the Condition 8. Have Ambiguous Wording 9. Be Double-Barreled 10. Lead the Respondent to a Particular Answer 11. Have Loaded Wording or Phrasing

13

GUIDELINES FOR COVER LETTERS AND QUESTIONNAIRE INTRODUCTIONS

1. Identify the Surveyor/Sponsor of the Survey 2. Describe the Surveys Purpose 3. Indicate How Prospective Respondents were

Selected
4. Request the Prospective Respondents Participation in the Survey/ Provide incentive 5. Screen the Prospective Respondent
14

Questionnaire Organization
Questionnaire organization is the sequence of statements and questions that make up the questionnaire. It is important because the questionnaire appearance and ease of flow affect the quality of the information gathered.
Ch 11 15

Questionnaire Validation
Use or adapt existing questionnaires
Validated (and possibly harmonised)

New questionnaires
Not validated Needs to be tested (pilot)

Open or Closed?
Closed Advantages:
Simple and quick Reduces discrimination against less literate Easy to code, record, analyse Easy to compare Easy to report results

Open or Closed?
Closed Advantages:
Simple and quick Reduces discrimination against less literate Easy to code, record, analyse Easy to compare Easy to report results

Closed questions
Disadvantages:
Restricted number of possible answers Loss of information

Possible compromise:
Insert field others

Types of fixed alternative questions

Single dichotomy or dichotomous-alternative
questions Are you currently registered in a course at the University of Brawijaya ? Yes____ No____

Respondent chooses one of two alternatives

(yes/no; male/female) What scale would this data create?

Types of fixed alternative questions

Multi-choice alternative
Respondent chooses from several alternatives Many types

Check List
Provide multiple answers to a single question Should be mutually exclusive and exhaustive

Open questions
Advantages:
Not directive Allows exploration of issues to generate hypothesis
qualitative research, focus groups, trawling questionnaires

Used even if no comprehensive range of alternative choices Good for exploring knowledge and attitudes Detailed and unexpected answers possible

Open questions
Disadvantages:
Interviewer bias Time-consuming Coding problems Difficult to analyse! Difficult to compare groups

Summary
A well designed questionnaire: Will give appropriate data which allow to answer your research question Will minimise potential sources of bias, thus increasing the validity of the questionnaire Will much more likely be completed

THANK YOU