Professional Documents
Culture Documents
Data collection
In this workshop we talk about all of the steps in the research process except Data Analysis and Findings.
What is a problem?
. . . any situation where a gap exists between the actual and the desired state. A problem does not necessarily mean that something is seriously wrong. It could simply indicate an interest in improving an existing situation. Thus, problem definitions can include both existing problems in the current situation as well as the quest for idealistic states in the future.
2. Preliminary Data Collection use of interviews, both unstructured and structured, to get an idea or feel for what is happening in the situation.
3. Literature Survey a comprehensive review of the published and unpublished work from secondary sources of data in the areas related to the problem.
What are some business problems you are aware of or have confronted?
than one problem is identified, separate and prioritize them in terms of who and when they will be dealt with. Identify and separate out measurable symptoms to determine root problem versus easily observable symptoms. For example, a manager may identify declining sales or lost market share as the problem, but the real problem may be bad advertising, low salesperson morale, or ineffective distribution. Similarly, low productivity may be a symptom of employee morale or motivation problems, or supervisor issues. Determine the unit of analysis = individuals, households, businesses, objects (e.g., products, stores), geographic areas, etc., or some combination. Determine the relevant variables, including specifying independent and dependent relationships, constructs, etc.
Types of Variables:
construct that influences or explains the dependent variable either in a positive or negative way. construct the researcher hopes to understand, explain and/or predict.
Dependent variable = also known as a criterion variable, it is a variable or Moderator variable = a variable that has an effect on the independent
dependent variable relationship. The presence of a moderator variable modifies the original relationship between the independent and dependent variables by interacting with the independent variable to influence the strength of the relationship with the dependent variable. that surfaces as a function of the independent variable and explains the relationship between the dependent and independent variables. Moderator variables specify when certain effects will occur whereas mediators speak to how or why such effects occur. Moreover, mediators explain how external events take on internal psychological significance.
Measurement = is the process of determining the direction and intensity of feelings about persons, events, concepts, ideas, and/or objects of interest that are defined as being part of the business problem. As part of measurement, researchers use predetermined rules to assign numbers or labels to: (1) individuals attitudes, behaviors, characteristics, etc.; (2) objects features or attributes; and (3) any other phenomenon or event being investigated. Rules tell researchers how to assign numbers or labels; e.g., assign the numbers 1 to 7 to responses based on the intensity of an individuals feelings, beliefs, etc.
Measurement involves two processes: (1) identification/development of constructs; and (2) scale measurement. The first process involves identifying and defining what is to be measured, while the second process involves selecting the scale to measure the construct(s).
Construct = also referred to as a concept, it is a abstract idea formed in the mind based on a set of facts or observations. The idea is a combination of a number of similar characteristics of the construct. Examples of constructs include: brand awareness, brand familiarity, purchase intentions, satisfaction, importance, trust, service quality, role ambiguity, etc. Scale measurement = using a set of symbols or numbers to represent the range of possible responses to a research question.
Brand Attitudes
Purchase Intentions
Satisfaction
Operational Description Percentage of respondents that have heard of a designated brand; awareness could be either unaided or aided. The number of respondents and their intensity of feeling positive or negative toward a specific brand. The number of people planning to buy the specified object (e.g., product or service) within a designated time period. To what extent do specific factors influence a person's purchase choice. The attitudes, opinions, interests and lifestyle characteristics of individuals providing the information. How people evaluate their post-purchase consumption experience with a particular product, service or company.
What is theory ??
Theory = a systematic set of relationships providing a consistent and comprehensive explanation of a phenomenon. In practice, a theory is a researchers attempt to specify the entire set of dependence relationships explaining a particular set of outcomes.
Theory is based on prior empirical research, past experiences and observations of behavior, attitudes, or other phenomena, and other theories that provide a perspective for developing possible relationships.
Theory is used to prepare a theoretical framework for the research.
Hypotheses = preconceptions the researcher develops regarding the relationships represented in the data, typically based on theory, practice or previous research. Examples:
The average number of cups of coffee students drink during finals will be greater than the average they consume at other times. Younger, part-time employees of Samouels restaurant are more likely to search for a new job.
Theoretical Framework = a written description that includes a conceptual model. It integrates all the information about the problem in a logical manner, describes the relationships among the variables, explains the theory underlying these relationships, and indicates the nature and direction of the relationships. The process of developing a theoretical framework involves conceptualization which is a visual specification (conceptual model) of the theoretical basis of the relationships you would like to examine.
Price
Independent Variable
Moderator Variable
Price
Group Exercise: Use the Samouels and Ginos restaurant database variables to develop a theoretical framework/conceptual model of the relationships that could be examined. Consider and evaluate several models, but be prepared to report your most interesting or thought provoking model.
Work Groups
Employee Commitment
Compensation Supervision
Work Groups
Intention to Search
Compensation
Potential Hypotheses: Commitment is positively related to supervision, work groups and compensation. Intention to Search is negatively related to supervision, work groups & compensation.
Variable Type
Metric Metric Metric Metric Metric Metric Metric Metric Metric Metric Metric Metric Nonmetric Nonmetric Nonmetric Nonmetric Metric Metric Metric Nonmetric Nonmetric Nonmetric Nonmetric Nonmetric Nonmetric Nonmetric Metric Nonmetric
Relationship Variables
Classification Variables
(2)
(3)
RESEARCH DESIGN
Role of Qualitative Research:
Search of academic, trade and professional
literature (both traditional & Internet). Use of interviews, brainstorming, focus groups. Internalization of how others have undertaken both qualitative and quantitative research. Use of existing questionnaires/constructs.
RESEARCH DESIGN
sample to population. Facilitates examination of large number of representative cases. Structured approach to data collection. Enables extensive statistical analysis.
DATA COLLECTION
Approaches:
Observation
Self-Completion Mail/Overnight Delivery/Fax Electronic Interviewer-Administered Face-to-Face Home, Work, Mall, Focus Groups Telephone
DATA COLLECTION
Budget Knowledge of issues qualitative vs. quantitative Respondent Participation Taste Test; Ad Test Card Sorts; Visual Scaling Time Available
DATA COLLECTION
Types of Data:
Primary Secondary
PRIMARY DATA
Informal discussions; brainstorming Focus groups Observational Methods Structured & Unstructured Surveys Experiments
Focus Groups = bring a small group of people (10-12) together for an interactive, spontaneous discussion of a particular topic or concept. Discussion is led by a trained moderator and usually lasts 1 hours. Typical Objectives:
To identify and define problems. To generate new ideas about products, services, delivery methods, etc. To test advertising themes, positioning statements, company and product names, etc. To discover new constructs and measurement methods. To understand customer needs, wants, attitudes, behaviors, preferences and motives.
Primary Data
2004 Wireless Retail Sales Satisfaction Study Sales Staff 44% Price/Promotion 28% Store Display 14% Store Facility 14%
Source: J.D. Power and Associates, 2002, 2003 & 2004.
Primary Data
Primary Data
22%
19%
14%
5%
CONSIDERATIONS:
Purpose of Questionnaires:
To obtain information that cannot be easily
QUESTIONNAIRE DESIGN
Steps in Questionnaire Design:
1. Initial Considerations problem, objectives, target population, sampling, etc. 2. Clarification of Concepts select variables, constructs, measurement approach, etc. 3. Developing the Questionnaire
Length and sequence. Types of questions. Sources of questions.
QUESTIONNAIRE DESIGN
Open-ended Questions = place no constraints on respondents; i.e., they are free to answer in their own words and to give whatever thoughts come to mind.
Closed-ended Questions = respondent is given the option of choosing from a number of predetermined answers.
QUESTIONNAIRE DESIGN
Examples of Open-ended Questions: How do you typically decide which restaurant you will
eat at?
QUESTIONNAIRE DESIGN
Open-ended Questions
Typically used in exploratory/qualitative studies. Typically used in personal interview surveys involving
small samples. Allows respondent freedom of response. Respondent must be articulate and willing to spend time giving a full answer. Data is in narrative form which can be time consuming and difficult to code and analyze. Possible researcher bias in interpretation. Narrative is analyzed using of content analysis. Software is available (e.g., NUD*IST).
QSR NUD*IST stands for Non-Numerical Unstructured Data Indexing and Theorizing. It is a popular computer software package used by researchers to analyze text from focus group or interview transcripts, literary documents and so on. It examines non-textual data such as photographs, tape recordings, films and so on. Users can us it to index and link several documents in a structured way to produce categorical data in a form amenable to further analysis. NUD*IST output can be exported to software programs such as SPSS and Excel. For more information about QSR NUD*IST and its related product NVIVO visit their website (http://www.scolari.co.uk/qsr/qsr_n4.htm).
QUESTIONNAIRE DESIGN
Closed-end Questions:
Single Answer Multiple Answer Rank Order Numeric Likert-Type Scales Semantic Differential
4.
5.
QUESTIONNAIRE DESIGN
Closed-end Questions
response categories. Data can be pre-coded and therefore in a form amenable for use with statistical packages (e.g., SPSS, SAS) data capture therefore easier. More difficult to design but simplifies analysis. Used in studies involving large samples. Limited range of response options.
QUESTIONNAIRE DESIGN
Broad Considerations
Sequencing of questions. Identification of concepts. How many questions are required to capture
each concept. Question wording. Overall length of questionnaire. Placing of sensitive questions. Ability of respondents. Level of measurement. Open-ended versus closed-end questions.
QUESTIONNAIRE DESIGN
Questionnaire Sequence
Opening Questions Research Topic Questions Classification Questions
Tonight we are talking with individuals who are 18 years of age or older and have 50 percent or more of the responsibility for banking decisions in your household. Are you that person? __ Yes __ No
Rapport Questions:
. . . are used to establish rapport with the respondent by gaining their attention and stimulating their interest in the topic.
Reliable delivery Technical sales Support Inside sales representatives Field sales representatives Complaint resolution Ordering/Invoicing Website design
QUESTIONNAIRE DESIGN
Concepts
Concept Identification Conceptual definition e.g., Service Quality. As perceived by customers, it is the difference between customers expectations or desires of a vendor and their perceptions of the actual situation (their experiences).
Working Definition for Concept Decompose definition into components. Search for items that are measurable.
QUESTIONNAIRE DESIGN
communicate and listen to consumers; sincerely empathize with customers in interpreting their needs
and wants; problems;
be tactful in responding to customers questions, objections, and create an impression of reliability in performing services; create an image of credibility by keeping promises; demonstrate sufficient technical knowledge and competence;
QUESTIONNAIRE DESIGN
QUESTIONNAIRE DESIGN
Use Simple Words. Be brief. Avoid Ambiguity. Avoid Leading Questions. Avoid Double-Barreled Questions. Check Questionnaire Layout. Prepare Clear Instructions. Watch Question Sequence.
QUESTIONNAIRE DESIGN
Recently a survey was conducted by the United Nations using a sample from several different countries. The question asked was: " Would you please give your opinion about the food shortage in the rest of the world?" The survey was a huge failure. Why?
In Africa they did not know what 'food' meant. In Western Europe, they did not know what 'shortage' meant. In Eastern Europe they did not know what 'opinion' meant. In South America they did not know what 'please' meant. And in the U.S., they did not know what 'the rest of the
world' meant.
QUESTIONNAIRE DESIGN
Avoid Position Bias:
Position Bias: How important are flexible hours in evaluating job alternatives? What factors are important in evaluating job alternatives? No Position Bias: What factors are important in evaluating job alternatives? How important are flexible hours in evaluating job alternatives?
QUESTIONNAIRE DESIGN
Double-Barreled Questions:
To what extent do you agree or disagree with the following statements?
Harrods employees are friendly and helpful. Harrods employees are courteous and knowledgeable.
QUESTIONNAIRE DESIGN
Branching Questions:
. . . are used to direct respondents to answer the right questions as well as questions in the proper sequence.
telephone service in the past 30 days? If No, go to question #10. If Yes , were the advertisements on radio or TV or both? If the advertisements were on TV or on both radio and TV, then go to question #6? If the advertisements were on radio, then go to question #8.
QUESTIONNAIRE DESIGN
Issues Self-Completion Instructions:
QUESTIONNAIRE DESIGN
How to increase respondent participation. How to screen out respondents that are not wanted and
still keep them happy. What to say when respondents ask how to answer a particular question. When concepts may not be easily understood, how to define them. When answer alternatives are to be read to respondents (aided response) or not to be read (unaided response). How to follow branching or skip patterns. When and how to probe. How to end the interview.
QUESTIONNAIRE DESIGN
Identify response bias for below questions:
1. Do you advocate a lower speed limit to save human lives?
2.
3. 4.
When you visited the museum, how many times did you read the plaques that explain what the exhibit contained?
About what time do you ordinarily eat dinner? How important is it for stores to carry a large variety of different brands of this product?
5.
6. 7.
Would you favor increasing taxes to cope with the current fiscal crisis?
Dont you see some danger in the new policy? What small appliance, such as countertop appliances, have you purchased in the past month?
8.
9.
When you buy fast food, what percentage of the time do you order each of the following types of food?
Do you like orange juice?
QUESTIONNAIRE DESIGN
Comments on Questions:
1.
2. 3. 4. 5. 6. 7.
8. 9.
A loaded question because everyone wants to save lives. Also, it presumes that lower speed limits saves lives. Too specific because respondents likely cannot remember the exact number of times. Ambiguous because dont know if dinner is lunch or evening. Not specific enough about types of stores. Overemphasis because refers to crisis. Leading question because uses danger in sentence. Answers likely to relate only to countertop appliances and not all small appliances. Over generalization because does not specify time period. Ambiguous because may like orange juice for themselves, or for their kids, but really do not know.
QUESTIONNAIRE DESIGN
Pre-testing Questionnaires:
Objective: to identify possible shortcomings of questionnaire. Approaches informal or formal. Can assess:
clarity of instructions cover letter clarity of questions adequacy of codes and categories for pre-coded questions quality of responses likely response rate
ability to perform meaningful analyses time to complete the questionnaire cost of data collection which questions are relevant whether key questions have been overlooked sources of bias
Scale Development
Marketing Bearden, William O. and Richard Netemeyer, Handbook of Marketing Scales, Sage Publications, 2nd ed., 1998. Summarizes over 130 marketing related scales.
Bruner, Gordon Paul Hensel, Marketing Scales Handbook, Chicago, Ill., American Marketing Association, 1992. Includes almost 600 scales. General Robinson, John P., Phillip R. Shaver and Lawrence S. Wrightsman, Measures of Personal and Social Psychological Attitudes, San Diego, CA: Academic Press, 1991. Contains over 150 published scales in 11 different areas. Buros Institute of Mental Measurements website has reviews of published tests and measurements. www.unl.edu/buros
Decision Analyst
www.decisionanalyst.com
Decisive Technology
Survey Builder
www.surveybuilder.com
www.decisive.com
SurveyPro
www.surveypro.com
Perseus Development
www.perseusdevelopment.com
Socratic Technologies
SurveySez
www.surveysez.com
WebSurveyor
www.websurveyor.com
www.sotech.com
SPSS
www.spss.com
MEASUREMENT SCALES
Types of Scales:
Metric (interval & ratio) Likert-type Summated-Ratings (Likert) Numerical Semantic Differential Graphic-Ratings Nonmetric (nominal & ordinal) Categorical Constant Sum Method Paired Comparisons Rank Order Sorting
When I hear about a new restaurant , I eat there to see what it is like.
Strongly Agree 1
Strongly Disagree 5
Example:
My sales representative is . . . . SD D Courteous ___ ___ Friendly ___ ___ Helpful ___ ___ Knowledgeable ___ ___ N ___ ___ ___ ___ A ___ ___ ___ ___ SA ___ ___ ___ ___
I always eat at new restaurants when someone tells me they are good.
Strongly Agree 1 Agree Somewhat 2 Neither Agree or Disagree 3 Disagree Somewhat 4 Strongly Disagree 5
This approach includes a separate labeled Likert scale with each item (statement). The summated rating is a total of the responses for all the items divided by the number of items.
Very Satisfied Somewhat Satisfied Neither Satisfied nor Dissatisfied Somewhat Dissatisfied Very Dissatisfied
Sorting:
A scaling technique in which respondents are asked to indicate their beliefs or opinions by arranging objects (items) on the basis of perceived importance, similarity, preference or some other attribute.
Scale Development
Number of items (indicators) to measure a concept? Number of scale categories? Odd or even number of categories?
(Include neutral point ?) Balanced or unbalanced scales? Forced or non-forced choice? (Include Dont Know ?) Category labels for scales? Scale reliability and validity?
Scale Development
Balanced vs. Unbalanced Scales?
Balanced: To what extent do you consider TV shows with sex and violence to be acceptable for teenagers to view? __ Very Acceptable __ Somewhat Acceptable __ Neither Acceptable or Unacceptable __ Somewhat Unacceptable __ Very Unacceptable Unbalanced: __ Very Acceptable __ Somewhat Acceptable __ Unacceptable
Scale Development
Forced or Non-Forced?
How likely are you to purchase a laptop PC in the next six months? Very Very Unlikely Likely 1 2 3 4 5 6 __ No Opinion
Scale Development
Category Labels for Scales?
Verbal Label: How important is the size of the hard drive in selecting a laptop PC to purchase? Very Somewhat Neither Important Somewhat Very Unimportant Unimportant or Unimportant Important Important 1 2 3 4 5 Numerical Label: How likely are you to purchase a laptop PC in the next six months? Very Very Unlikely Likely 1 2 3 4 5 Unlabeled: How important is the weight of the laptop PC in deciding which brand to purchase? Very Very Unimportant Important
___
___
___
___
___
MEASUREMENT SCALES
Capabilities of Respondents. Context of Scale Application. Data Analysis Approach. Validity and Reliability.
MEASUREMENT SCALES
Validity
Reliability
Measurement Error = occurs when the values obtained in a survey (observed values) are not the same as the true values (population values).
RESEARCH DESIGN
Types of Errors:
Nonresponse = problem definition, refusal, sampling, etc. Response = respondent or interviewer. Data Collection Instrument: Construct Development. Scaling Measurement. Questionnaire Design/Sequence, etc. Data Analysis. Interpretation.
SECONDARY DATA
SECONDARY DATA
Availability Relevance
Accuracy
Sufficiency
Data Analysis
Methods:
Dependence
Interdependence
Learning Checkpoint:
Define a research problem to be studied. Identify the topics /concepts that will be covered
to answer research questions. Identify the types of questions and/or scaling you will use. How will you evaluate the questions/scales you use? Determine the best way to collect the data. Present group suggestions; defend.