Research Design

Dr. A. K. Dey

Research Design: Dr. Dey

Learning Outcomes
Types of research designs Distinctions between Exploratory, Descriptive & Experimental designs Principles of good research design Types of experimental designs Characteristics of sound measurement: Validity, Reliability & Practicality Test marketing
Research Design: Dr. Dey 2

Research Design

The research design is the master plan specifying the methods and procedures for collecting and analyzing the needed information.

Research Design: Dr. Dey

Types of Research Design

Although every problem and research objective may seem unique, there are usually enough similarities among problems and objectives to allow decisions to be made in advance about the best plan to resolve the problem. There are some basic marketing research designs that can be successfully matched to given problems and research objectives.

Research Design: Dr. Dey

Types of Research Design

Three traditional categories of research design: Exploratory Descriptive Causal/Experimental The choice of the most appropriate design depends largely on the objectives of the research and how much is known about the problem and these objectives.

Research Design: Dr. Dey

Research Design: Some Observations

The overall research design for a project may include one or more of these three designs as part(s) of it. Further, if more than one design is to be used, typically we progress from Exploratory toward Causal.

Research Design: Dr. Dey

Uncertainty Influences The Type Of Research

CAUSAL OR DESCRIPTIVE

COMPLETELY CERTAIN

ABSOLUTE AMBIGUITY

EXPLORATORY

Research Design: Dr. Dey

Degree of Problem Definition

Exploratory Research (Unaware of Problem)

Descriptive Research (Aware of Problem)

Causal Research (Problem Clearly Defined) Will buyers purchase more of our products in a new package? Which of two advertising campaigns is more effective?

Our sales are declining and What kind of people are buying we dont know why. our product? Who buys our competitors product? Would people be interested in our new product idea? What features do buyers prefer in our product?

Research Design: Dr. Dey

Basic Research Objectives & Research Design

Research Objective Appropriate Design

To gain background information, to define terms, to clarify Exploratory problems and develop hypotheses, to establish research priorities, to develop questions to be answered To describe and measure marketing phenomena at a point Descriptive in time To determine causality, test hypotheses, to make if-then Causal/Experimental statements, to answer questions

Research Design: Dr. Dey

Research Design: Exploratory Research

Exploratory research is most commonly unstructured, informal research that is undertaken to gain background information about the general nature of the research problem. Exploratory research is usually conducted when the researcher does not know much about the problem and needs additional information or desires new or more recent information.

Research Design: Dr. Dey

10

Research Design: Exploratory Research

Exploratory research is used in a number of situations: To gain background information To define terms To clarify problems and hypotheses To establish research priorities

Research Design: Dr. Dey

11

Research Design: Exploratory Research

A variety of methods are available to conduct exploratory research: Secondary Data Analysis Experience Surveys Case Analysis Focus Groups Projective Techniques

Research Design: Dr. Dey

12

Research Design: Descriptive Research

Descriptive research is undertaken to provide answers to questions of who, what, where, when, and how but not why. Two basic classifications: Cross-sectional studies Longitudinal studies

Research Design: Dr. Dey

13

Descriptive Research: Cross Sectional Studies

Cross-sectional studies measure units from a
sample of the population at only one point in time. Sample surveys are cross-sectional studies whose samples are drawn in such a way as to be representative of a specific population. On-line survey research is being used to collect data for cross-sectional surveys at a faster rate of speed.

Research Design: Dr. Dey

14

Descriptive Research: Longitudinal Studies

Longitudinal studies repeatedly draw sample
units of a population over time. One method is to draw different units from the same sampling frame. A second method is to use a panel where the same people are asked to respond periodically. On-line survey research firms recruit panel members to respond to online queries.

Research Design: Dr. Dey

15

Descriptive Research: Longitudinal Studies

Two types of panels:

Continuous panels ask panel members the

same questions on each panel measurement.

Discontinuous (Omnibus) panels vary

questions from one time to the next. Longitudinal data used for: Market tracking Brand-switching Attitude and image checks
Research Design: Dr. Dey 16

Research Design: Causal/Experimental Research

Causality may be thought of as understanding a phenomenon in terms of conditional statements of the form If x, then y. Causal relationships are typically determined by the use of experiments, but other methods are also used.

Research Design: Dr. Dey

17

Experiments
An experiment is defined as a research process that allows study of one or more variables which can be manipulated under conditions that permits collection of data that show the effect of such variables in unconfused fashion Independent variables: those over which the researcher has control and wishes to manipulate i.e. package size, ad copy, price. Dependent variables: those over which the researcher has little to no direct control, but has a strong interest in testing i.e. sales, profit, market share. Extraneous variables: those that may effect a dependent variable but are not independent variables.

Research Design: Dr. Dey

18

Experimental Design
An experimental design is a procedure for devising an experimental setting such that a change in the dependent variable may be solely attributed to a change in an independent variable. Symbols of an experimental design: O = measurement of a dependent variable X = manipulation, or change, of an independent variable R = random assignment of subjects to experimental and control groups E = experimental effect
Research Design: Dr. Dey 19

Basic Principles of Research Design

Principle of Replication
Experiments should be repeated more than once Increases statistical accuracy & precision

Principle of Randomization
Protects against effects of extraneous factors Variations can be termed as Chance

Principle of Local Control

Effects of extraneous factors can be minimized by using best methods & measuring instruments

Research Design: Dr. Dey

20

Important Experimental Designs

Informal
Uses less sophisticated form of analysis Allows study of one variable at a time

Formal
Uses precise statistical methods for analysis Allows study of more than one variables at a time Interaction between variables can be studied

Research Design: Dr. Dey

21

Informal Experimental Designs

After only Before - After Before - After with Control Group Four Group Six Study After Only with Control Group Ex Post Facto

Research Design: Dr. Dey

22

Formal Experimental Designs

Completely Randomized Randomized Block Latin Square Factorial

Research Design: Dr. Dey

23

Informal Experimental Designs

After Only
Dependent variable is measured only after independent variable is introduced Purchase of Pepsi through coupon redemption in advertisement Not a good design because Before the event response is not measured Sometimes used for new products when Before measurements is known to be zero

Research Design: Dr. Dey

24

Informal Experimental Designs

Before After Design
Experimental Group

Before measurement Exptal variable Introduced After measurement

X1 Yes
X2

Effect of Experimental variable =

( X 2 X1 ) Some of the aspects that may cause distortion are

History Effect Maturation Pre Test Effect Variety introduced by the researcher

Research Design: Dr. Dey

25

Informal Experimental Designs

Before After with Control Group Design
2 1

Before measurement Exptal variable Introduced After measurement

( X Control X ) (Y Y ) Experimental Group Group Y1 X

2 1

Yes X2

No Y2

Net Effect of Experimental variable = Some problems are Mortality Effect Interaction Effect
Research Design: Dr. Dey 26

Informal Experimental Designs

Four Group Six Study Design
Experimental Gp1 Before Measurement Experimental Variable X1 Yes Gp2
1 2

Control Gp1 Y1 No Y2 Gp2

1 2

( X 1 Y1 )

( X 1 Y1 )

Yes X3

No Y3

After Measurement X2

For Gp2 =

1 2

( X 1 Y1 )
27

Research Design: Dr. Dey

Informal Experimental Designs

Experimental Group1:
Experimental Variable, Before Measurement, Interaction of Before Measurement with Experimental Variable & Uncontrollable variables

Experimental Group2:
Experimental Variable & Uncontrollable variables

Control Group1:
Before Measurement & Uncontrollable variables

Control Group2:
Uncontrollable variables

Research Design: Dr. Dey

28

Informal Experimental Designs

After only with control group Exptal Control Group Group Before Measurement No No Exptal Variable Yes No After Measurement Yes ( X ) Yes ( Y1 )
1

Effect of Experimental Variable =

Research Design: Dr. Dey

Y1 X 1
29

Experimental Designs - CRD

Completely Randomized Design (Describe the experiment of regular & special trainings) Always use simple random for any selection Involves two principles: Replication & Randomization For analysis One way ANOVA is used Shortcoming: Can not control the effect of extraneous variable: Quality of training by different trainers

Research Design: Dr. Dey

30

Experimental Designs - RBD

Randomized Block Design
Improvement over Completely Randomized Design: Principle of local control is also applied ANOVA is used for analysis Describe the experiment: To measure the effect of four different tests on students having different IQ level
Divide students according to IQ level One student from each group is selected randomly Tests can be taken in random order: eliminates effect of fatigue or experience gained by taking repeat exams
Research Design: Dr. Dey 31

Experimental Designs - Latin Square

Latin Square Design
To evaluate the effect of three alternative shelf arrangements (A, B & C) in each of the three stores (1, 2 & 3) in each of the three time periods (First, Middle & Last week) Time Period 1 First Week Middle Last Week A C B STORES 2 C B A 3 B A C
32

Research Design: Dr. Dey

Experimental Designs Latin Square

Latin Square: A pattern in which each letter appears only once in either a row or a column Always a square: Current example is 3x3 Comparison based on such design will be free from following two extraneous factors
Difference between Rows (Time periods) Difference between Columns (Stores)

For analysis- two way ANOVA Helps in studying the effect of three variables simultaneously

Research Design: Dr. Dey

33

Experimental Designs - Factorial

Effect of two or more variables can be studied simultaneously Interaction effect can be studied Describe the experiment of Flavours & Sugar Intensity for finding the choice of consumers for a new soft drink Helps in studying effect of two variables simultaneously Two way ANOVA is used for analysis Latin Square Design To study effect of three variables
Research Design: Dr. Dey 34

Experimental Designs Different Representation

After-Only Design: X O1 One-Group, Before-After Design: O1 X O2 Before-After with Control Group: Experimental group: O1 X O2 Control group: O3 O4 Where E = (O2 O1) (O4 O3)

Research Design: Dr. Dey

35

Experimental Designs & Scientific Methods

Scientific Method: Validity & Reliability Validity: Ability to measure the effect of the claimed variable (e.g., TV viewer ship) Reliability: Reproducibility of the results if the experiment is repeated again under similar situation Experiments are more scientific method than Descriptive & Exploratory methods of data collection
Research Design: Dr. Dey 36

How Valid Are Experiments?

An experiment is valid if: the observed change in the dependent variable is, in fact, due to the independent variable (internal validity) if the results of the experiment apply to the real world outside the experimental setting (external validity)

Research Design: Dr. Dey

37

Characteristics of sound measurement

Characteristics of a good measurement tool
Accurate counter or indicator of what is being measured Easy and efficient to use

Three major criteria for evaluating a measurement tool

Validity Reliability Practicality
Research Design: Dr. Dey 38

Validity
External & Internal validity Internal validity: One aspect
Ability of a research instrument to measure what it is purported to measure Does the instrument really measure what its designer claims it does?

Classification of validity
Content validity Criterion related validity Construct validity
Research Design: Dr. Dey 39

Validity
Content Validity Degree to which the content of the items adequately represents the universe of all relevant items under study To measure Corporate Image what knowledge, attitude and opinions are relevant to be included for measurement It is judgmental or panel evaluation
Research Design: Dr. Dey 40

Validity
Criterion-Related Validity
Reflects the success of measures used for prediction or estimation Predictive Concurrent Opinion questionnaire that can correctly forecast the outcome of a union election has predictive validity An observational method that can correctly classify families into income classes has concurrent validity May sound very easy but for some variables it may prove to be difficult to secure correct figure for example Income of a family
Research Design: Dr. Dey 41

Validity
Construct Validity To measure attitude, aptitude, personality tests and infer is very difficult No empirical validation seems possible This is typical construct validity To evaluate construct validity both theory and measuring instrument need to be considered To determine the effect of ceremony on organizational culture
Research Design: Dr. Dey 42

Reliability
Example: Weighing Scale
It is Valid & Reliable if it measures your weight correctly If it consistently overweighs you by 3 Kgs, it is Reliable but not valid If it weighs erratically, it is neither valid nor reliable

Reliable instruments are Robust, free from random errors, and work well under different times and different conditions
Research Design: Dr. Dey 43

Reliability
Three perspectives of reliability Stability, Equivalence & Internal consistency Stability Producing consistent results with repeated measurements under similar environment Is concerned with personal and situational variations from one time to another Test-retest
Research Design: Dr. Dey 44

Reliability
Equivalence
How much errors can be introduced by different investigators or different samples Is concerned with variations at one point in time among observers or samples Parallel forms

Internal Consistency
Tests the homogeneity among the items Split-half Kuder Richardson Formula 20 Cronbachs alpha

Research Design: Dr. Dey

45

Practicality
Measurement process has to be
Scientifically: Valid & Reliable Operationally: Practical

Practicality has been defined as

Economy Convenience Interpretability
Research Design: Dr. Dey 46

Types of Experiments
Two broad classes: Laboratory experiments: those in which the independent variable is manipulated and measures of the dependent variable are taken in a contrived, artificial setting for the purpose of controlling the many possible extraneous variables that may affect the dependent variable Field experiments: those in which the independent variables are manipulated and measurements of the dependent variable are made on test units in their natural setting
Research Design: Dr. Dey 47

Problems with Experimental Designs

Lack of theoretical base: Unless enough data, Hypothesis formulation may not be possible Time factor Cost of conducting experiments is high Administrative problems of coordination & execution
Research Design: Dr. Dey 48

Experiments: Test Marketing

Test marketing is the phrase commonly used to indicate an experiment, study, or test that is conducted in a field setting. Two broad classes: To test the sales potential for a new product or service To test variations in the marketing mix for a product or service

Research Design: Dr. Dey

49

Types of Test Markets

Standard test market: one in which the firm tests the product and/or marketing mix variables through the companies normal distribution channels Controlled test markets: ones that are conducted by outside research firms that guarantee distribution of the product through pre-specified types and numbers of distributors

Research Design: Dr. Dey

50

Types of Test Marketscont.

Electronic test markets: those in which a panel of consumers has agreed to carry identification cards that each consumer presents when buying goods and services Simulated test markets: those in which a limited amount of data on consumer response to a new product is fed into a model containing certain assumptions regarding planned marketing programs, which generates likely sales volume

Research Design: Dr. Dey

51

Test Markets

Test marketing is used in both consumer markets and industrial or B2B markets as well. Lead country test market: test marketing conducted in specific foreign countries that seem good predictors for an entire continent

Research Design: Dr. Dey

52

Criteria for Selecting Test Market Cities

Representativeness: Do demographics match the total market? Degree of isolation: Cities are isolated markets; Or not isolated. Ability to control distribution and promotion: Are there pre-existing arrangements to distribute the new product in selected channels of distribution? Are local media designed to test variations in promotional messages?
Research Design: Dr. Dey 53

Test Marketing
Pros: Allows most accurate method of forecasting future sales Allows firms the opportunity to pretest marketing mix variables Cons: Does not yield infallible results Are expensive Exposes the new product or service to competitors Takes time to conduct
Research Design: Dr. Dey 54