Session 6 Observation, Experimentation and Survey Methodologies

Observation Any form of behaviour can be observed Uninvolved observation is the purest form of research It is the only methodology for certain groups of respondents (children , animals) Observation may be participative (in which the observer is in an assu med or open role) or may be through mechanical means

The problems with observation! Getting people to admit what they do (particularly if it is socially undesirable !) Interpreting what they do Translating findings into managerial actions Gettin g people to accept changes in their pattern of behaviour.

An ethical issue Is it ethical to observe a person mechanically when they do not know they are be ing observed or have given their consent? (As via a video) Even worse when you g o into a situation in an assumed role (It might be called spying!) However if yo u tell them they are to be observed will they still behave in the same normal way? (The Hawthorne Effect).

Some forms of observation Monitoring of web site traffic Content analysis of advertisements, etc Garbage a nalysis Physical inventories (A.C.Neilson) Monitoring physiological reactions (p sychogalvanometers, voice pitch analysers, pupilometers, eye trackers) Bar code scanners

Experimentation Derived from and based upon physical science studies Principle is to keep all va riables constant except one (the independent variable) in order to chart its eff ect on the other variable (the dependent variable) which is then measured. Simpl e in theory, impossible in practice

Some of the problems with experimentation Keeping all other variables constant while you manipulate one only Interaction b etween variables Unexpected competitive reactions (price reduction might be matc hed!) Selection of test beds (same reactivity?) Effect is very often not equal t o simple economics (advertising for example)

Some alternative forms of experimental design In parallel experiment: Two test beds used simultaneously to get rid of the prob lem of time difference In sequence experiment: Use of only one test bed to get r id of the problems of differential reactivity of test beds (but has the alternat ive problem of time difference!) Double Blind trial: Where neither subject nor a dministrator knows which alternative is being used

Experimental Validity Internal validity: Are the observed changes solely due to the dependent variable or are there other factors which might be causing the observed effect External validity: To what extent can the results of the experiment be generalised outsid e the experiment to a wider set of situations

Different authors, different emphases Read Zikmund: Chapter 12. This author is a far bigger fan of business experiment s than I am! Personal experience has taught me there are so many possible errors , and things which can go wrong, that they are far too risky to consider, and qu estionable in terms of the results they generate!

Surveys: A problem of communication Idea I wish to get information about encoding into words h received via hearing decoded by recipient something is s of this) A response is formulated within the brain it is mmunicated via speech received via hearing recorded by the is understood !!! 12 stages to ask one question!

communication via speec understood (On the basi encoded into words co interviewer something

Surveys: The Metrics With whom: Definition of target market, sample size and selection. How: Methods of contact With what: Questionnaires and schedules Medium: Electronic or printed Time and Cost: What is the budget?

Errors in survey design Random sample design / Self selection bias (Readers of the Bangkok Post say .) Res ondent error: Non response error / refusals / leading to substitution? Response bias (telling lies / semantics misunderstanding / fading memory)

Response Bias The yes respondent says yes to everything! A desire to please Extremity / central tendency in responses Interviewer bias (through age, dress, body language, voice , etc) Auspices (the credibility and stated position of the survey sponsors) Soc ial desirability / status / image bias (responding to maintain a social position and / or non admission of social problems)

Different types of survey Cross sectional survey: A snapshot of customer behaviour at one particular instant in time. May be repeating similar studies done at different times Longitudinal surveys: May use totally the same sample of respondents (a panel) or a rotating panel (NRS / TGI) or a totally different sample on each occasion. The latter has the problem with comparability!

Different forms of contact Electronically / Telecommunications (SMS, Internet, voice activation telephone, call centre (CAT) research, interactive TV, mobile telephone, fax, e-mail, etc) Mail contacts Personal interviewing (See excellent summary on Page 228: Zikmund)

Evaluation of electronic methods Positives: Fast, cheap, geographically flexible, high respondent anonymity, ofte n good cooperation, adaptable to IT technology Negatives: Precise and simple wor ding, limited areas for discussion, limited times for contact, allows cheating, certain groups difficult to contact, limited length, sample self selecting

Evaluation of mail surveys Positives: Can be cheap, sensitive areas can be addressed (nobody present), all family member completion, time period (diary) research, geographical flexibility , time to consider replies, question precision, can be up to 6 / 8 pages long Ne gatives: Cheating, misunderstanding, slow, low response rate, biased response ra te, not versatile, no interviewer available for clarification,

Response rates: Incentives or not? Do incentives increase response? YES Group or individual incentive? DEPENDS Pre or post completion? Related / unrelated to survey? Value? THANK YOU OR BRIBE! Hy giene factors which will improve response without incentives!

Hygiene factors to increase response (mainly mail surveys!) High quality presentation in all aspects of survey design (paper, layout, interv iewer appearance, etc) Covering letter with assurances of anonymity, contact num ber for queries, clearly stated objectives, sharing of results Reply paid envelo pe (stamp not Freepost) Follow up for late respondents

Personal Interviews Very flexible Only as good as the interviewer! Can gather a large volume of rmation Can probe and go into difficult areas Expensive or very expensive! Lack of respondent anonymity Less chance of respondent lying Location / timing can in fluence response

What makes a good interviewer? The person has to accept they are simply a pipe in the flow of information They sh ould, in all aspects ,be neutral (dress, manner, attitude, ethnicity, etc) They should be (a) trained in interviewing techniques and (b) trained in this particu lar survey Remember there is always interviewer bias people like people like the mselves!

Different types of survey instrument Must be suited to the survey methodology Questionnaires / Schedules. Self comple tion / Monitored completion / recorded completion Length and organisation is vit al to improve response. Level of structure determined by question typology Remem ber you have to analyse it subsequently, so keep it simple!

Schedules Very loose type of questionnaire where there is little or no structure Used by T V interviewers, who make up the questions based on a series of topics to be covere d Best use when dealing with executives, where informality is the key. No sequen ce / structure to contact Difficult to analyse, other than as narrative.