Research Methodology Notes

Section : A

Naiha Abid
Prof. Amina Zulfiqar

Table of Contents
I. A. B. Research Design ............................................................................................................................ 3 Research Process ....................................................................................................................... 3 The research Design .................................................................................................................. 4

II. Sampling ........................................................................................................................................ 7 A. B. C. 1. 2. Important statistical terms ....................................................................................................... 7 Sampling Process ...................................................................................................................... 7 Types Of sampling .....................................................................................................................8 Probability sampling .............................................................................................................8 Non probability sampling ..................................................................................................... 9

D. Sampling Errors ...................................................................................................................... 10 III. Data Collection Methods ............................................................................................................ 11 A. B. Types of data collection methods ............................................................................................11 Multi Methods ......................................................................................................................... 15

IV. Measurement and scaling........................................................................................................... 16 A. B. C. E. F. G. I. V. A. B. Scales........................................................................................................................................ 16 Rating Scales ........................................................................................................................... 17 Ranking scales ......................................................................................................................... 19 Bad Questions ........................................................................................................................ 21 Question sequence ..................................................................................................................22 Questionnaire layout ...............................................................................................................22 Questionnaire translation .......................................................................................................23 Criteria for Good Measures ........................................................................................................ 24 Measuring Reliability ..............................................................................................................24 Measuring Validity .................................................................................................................. 25

D. Questions format .................................................................................................................... 20

H. Questionnaire pretesting ........................................................................................................22

2|P age

I. Research Design
A. Research Process

1. Observation First Step of the Research Process area of attention realized 2. preliminary data gathering Collecting First hand information Interviewing Literature surveys Usually costly

Relationship among variables recognized 5. generation of hypothesis Developing of a hypothesis statement based on literature review Usually tested at the end of research Can be directional or non directional hypothesis 6. scientific research design discussed later in this document 7. data collection analysis and interpretation Data is collected at this stage Once gathered, it is analyzed and interpreted 3|P age

3. problem definition scope of problem defined issues that need to be focused on 4. theoretical framework Variables Clearly identified and labeled Dimensions of study defined

8. Deduction Conclusions are made out of analysis and interpretation The hypothesis is tested Whether Hypothesis sustained or not Is Research question answered? Further research required?

10. report presentation Report made listing conclusions and drawings from research Contains recommendations and suggestions might interest to the stake holders of the research area 11. managerial decision making decisions are made based on the results derived from the research

9. report writing Conclusions and findings are listed

B.
1. Purpose of Study

The research Design

Exploratory study The research is exploratory if it is the following: Situation is unknown Exclusively based on Preliminary information Comprehensive study of the topic Focus groups and comprehensive interviews are appropriate Example: Ethical values of different Cultures to manage workforce diversity Descriptive Study The research is said to be descriptive if the following occurs: Description of features of the variables Data is quantitative ( coded or otherwise) for calculation of Mean, Percentages, Std. Deviation

A descriptive study also does the following: Understand characteristics Think systematically Offer ideas for further probe and research Helps in decision making

Testing Hypothesis Explain nature of relationships among the variables and their dimensions Differences and interdependences among factors Example: sales volume promotion efforts

4|P age

Case Study conceptual analysis of similar situations for generalization Difficult to find same type of problem in comparable setting 2. Type of Investigation Co-Relation Identify important factors associated with problem. A Co-relation means that two factors are related to each other in any possible way. Example: are smoking and cancer related?

Causal Relationship Establish definite cause of a problem. A causal relationship means that one things leads to another. Example: does smoking causes cancer? 3. Researcher Interference

Minimal Interference When research studies in natural settings without changing or altering any factor whatsoever Example: training effectiveness based on data Researcher has some Interference Manipulation, control or simulation of the environment The analyst controls variables or factors Example: effect of light on worker output 4. Study Setting Non contrived natural setting- like field experiment

Contrived Controlled settings Either specific or number of settings

(1) Field Study Non-contrived setting as no changes were made Example: rates of interest related to extent of deposits? (2) Field Experiment Controlled settings Some factors are manipulated (3) Lab Experiment independent variable changed to see effect on dependant Contrived settings and maximum interferences

Example: to study relationship of rate of interest on inclination to save the rates of interest in various branches are changed 5|P age

5.

Units of Analysis 4. Organizations Differences among functions of management in various organizations 5. Cultures Nations as unit of analysis

1. Individuals Study motivation of employees 2. Dyads Interaction of supervisor-subordinate pair 3. Group Pattern of misuse by various departments 6. Time Horizon

Cross Sectional One shot study-one time or period Example: data study of stock market April-June Longitudinal Study of information at more than one period of time e.g.: change in behavior of employees before and after management change.

6|P age

II. Sampling
A. Important statistical terms
1. Population Population refers to the entire set of interest of the researcher. 2. Sample Sample is a limited number of subjects chosen from the population for research. It is a subset of population. Sample is usually taken to get information when it is impossible to research due to a large population More accuracy i.e. Can Do A Better Job of Data Collection When its impossible to study the whole population

Less costs Less field time

B.
1. 2. 3. 4.

Sampling Process

Define the Target Population: The population to be studied is the target population. Initially it needs to be defined. Define the Sampling frame List the sources from where the sample is drawn The basis on which samples are selected Determine the Sample Design Probability or Non-probability sampling Determine the appropriate sample size The amount of sample should be large enough to represent the entire population yet small enough to make it easier to analyze and research. Execute the Sampling Process Method of selecting sampling units from sampling frame

5.

7|P age

C.

Types Of sampling

1. Probability sampling A probability sample is one in which each element of the population has a known non-zero probability of selection. Each subject has a known probability of being selected. Allows application of statistical sampling theory to results to: Generalise Test hypotheses Probability samples are the best, they ensure Representativeness Precision Simple random sampling Merely choosing anyone from anywhere, no order no criteria Each element in the population has an equal probability of selection AND each combination of elements has an equal probability of selection Names drawn out of a hat Random numbers to select elements from an ordered list

Systematic sampling Developing a pattern to select the sample, every 4th element or every tenth element etc Involves drawing every nth element in the population starting with a randomly chosen element between 1 and n

Stratified sampling Probability sampling procedure in which simple random sub samples that are more or less equal on some characteristics are drawn from within each stratum of the population Each group is internally homogenous but there are comparative differences between groups. Double sampling Selection of a sample for research and then a subset of the sample for further research A sampling design where initially a sample is used in a study to collect some preliminary information of interest and later a subsample of this primary sample is used to examine the matter in more detail.

Multi-stage sampling First step cluster sampling (for geographical location); next particular area would be chosen and at third stage particular units would be chosen.

8|P age

Cluster sampling Groups that have heterogeneous members are first identified, and then some are chosen at random. Each group is internally heterogeneous but groups are homogeneous on some characteristic. All the members in each of the randomly chosen groups are studied. Target population is first divided into clusters and then random sample of clusters is drawn 2. Non probability sampling When probability of being chosen is unknown Cheaper but unable to generalise potential for biasness

Types of Non-Probability Sampling (1) Convenience Sampling Ease of Access Sample is selected from elements of a population that are easily accessible to the researcher Snowball sampling (friend of friend.etc.) (2) Purposive Sampling

Judgemental Select subjects who are in the best position to give the required information. Experts etc Quota sample Predetermined proportions of subjects are sampled from different groups.

9|P age

D.
1. Error 1 Random Sampling Error

Sampling Errors

This is defined as the difference between the sample result and the result of a census conducted using identical procedures and is the result of chance variation in the selection of sampling units If samples are selected properly the sample is usually deemed to be a good approximation of the population and thus capable of delivering an accurate result Usually, the random sampling error arising from statistical fluctuation is small, but sometimes the margin of error can be significant Increasing the sample size will reduce this error.

2. Error 2 Systematic (Non-Sampling) Errors These errors result from factors such as: an improper research design that causes response error , errors committed in the execution of the research, errors in recording responses and non-responses from individuals who were not contacted or who refused to participate

Both Random sampling errors and systematic (non-sampling) errors reduce the representativeness of a sample and consequently the value of the information which is derived by business researchers from it

10 | P a g e

III. Data Collection Methods

A.
1. Questionnaires

Types of data collection methods

Questionnaire Design (1) Definition A reformulated, written set of questions to which the respondent records his answers, usually within rather closely defined alternatives (2) When to use a Questionnaire When researcher knows precisely what information is needed When large numbers of people are to be reached in different geographical regions When groups of people can be assembled in conference rooms to which questionnaire can be administered and collected immediately. (3) Principles of Questionnaire Design Measurement Word content General Appearance and Get-up (4) Questionnaire Administration and Data Collection

(5) Advantages of a Questionnaire Helps researched obtain data fairly easy Information from questionnaires is easily coded Benefits the scientific community if the measure are well validated and are reliable

Two types of questionnaires (6) Personally administered To groups of people Possible to establish right amount of rapport with respondents

Return rate is high Doubts of the respondents can be clarified at the spot

11 | P a g e

(7)

Mail Surveys Advantages

Wide geographic coverage No face to face interaction awkwardness Can be answered at home and at own pace Disadvantages

Low return rates Clarifications cant be made Respondents cant be observed at all Representatives of samples difficult to establish because return rates are low

Guidelines (1) Purpose To get at objective facts or subjective feelings and perceptions (2) Wording and Language Would respondent understand the words in the questionnaire? Are the questions ambiguous? Are there double barreled questions? Are there loaded questions? (3) Type of Question

Are the questions recall-dependent? Does any question prompts social desirability? Does the length of question exceed 20 words or one line of print?

Open ended Freedom to answer a question any way the respondent wants

Could be a problem for the researcher to encode the response

Closed ended Easy to interpret the code Alternatives that are mutually exclusive Respondents can make quick decision and collectively exhaustive are given. (4) Form of question Both positively and negatively worded to reduce response bias or halo effect

12 | P a g e

(5) Sequencing of question The funnel approach: From general to specific From easy to difficult (6) Classification of Data Personal Information or Demographic Data Whether personal information is required? What data is required? Income, marital etc Should a range be given, if yes what are the appropriate ranges? Personal and demographic should be asked in the beginning or end? Interviews Structured Saves times helps to code information better inter-rater reliability could be established through trained interviewers Unstructured Useful during exploratory states of a research helps develop better theoretical framework helps develop better questionnaires and structured interviews (2) Disadvantages

2.

Face to face or direct interviews (1) Advantages Can change or adapt/ adopt questions Can pick up non verbal cues Can clarify doubts

Costly Geographic limitations Reliability of measure may suffer

Interviewer bias (different interpretation) Inter-rater reliability (differences in interpretations between interviewers) Voice inflection and interviewer conditioning the subjects may introduce response biases Confidentiality rather difficult to be assured of

Telephonic interviews (1) Advantages Discomfort of face to face interviews avoided Number of calls per day could be high (2) Disadvantages Time may be limited for the respondent. Interviewee could put the phone down and refuse to answer

13 | P a g e

Tips for interviews 3. Interviews can be taped with the permission of the interviewee Cameras can be used to capture the non verbal cues of the interviewee and can be studied at leisure by researcher Training of interviewees necessary to establish inter-rater reliability Observation

Types of observation (1) Participant Participates in the environment and makes an observation. (2) Non participant Merely observes without interfering or making amendments. Advantages Sample size could be increased at less costs Respondent bias could be avoided Both bevavirual and non-verbal reactions could be studied Disadvantages Observer fatigue could set in Observer bias 4. Motivational Research

Inter-observer reliability problems Costs of observer training could be high

Types of Motivational research (1) (2) (3) (4) (5) In-depth probing Word association Sentence completion Thematic Apperception Test (TAT) Ink blot Tests

5. Panel Studies Mostly used in Market Research Examples: advertising effects, Product improvements, Brand comparison etc Types of panel studies Static Same members throughout Dynamic Members substituted from time to time 14 | P a g e

Advantages Panel gains expertise and can give good data Longitudinal study could establish cause-effect relations Disadvantages Could be expensive Attrition of members with time Panel could lose interest after a while

B.

Multi Methods

Data from different sources and through different methods could improve the goodness of the data. Interview Observation Questionnaire Same-source and other-source data Objective measure ( physical measurement, counting, etc) Example: Performance data get from all the above sources. See how they correlate

15 | P a g e

IV. Measurement and scaling

A. Scales
1. Nominal Allows researcher to assign subjects to certain categories or groups, the Categories should be mutually exclusive and exhaustive. It Splits data into groups. Example: Gender Male Female 2. Ordinal Allows researcher to characterize the variables in such a way so as to denote differences among various categories and also rank them in some order. Ranks data in some order. Example: Studying for 2 hours daily is good, for 4 hours is better and studying 6 hours a day is best. 3. Interval An interval scale allows us to perform certain arithmetic operations on the data collected from the respondents Sets data on a continuum. Example: very low- very high Negative extreme to positive extreme 4. Ratio The ratio scale overcomes the disadvantage of the arbitrary origin point of the interval scale, in that it has an absolute (in contrast to arbitrary) zero point, which is meaningful measurement point Starts with zero and indicates proportion Example: how many organizations did you work for before joining this system?

16 | P a g e

B.

Rating Scales

Asks the respondent to estimate the magnitude of a characteristic, or quality, that an object possesses The respondents position on a scale(s) is where he or she would rate an object

There are 11 rating scales 1. Category Scale A category scale is a more sensitive measure than a scale having only two response categories - it provides more information. How important were the following in your decision to visit Switzerland? (Check one for each item) VERY SOMEWHAT NOT TOO IMPORTANT ___________ ___________ ___________ ___________

IMPORTANT CLIMATE ___________

IMPORTANT ___________

COST OF TRAVEL

___________

___________ ___________

FAMILY ORIENTED ___________ FAMILIARITY WITH AREA

___________

___________

2. Likert Scale It is an extremely popular means for measuring attitudes. Respondents indicate their own attitudes by checking how strongly they agree or disagree with statements. Response alternatives: strongly agree, agree, uncertain, disagree, and strongly disagree. 3. Numerical Scale Numerical scales have numbers as response options, rather than semantic space or verbal descriptions, to identify categories (response positions). 4. Semantic Differential Scale A series of seven-point bipolar rating scales. Bipolar adjectives, such as good and bad, anchor both ends (and poles) of the scale. A weight is assigned to each position on the rating scale. Traditionally, scores are 7, 6, 5, 4, 3, 2, 1, or +3, +2, +1, 0, -1, -2, -3. n n n Exciting ___ : ___ : ___ : ___ : ___ : ___ : ___ : Calm Interesting ___ : ___ : ___ : ___ : ___ : ___ : ___ : Dull Passive ___ : ___ : ___ : ___ : ___ : ___ : ___ Active 17 | P a g e

5.

Itemized Rating Scale

6. Fixed or Constant Sum Rating Scale Divide 100 points among each of the following brands according to your preference for the brand: 1. Brand A _________ 2. Brand B _________ 3. Brand C _________ Magnitude of each alternative known Equal value can be indicated. Too many can confuse 7. Staple Scale Modern versions of the Staple scale place a single adjective as a substitute for the semantic differential when it is difficult to create pairs of bipolar adjectives. The advantage and disadvantages of a Staple scale, as well as the results, are very similar to those for a semantic differential. However, the staple scale tends to be easier to conduct and administer. 8. Graphic Rating Scale A graphic rating scale presents respondents with a graphic continuum. Easy to construct and simple to use, can discern fine distinctions, but not very reliable, can be treated as interval data. 9. Simple Attitude Scaling In its most basic form, attitude scaling requires that an individual agree with a statement or respond to a single question. This type of self-rating scale merely classifies respondents into one of two categories. THE PRESIDENT SHOULD RUN FOR RE-ELECTION _______ AGREE 10. 11. ______ DISAGREE

Dichotomous Scale Consensus Scale

18 | P a g e

C.

Ranking scales

Ranking tasks require that the respondent rank order a small number of objects in overall performance on the basis of some characteristic or stimulus. Following are the ranking scales Rank order scale Rank the following in order of preference, 1 being the most preferred and 3 the least: Brand A _________ Brand B _________ Brand C _________ Paired comparison In paired comparisons the respondents are presented with two objects at a time and asked to pick the one they prefer. Ranking objects with respect to one attribute is not difficult if only a few products are compared, but as the number of items increases, the number of comparisons increases geometrically (n*(n -1)/2). It causes fatigue and respondent no longer carefully discriminate among them. Example: Which support provides you more motivation to complete your task? Friends support family support

Comparative scaling Provides a benchmark to assess attitudes toward current object, event or situation understudy. Example: Please indicate how the amount of authority in your present position compares with the amount of authority that would be ideal for this position. TOO MUCH ABOUT RIGHT TOO LITTLE

Forced choice A Forced Choice enables respondents to rank objects relative to one another, among the alternatives provided. Example: Rank the following channels you like to watch in the order of preference, assigning 1 to the most preferred choice and 5 to the least preferred. Duniya Geo Express Samaa

Two important guidelines are: 1. Relevancy ask only questions that are relevant to the research objective 2. Accuracy ask only questions that will provide accurate information 19 | P a g e

D.

Questions format

Need to decide how what is to be asked should be phrased Alternative question formats are shown below Each format has its advantages and disadvantages

1. Open ended Respondent answers in his own words Example : Why do you like studying at UCP? 2. Closed ended Respondent selects one or more options from pre-determined set of responses. Simple dichotomy Closed ended question with only two response alternatives Example: What do you consult while preparing for the examination? PPT-slides prescribed course literature Multiple Choice Closed ended question with more than two response alternatives (1) Checklist question Multiple choice questions in which respondent can select more than one of the response alternatives. What sources of information would you use to compile your research paper: Local bookstores University libraries Research Institutes The Digital library Technical Journals Magazines and Newspapers

Response categories provided for each close-ended question should be mutually exclusive and exhaustive
Mutually Exclusive Response categories must be such that the same respondent cannot be classified into more than one category. Example: the categories $1,000-5,000 and $5,000-10,000 are not mutually exclusive. Mutually Exhaustive Response categories should include all possible response options. Sometimes this is achieved by including a response option like Other (Please specify).

20 | P a g e

E.

Bad Questions

Avoid complexity: use simple, conversational language. Avoid leading and loaded questions. Avoid ambiguity: be as specific as possible. Avoid double barreled items. Avoid making assumptions. Avoid burdensome questions.

1. Leading A question that suggests or implies a certain answer The bandwagon effect Example: Most Saudis have stopped eating junk food. Do you eat junk food? Partially mentioning some alternatives Examples: which fast food restaurant do you prefer, Pizza Hut or others? Questions with the phrase: Dont you think that ...

2. Loaded A question that is designed to suggest a socially desirable answer, usually it is emotionally charged. Choice of words Using emotionally-charged words, Example: In your opinion is it fair that the Security Dept should be harassing students with parking tickets? Example: Do you think that older people should be laid off? Framing question such that honest answer is painful or embarrassing use counter biasing statement.

Example: There are pros and cons to retaining senior citizens in the workforce. To what extent do you think firms should continue to keep the elderly on their payroll? 3. Double barreled Introduces bias because it covers two issues at once I am satisfied with my masters level performance. My bachelors performance was also satisfying Highly satisfied Somewhat Satisfied Neither satisfied nor dissatisfied Somewhat dissatisfied Completely dissatisfied

21 | P a g e

F.

Question sequence

Two related issues: 1. Order of questions in questionnaire. 2. Order of answer alternatives for specific questions. Both can lead to order bias. Order of questions Use simple, interesting opening questions Example: asking for respondents opinion on an issue. Ask general questions before specific questions funnel approach Use branching questions (filter & pivot) with care. Ask for classification information last. Place difficult or sensitive questions late in questionnaire. Finish asking questions on one topic before moving to another.

G.

Questionnaire layout

Keep questionnaire short if possible, but not too short that you sacrifice needed information Do not over crowd questionnaire Provide decent margin space Use multiple- grid layout for questions with similar responses Use good quality print paper. Use booklet form if possible Carefully craft the questionnaire title: 1. Captures respondents interest. 2. Shows importance of the study. 3. Shows interesting nature of the study.

H.

Questionnaire pretesting

1. Pretesting Process Seeks to determine whether respondents have any difficulty understanding the questionnaire and whether there are any ambiguous or biased questions. 2. Preliminary Tabulation A tabulation of the results of a pretest to help determine whether the questionnaire will meet the objectives of the research

22 | P a g e

I.
1.

Questionnaire translation

Back Translation Taking a questionnaire that has previously been translated into another language and having a second, independent translator translate it back to the original language. A questionnaire developed in one country may be difficult to translate because equivalent language concepts do not exist or because of differences in idiom and language

23 | P a g e

V. Criteria for Good Measures

Major criteria for evaluating measurements are reliability and validity. Reliability is an indicator of a measures internal consistency. Validity is the accuracy of a measure or to the extent to which a score truthfully represents a concept. In other words, we are accurately measuring what we think we are measuring.

A.

Measuring Reliability

Internal consistency represents a measures homogeneity or the extent to which each indicator of a concept converges on a common meaning. Two methods used:

Split half A method for assessing internal consistency by checking the results of one-half of a set of scaled items against the results of the other half Coefficient alpha The most commonly applied estimate of a multiple-item scales reliability 1. It represents the average of all split half reliabilities for a construct. Ranges from 0 to 1. When the coefficient is below 0.6, the scale has poor reliability. IN SPSS (Analysis -- Scale -- Reliability Analysis and Select items)

Test- Retest Reliability Administering the same measure to the same respondents at 2 separate points in time to test for stability To check the stability of the measure over time Problems: First measure will influence the results when used for the second time. Change in attitude because of long time duration between the two measures.

24 | P a g e

B.
The 4 basic approaches to establish validity are:

Measuring Validity

1. Face Validity A scales content logically appears to reflect what was intended to be measured. 2. Content Validity Is the degree that a measure covers the breadth of the domain of interest? 3. Criterion Validity The ability of a measure to correlate with other standard measures of similar constructs or established criteria 4. Construct Validity Exists when a measure reliably measures and truthfully represents a unique concept; consists of several components including face, content and criterion validity.

25 | P a g e