WHAT THEORY IS NOT?

• Goal of scientific method is to create explanation

• Exercise:
• How much the construct definition is clear?
• Does theory creates explanation is only a statistical model?
• BSC is only a tools, and strategic map is experimental, yet in scientific method you
need theory development and justification
• Each hypothesis needs explanation, and just creating multiple hypothesis out of
literature is not enough
• Proposition is in the form of:
• i‘th rational
• i‘th proposition
• Theoretical could be at the end of literature review, or in a separate section
PROPOSITION AND THEORETICAL FRAMEWORK
• We need to be able to explain why we put this variable in our model, and explain it in
terms of argument based on theory
• It is rare that number of variables would be high, and you have theoretical argument
behind it
• We may reach a point that we would not be able to create theoretical argument
• If you are proposing a hypothesis you need to be able to explain it
• Literature should exist for explaining the relation, and you need to have argument based
on
• Explanation is very important part of each model, and for any piece that is added to the
literature
• Your model should answer to the question of why?
• You must also have evidence and empirical support for your theoretical argument
• e.g. Since reference XXX has showed this, an on the other hand references XXX has
shown that …., as a result we should be able to show that
5A m.45
LEVEL OF THEORY CONT.
• Theories have levels:
• Individual level (organizational psychology)
• Relation between individual level variables
• E.g. job satisfaction
• Group level
• Relation between group level phenomenon
• Group performance for example is not equal to sum of performance of each, and that is
why we call it group level
• Another example could be group learning
• Organizational level:
• Separating different organizations
• Unit of measurement would be organization
• When it would be measured at individual level then we could not infer organizational level
results
• Level shows which different groups are separated, and differentiated
• For data collection and hypothesis is important to know level
• Theory describes relation between individual level variables, and creates prediction
LEVEL OF THEORY CONT.
• Industry level
• Cultural level
• Unit of analysis would be country
• We could have sub-culture and culture defines as common feature of all sub-
cultures
• To know specific culture you need to first understand subculture features and
find the joint features of them
• If there would be huge difference gap between individual level behavior,
researcher may cast doubt on the existence of such thing
• For example usually time may be needed for team effect to establish
• Errors in these theories (ecological fallacy):
• Find a phenomenon at higher level and extend it to the level at the bottom
• Find phenomenon at lower level and extend it to higher level
• Find in individual level and generalize it to different societies
LEVEL OF THEORY CONT.
• Another example is that from specific risk level of each person results in promising
result, yet we can not infer the risk level of group will result in promising result
• There should be another research question whether we can extend ecologically
this finding that is taken in individual level
• Usually a theory is not proved in our research context, but we only find evidence to
support it
• Individual level theories will find out that interaction between people does not have
effect on their results
• Some theories incorporate all three levels, in other word are multilevel
• E.g. specific individual level phenomenon has effect on group level phenomenon
• By checking only one organization you can not generalize to the other organization
cultures
• Level of theory as a result tells us the relation is between phenomenon at what
level
TERMINOLOGY
• Moderator
• Whether relation is strong or weak is dependent upon the strength of this variable
• E.g. personality of a person, composition of group for the performance of the group,
culture
• Most of the time contextual variables are in this group
• Mediation
• Are intermediate effects
• The relation may not always be direct, and effects may have mediators
• When you have multiple mediators at the middle you may assume that there is no effect
due to the insignificance effect of antecedent on consequent, yet it exists yet mediated
by multiple mediators
• E.g. relation of multiple variables on performance
• Mind would be more complex and not linear anymore
• Phenomenon will have effect on each other through multiple mediators
• E.g. training will have mediators to be actualized
• Delay is one issue, and moderator does not exist to make things happen
• Exercise: try to create a proposition= prediction about the relation between two
phenomenon, we will then talk about creating hypothesis by converting it to a statistical
statement
ARGUMENT DEVELOPMENT
• When you develop your argument, you are confirming your own position, building your
case
• Use empirical evidence, such as facts and statistics, to support your claims
• Appeal to your audience's rational and logical thinking.
• Argue your case from the authority of your evidence and research
• Your list of strengths and weaknesses can help you develop your argument
• Prioritize the strengths and weaknesses for each position;
• decide on the top three to five strengths and weaknesses
• using a technique for developing content ideas, e.g., clustering, association, journalist's
questions, begin to expand your understanding of each of the items on your list
• Evaluate each item as to how you can support it—by reasoning, providing details, adding
an example, by using evidence.
• prioritize your list of strengths and weaknesses, this time noting what supporting
comments need more work, more evidence, or may be irrelevant to your argument
ARGUMENT DEVELOPMENT
• Clear thinking requires that you state your claim and support it with concrete,
specific facts. This approach appeals to our common sense and rational thinking
• When our logical thinking states specific facts (called premises) and then draws
a conclusion, a generalization, we call this inductive thinking.
• In deductive reasoning, our logical thinking starts with the generalization. As we
apply our generalization to a specific situation, we examine the individual
premises that make that generalization reasonable or not.
• Ethics: Do not misrepresent evidences, and research enough
• Think of ethics as the force of character of the speaker as it is represented in
oration or writing
• Emotion: Using emotions as a support for argument can be tricky
• Try to write as objectively as possible
SOME FEEDBACKS
• When you have not seen original and have seen reference to it in another paper :
• first priority is to go and check original statement
• Second if not possible write that somebody else has quoted this from someone
else's paper
• If the statement does not have evidence it should not be biased, and it should be
written in the form of “it seems”
• Any statement should have methodology, and have probabilistic statement
• We talked about proposition that is prediction of the relation between two variables
• If you use a theory, and try to use argument to explain a phenomenon, you used
deduction method
• In contrast, in exploratory, you use some facts to extract an explanation
• Qualitative research helps you to get better understanding, but has lower
generalizability
• Qualitative research usually have higher depth
RESEARCH TYPES
• Research could be categorized as:
• Qualitative research
• Classification of visions, one can quantitate it by counting visions that have
customer in them
• Strategy research is also in this form
• One of the most important methods is interview
• Observation that is descriptive method is also qualitative
• Quantitative research
INTERVIEW
• Interview is about perception of people
• When we have objective phenomenon and have access to it we need to do
objective, and not interview that needs perception
• For example you should never find weight of a person by asking it from other
people instead of measuring it
• Goal of qualitative approach is survey from informants
• You can use qualitative approach and then use the result for quantitative approach
to reconfirm and generalize it
• Another approach could be to use result of quantitative and then use qualitative
approach that is not generalizable, yet helps you get better understanding of the
phenomenon
• Sampling is the first step
• Sample is on informants people in here, since we need to understand
phenomenon and mechanism
INTERVIEW
• Second is about the guideline of interview
• Creation of questions (structured, semi structured, and unstructured)
• Whether we can record it, what to do when not allowed
• How team works
• To what extent we need to describe the objective
• Ethical issues such as result that could be altercation between firm and
employee
• Follow up questions
• Can we interrupt the person
• Analyzing is not only for quantitative approach, and for qualitative approach is also
important
• Embirgo is a software for this that can find specific phrases in it, and you can infer
the context of text by finding specific phrase, theme, and codings
• Transcription of the interview whether summery and conceptual or word by word
QUALITATIVE APPROACH
• Result is model of category of concepts
• It could be a typology
• E.g. symbols of teamwork in different cultures, that shows the typology of
understanding people in different cultures from team work
• Model could be timely or any other ways
• Concept and construct of an interview in one hour could be 60, and to generalize
you need to restrict the scope
• We will talk about theme analysis in next sessions
SAMPLING QUALITATIVE
• Context of our study is that we have huge population and we get small sample for
understanding phenomenon and not for generalization
• We want to understand the phenomenon, and we need people that are informants,
but for their position or background they watch the phenomena from different
aspects
• We should include women as well as men in our sampling
• Our objective in qualitative work is to go over people who have experienced it
• It is not important whether people think right or wrong, but we want to understand
how they think
• If you want to generalize you need to diversify sample
• You need to take sample so that it would be related to the objective of the research
• Questionnaire could be explanatory for qualitative study, or ask people to rate for
quantitative study
SAMPLING QUALITATIVE
• Interview has advantage of feedback, follow up question, and validation while being
attentive for bias
• This is usually done at the end of the interview in summarizing and closing
session
• Also trust building will be conducted through interview
• Category of interviews:
• Structured: you have chosen the questions, and you only ask them
• Used when you have a model and just you want to validate it, and not digress
• Try to cover the scope with couple of questions
• Semi structure: it has questions, but you can ask questions that are not in it
• Unstructured: you want to create no bias before, and it is very in depth, since it is
looking for crude analysis of the phenomenon
• Generally for interview you must have prepared yourself, at least from state of mind,
and be prepared that when there is no question you would be able to incorporate
phenomenon that previously you have not included in your model
INTERVIEW
• Interview is not about discussion and persuading and you may not like what person
is communicating, and you just should be able to understand what the person is
saying, and not bias a person or argue with interviewee
• You should create intimacy for trust and you should avoid digression from main
subject resulting from intimacy
• Stress, behavior, emotions, and body language may be important in some research
subjects such as commitment and organizational citizenship
• Qualitative approach needs patient and ears to listen even unrelated things when
emotional stickiness exists
• Unstructured as we explained the researcher know the objective of the research,
but avoids any assumption before interview, just prepares his mind:
• It is mostly useful only for professionals
• Interviewer should have enough mental and communicative power to lead the
interview
INTERVIEW
• Interview has the objective of in depth understanding
• As a result there is limited quantitative and rating questions
• Guidelines:
• Start interview with explanation about the objective of the interview
• Also explain confidentiality terms, to create trust
• Try to explain high level research scope to create the trust
• Try to record the interview, and get permission before it
• Try to start from general questions and then little by little move over specific
questions [explain this process to avoid time waste perception]
• Body language is very important, show attention, and eye contact
• For note taking when recording is not allowed try to attend in teams for
assigning one person to have eye contact and the other take notes
• Interview questions should have the nature of asking for explanation (explanatory
questions)
ACADEMIC WRITING
• In paragraph writing you need to comply the same structure that you learnt in
TOEFL, you need to have a topic first, and then provide support, explanation and
example to help reader to proceed smoother
• In correct writing you need to make the reader ready step by step
• You need to provide the definition of the concepts and constructs first before using
them
• You need to start from general explanation first, and then little by little become
specific, and the extreme of being specific is in theoretical framework
WHAT THEORY IS NOT?
• Hypothesis is not theory, and theory should have explanation
• List of variables is not a theory
• Data is not a theory, yet it should support theory
• Theory discusses about why, and data only supports the theory
• Theory should provide explanation and create understanding
• When strong theory exists that explain phenomenon, using the exploratory
approach could be deemed as irrelevant
• Collecting data is researchers task, and interpretation of the data by researcher is
called theory
• Interpretation of what has happened that resulted in the data is a theory
• Description of constructs solely is not a theory, and theory must show the relation
between them and explain them
• If you just provide constructs, it would be dictionary, and they may not have value
until we show relationship between them
WHAT THEORY IS NOT?
• Just relation is not enough, in the form of diagram, and you need to provide
explanation
• You need to provide description of the relation, and what each of these relationship
mean
• Diagram should be described in the text
• Hypothesis tries to predict
• Hypothesis is not substitutable with theory, since theory provides explanation
• You must have theory, explanation and then prediction and hypothesis
• In a theoretical paper both are necessary
• Different journals have different requirements:
• Empirical work
• Theoretical work
• Both empirical and theoretical work
CONTRAST THEORY AND METHODS
• Field study is important in management and management student learn both
qualitative and quantitative approach, and these are teachable
• Theory needs in depth understanding, and person should be visionary, and creating
theory is art in contrast with research methods that are teachable
• Theory should:
• Provide explanation, and interpretation
• Show clear relationship between variables and mediator and moderator
variables
• Models is general theory basis, yet theory should be more specific and provide
explanation
• Propositions together could create a theory or not
• Theory is viewed as social process by some practitioners
• Technical approach and research instrument is not core of the research, yet core of
research is theory
THEORY ROLE IN METHOD SELECTION
• Based on journal usually you need to restructure your research paper
• Method should be matched with phenomenon under study, and after you defined
the constructs you need to match the correct method
• Theory should create prediction, so that you would be able to test it
• Theory should answer the question of WHY?
• Theory should give you Reason, by argument, and give you explanation
• At the beginning of the paper you need to talk about theory, and at the end in the
discussion also talk about theory, explaining why alternative explanation could not
be true
• Good research should have based its main task over the foundation of strong
theory and not the weak one
• Qualitative approach helps you to add new constructs to the previous theories, and
create explanation for them
INTERVIEWING CONT.
• We have different questions:
• Introducing: general to start
• Follow ups: you ask for more explanation
• Interpreting: ask whether your interpretation is right, in the form of confirmatory
• Transcription of the interview even when you record it is recommended
• Writing emotion of the person, since it may not be clear in record
• Two approach of transcription:
• Word by word
• Just writing concept: it is dangerous when you have multiple people, due to the
idiosyncratic prioritizing
THEME ANALYSIS
• Steps:
• Create the transcript
• Concepts will be identified in the text that is called theme
• Then tries to categorize the concepts
• Build model over category of concepts
• Concepts could have different levels
• The most detailed entity could be deemed as concept
• All the concept of the text would be extracted in theme analysis, and we will try to
create model by them
• Then these concepts are categorized to create bigger groups
• Then you go over model, and they try to develop model based on category of
concepts
• Word by word the interview would be transcripts first for the purpose of theme
analysis
THEME ANALYSIS
• In qualitative analysis your explanation should be supported by quotes
• Number of categories depends on the sample size
• Generalizing the qualitative model would be done through quantitative work
• As we said in interviewing after transcripting the interview, theme analysis process
should be conducted
• Result of interview would be explanation of informant
• Transcripts may have common elements that we identify as concept
• We may be able to categorize the concepts
• Repetition is the key here
• Evidence would be find from the text of transcript, analogous to quantitative
approach evidence is searched in the data
• Interview is usually used for exploratory, and from it we could extract theoretical
framework based on the interview
THEME ANALYSIS
• In qualitative analysis we do not care about the generalizability
• After finding category of concepts we use questionnaire to generalize it via quantitative
analysis
• Each concept that you talk about will also include the evidences from transcripts
• Quantifying and getting correlation through software for transcripts is also another option
• Yet you must have big sample in order to be able to do this
• After grouping the concept we will shift our focus over the instrument of measurement to
be able to measure them
• Other research question may need:
• Different research design
• Different research questions
• In qualitative work you can not talk big, since it has limited sample
• We can just say that these variables seem to be important
• In qualitative work sample size is not important, since an informant may say something,
yet in quantitative work you will find out that this effect really exists
THEME ANALYSIS
• Some people say to sort the results in term of number of citing, yet some say that it
is not right since minority would be at the end
• A good way to report the result of qualitative is based on relevance to the research
questions
• In quantitative work usually the results are reported based on count
• Qualitative work is critically subjective
• Each person could have different explanation
• Permissibility of the model is to the point that you have reason for it
• Summarizing result of qualitative approach is subjective, and you must have reason
for them, and propose theories to be tested
• It is starting point
• You should not emphasize on results you present in qualitative work
• Qualitative research in measurement, and model development are the beggining
GROUNDED THEORY
PURPOSEFUL SAMPLING
• Convenience sampling
• Based on availability
• Problem: biased
• Maximum variation sampling
• Wide variations in sample, also used in focus group
• In each group homogeneous
• Snowball sampling:
• Identifying some relevant to study
• Ask them to locate other informants, one informant refers researcher to other
• Variation is limited
• Sampling contrasting cases:
• two or more population groups with distinct characteristics
QUANTITATIVE APPROACH
• We are dealing with numbers
• We try to measure phenomenon, and take the results into mathematical model
• It is positivist approach
• It is kind of limitation in this type of researches
• Theoretical framework should be basis for statistical analysis
• After extracting the theoretical framework we need to find statistical model that fits
with it
• Many phenomenon are objective, and many of them are constructs that are not
directly measurable such as:
• Attitude
• For each phenomena in quantitative work we need operational definition
OPERATIONAL DEFINITION
• Find the manifestation of phenomena in term of result of that phenomena
• For example self esteem, job satisfaction, attitude
• We need to find how it is descripted, and how people identify it, and this is
created through qualitative work
• You need to understand how researcher operationally defines construct
• You may find different questionnaire is literature with different operational
definitions of the same construct
• Proper research you need to have good problem, and you then check whether for
your theoretical framework there is measurement instrument available
• After defining the phenomena operationally, then you will face with question of what
indexes to use to measure
• Measurement has limitation for research
• At the end of the paper you can point them out
• Operational definition is part of these limiations
HOW TO MEASURE
• There are two different measurement approaches, you can ask a team to provide one
evaluation for team, or ask each person individually to provide their evaluation
• When we do not have objective approach we go over questionnaire
• If it is objective go over measure of objective
• If objective measurement is not available then you can use questionnaire
• How we can do the measurement?
• Categorical, nominal:
• frequency is important, we look at them as objects
• Ordinal:
• You may not be able to measure difference between them for example first and
second person
• Interval, ratio:
• Difference between them is same
• Ratio has zero, yet in interval zero does not exist
• for example self-esteem equal to zero has no meaning
• Different statistical approaches need different type of variables
HOW TO MEASURE?
• Scale of questionnaires:
• Five point scale
• It could be seven point or ten point, and there are different approaches that we
will discuss later
• These are ordinal and at most it could be interval
• Significance of variable whether you assume it ordinal or interval would be
different
• Researchers assume them optimistically interval, yet it is questionable
sometimes, since they may be really ordinal
• Some people explain for all of the points and some only put it at the top
• Ratio usually is used for objective measurement
TYPE OF VARIABLES
• Type of variables:
• Dependent: measure of the phenomena that we want to measure, for example
job satisfaction which is a construct
• Independent: measures that will affect our subject of interest; these variables
are used for prediction
• Mediator variable: variable that effect is happening through it
• Moderator variable: The level of this variable determines the magnitude of one
variable affect on the other variable
• Different type of researches exist for the main variable of interest (e.g. customer
loyalty):
• Antecedent searcher (what will affect it)
• Consequent searcher (why it is important)
• Find the moderator and mediators
• how these relationship will become stronger or weaker by changing this
variable, for example check what variable will make satisfaction stronger to
affect consumer loyalty
OPERATIONAL DEFINITION
• You need to create operational definition for each phenomena that you want to use
for your research
• Questionnaire:
• Will have at least 3 items, and they should be different description of the
phenomenon
• for example four people may have different definition of the transactional
leadership
• Path analysis would be done then for analysis of questionnaires
• Since answering question could be different based on priority, and as a result
path analysis tries to calculate weighted average
• There are items in questionnaire that try to conceptually measure your
phenomenon
• When questionnaire designed
• They will used pool of different peoples accent, and try to used them
• Questionnaire is operational definition of the phenomena in different accents
QUESTIONNAIRE
• Proposition is what our theory predicts
• relation that you have extracted in your theory
• Hypothesis:
• Will start after measurement
• Is proposition in statistical language
• Hypothesis type:
• Mean: that these two groups have different means, so are from different
population (H0: two population are the same): use t-test
• Relation: show that these two relation exists, for example one is antecedent of
another (H0: the relation does not exist)
• Null hypothesis:
• Positivist approach: we have theory, and we can not recover theory, but we try to
find data to reject hypothesis, but if we could not reject it, we say that it could be
true, and we do not prove it
• They say our hypothesis is supported by data, yet has error
QUANTITATIVE ANALYSIS
• Cross validation:
• Check theory in different contexts
• Tests could be non-statistical such as:
• Modeling
• Simulation
• Compare the results
• Some phenomenon can have multiple operational definition:
• E.g. organizational effectiveness: organizational goal, profit or so on
• When we want to measure phenomenon we need to first find out at what level it is
(organization, group, individual or …), and w need to match our measurement with
that level
• For example if you take data at individual level and get the weighted average it
would be measurement error
QUESTIONNAIRE
• Validity: measure with multiple methods and evaluate which one measures better
• There could be multiple instruments
• Questionnaire structure:
• Introduction
• Tells objective of research
• Identity is sometimes necessary, but most of the time identity will jeopardize
the validity
• You will put that identity will be confidential
• Social desirability can create bias, due to the norm of the society
• This should be controlled
• You need to show that no matter whether wrong or right, your answer would
be useful, and there is no wrong or right questions
QUESTIONNAIRE
• How interesting is survey, when questionnaire will be filled, and how you collect it is
very important
• Rhythm of questioning is important, and you need to have strong positive effect
at the beginning
• Questionnaire should not be boring, and should be interesting, to increase
reliability
• Questionnaire filling is thinking process, and answering each part will be
dependent on the previous
• Sections:
• You can use standard one that is used in other research or packages
• Research instruments have copy right
• Reliability of questionnaire would be the same as that commercial instrument
QUESTIONNAIRE
• Sections:
• You will have sections about phenomenon that your theoretical framework asked
to collect
• You should not ask about everything you want, because it makes your
questionnaires larger, and more than 30 minutes it take results in unreliability
• Questionnaire should be selective
• Path analysis should be used for personally developed questionnaires
• Each section should have theoretical justification about why you are measuring it
• Any customization of questionnaire for new concept means creating new
questionnaire and validity of it should be checked
• How many options should be available?
• Phenomenon may not need detail scale of seven, and it could be lower
• Some may find out that people have more opinions, and based on interview
we found out that more than five options would be needed to create more
degree of freedom
QUESTIONNAIRE
• Scale:
• When scale would be detail, it creates complexity for the person who wants to
answer them, and creates thinking cost and load
• Based on the interview conducted within pilot study you will find out how many
would be suitable
• Changing scale may modify the results, due to peoples different understanding
• In pilot study you may find out that people can conveniently think in the form of
percentage
• Trade off of scale choice:
• If it would be more than a threshold it will result in complexity
• If it would be lower people may be able to check one options, and there would
be no variation
• Validity of questionnaire is also an issue
• For example specific scale may show better predictive validity
QUESTIONNAIRE
• Culture may result central bias, or underestimate or overestimate it
• Japanese and American do the latter two options respectively
• Sometimes central point is removed, so that people would become bias
• Five and seven scale usually have central point
• Research instrument:
• Protocol of interview
• Questionnaire
• At the end of the questionnaire is open section to allow people to add new things
• You can not used those for quantitative, and has qualitative value
• You may use this for theme analysis, but using it is usually rare
• If you put inappropriate items in questionnaires, the validity of result would be
questioned
• Complexity of terms used is very important, since people must have common
understanding
• If it would be vague then the results would be unreliable, for example identity could be
deemed as cultural identity, personal identity
• Or sometimes people have no idea about brain storming, and you ask them and result
would be unreliable
QUESTIONNAIRE –PILOT STUDY IMPORTANCE
• Pilot study is very important, since checking questionnaire with multiple person
helps you to remove the problems of vagueness
• You need to make sure whether the understanding is the same as you wrote it
• Each item should ask only for one concept, and not multiple one
• Multiple concepts in one question could jeopardize the validity
• Questionnaire items should not be long
• Long explanation in questionnaire can reduce validity
• You need to be economic in questionnaire, know what you want to ask, and ask it
• Bias is another issue:
• You should not emphasis on things, such as very, or never
• When you emphasis on one thing people will become bias
• Reverse items means you want to ask question, but you ask it negatively
• Some believe that it is not recommended to put reverse item
• Some believe that number of direct and reverse items should be equal
QUESTIONNAIRE BIASES
• In entering reverse item, you need to reverse them in entering it as data, since it has
negative meaning
• E.g. 2 means 4 out of 5 scale
• Questionnaire design is very difficult
• It is about how you measure a phenomenon
• You’d better think about how we can choose good questionnaire for measurement
• Validity and reliability is important in questionnaire
• If all my items are related to phenomenon the questionnaire is valid
• Means you are not measuring something else
• Construct validity:
• You need experts to confirm relevance of the items to the phenomenon you want to
measure
• Face validity:
• Are we making bias for people to answer or help them to answer it unbiased
• Predictive validity:
• Questionnaire should be able to predict the real value
QUESTIONNAIRE CREATION PROCESS

Sources: http://www.joe.org/joe/2007february/tt2.php
RELIABILITY OF QUESTIONNAIRE
• Reliability means the consistency or repeatability of the measure.
• especially important if the measure is to be used on an on-going basis to detect
change
• Forms of reliability:
• Test-retest reliability - whether repeating the test/questionnaire under the same
conditions produces the same results;
• Reliability within a scale - that all the questions designed to measure a particular
trait are indeed measuring the same trait.
• Other methods:
• You can use factor analysis to reduce the length of the questionnaire

http://www.daa.com.au/analytical-ideas/questionnaire-validity/
VALIDITY OF QUESTIONNAIRE
• Validity means that we are measuring what we want to measure.
• Types of validity:
• Face Validity - whether at face value, the questions appear to be measuring the
construct.
• largely a "common-sense" assessment, but also relies on knowledge of the way
people respond to survey questions and common pitfalls in questionnaire design;
• Content Validity - whether all important aspects of the construct are covered. Clear
definitions of the construct and its components come in useful here;
• Criterion Validity/Predictive Validity - whether scores on the questionnaire successfully
predict a specific criterion.
• E.g. does the questionnaire used in selecting executives predict the success of
those executives once they have been appointed; and
• Concurrent Validity - whether results of a new questionnaire are consistent with results
of established measures.

http://www.daa.com.au/analytical-ideas/questionnaire-validity/
PROPOSAL STRUCTURE
• Introduction
• Background, and literature
• Theoretical framework :
• summarizing the literature or something else
• Methods
• Design
• Phases of research
• Population and sample
• Instrument: Questionnaire, interview questions
• You need to show that instrument is available, and you are in selection
process, this helps for feasibility check
OTHER TYPES OF VALIDITY
• Other types of validity is:
• Convergent validity:
• Different measures lead to similar results of measurement
• Divergent validity (subset of conceptual validity):
• Construct is distinguishable from other construct (Discriminant validity)
• Distinguish between phenomenon (e.g. self efficacy and self esteem)
• Other pitfalls of questionnaire:
• Ambiguity
• Bias
• Low difference between options
• Difficulty in analyzing and understanding
• Overlap of options, and lack of mutual exclusion
RELIABILITY
• Test re-test:
• Means if you conduct the test again after some duration, you will get the same result
• Sample is separated into to half, and check them together
• This is a measure for precision of the measurement instrument
• Duration should be chosen based on the trade of :
• Forgetting the previous answer
• The duration that variables has not changed a lot
• Cronbach’s Alpha:
• coefficient of internal consistency: should be more than 0.6 and 0.7
• If it is standard even 0.6 is not good showing lack of fitness with culture as an
example
• Is indicator of precision, showing that indicators move together
• If for example temperature indicators do not move together they show existence of
third variable
• Saying that indicators more together up or down
WHAT MEASURES? HOW TO SELECT?
• For precision in literature you should not only tell about concept, theory and
relation but also read about the instruments, and you should show what were the
available measures
• You should look at the operationalization behind the instruments
• You should also show that what is your definition, and find the instrument that
measures this concept, and not other definitions
• Measurement should be valid and reliable to get reliable results
• Comparison of the instruments is very important, component, and match with
context is very important
• Path analysis helps to find better items for analysis
SAMPLING & STATISTICS IMPLICATION
• Qualitative work is not seeking to generalize, but quantitative work does
• Checking whole population is cumbersome due to the considerable resource
needed
• Sample should be representative of population
• Statistical inference:
• We take sample and extract statistics
• Generalize results to the population
• We want to check what is the probability that this statistics would be generalized
to the population
• For example you say that 95% probability we will have the same result
• Hypothesis is not confirmed, but we say within the range of errors our hypothesis
is correct (We say that reliable interval is this amount)
• Sampling should be correct, and it is very important, if sample would not be
representative, then the result would not be generalizable
SAMPLING AND STATISTICAL IMPLICATIONS
• Sample should be by congruent with level:
• E.g. if the phenomenon is organizational level, you need to have samples from
organizations, and you need to have sample from each cluster
• Sample could be multiple level:
• E.g. for each organization you may have sample from second level
• Sample should be random
• Random sample would be that you assign numbers to database of customers
• Assortment of people would be random, and you select out of them randomly
• You contact your sample based on sample size
• In this method you need to know population, and you know who are them
• Systematic sampling:
• When size of sample is identified
• Takes the first sample, and then goes forward in selection in steps of ten
• i.e. choose 10th, 20th, …
• If your data would be chronological, it could result in bias
• Advanced sampling thinks about population size
SAMPLING & STATISTICAL IMPLICATIONS
• Advanced sampling
• Thinks about the population structure
• Methods of Advanced Sampling:
• Stratified Sampling
• Clusteral sampling
• E.g. for example different generation of university, or different faculty, if you
believe each has specific feature, and important variable exists in those
clusters
• Variance of phenomenon results in greater sample size requirement
• Expected error also affects on the sample size
• There are different approaches for sampling, and they tell you based on
cluster what should be the size of sample given expected error

Another good source: www.cs.vu.nl/~stochgrp/aionetwerk/course.doc

SAMPLING & STATISTICAL IMPLICATIONS
• We need to have enough variation in the sample for the variable that we have variation in
the population
• In sampling we must have maximum variation for the phenomenon under study in the
sample, since if it would not have large variation, it means it is not variable
• E.g. very small variation in the salary and showing that is uncorrelated with the
dissatisfaction
• Theoretical understanding of population has direct effect on our clustering, since data are
sparse
• Example:
• Let say we have multiple companies and have variation at organizational level
• We have group level variation, mean how organizations are different in their variation
• Variation of individual level shows how inside each organization there is variation
• If you generalize from individual to organization, you must have enough variation in
organization
• For regression at least you need sample of 40
SAMPLING & STATISTICAL IMPLICATIONS
• For statistical inference, statistical hypothesis will be defined
• Normality of variable is assumed for statistical inference
• Distribution that will be used for social science:
• Z distribution (Normal distribution):
• sd=1, and mean=0
• Data that would be gathered should have normal distribution
• Skewness shows deviation from the normal distribution
• Variable is ratio
• t-distribution:
• Variable is ratio
• Chi- square
• Type of variables are nominal
• Statistical inference:
• Mean test: based on central limit theorem
CENTRAL LIMIT THEOREM
The Central Limit Theorem describes the characteristics of the "population of the
means" which has been created from the means of an infinite number of
random population samples of size (N), all of them drawn from a given "parent
population". The Central Limit Theorem predicts that regardless of the distribution
of the parent population:
[1] The mean of the population of means is always equal to the mean of the parent
population from which the population samples were drawn.
[2] The standard deviation of the population of means is always equal to the standard
deviation of the parent population divided by the square root of the sample size
(N).
[3] [And the most amazing part!!] The distribution of means will increasingly
approximate a normal distribution as the size N of samples increases.