Professional Documents
Culture Documents
2.1 Prepare
Define objectives Formulate hypotheses (final) Define unit of analysis Define variables Identify indicators & values (operationalise) Choose sampling procedure Make sample frame and draw sample Construct and pretest questionnaire Elaborate coding procedure Prepare tables, charts to be used for analysis Train interviewers Inform the population and relevant authorities
2.2 Implement
Collect and check data Encode the data Put data into matrix ICRA Learning Materials - Preparing a Formal Survey Guidelines - 3/10
2.3 Analyse
Analyse and extrapolate frequency distribution, average, mean, mode, standard deviation, etc. Carry out a statistical test of hypotheses (to establish the relationships between variables) Only during the preparation phase is there some room for iteration. Once the implementation has begun, there is very little one can do to readjust the process. Any major readjustment during the implementation would jeopardise the rigorously standardised procedure of measurement on which the whole concept of quantitative surveys is based.
obtained justify the time and effort needed for a quantitative survey. This learning resource focuses on the use of quantitative surveys to verify hypotheses developed previously from analysis of secondary information or primary information gathered with informal methods.
4 Hypotheses
A hypothesis, as the term is used here, is a proposition about a relationship between two or more variables. Within the ARD procedure, such hypotheses usually concern: ICRA Learning Materials - Preparing a Formal Survey Guidelines - 4/10 The numerical importance of different categories of farms in a typology, and/or The relationships between the distinguishing characteristics of these categories, (i.e. between the variables defining the typology). The ultimate objective of such a verification process is to check the findings made earlier in the ARD process with informal methods, which have usually been obtained at a limited number of farms or localities. It may be necessary to verify that these earlier findings are representative of the larger population for which the study aims to develop research recommendations. Four aspects need to be well defined in a formal survey used to verify (or falsify) a hypothesis. These are:
ICRA Learning Materials - Preparing a Formal Survey Guidelines - 5/10 to relate them to characteristics of the farm or household that we can observe. These measurable proxies for the variable are called indicators. For example, if you want to test the hypothesis that soil quality maintenance is poorer on large than on small farms, the first variable can e.g. be indicated by the average annual amount of money spent on this maintenance. This amount, of course, is not the maintenance itself, but it is an indicator. This one indicator may not be precise enough and you may need to combine it with other indicators: some farmers may compensate for low cash expenditure by spending more labour time on soil quality maintenance, for example. Sometimes one variable is therefore measured by a combination of various indicators, combined in what is called a scale. Finding indicators that are proxies for variables is called operationalisation. This is a complex step that needs a lot of attention. Suppose you want to investigate if farmers who own their land maintain it better than farmers who rent their land or are sharecroppers. Owning or renting may seem simple variables, but what if the ownership appears to be collective (e.g., a group of brothers), or if they rent from their parents (so that they will later inherit the land)?
4.3 Values
Indicators can take on different measurable values for different units at the same time or for the same unit at different times. These values are not always quantitative or numerical. Some may be nominal (e.g. male/female). Others only allow you to put the measured units in a certain order (e.g. from less to more), without you being able to say exactly how much less/more each farm scores compared to the others. Some indicators are actually numerical (e.g. average annual amount of money spent on soil quality maintenance per ha of cultivated land). Defining indicators and the values that these indicators can take on is equivalent to defining the operations by which a variable can be measured. In social science this is called an operational definition of the variable (to be distinguished from the conceptual definition of the variable itself). And the process of finding indicators and values is called operationalisation.
There are two types of sampling: Probability sampling. In this approach, each individual or sampled unit has a known but not necessarily equal probability of being selected (necessary to guard against biases, in particular those related to the accessibility of sites or persons). This type of sampling is essential if statistical inference on quantitative data is to be made from the results of the survey. Probability sampling depends on the availability of an up-todate, comprehensive sample frame that lists all the individuals or units in the population that can be sampled. Non-probability sampling. This type of sampling does not permit statistical inference, but can be very useful for selecting individuals for informal surveys, or for looking at different opinions, for getting an idea of the diversity in an area, etc. In this unit, we concentrate on random probability sampling and especially simple random sampling, multi-stage sampling, stratified sampling and grid sampling.
better technique. If houses are not near roads, the time and effort used in this sampling method should not be underestimated.
describing data failures. The sooner you start this process, the more chance you have to identify errors and omissions. You should consider: Layout. (Are all units of measurement standardised? How will you convert qualitative data into numerical/nominal data?) Pre-coding. (Which coding classes will you use? Do all possible answers fit within these classes?) Storing/registering data. (What software will you use? Who will enter the data? How long will this take?) Processing and analysis (How will you present the data? What tables/ graphs, etc? Who will do these analyses?). It is difficult to make major changes to a questionnaire once the survey is underway. Part of the work will be lost if all the respondents have not answered the questions in the way you expected, or when the answers cannot be processed. Therefore, it is a good idea to pre-test the questionnaire with a few initial interviews, in order to process these, verify and adjust the questionnaire (add or take out questions, clarify any questions that have not been understood by the respondents, change units of measurement, etc). The ease of data processing and analysis should also be tested at this stage: while there is still time to adjust the questionnaire. ICRA Learning Materials - Preparing a Formal Survey Guidelines - 10/10
9 Precautions in interviewing
As with all interviews (both formal and informal), it is important for the interviewer to: Respect the local customs and uses, and working times when the timing of interviews is being arranged. Avoid lengthy questionnaires that take more than about 40 minutes to fill in. Include only those questions that are necessary for your objectives. Try to fit in avoid showing fancy clothes or cars where possible. Know at least some expressions (e.g. simple greetings) of the local language. Introduce yourself. Explain to the respondent the purpose of the survey, why or how he/she has been selected for interview. Clarify what is expected from the respondent and how the interview will be conducted, (e.g. so that the respondent is free to express his/her opinions), and what use will be made of the interview results. Confirm the importance and value of the answers that are given. Go through the appropriate local authorities. Try not to be too formal even where a questionnaire is being applied, the interview should more resemble a conversation than just a set of questions. Focus on what his counterpart says rather than just follow the prepared list of questions; even when using a closed questionnaire, you should leave some space for answers to the respondent. Seek the logic behind the answers, rather than merely record what is said. Be careful to explain or rephrase questions that are not clear or not understood (e.g. using unfamiliar units of measurement).
10 Acknowledgements
This learning resource was prepared for ICRA (www.icra-edu.org) by Jon Daane and Richard Hawkins, using information from Development Oriented Research in Agriculture: An ICRA Textbook by Hal Mettrick; published by ICRA, 1993. It forms part of the ICRA learning resources, written for use in capacity building in Agricultural Research for Development. You are welcome to use these materials for
non-profit purposes; we only ask you to send us a short email at Secretariat.ICRA@wur.nl, or leave a comment on our webpage telling us how you have used them and how you think they could be improved Thank you!