Synthesis Paper on

Research in Development Arenas

Submitted to Professor Dr. Johan Bastiaensen Institute of Development Policy and Management (IOB) University of Antwerp, Antwerp, Belgium

Submitted by S.M. Manzoor-E-Khoda Master of Globalisation and Development Student Id: 20123544 Session: 2012-2013 Institute of Development Policy and Management (IOB) University of Antwerp, Antwerp, Belgium

Date of Submission: 23 November 2012

Content

Page Introduction The Role of Applied Research in Development Arenas Epistemological Issues and Boundary Setting Epistemological Issues Boundary Setting Choice of Research Method Rigour in Development Research Conclusion References 3 4 5 5 7 8 10 11 12

Introduction:
Wikipedia refers to more than fifty different dimensions of development, which resonate with the argument of Thomas in Sumner and Tribe (2008: 10) that the concept of development is inherently complex and confusing. In spite of disagreement among scholars regarding the definition, Sumner and Tribe (2008: 10) recognise a mutual understanding among them in terms of emphasizing on the change of the human condition in a variety of aspects. For instance, Chambers states, The objective of development is wellbeing of all. He also stressed on development as the experience of good quality of life (Chambers, 2004: 10). In fact, quality of life is heavily depended on various economic, social, political and environmental phenomena in contemporary global world, as these are interwoven. In order to ensure the quality of life, the policy initiators need the support of development researchers. Accordingly, to contribute at the policy level, development activists need the help of rigorous research based findings. Especially, battling the threat of climate change, contemporary world economic crisis, the escalation of conflicts and violence throughout the Arab world and the possibility of new wave of democratization have made the role of development policy research very important. Working effectively and efficiently in development arenas is not an easy task. Conversely, it is extremely difficult to formulate an effective and sustainable development program or policy without deep contextual investigation and research on the specific development issues. Therefore, development policy research has a great scope to influence the authority to introduce, change or discard policy initiatives and thus to contribute to the sustainable global development. However, the issues and concepts regarding development are always contextual and the terms are value laden. Consequently, development policy researchers need to have adequate knowledge and proficiency to underpin the intellectual challenge in formulating appropriate development policies. Therefore, an understanding of the epistemological and ontological foundations of development research is an imperative. The main objective of this paper is to discuss about the epistemological and the methodological issues that development policy research entails. More specifically, this paper will highlight the role of applied research in development arenas, epistemological issues and boundary setting, methodological issues and the rigour that such research demands.

The Role of Applied Research in Development Arenas:

In accordance with the expressed priorities and needs of decision-makers, the role of applied research in development arenas is extremely valuable since it promotes knowledge-based policy formulation. Through policy research, development researchers tend to evaluate the working policies and the existing knowledge, tries to mitigate the inherent problems of the current development policies, and strive to recognise the policy demand of a particular group of beneficiaries or stakeholders. Thus they incline to help building explorative knowledge for the decision makers in various development fields of their interest. Policy research often attempts to excavate the deeper causes of social problems and opportunities by considering the short-term and long-term consequences of the policy implementations. The nexus between research and policy is essential to avoid policy failures. Nevertheless, as discussed earlier, this is a vigorous and complex task, due to the different interwoven and contextual development perspectives. Thomas (1998: 5) claims that, in a way each development policy externally influences and intervenes in the social process by working towards broad social goals. However, these social goals are subject to value based conflicts, where multiple actors and agencies are involved (Thomas, 1998: 4). Thomas supports his argument by mentioning an example where the conflicts of desirable development goals emanate from different perspectives between the Greenpeace and the Botswanas government (Thomas, 1998: 5). He also mentions that the development organisations and the donor agencies often tend to propose policy papers as prescription. Differing with this view he asserts that, it is essential to understand that policy research is not just about gathering information, but a continuously creative and developing process of conceptualization and re-conceptualization (Thomas, 1998: 13). Sound development policy should be developed by and with the people in a participatory way, which enhances the local capacity to implement the initiatives as per the policy recommendations. Therefore, Thomas (1998: 6) emphasizes developing policy research in considering public action. He also states that even [...] investigations are part of the policy process (Thomas, 1998: 5). From Thomass (1998: 7) perspective, it is clear that policy researches demand no less rigour than academic research. Hence, it is important to understand the actors perspectives in development policy.

Understanding the actors involved in the process is also crucial to conceptualise the process of both policy research and investigation. Again, there are multiple actors, including government, donor agencies, development organizations, political parties, civil society and the targeted local communities, involved in the development process. This complex and multifarious process demands deeper and wider investigation. Subsequently, such rigorous investigation demands adequate resources and time; but often the reality compelled the researcher to complete it rapidly. Thomas (1998: 11) suggests that policy research strives to influence development policies or public actions, with limited logistic support and time. Here, he also ascertains about the risks associated with unduly rapid research. He terms rapid research as finding out fast and claims that it is often dangerous as there are possibilities to produce shallow reports to be referred as rubbish (Thomas, 1998: 9). Sometimes few rapid research techniques may be used inefficiently, or it may be channelled into the researchers biased way of thinking. Therefore, such policy research initiatives often invite questions regarding their validity and reliability. Hence, the issue of epistemology, which creates the level of conscience, plays crucial role in development policy research.

Epistemological Issues and Boundary Setting Epistemology:

Epistemology refers to the study of knowledge and was introduced by the Scottish philosopher James Frederick Ferrier (Wikipedia, 27/10/12). The term is derived from Greek words, episteme and logos meaning knowledge and reason respectively. Epistemology is a branch of philosophy that focuses on the knowledge-gathering process and developing new theories. It deals with the methods of theory building and its validation. According to Blaikie epistemology is the possible ways of gaining knowledge of social reality, whatever it is understood to be. In short, claims about how what is assumed to exist can be known (Blaikie in Grix, 2002: 177). In order to conduct an effective development policy research, knowledge of the ontological and epistemological assumptions is essential. It helps to comprehend the interrelationship of the key components of research including methodology and methods and to avoid confusion about theoretical perspectives to social phenomena. Here, according to Grix, ontology is about what we may know, what is the reality, and epistemology is about how we come to know what we know (Grix, 2002: 177). 5

In the philosophy of social science, there are two strong epistemological approaches, namely positivism and non-positivism. For knowledge creation, positivist epistemology refers to empiricism that claims knowledge from experience; whereas, non-positivist epistemology refers to constructivism or instrumentalism that involves perceptions as instruments to explain the reality (Sumner & Tribe, 2004: 3). The ontological stance of positivism or empiricism suggests that the reality exists and universal truths are observable (Sumner & Tribe, 2004: 4). [] [E]mpiricism is defined as a research approach predicated on an observation based model for determining the truth or validity of knowledge claims in which brute data are assigned a special role (Kanbur and Schaffer, 2007: 185). In positivist approach, brute data is numerical in nature and has single meaning. It is subject invariant and testable. Hitherto, the universal truth is sought in an objective and independent manner from a distance, without being influenced by the values and perceptions of the researcher. Thus, in positivism, predominantly quantitative data is sought, which is measurable in providing evidence. It considers language as a neutral vehicle of the actual sense (Sumner & Tribe, 2004: 4). On the contrary, non-positivism or constructivism embraces different ontological stance, which suggests that reality or truth is constructed by social phenomena and exists only through our experience. Non-positivism argues that multiple realities exist which are intangible, local and specific in nature (Sumner & Tribe, 2004: 5). In this approach, it is claimed that the concept of single truth is meaningless. Traditionally non-positivism deals with descriptive and explanatory qualitative data, in search of a subjective meaning and explanation of the reality, based on discourse (Sumner & Tribe, 2004: 5). The non-positivist approach includes interpretivism, hermeneutics, and social constructionism. Defenders of interpretivism claim that human action can be understood (Verstehen) through the human sciences. They also argue that human or social action is inherently meaningful, and grasping the meaning of constituents of that action is a must to understand the human action (Schwandt, 1994: 191). In defining empathic identification, Dilthey claims that, understanding the subjective consciousness or intent of the actor is essential to comprehend the meaning of human action (Dilthey in Schwandt, 1994: 192). In addition, Schwandt explains, It is an act of psychological reenactment- getting inside the head of an actor to understand what he or she is up to in terms of motives, beliefs, desires, thoughts, and so on (Schwandt, 1994: 192). Another way of interpretivism, the phenomenological analysis is mainly eager to understand the constitution of intersubjetive social life world through the synthesis of conversation and 6

interaction. In describing the other pattern of interpretivism, the language games, Schwandt (1994: 192) claims that, there are certain rules in language also, like other games, which make the human action meaningful. Philosophical hermeneutics claims that the understanding is not a rule-governed procedure, rather understanding is interpretation. Rejecting the views of interpretivism, Gadamer clarifies that understanding is a inherent structure of our life experience. He also claims that clear understanding requires the engagement of ones sociohistorically inherited biases (Gadamer in Schwandt, 1994: 194). According to Bernstein, philosophical hermeneutics suggests that understanding of the human action is achieved through participative, dialogic and logical conversation. Moreover, the meaning of reality is produced through the conversation and language, not by the analysis of an interpreter (Bernstein in Schwandt, 1994: 195). According to Schwandt (1994: 197), social constructionists believe that knowledge is the construction of our active mind, and consequently we are all constructionists. It argues that we do not necessarily represent or discover the knowledge; rather we construct it through various social interactions against a backdrop of historical and socio-cultural dimensions. Potter claims that the real world is constituted by the way of people talking, writing and arguing about it (Potter in Schwandt, 1994: 197).

Boundary Setting:
Boundary setting is a metaphor that refers to the overall conceptual framework of a development research, an outline to guide the researcher in reaching the targeted goal. The boundary is the conceptual line that demarcates what is inside the system from what is outside the system, in its environment. In other words it separates what is directly relevant for a particular purpose from what is not of direct relevance or interest (Blackmore & Ison, 1998: 41). For development policy researchers, boundary setting is essential as it assists to simplify the working procedure, to emphasize the important tasks ignoring others. It is particularly helpful in delimitating the geographical boundary, identifying the stakeholders, distributing and managing the responsibilities, and anticipating the expected impacts of the intervention. Even employing the concept of boundary can guide and shape the thinking process of the researcher (Blackmore & Ison, 1998: 41-42).

Conceptual boundaries could be distinguished as open or closed systems. From the empiricist epistemological position, a research project could be very specified, fixed about the methodological procedure, tolerating no deviation from the anticipated outcome. This could be seen as a closed system with fixed boundaries. Whereas, a social constructionist approach could offer more flexible way to conduct a research in terms of methodology, targeted outcome, and could be related with an open system with a permeable boundaries. In setting boundaries, Blackmore and Ison (1998: 43) suggest about four set of questions as a checklist, such as, questions regarding the source of motivation, control, expertise and legitimation. However, Blackmore and Ison (1998: 46) also advice to remain careful about the trap of boundaries, which could unnecessarily constraint and narrow the thinking process and provide a false impression that the researcher could position himself outside the system in maintaining objectivity. Therefore, they prefer interdisciplinary or holistic approach in project designing. Blackmore and Ison (1998) claim that various social phenomena, such as, values, beliefs, and perceptions are often dynamic. So development policy researchers, in the process of finding out fast need to consider boundary setting as an iterative process and remain conscious about keeping the boundaries open. This will eventually help to conceptualise and reconceptualise the investigation process in a changing context.

Choice of Research Method:

Those who wish to succeed must ask the right preliminary questions (Aristotle, Metaphysics II). In development policy research, selecting the suitable research methodology is a challenging task. According to Bechhofer, ... the research process is not a clear-cut sequence of procedures following a neat pattern but a messy interaction between the conceptual and empirical world, deduction and induction occurring at the same time (Bechhofer, 1974: 73). Choice of research method largely depends on the research question, the type of information needed to accomplish the main research objectives, and at the same time it is heavily influenced by the researchers epistemological stance, as well as, by other practical factors. These practical factors often include the availability of funds and ease of access to data, rather than philosophical considerations. According to Potter and Subrahmanian (1998: 19), different types of policy measures need different types of information, as well as, different types of research questions, depending on which research methods are selected. Again, evolving policy process may require the adjustment of research method accordingly. 8

Potter and Subrahmanian (1998: 21) also suggest differentiating the research questions according to their dimensions. The normative or evaluative question requires a mixture of evidence and subjective judgement. And a descriptive or explanatory question requires separate out the evidence from the judgement. Potter and Subrahmanian (1998: 23) claim that there are four broad questions -- what if, why, how and what -- in policy research. To answer, each of these questions requires different types of research methods. Accordingly, answering the questions with what if require experiment, scenarios, multi-variable modelling, qualitative interviews; and questions with why require case studies, experiments and semi-structured interviews. Sequentially, questions with how need simple modelling and semi-structured interviews; whereas, questions with what need surveys, archives and administrative statistics to find the answer. From the epistemological perspective, generally quantitative methods are leaned towards positivist approach; whereas, non-positivist or social constructionist approach is more associated with qualitative methods. According to Sumner and Tribe (2004: 3), epistemology provides the credibility which legitimises knowledge through a rigorous and full range of methodology. In positivist approach, predominantly objective quantitative data is preferred which could be verified through quantitative testing of hypothesis. On the contrary, constructivist approach associated with subjective and discourse based qualitative data. In positivist approach, research is conducted in a specified methodological framework, where truth is not negotiated. Quantitative data are collected to construct statistical model in an attempt to explain the reality. Usually it collects data through surveys and use descriptive and inferential statistics to analyze. On the other hand, in non-positivist approach qualitative methods are flexible with multidisciplinary bricolage. Here, more detail, explanatory and descriptive data are collected through some understanding process, such as, in-depth interviews, participant observation, discourse analysis, life history, and case studies, to interpret or construct the reality. However both the methods, quantitative and qualitiative, have some drawbacks. [] the quantitative analysis would not be very effective at describing the local politics at village level that led to the formation of the committee or the details pertaining to deliberations within it [] (Rao & Woolcock, 2003: 167). On the other hand, qualitative methods are subjective, unrepresentative and unreliable (Sumner & Tribe, 2004: 13).

Against this backdrop, Sumner and Tribe (2004: 14) argue that multiple methods are useful to corroborate and ensure validity, not providing proof but improving consistency across methods in a process of triangulation. Bryman (2004: 457) suggests that there are several ways to facilitate quantitative research through qualitative research, such as, providing hypothesis and aiding measurement. Even quantitative research can assist the qualitative research through the selection of interview respondent. The multi-strategy research method is very challenging and demands efficiencies on both methodologies. In this context, rigorous efforts should be made not to mix the weakness of the methods.

Rigour in Development Research (perspectives on validity):

Rigour in development policy research denotes quality assurance. It is critically associated with being thoroughly systematic in the overall research process to ensure the validity. According to Sumner and Tribe (2004: 7), rigour is essential to distinguish between intentional misinterpretation for biased results and acceptable or acknowledged bias in values or data. Rigour is crucial to validate the research findings. Maxwell (2005: 106) refers to the correctness or credibility of a research outcome by the validity. In policy research, validity is essential to contextualise the knowledge claims and to convince the audience about the outcome. According to Maxwell (2005: 105), [] validity depends on the relationship of your conclusions to reality, and there are no methods that can completely assure that you have captured this. Maxwell (2005: 105) claims that validity is relative, not absolute. The criteria of validity depend upon the epistemological perspective. White in Sumner and Tribe (2004: 12-13) argues that the traditional presumption that quantitative methods are more rigorous is not always correct. Rather different techniques are appropriate in different settings. Efficient synthesis between qualitative and quantitative methods could be more effective. Rigour is not related to the technique; rather the application of the technique is more important. It is assumed that validity is not a property of designs or methods, rather related to inference (Shadish et al., 2002: 34). In quantitative research, there are four typologies of validity. These are statistical conclusion validity, internal validity, construct validity, and external validity (Cook & Campbell in Shadish et al., 2002: 37). Each types of validity are concerned with different types of validity threat. Threats to statistical conclusion validity is related to low statistical power, unreliability of measures and treatment implementation, restriction of range, 10

heterogeneity of units etc. (Shadish, et al., 2002: 45). And, threats to internal validity is related to selection, history, maturation, regression, etc. (Shadish, et al., 2002: 55). Accordingly, there are certain threats to construct validity and external validity, which are construct confounding, mono-method bias, and by interaction of the causal relationship with units and context-dependent meditation respectively (Shadish, et al., 2002: 73, 87). However, there are different techniques to deal with these threats, which are randomization, increasing the sample size, triangulation, homogeneity of units, and monitor treatment, etc. (Shadish, et al., 2002). The validity threats in qualitative research come from researchers bias and effect of the researcher on the individuals studied, which is called reactivity (Maxwell, 2005: 108). In contrast, Maxwell (2005, 110-113) also suggests some techniques to deal with these validity threats, which are intensive, long term involvement, rich data, respondent validation, intervention, quasi-statistics, comparisons etc.

Conclusion:
Development policy research, in the context of developing countries, often needs vertically in-depth analysis to find out the root cause of the problem and the causal relationship with other socio-cultural phenomena. Hence, such policy research demands the non-positivist or social constructionist approach from the epistemological stance. Besides, an effective and quality research depends upon the researchers efficiency and integrity. Therefore, thorough knowledge on epistemological issues, boundary settings, research methodology and capability of linking all these issues are extremely necessary.

11

References:
Bechhofer, F. (1974) Current approaches to empirical research: some central idea in: Rex, J. (ed.) Approaches to sociology, Routledge, London.

Blackmore, C. and Ison, R. (1998) Boundaries for Thinking and Action, in: Thomas, A., Chataway, J. and Wuyts, M. (eds.) Finding out Fast: Investigative Skills for Policy and Development, London, Sage Publications, 41-66.

Bryman, A. (2004) Combining Quantitative and Qualitative Research in: Bryman, A. (ed.) Social Research Methods, Oxford, Oxford University Press, 451-465.

Chambers, R. (2004) Ideas for development: reflecting forwards IDS Working Paper No 238, University of Sussex, UK.

Grix, J. (2002) Introducing Students to the Generic Terminology of Social Research Politics, 22(3): 175-186, http://onlinelibrary.wiley.com/doi/10.1111/1467-9256.00173/pdf (last consulted: October 28, 2012)

Kanbur, R. and Shaffer, P. (2007) Epistemology, Normative Theory and Poverty Analysis: Implications for Q-Squared in Practice, World Development 35(2): 183-196

Maxwell, J.A. (2005) Validity: How Might You Be Wrong? in: Maxwell, J.A. (ed.) Qualitative Research Design: An Interactive Approach, London, Sage Publications, 105-116.

Potter, S. and Subrahmanian, R. (1998) Conceptualizing Policy-Related Investigation: Information Needs and Policy Change, in: Thomas, A., Chataway, J. and Wuyts, M. (eds.) Finding out Fast: Investigative Skills for Policy and Development, London, Sage Publications, 19-24. 12

Rao, V. and Woolcock, M. (2003) Integrating Qualitative and Quantitative Approaches in Program Evaluation, in: Bourguignon, F.J. and Silva, L.P.d. (eds.) The impact of economic policies on poverty and income distribution: Evaluation techniques and tools, New York, Oxford University Press, 165-186.

Schwandt, T.A. (1994) Three Epistemological Stances for Qualitative Inquiry: Interpretivism, Hermeneutics and Social Constructionism, in: Denzin, N.K. and Lincoln, Y.S. (eds.) Handbook of Qualitative Inquiry, 2nd edition, London, Sage Publications, 189214.

Shadish, W., Cook, T.D. and Campbell, D. (2002) Experimental and Quasi-experimental Designs for Generalized Causal Inference, Boston, Houghton Mifflin.

Sumner, A. and Tribe, M. (2004) The Nature of Epistemology and Methodology in Development Studies: What Do We Mean by Rigour? DSA Annual Conference, Bridging research and policy, Church House, London.

Sumner, A. and Tribe, M. (2008) International Development Studies: Theories and Methods in Research and Practice, London, Sage Publication Ltd.

Thomas, A. (1998) Introduction, in: Thomas, A., Chataway, J. and Wuyts, M. (eds.) Finding out Fast: Investigative Skills for Policy and Development, London, Sage Publications, 1-18.

http://en.wikipedia.org/wiki/James_Frederick_Ferrier (last Consulted: October 28, 2012)

13