Research Methods

The Critical Literature Review (CLR)

www.colinwatsonleeds.co.uk These slides are available on my website

Teaching and Learning Outcomes

At the end of this session you will understand:

Why the critical literature review must inform research project problem definition, design decisions and justification Critical analysis, critical thinking and synthesis Uses of literature in developing structure/supported argument Sources of literature available to support a research project How to plan effective and efficient literature searches How to record and catalogue literature searches How to document and present a critical literature review

Recall

The process of doing a dissertation

Phase 1: Phase 2: review Phase 3: Phase 4: Phase 5:

Getting Started Writing a Critical Literature

Devising Research Questions Research Process Writing it up.

Note: it is possible to loop back at various stages.

Note

For the Research Proposal you set out the first 3 steps and state your PLANS for Phase 4, regarding data collection & analysis. For the final year dissertation you will cover all 5 steps! The Critical Literature Review (CLR) is a vital phase in these processes.

Note

A CLR forms Section 2 of the Research Proposal max. 40 marks (See module handbook) An extended CLR typically forms one of the earlier chapters of the final year dissertation The academic journal articles that we recommend provide embody CLRs and act as models of good practice in How to write them (to differing degrees).

Seminar Pre-reading
Required (Directed)
Bryman&Bell (ch.23) Saunders (2003: Chapter 3 & Case 3) Jankowicz (2002: Chapter 8) Recommended (Self-Directed) Creswell (2003: Chapter 2) Burton (2000: Part III)

Questions a Literature Review can answer

What are the major issues and debates about the topic? What are the key sources? Who are the key authors? Who influenced them? What are the key concepts, models, principles, theories and ideas? What are the main questions and problems that have been addressed to date?

Questions a Literature Review can answer

What are the origins of the topic? How has it been defined?

Official sources; academic definitions PESTLE?

What is the context to the topic?

How is knowledge on the topic structured and organised?

A hierarchy with seminal articles referred to time and time again ?

Questions a Literature Review can answer

What strategies/designs/methods of data collection have past researchers used? Might they influence your choices? What ethical or access issues did researchers come across? Can you learn from this? Did they suggest opportunities, so that you can take over where they left off?

The Critical Literature Review Why is it necessary?

Sets the context of your research topic Identifies the key texts/ideas Demonstrates your knowledge and critical understanding of the topic Refines parameters of your study Justifies need to conduct a study

The Critical Literature Review What does it do?

It provides justification of your central research question and research objectives/purpose It helps you refine your central research question and research objectives/purpose It highlights research possibilities that have been overlooked implicitly in research to date To discover explicit recommendations for further research

Emotional labour: An example of an under-researched area

Previous writers have adopted

Examples of
Lit. Reviewed

a feminist epistemology Legitimately borrowing ideas and concepts from another field

Wider
Significance hinted at

Source: Constanti and Gibbs, 2004. HE teachers and Emotional Labour (Emerald)

The Critical Literature Review What does it do (cont.)?

It will help you to avoid simply repeating work that has been done already It will hep you to integrate both existing research within your central research question and research objectives/purpose

The Critical Literature Review Be Critical!

Simply describing the literature is of NO value

Your review MUST not simply describe/repeat what we already know It MUST critically analyse in relation to your research area How do key ideas relate together and relate to your research area Assess the credibility & value of previous work Whats the justification for your research? Analyses critically! Think critically! Synthesise

The Critical Literature Review Sources of Literature

Use a wide range of Resources Remember we want depth and quality That means empirical, published, peer reviewed Avoid superficially glossy but nonetheless trivial non-academic sources Contact your subject specialist tutor Download and work through S.Watsons Students Guide to using electronic journals at LMU (from the e-searching page on my site)

Literature sources available to you

PRIMARY e.g. reports, theses, emails, conference Reports, company reports, some govt. publications, Unpublished manuscript sources SECONDARY e.g newspapers, books, journals, internet, Some govt. publications
TERTIARY e.g indexes, abstracts, catalogues, encyclopaedias, dictionaries, bibliographies, citation indexes.

The Critical Literature Review Planning The Review

Locating and Including All The Key Literature isnt Easy But doing so is expected at UG level

Plan; be Efficient, be Effective Many searches, many approaches Athens etc, Internet Searches, Learning Centre Bound Copies, Visit British Library, Conference Proceedings Network with and Speak to Authors, other Researchers, Practitioners, Professionals, Your Supervisor Look for Published Review Articles Use Reference Lists to guide your searches Search abstracts Focus on key Authors If you can, go to conferences, local research events

Searching for e-journals in Emerald

Find the A-Z Index and scroll To E for Emerald Fulltext

Searching for e-journals in Emerald

Emerald Fulltext
You will need ATHENS password to enter

Searching for e-journals in Emerald

A search on The term qualitative Yeilds 8,709 Items. Many articles can be viewed In pdf format

Searching for e-journals in Emerald

Some software
allows you to copy and paste sections Into Word or Articles Can then be Read on screen or saved To disk/drive or printed out

Powerpoint Scroll to the

Foot of the page to find the full journal reference

Advanced searching Common link terms that use Boolean logic

Link term Purpose Example Outcome

AND narrows search Recruitment and interviewing and skills Only articles containing all three key words selected

Eg PUBLIC AND RELATIONS 7,845 hits using ADVANCED SEARCH in EMERALD

Advanced searching Common link terms that use Boolean logic Link term Purpose Example Outcome OR widens search Recruitment or selection Articles with at least one key word selected

Eg PUBLIC OR RELATIONS 27,015 hits using ADVANCED SEARCH in EMERALD

Advanced searching

Common link terms that use Boolean logic

Link term Purpose Example Outcome NOT excludes terms from search Recruitment not selection Selects articles containing the key word recruitment which do not contain the key word selection

Eg PUBLIC NOT RELATIONS 13,593 hits using ADVANCED SEARCH on EMERALD

Advanced searching

Common link terms that use Boolean logic

Link term Purpose Example Outcome *(truncation) Uses word stems to pick up different words Motivat* Selects articles with: Motivate Motivation Motivating

Eg RELATION* 34,446 hits using ADVANCED SEARCH on EMERALD

Advanced searching

Common link terms that use Boolean logic

Link term Purpose Example Outcome ?(wild card) Picks up different spellings behavio?r Selects articles with: Behaviour behavior

Eg labo?r 8,130 hits using ADVANCED SEARCH on EMERALD

The Critical Literature Review Cataloguing and Documenting

Catalogue from the start Write from the start, progress in stages

Write your review in note form as you go Apply the Lit. Review Template on each article Find relationships draw them on a Reference Tree or Mind Map Meet with your tutor to discuss your Proposal (see module calendar) Develop a logical progression of critical arguments, links/synthesis, evidence Use software to catalogue revisions, comments and build a reference list as you go, e.g. ENDNOTE; MS ACCESS.

Endnote: bibliographic software

Allows you to search many

libraries online
and build up reading lists very quickly indeed

and allows you to

save your searches in easily retrievable form and to print out the lists in the Harvard (and many other) Referencing system Visit Endnote.com

Writing up your CLR Use Structure, Be argumentative!

Your writing style must be about impact and clarity You must Justify your Proposed Research Look closely at the articles provided and any you find independently How do they create an impact? How do they present arguments and supporting evidence; Clarity How do they reference the literature? Look for the way academics use roadmaps, signposts, synthesis, summaries and conclusions Finish with your conclusions; Impact

TODO for next week

Make a note of this

Read the entire module handbook Read and use S.Watsons Guide to e searching (available on my website) Find a suitable article B and bring a print out along to the class for the tutor to see it a week before the presentation. As audience members read the article A that forms part of next weeks presentation (see handbook for details) and be prepared to take part in discussions. From now on, apply the Literature Review Template to all academic articles you come across, (not just for the presentation) and save the results systematically.

The Critical Literature Review Outcomes of the Review

Skills Developed

distinguishing what has been done from what needs to be done discovering important variables relevant to the topic synthesising and gaining a new perspective identifying relationships between ideas and practice establishing the context of the topic or problem rationalizing the significance of the problem

The Critical Literature Review Outcomes of the Review (Contd.)

Skills Developed (Contd.)

enhancing and acquiring the subject vocabulary understanding the structure of the subject relating ideas and theory to applications identifying the main methodologies and research techniques that have been used placing the research in a historical context to show familiarity with state-of-the art developments

The Critical Literature Review Exercise

Case 3 (Saunders, 2003)
1. 2.

3.

4.

How do you think Geoffs view of the purpose of undertaking a literature review changed? What specific skills did Geoff develop in the course of preparing the review? Why do you think Geoffs friend recorded everything she had read in a database rather than just those articles she felt she would definitely be of use in her dissertation? How could Geoff have made better use of Internet search engines in researching his topic?

Appendices

Questions a literature review can answer

What are the major issues and debates about the topic?
What are the key sources? What are the key concepts theories and ideas? What are the epistemological and ontological grounds for the discipline? What are the main questions and problems that have been addressed to date?

Literature search and review on your topic

What are the political standpoints?

How is knowledge on the topic structured What are the origins and and organised? definitions of the topic?

Applying the Lit. Review Grid

For any academic article create fields in a database for key reference information:

Author(s) Date Journal Pages Location of the article

And then set up fields for the following

Applying the Lit. Review Grid

Area, Topic, Purpose What is the broad research area? What is the topic? Is it clearly defined? Does the author show how it fits within the research area? What is the overall purpose of the research?

Applying the Lit. Review Grid

Background & Context Into what background and context does the research fit? Provide examples from PESTLE where relevant (Political, Economic Legal, Socio-Cultural, Technological, Legal, Ethical). What is changing in the real world context or environment to make this research important, if not urgent?

Applying the Lit. Review Grid

Research Question(s) What are the general research questions? What are the specific research questions? Is it clear what data are required to answer each question?

Applying the Lit. Review Grid

Literature Review in the article What literature is relevant to this study? (provide authors, dates) What is the relationship of this study to this literature? In particular can you find where the author(s) refers to :

A point of view that they agree with, accede to,or wish to defend? A new point of view which they seek to propose? A point of view which they concede has certain merits, but which needs to be qualified in some way? A point of view which needs to be reformulated such that a new version makes for a better explanation?

Applying the Lit. Review Grid

Literature Review in the article In particular can you find where the author(s) refers to :

A point of view which needs to be reformulated such that a new version makes for a better explanation? A point of view which they dismiss on account of its inadequacy, irrelevancy, or incoherence? Authors try to reconcile two positions, which seem at variance, by appeal to some third principle? Gaps in the literature.

Applying the Lit. Review Grid

Literature Review in the article (Conceptualisation) What are the key concepts referred to by each of the authors? How do they differ in their useage of these terms? Where do they agree? Has anyone attempted to fit them into a model or framework? Definitions borrowed; definitions created?

Applying the Lit. Review Grid

Epistemology, Perspectives Is there a particular perspective behind the research, e.g. Positivist, Interpretivist, Feminist, Critical Theory, other?

Applying the Lit. Review Grid

Substantive Theory What is the role of theory in the study? Do they make use of any recognised Grand Theory? Is the purpose description, or explanation, or some other purpose? Are new theories generated? Are old theories tested? If the purpose is theory testing (verification), what are the hypotheses and what is the theory behind them? Does the article make use of Models? Frameworks? Principles Are they tested?

Applying the Lit. Review Grid

Research Strategy Does the study use quantitative or qualitative strategies? or a hybrid of both?

Applying the Lit. Review Grid

Research Design Experimental? Cross-sectional? Longitudinal? Case Study? Comparative? Ethnographic? Survey? Discourse Analysis? Other?

Applying the Lit. Review Grid

Methods of Data Collection and Analysis Does the research rely on secondary data, primary data, or a mixture of both? Who, or what is studied? From whom is data collected? What is the sampling plan, sample size, and the basis for sample selection? How was the data collected? What instruments (if any) were used e.g. questionnaires. Were existing instruments used for the study, or were new ones developed? If existing instruments were used, what is known about them?

Applying the Lit. Review Grid

Methods of Data Collection and Analysis (contd) What data collection methods were be used, e.g. experiments, interviews, secondary data, questionnaires? What, in general are the advantages and disadvantages of the particular methods chosen? How did these methods affect the quality of the data (reliability and validity issues)? Does the study tend towards a) pre-specified questions, a tightly structured design and pre-structured data, or towards b) general open-ended questions with loose design and data not pre-structured? How was the data analysed? What computer packages if any were used?

Applying the Lit. Review Grid

Consent, Access, Ethics What issues of consent were involved in carrying out the study and how were they dealt with? What issues of access were involved in carrying out the study and how were they dealt with? What other ethical were involved in carrying out the study and how were they dealt with?

Applying the Lit. Review Grid

Research reporting Does the article constitute a logical and coherent argument, with interconnected sections do the parts fit together? Is the research reporting clearly written? Well organised, easy to follow? Does the author point out the limitations of his/her research? Is the research presented in an appropriate academic form, with proper referencing and bibliography? Does the author suggest any practical implications, for stakeholders and others that follow on from the research?

Applying the Lit. Review Grid

Research Process Provide a stage by stage diagram (on a separate sheet) that maps out the steps actually taken by the researcher or research team. How does this compare to the idealisations of the process give in the textbooks eg B&B ch 3 (main steps in Quants) ; B&B ch.13. (main steps in Qual.). Were any important steps missed out, e.g. piloting, pre-testing?

Applying the Lit. Review Grid

Research opportunities Does the research suggest areas or topics that need to be researched in the future? Is replication possible? Given your responses above, what are there be implications for your own Research Proposal?

The Critical Literature Review Its Central Importance

2. Informs

Research Problem

Literature Review

Research Design

Research Findings

1. Informs

3. Reforms 4. Contributes to

The Critical Literature Review Cataloguing and Documenting

The Critical Literature Review defined: The selection of available documents (both published and unpublished) on the topic, which contain information, ideas, data and evidence written from a particular standpoint to fulfil certain aims or express certain views on the nature of the topic and how it is to be investigated, and the effective evaluation of these documents in relation to the research being proposed. (Hart, 1998)