Research Methods Course Design and Teaching

Workshop 2: Teaching Critical Reading and Writing Skills

Mike Wallace

Aims of the Workshop Series

1. Build management research capacity by supporting academics in their secondary role as trainers 2. Promote mutual support networks for sustainable development 3. Inform practice through inputs, networking (substantive outputs)

Aims of Workshop 2
1. Share evidence of students learning experience, implications for research methods course design and teaching 2. Review teaching to evaluate and develop arguments 3. Explore structured approach to teaching critical reading and writing skills 4. Support planning and introduce preparation for Workshop 3

Todays Sessions
1a. Learning through research methods courses (aim 1) 1b. Teaching evaluation and development of arguments (aim 2) 2. Structured approach to teaching critical reading and writing skills (aim 3) 3. Engagement with literature, reviews, integration in dissertation/thesis (aim 3) 4. Practical implications, planning, preparation (aim 4)

(Workshop 1) Nature of Learning

1. No adult starts with nothing 2. Talking and writing about performance and acquiring skills are not the same as performing 3. Learning skilful performance entails trying it out in the real situation, so preparation can have limited impact 4. Transferable skills cannot be transferred without further learning

Implications for Learning Support

1. Learning starts from where the learner is 2. Articulating principles and practising skills are not sufficient 3. Integrating knowledge and skills requires real experience 4. Learning is never complete, so it needs fostering as a habit

Stages in learning
Existing performance of management research tasks

Components of Learning Support

Previous experience Task performance Diagnosis From research, professional knowledge, theory, policy From personal theories of action General principles Specific information necessary for tasks General research and personal development Specific for research tasks Specific for learning activity Range of tasks Specific tasks

Challenge to existing performance Increased awareness, and justification for development Rationale for development Practical ideas on how to develop Practice in developing performance Improved performance of management research tasks Practical information Skills Critical understanding

Integration into skilful performance

Group Exchange: Students Learning Experience

(Preparatory task to interview a high-achieving and a low-achieving research student about the quality of their learning experience as participants in research methods training courses)

1. Those who have completed this task:

report on what you have learned with implications for improving provision 2. Identify one overall implication for improving research methods training

(Workshop 1) The Logic of Enquiry

Two-way critical academic discourse: as a reader, evaluating others attempts to communicate and convince through developing their argument as a researcher and writer, developing ones own argument that will communicate with and convince the projected audience

Components of an Argument
the conclusion rests on claims to knowledge, assertions that something is, or should be, true claims to knowledge are backed by evidence, the warrant for the conclusion evidence varies, e.g. from literature / own work: - research findings - professional experience - a definition of a theoretical idea

Promoting the Development of a Critical Frame of Mind

1. How are your PG students taught to engage critically with literature, to evaluate arguments and to develop their own argument? 2. Any problems or issues to share?

Critical Reading & Self-Critical Writing

The book is built on two key ideas (p vii-viii): The logic of enquiry - two-way constructively critical academic discourse involving the evaluation and development of arguments Arguments comprise a conclusion (claims to knowledge) and a warrant ( evidence backing these claims to knowledge

Complete the exercise on page 3 of the handout (book p 12-13) How may learning from the parallel between critical reading and selfcritical writing be promoted?

Three-Part Book Structure (p viii-x)

1. Getting started on critical reading and self-critical writing 2. Developing a mental map for navigating the literature, analysing texts in depth, writing critical reviews of them 3. Structuring critical reviews of the literature, incorporating them into a dissertation, taking forward the skills of critical reading and self-critical writing in an academic career

Part 1 (Chapters 1-5): becoming a critical reader and self-critical writer

Insights and techniques Target written product

critical reading for self-critical writing critical choice of texts to read developing an argument Critical Synopsis Questions * Critical Synopsis of a text *

Critical Summary (one text) * Comparative Critical Summary (several texts) * * worked example

Part 2 (Chapters 6-10): developing an in-depth analysis

Insights and techniques Target written product

mental map Critical Analysis Questions * Critical Analysis of a text *

Critical Review (one text) * Comparative Critical Review (several texts) * * worked example

Part 3 (Chapters 11-14): constructing a critical review of the literature

Insights and techniques
structuring a Critical Literature Review via Critical Analyses and Critical Synopses * integrating Critical Literature Reviews into the structure of a dissertation transferring critical reading and self-critical writing skills to the professional academic sphere

Target written product

self-contained Critical Literature Reviews structure for a dissertation incorporating several Critical Literature Reviews academic journal articles incorporating a Critical Literature Review research grant proposals incorporating a Critical Literature Review peer reviews and assessment reports

* worked example

Key Ideas in Part 1: Frontline v Support Literature (Ch 2)

Main sources at postgraduate level: frontline refereed journal articles, books and chapters on theory, research, practice, policy Useful starting points: support textbooks, readers, encyclopaedias, popular and professional journals (Internet opportunities and dangers)

Critical Reading (Ch 3): Five Critical Synopsis Questions (p31-4) A Why am I reading this? B What are the authors trying to do in writing this? C What are the authors saying thats relevant to what I want to find out? D How convincing is what the authors are saying? E In conclusion, what use can I make of this?

(Preparatory task to choose a front-line refereed journal article or a chapter reporting on some aspect of management research, read and bring it)

Complete the Critical Synopsis form on page 4 of the handout for the research report article or chapter you brought (book p35) What are the implications for teaching students to use it (until asking the questions becomes a habit)?

Self-Critical Writing (Ch 4): Structure for a Critical Summary (p44)

warrant

Title Introducing the text, informed by answer to Critical Synopsis Question: A Why am I reading this? Reporting the content, informed by answer to Critical Synopsis Questions: B What are the authors trying to do in writing this? and C What are the authors saying thats relevant to what I want to find out? Evaluating the content, informed by answer to Critical Synopsis Question: D How convincing is what the authors are saying? Drawing your conclusion, informed by answer to Critical Synopsis Question: E In conclusion, what use can I make of this?

Comparative Critical Summary (Ch 5): Structure (p49)

warrant

Title Introducing the text, informed by answers to Critical Synopsis Question A for all texts Reporting the content, informed by answers to Critical Synopsis Questions B and C for all texts Evaluating the content, informed by answers to Critical Synopsis Question D for all texts Drawing your conclusion, informed by answers to Critical Synopsis Question E

Key Ideas in Part 2: Mental Map for Navigating the Literature (Ch 6-7)
One Two Three Four Five set of tools for thinking dimensions of variation amongst knowledge claims kinds of knowledge types of literature intellectual projects

One set of Tools for Thinking: the Key to the Mental Map
Concepts Perspectives Metaphors Theories Models Assumptions Ideologies

Two Dimensions of Variation of Claims

Degree of certainty
Low Low High

Low vulnerability weak claim, minimal generalization Moderate vulnerability weak claim, extensive generalization

Degree of generalization

Moderate vulnerability strong claim, minimal generalization High vulnerability strong claim, extensive generalization

High

Three Kinds of Knowledge (+ Key)

Theoretical knowledge
developed through systematic reflection

Tools for thinking

concepts

Research knowledge
developed through systematic investigation

perspectives metaphors theories models assumptions ideologies

Practice knowledge
developed through taking action

Four Types of Literature

1. research - systematic enquiries into
policy and practice by professional researchers or practitioners, results are made public 2. practice - by informed professionals who evaluate others practice and by practitioners who evaluate their practice 3. policy - policy-makers desired changes in practice (negative evaluation of present) 4. theoretical - ideas and models for interpreting and explaining practice

Five Intellectual Projects

Project UnderstandIng Rationale Understand through theory and research Value stance Disinterested Typical question What happens and why? What is wrong with what happens?

Evaluate through Critical evaluation theory and research Action Inform policy-makers through research and evaluation Improve practice through training and consultancy Improve own practice through evaluation and action

Critical

Positive towards How effective are policy and actions to improve improving practice practice? Positive towards How may this policy and programme improving practice improve practice? Critical of practice, How effective is positive about my practice? How improving may I improve?

Instrumentalism Reflexive action

Critical Synopsis and Critical Analysis Questions (Ch 8) (p92)

A 1. B 2. 3. C 4. Why am I reading this? What review question am I asking of this literature? What are the authors trying to do in writing this? What type of literature is this? What kind of intellectual project is being undertaken? What are the authors saying thats relevant to what I want to find out? What is being claimed that is relevant to answering my review question?

D 5. 6. 7. 8. 9. E 10.

How convincing is what the authors are saying? How far is there backing for claims? How adequate is any conceptual or theoretical orientation to back claims? How far does any value stance adopted affect claims? How far are claims supported or challenged by others' work? How far are claims consistent with my experience? In conclusion, what use can I make of this? What is my overall evaluation of this literature in the light of my review question?

(Preparatory task to choose a front-line refereed journal article or a chapter reporting on some aspect of management research, read and bring it)

Complete the Critical Analysis form for the research report article or chapter you brought (book Appendix 3) What are the implications for teaching students to use it (until asking the questions becomes a habit)?

Useful Sources of Assistance

Mental map Ch 6-7 Types of literature and potential limitations Table 7.1 (p81-2) Guidance on using Critical Analysis form (p93-99)

Critical Review Structure (Ch 10) (p117-8)

warrant

Title - keywords Introduction state purpose (review questions critical analysis Q1) Summary of research design purposes (Q2, 3), relevance to review questions (Q1), procedure Main findings and claims up to 5 claims relating to review question (Q4), range of contexts to which applied Evaluation of claims for context from which derived, applicability to other contexts (Q5-9) Conclusion overall evaluation, summary answer to review question (Q10) References

Key Ideas in Part 3: Literature Review Structure Based on Critical Analyses and Critical Synopses (p134-5)
Introduction purpose, justification, scope, limitations, signposting Based on answers to Critical Analysis Questions for central texts Based on answers to Critical Synopsis Questions for more peripheral texts Sections building up the warrant of your argument

Introduction to texts being reviewed Authors main claims relevant to review question Evaluation of authors main claims Final section setting out conclusion of argument Summary of evaluation answering review question References

2, 3 4 5-9 10

B C D E

Extended Structure for Multiple Review Questions (p139-40)

Introduction purpose, justification, scope, limitations, signposting Section addressing the first review question Based on answers to Critical Analysis Questions for central texts Based on answers to Critical Synopsis Questions for more peripheral texts Subsections building up the warrant of your argument for this section Introduction to texts being reviewed Authors main claims relevant to first review question Evaluation of authors main claims Final subsection setting out conclusion of argument Summary of evaluation answering first review question

2, 3 4 5-9 10

B C D E

Section addressing second

review question Based on answers to Critical Analysis Questions for central texts Based on answers to Critical Synopsis Questions for more peripheral texts

Subsections leading to conclusion of argument answering second review question

Section addressing

third review question Based on answers to Critical Synopsis Questions for more peripheral texts

Subsections leading to conclusion Based on answers of argument answering third to Critical Analysis review question Questions for

central texts

Conclusion for whole literature review References

Logic of Argument in a Dissertation (p153-5)

Introduction Literature review Research design Findings and discussion Conclusion
Central question Aims Review questions Research questions Instrument items Answer research questions, implications for literature

Evaluation of study, including achieving aims, contribution to answering central question

Identifying the Focus of the Enquiry

Introduction
Central question (substantive topic, general terms)

Aims substantive (to study topic in specific context) theoretical (conceptual tools to study topic) methodological (approach and methods to study topic)

Reviewing Literature to Inform the Empirical Work

Literature review
Review questions substantive aim (issues connected with topic) theoretical aim (framework and concepts to study topic)

Research questions (substantive topic in specific context)

Building towards Data Collection

Research design
Review questions methodological aim (approach and methods to study topic)

Instrument items answer research questions (substantive topic in specific context) employing concepts of theoretical framework

Presenting and Analysing Data

Findings and discussion
findings for instrument items answer research questions and contribute to achieving substantive aim

implications of answers to research questions for literature reviewed (substantive topic) impact of theoretical framework and data collection methods on the findings

Drawing warranted conclusions

Conclusion
1. summative claim about what found out 2. how far research questions answered and substantive aim achieved how far theoretical aim achieved by using the theoretical framework how far methodological aim achieved through the design and data collection instruments evaluation of theoretical framework, research design, overall success, and what learned from the experience 3. certainty with which findings from specific context can be generalized in contributing to answering the central question 4. recommendations for research, policy and practice supported by findings

A Logic Checksheet (Appendix 4)

Can be filled in gradually and revised as you go along Starts with the broad central question about the substantive focus First part encourages linkage between the central question, review questions, research questions and instruments in focusing the research

Electronic Resources
Three blank forms (which can be used as masters) can be downloaded from the website: www.sagepub.co.uk/wallace Critical Synopsis form Critical Analysis form Logic Checksheet form

Preparing for Workshop 3

1. Gather information on how students are taught quantitative methods, and identify any problems or issues with provision 2. Identify one particularly successful aspect of quantitative methods teaching, which you would be willing to share