Evaluation systems and evaluation practice: towards a framework for research evaluation. Maria nedeva, MIoIR the university of manchester warsaw, presents a framework. Broad conditions for evaluation what to watch for science / innovation systems.
Original Description:
Original Title
20110114 Dr Maria Nedeva - Evaluation Systems and Evaluation Practice
Evaluation systems and evaluation practice: towards a framework for research evaluation. Maria nedeva, MIoIR the university of manchester warsaw, presents a framework. Broad conditions for evaluation what to watch for science / innovation systems.
Evaluation systems and evaluation practice: towards a framework for research evaluation. Maria nedeva, MIoIR the university of manchester warsaw, presents a framework. Broad conditions for evaluation what to watch for science / innovation systems.
research evaluation Maria Nedeva, MIoIR the University of Manchester Warsaw, December 3, 2010 Outline of presentation Science/innovation systems and evaluation systems Evaluation framework Evaluation practice Broad conditions for evaluation What to watch for Science/innovation systems Science/innovation systems Research spaces and fields Research organisation(s) FO1 FO2 FO3 KC1 KC2 KC3 KP2 KP1 KP3 KEY: KC = knowledge community KP = knowledge pool FO = funding organisation KCs interactions with different research spaces Research space Research field Research fields and policy spaces Research space is a policy space Relationships between regulators, funders and performers of research Funders can be layered e.g. Government organisations, executive agencies etc. National Evaluation System: definition Research evaluation systems are organised sets of procedures for assessing the merit of research undertaken in publicly funded organisation that are implemented on a regular basis, usually by state or state-delegated agencies...[NESs vary] in terms of their frequency, formalisation, standardisation and transparency. R. Whitley, 2007 (The Changing Governance of the Sciences) National Evaluation System Permeates research spaces and fields Institutionalised: rules, responsibilities, procedures, consequences Repeated National Evaluation System: organisation Who commissions Who organises Who carries out Who applies the results Example the RAE in the UK Organisation of science and evaluation systems Answers depend on the organisation of the research space (national) As a rule of the thumb: Political organisations commission (Ministries, Govnt Departments) Intermediaries (agencies) organise Multiple actors carry out (contractors, panels, experts, in- house specialists) Political organisations and agencies apply Organisation of science and evaluation systems NESs are conceived, developed and governed in the Research Space They can target social phenomena in research spaces and research fields Generally evaluations are about performance management: of funders, research organisations, research teams, researchers and knowledge Evaluation as management practice Information Judgement Action Information About what? Object Types of information How collected? Methodologies Sources Who collects? In-house Outsourced Judgement Criteria Standards Multiple criteria Layered Depend on dominant ideological beliefs Action Funding decisions Regulation Change Closure Evaluation practice Distortions Indicator based Peer review based Evaluation methodology(ies) Broad conditions of evaluation Fit between the overall and localised evaluation practices Time-lines Sharing of practice Experience Comparison Relying on others Pitfalls Over- evaluation Distortions of the science system knowledge management Assumptions Thanks for listening! Any questions? Maria.Nedeva@mbs.ac.uk