Two-tier Multiple Choice Questions (MCQs) - How effective are they- A studentteachers perspective

IOSTE - NW Europe, June 20-21 2011 Reading, UK,

Alison Cullinane, Maeve Liston, NCE-MSTL; University of Limerick Abstract: Frame of the research Assessment of Biology at second level in Ireland has come under scrutiny in recent years with a change in curriculum and assessment in 2002 and now a new proposed curriculum change announced (NCCA 2007). With the emphasis towards high-stalks testing and summative examinations, it has been shown that formative assessment techniques are missing from many classrooms even though formative assessment has proved time and again it can significantly improve student learning, (Black & Wiliams, 1998; Keeley. 2008). To combat this, an instrument called two-tier multiple choice questions (MCQs) were developed for the Irish Biology classroom in the area of photosynthesis and respiration to (i) aid formative assessment in the classroom, (ii) promote higher thinking among students and (iii) identify misconception students may have in these areas at secondary school level. This paper presents findings from a study which is part of a larger study being conducted by the author in the NCE-MSTL, at the University of Limerick on formative assessment in the Irish classroom. This aspect of the study aims to develop a method of formative assessment for the Irish curriculum and then gather information on pre-service teachers attitudes towards the formative assessment instrument; twotier MCQs. The pre-service teachers participating in this study in a four year science education degree programme in the University of Limerick.

The research Many methods of formative assessment were researched for this project. The criteria which were considered necessary were (i) the ease of implementation into everyday 1

teaching and learning by Irish teachers and (ii) corresponding and complimenting the aims of the syllabus and assessment procedures of the Leaving Certificate Biology examination. The research questions: 1. Do pre-service teachers at third level find two-tier MCQs to be a good method of formative assessment? 2. Would they use the instrument as a method of formative assessment in their classrooms in the future? 3. Did the two-tier MCQs identify any misconceptions held by the pre-service teachers? Methodology The two-tier MCQs used in this study were developed specifically for the Irish Biology Curriculum in the area of photosynthesis and respiration. These topics were chosen after an extensive literature review, where several students surveyed from around the world held misconceptions in these two topics, (Haslam and Treagust 1987; Griffard and Wandersee 200; Mann and Treagust 1998; Storey 1989). The misconceptions identified by these researchers were used to develop the questions distracters. Adding to this decision was research carried out by Liston, (2009 unpublished) where it was discovered teachers in Ireland had particular difficulty teaching these two topics. Thirteen questions were finalised for study (photosynthesis = 6 and respiration = 7). Questions were adapted guided from previous research, conducted on two tier MCQs, (Mann & Treagust 1998 and Haslam & Treagust 1987). Third level students were used in this study as the author could not gain access to second level students in the time frame of the study. All students who participated had studied Biology at Leaving Certificate at second level as this was a perquisite for taking part in the study. The sample of students who volunteered for the study (n=17) were from 1st year (11) and 3rd year (6) of the degree programme. The students were asked to complete the two-tier MCQs and then complete an evaluation form on the instrument. The evaluation form sought their opinion on the instrument; particularly in the area of the instrument as a method of formative assessment and also as an instrument to diagnose misconceptions. 2

Data Outline and analysis Results from the data show the pre-service teachers had a very positive attitude towards the instrument. From the pre-service teachers who answered the questions on the evaluation form, 100% of the pre-service teachers thought (i) the instrument challenged them more then traditional MCQs, (ii) that the instrument would be good in diagnosing misconceptions, (iii) that they would have benefited by the use of the two-tier MCQs when they were in secondary school themselves, and (iv) 94% of the pre-service teachers felt they would use the instrument in their classroom in the future. The third research question asked did the pre-service teachers hold any misconceptions and from the analysis of the two-tier MCQs undertaken by the preservice teachers, various misconceptions were apparent, despite these students being A grade students. The results do show though; misconceptions were less among the third year pre-service teachers. Interpretation and Implications This paper presents an initial investigation into the use of two-tier MCQs as an instrument for formative assessment. The positive attitudes of the pre-service teachers in third level are a good indication of A) how teachers and B) students at secondary level will respond to the instrument, particularly the attitude of the 1st year third level students who would have recently completed their second level education in Biology. An interesting point noted in this study was that the instrument was able to identify misconceptions held by the pre-service teachers from their given answers to the questions. Further work To add to this study practicing teachers in Irish classrooms will be asked to evaluate the two-tier MCQs and give their opinion on the instrument as a method of formative assessment in biology on the topics of photosynthesis and respiration. References,

Atkin, J. M., Black, P. and Coffey, J. (2001) National Research Council: Classroom Assessment and the National Science Education Standards Washington D.C.: The National Academies Press. Black, P. and Wiliam, D. (1998) 'Assessment and Classroom Learning', Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. Clarke 2005, Griffard, P. B. and Wandersee, J. H. (2001) 'The Two-tier Instrument on Photosynthesis: what does it diagnose?', International Journal of Science Education, 23(10), 1039-1052. Harlen, W. and James, M. (1997) 'Assessment and Learning: differences and relationships between formative and summative assessment', Assessment in Education: Principles, Policy & Practice, 4(3), 365-379. Haslam and Treagust (1987) 'Diagnosing secondary students misconceptions of photosynthesis and respiration in plants using a two-tier multiple choice instrument', Journal of Biological Education, 21(3), 203-211. Keeley, P. (2008) Science Formative Assessment - 75 practical strategies for linking assessment, Instruction and learning. , California: NSTA press & Corwin Press Liston, M. 2009. A National Survey of Science Teachers in Ireland NCE-MSTL, (unpublished). Mann, M. and Treagust, D. F. (2000) 'An instrument to diagnose conceptions of breathing, gas exchange and respiration ', in Paper presented at the annual meeting of the National Association for Research in Science Teaching, New Orleans, L April 28 - May 1, 2000, 18. NCCA (2007) 'Moving Forward: Senior Cycle', National Council for Curriculum and Assessment. [Date accessed 17 December 2010] (http://www.ncca.ie/uploadedfiles/SCbooklet.pdf)