Workplace Based Assessments

September 2010

The Royal College of Surgeons of England 35-43 Lincolns Inn Fields London WC2A 3PE Telephone: 0207 973 0301 Fax: 0207 430 9235 Web: www.asit.org Email: president@asit.org Author: Mr Roddy OKane On behalf of the ASiT Executive and Council

ASiT - the pursuit of excellence in training 2

1 1.1

Introduction The Association of Surgeons in Training (ASiT) is undertaking a review of surgical training and the many issues that bear influence on it. This work makes up one part of this review. It will give consideration to workplace based assessments (WBAs) and their impact on surgical training. It is of course only one component of the surgical curriculum, and this review should not be considered in isolation but must be considered with all the other aspects relating to surgical education and training. ASiT welcomes the opportunity to comment on WBAs and hopes that our opinions can be used constructively towards a common goal of improving WBAs for the trainees, trainers and the public. This report has drawn from the opinions of the ASiT Executive and Council. Overview The Intercollegiate Surgical Curriculum Programme (ISCP) was implemented in August 2007 for all 9 surgical specialties. The curricula and associated assessment systems have been approved for all 9 specialities by the Postgraduate Medical Education and Training Board (PMETB). The aim of the ISCP is to produce consultant surgeons who are able to provide high-quality surgical care for their patients. Assessment is one of the four components of the curriculum. Workplace Based Assessments (WBAs) make up one component of the assessment system utilised by the ISCP.1 The WBA methods include: Mini-Peer Assessment Tool (Mini-PAT) Mini-Clinical Evaluation Exercise (Mini-CEX) Case Based Discussion (CBD) Surgical Direct Observation of Procedural Skills (Surgical DOPS) Procedure Based Assessments (PBA) They are designed to Provide formative feedback Encourage dialogue between trainer and trainee Provide a reference point within the trainees stage of training Provide evidence and contribute to the summative assessment for the annual review of competence progression (ARCP)

1.2

2 2.1

2.2 2.3

2.4

2.5

ASiT - the pursuit of excellence in training 3

2.6

They are trainee led; the trainee chooses the: Timing Case Assessor for each WBA. Completion of WBAs is the responsibility of the trainee. Guidance for the trainee in relation to WBAs and their use is provided by the assigned educational supervisor (AES). Minimum numbers of WBAs to be undertaken during training have been stipulated in the ISCP.2 The ISCP states that WBA is principally a formative assessment.3 They may also contribute to the ARCP (summative assessment).

2.7

2.8

3 3.1

The current use of WBAs in surgical training There is considerable dissatisfaction amongst surgical trainees with regards to WBAs. There are many underlying issues that have given rise to this dissatisfaction. Formative Vs Summative assessment

3.2

Many trainees perceive WBAs as a summative assessment. Having previously been through assessments systems in both undergraduate and postgraduate training that were predominantly summative, they have been conditioned to a culture where they consider all assessment types as high stakes, and summative in nature. The formative approach is often neglected by the trainer. Trainees and trainers alike describe these as tick box exercises. They fail to give feedback and as such lose out on the opportunity to direct the learning for the trainee, the prime function of these assessments. The design of these assessment tools also serves to promote a summative interpretation of their use as detailed by Eraut.4 Trainees and trainers tend to focus on the scoring of these assessments. This serves in promoting a summative interpretation. Often no guidance is given by the trainer, completely removing the formative educational benefits. Time Constraints

3.3

3.4

3.5

Finite resources of trainer time and working time restrictions for trainees have resulted in difficulty undertaking these assessments in the correct formative manner. ASiT - the pursuit of excellence in training 4

3.6

Whilst simple scoring measures and tick boxes allow for user-friendly and speedy completion they serve to remove the formative nature of the assessment and so miss the fundamental goal of WBAs. As the responsibility rests entirely on the trainee to complete WBAs, trainees perceive a lack of interest from trainers in performing them. The balance of ownership of the assessment lies entirely with the trainee, disenfranchising the trainer from the process. Poor expertise in the utilisation of formative assessments

3.7

3.8.1 The speed of implementation of a new educational culture in surgical training has not given consideration to lack of understanding of the nature and purposes of these assessments. 3.9 Trainers and trainees alike have little understanding in how to undertake these types of assessments. Opportunity to learn about these assessments is promoted and taught in courses such as the Training and Assessment in Practice (TAIP) course at the Royal College of Surgeons of England. Trainees report that many trainers completing their assessments have not undertaken this course or similar courses. Limited number of PBAs for some specialties 3.10 Trainees have reported a lack of PBAs and surgical DOPS in relation to some surgical specialties. Competence and Excellence 3.11 The Tooke report alluded to the new culture of mediocrity, which is facilitated by a competency-based curriculum.5 His statement that good enough is not good enough should enshrine throughout the surgical curriculum. This of course includes WBAs. The scoring system seen on WBAs only serves to promote this mediocrity and does not serve to promote the aspiration to excellence described by Tooke. Trainer ownership dissociation 3.13 Whilst the responsibility of assessment lying with the trainee fits with contemporary androgogical theory, an unintended consequence is that the trainer has no onus to implement these in their correct manner.

3.12

ASiT - the pursuit of excellence in training 5

Mini-PAT 3.14 Trainees report greater acceptance for this specific type of WBA. It is generally considered more valid by trainees. Conclusions & Recommendations ASiT continues to support the use of WBAs in surgical training. Assessment is a key driver in learning.6 If support and a positive attitude to WBAs is engendered in the surgical community, and time and infrastructure facilitates WBAs, then one can see the value of these in further enhancing learning and development of expertise in the practice of surgery. ASiT believes that the design of these assessments should be examined and amended. To remove the tick box culture ASiT believes we should remove the tick boxes. To promote and engender a new learning culture in the surgical profession it may be necessary to completely remove the scoring system within the assessment. This will focus the attention onto the primary formative function of WBAs. Attendance at a TAIP course should be mandatory for all trainers involved in using WBAs. To facilitate this, the TAIP course should be run in various centres throughout the country. Trainees in the latter stages of training should also be encouraged to undertake the course so that cultural change in relation to these types of assessments is promoted for future generations of surgeons. Further development of PBAs and surgical DOPs for surgical procedures throughout all the surgical disciplines is required. The NHS needs to recognise the importance of training provision and that adequate time is required for Consultant trainers to carry out WBAs. ASiT recognises that the implementation of these forms of assessments will require a paradigm cultural shift within the surgical community. It would be unrealistic to expect this transformation to occur overnight. It may require generational change to take full effect (i.e. have positive support and promote excellence in learning).

4 4.1

4.2

4.3

4.4 4.5

4.6

ASiT - the pursuit of excellence in training 6

5 1

References ISCP Assessment System Available at: http://www.iscp.ac.uk/Assessment/Assessment_Framework.pdf Guidance on the frequency, timing and use of workplace-based assessments Available at: http://www.iscp.ac.uk/documents/wba_guidance.pdf The Intercollegiate Surgical Curriculum. Educating the surgeons of the future. Available at: http://www.iscp.ac.uk/documents/iscp_educating_future_surgeons. pdf Eraut M. Evaluation of the Introduction of the Intercollegiate Surgical Curriculum Programme. 2009 Aspiring To Excellence. Independent inquiry into Modernising Medical Careers. Tooke, J. 2008 McLachlan, J. C. 2006, "The relationship between assessment and learning", Med.Educ., vol. 40, no. 8, pp. 716-717.

4 5 6

ASiT - the pursuit of excellence in training 7