Preface to the ISCP Evaluation Report

The case for a major review of surgical training in this country had been steadily building since the late 1990s. The introduction of the Calman reforms during 1996 saw improvements to higher surgical training but left basic surgical training unreformed. Many aspiring surgeons spent years waiting to enter specialty training, going from job to job, often referred to as the lost tribe. Additionally other external factors were starting to impact on traditional surgical training as a whole and were increasing the pressure for change, for example:

The European Working Time Directive (EWTD); Increasing public expectations for accountability and transparency; and New working practices and changes to service delivery.

The decision to review existing surgical curricula began in 2002. In 2003 Modernising Medical Careers provided additional impetus to develop new curricula. The Chairman of JCHST, now JCST, and the Specialist Advisory Committee (SAC) Chairs, together with their delegated editors, led the process of curriculum creation and the Intercollegiate Surgical Curriculum Project (ISCP) was established in March 2003. Practising surgeons, trainees, educationalists, and other specialists were involved in all aspects of curriculum development. The curriculum was designed to integrate four key domains in surgical practice: clinical judgement, technical and operative skills, specialty-based knowledge, and generic professional skills. For the first time, the evolving curriculum articulated stages of training and the standards to be achieved, in the four domains, at each stage. This was an innovative approach to surgical education and training and one that was very different from existing practice. As the curriculum developed it was decided to integrate the curriculum with a web based training management system that would promote and support good educational practice. The curriculum development went through a pre-pilot and pilot phase prior to launching. At an early stage the services of Professor Michael Eraut, an expert in professional and workplace learning, were enlisted to act as a participant observer who would produce an independent report at the end of the prepilot phase (http://www.iscp.ac.uk/Documents/EvaluationReportPhase2.pdf). The findings of the report highlighted the importance of undertaking contemporaneous research and evaluation. The ISCP Evaluation Task Group was established under the chairmanship of Professor Irving Taylor in 2006. It reported to the Curriculum Project Management Group chaired by Professor Gordon Williams. A grant of 50,000 was allocated from the ISCP development budget to fund small research projects to contribute to the overall debate. Six areas of research were identified:

 access to appropriate experience for trainees; provision of appropriate resources and support for trainers; the effectiveness and cohesion of the underpinning infrastructure, for both provision and regulation; organisational issues around time and resources that impact on the feasibility of delivery of the new curriculum; the effectiveness of educational and assessment tools and resources developed as part of the project; the effectiveness of the change strategy for implementing the new curriculum. It was intended that the research projects would be carried out by the deaneries in partnership with their schools of surgery and an academic institution that would act as a research agent. Deaneries were asked to bid to conduct a research project against one of the six research areas. Four projects were chosen, as shown below.
Organisation Kent Surrey and Sussex Deanery School of Postgraduate Medical and Dental Education, Cardiff University and Wales Training School of Surgery University of Southampton Warwick Medical School Title of Project An evaluation of the KSS methods for introducing ISCP through the establishment of Surgical faculty groups within Trusts The Intercollegiate Surgical Curriculum: an evaluation of the online learning agreement and web-based resources

Do Current Surgical Training Posts Provide Appropriate Experience and Support for Future Surgical Trainees? The Effects of Changes to the Modern NHS upon Continuity of Patient Care, Surgical training and Overall Patient Care

The original time frame for data collection was February July 2007, however this was adjusted in the light of the project partners proposals. The research therefore spanned both the pilot phase and initial implementation. Professor Michael Eraut was chosen as the external academic advisor across the various research groups to provide advice on the main themes of the project overall, and to facilitate cross-fertilisation of ideas. Alongside the ISCP funded research, both the ISCP and PMETB conducted evaluation of trainees experiences. Professor Michael Eraut was commissioned to produce an overall summary on behalf of ISCP drawing on the results of the ISCP research projects and data from the ISCP and PMETB surveys.

Professor Irving Taylor Chairman, ISCP Evaluation Task Group

Francine Alexander Interim Head of Education RCS England

ii

Acknowledgements This report is based on evidence collected by a large number of people. The quantitative data comes from four sources: The JCST Quality Assurance team led by Nigel Stripe, the PMETB website, the ISCP website managed by Jeremy Brooks-Martin and an opportunistic study of London trainees attending Anatomy days at RCS by John Masih. The qualitative evidence comes from four research studies commissioned by the ISCP Evaluation Task Group, chaired by Professor Irving Taylor, following an invitation to all the Deaneries. Given the wide variations in trainees experiences, it has been important to present the experiences of trainers and trainees in different parts of the ISCP partnership both collectively and individually through their own voices. For this data, I am indebted to the fine research undertaken by these studies, especially to their leading researchers: Robert Padwick, Deborah Markham and Neil Johnson at Warwick Mary Seabrook, Pam Shaw and Mary Hayes at KSS Stephen Brigley, Louis Fligelstone, Lynne Allery, Janet MacDonald and Lesley Pugsley at the Wales postgraduate deanery James Gilbert, Karen Nugent and Debra Humphris at Southampton I have referred to these studies when they were the primary source for one or more pages, but not on every occasion when their work was used because that would have made the text difficult to follow; and there were also places where the context required a rather different wording than the original. I take responsibility for such changes and have checked with the authors that they are not unhappy with my interpretations. I also gained many new insights through elite interviews with experienced surgical educators, who also vetted my use of their interviews. All the data collected by all these sources promised anonymity to their informants. An important part of my contribution has been an attempt to provide a bigger picture than could be provided from any single piece of research in order to show how variations can be incorporated into a more complex but understandable picture. Thus my analysis seeks to show how different findings connect with each other, and how we might plan policy pathways that address some of the considerable difficulties that confront surgical education today. In seeking this more ambitious interpretation of my brief, I have been greatly helped by three former members of the Raven Department of Education at RCS Eng: Natalie Briggs, Andrea Kelly and John Masih. Finally, I would like to thank those members of education related ISCP and RCS committees, especially Professor Irving Taylor and Francine Alexander who gave me considerable advice and support during the final period of my evaluation. None of the helpful people listed above are responsible for my final text and recommendations; so any mistakes, misunderstandings or disagreements are my own responsibility and critical comments should be sent to myself alone. Michael Eraut, Professor Emeritus, University of Sussex

iii

CONTENTS
Preface Acknowledgements Contents Summary Introduction Data sources for Pilot Evaluation Changes in the Surgical Education Context outside ISCP The Impact of MMC and EWTD Less time for training for both trainees and trainers Providing continuity of patient care Providing continuity of training Organisational factors and targets Early implementation of the ISCP curriculum Educational Supervisors and Learning Agreements The ISCP website Initial response to ISCP Learning contexts and access to learning Elective Surgery Emergency Surgery and Trauma Clinics for Outpatients Ward work Simulators, models, endoscopy and recordings Formal educational activities When is Service Work a learning opportunity? Apprenticeship, informal learning and coaching Competence and Assessment Portfolios ARCP (formerly RITA) Teamwork and relationships Handover of patients Support for Trainers Distributed Apprenticeship and Organisational Practices Risk analysis and new organisational practices The training of surgical trainers and surgical teams References Recommendations _____ Appendices A to E Page i Page iii Page iv Page 1 Page 7 Page 7 Page 9 Page 9 Page 10 Page 11 Page 13 Page 14 Page 16 Page 16 Page 19 Page 21 Page 27 Page 28 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 37 Page 41 Page 44 Page 44 Page 46 Page 49 Page 51 Page 55 Page 55 Page 57 Page 58 Page 59

Pages 60 -64

iv

TABLES
Table 1 % of validated trainees with an AES and a learning Agreement Table 2: Meetings about progress and formal assessment in General Surgery Table 3: RCS Quality Assurance survey data on Supervision Table 4: PMETB data on clinical supervision, informal feedback and ethics Table 5: QA data on Appraisal and Feedback Table 6: The balance between constructive and undermining Feedback Table 7: From whom do trainees get their support, and how much? Table 8: QA data (1-5 scale) on Induction Table 9: QA data (1-5 scale) on Response to Trainees Concerns Table 10: QA data (1-5 scale) on Organisational Issues Table 11: Distribution of trainee time across patient contexts Table 12: Issues relating to Learning in the Workplace Table 13: Availability and Quality of Formal Educational Opportunities Table 14: Formal Teaching and Audit Table 15: The Informal Audit of critical events and near misses Table 16: Frequency of on the spot discussions of clinical/professional episodes Table 17: Frequency of discussions about a trainees work Table 18: Frequency of discussions about a trainees general progress Table 19: QA data (1-5 scale) on Assessment Table 20: Trainee use of Portfolios and Logbooks Table 21: Handover Arrangements Table 22: Frequency of topics discussed at KSS Faculty Group meetings Page 16 Page 17 Page 22 Page 22 Page 23 Page 23 Page 24 Page 24 Page 25 Page 25 Page 27 Page 28 Page 34 Page 34 Page 34 Page 38 Page 38 Page 39 Page 43 Page 44 Page 50 Page 63

FIGURES
Figure 1: Trainees perceptions of the balance of their time Figure 2: Learning from Unsupervised Work Page 27 Page 36

Evaluation of the Introduction of the Intercollegiate Surgical Curriculum Programme

Professor Michael Eraut, University of Sussex Summary
1. External factors affecting the introduction of ISCP (pp9-15) The biggest challenge for this evaluation was that the changes introduced by ISCP were almost swamped by other changes already in progress. The European Working Time Directive (EWTD) will reach its final figure of 48 hours a week in 2009, and this has significantly reduced the time that trainees can spend in both formal and informal learning environments. This situation has been further exacerbated by the growth of Sub-specialties, which limit the range of consultants domain of expertise. The combination of both these factors has led to the dissolution of the firm structure and the introduction of shift working for both trainers and trainees in several branches of medicine. In surgery, the overall work patterns of trainers and trainees in a shift system cannot be matched; so the time they can be in the same place together has been significantly reduced. Modernising Medical Careers (MMC) was intended to reduce the time taken to become a consultant by 2 years, which might have been possible if other factors had stayed the same. However, the understandable focus on reducing waiting lists for elective surgery has led to the disappearance of training lists from many hospitals to improve their productivity. Moreover, the recruitment of the first cohort of the new MMC surgical trainees through MTAS coincided with the introduction of ISCP and created a very negative mood among surgical educators, which did not help evaluators seeking data on the progress of ISCP. Although ISCP was able to negotiate the restoration of the two lost years, the effect of all the other changes was a steady reduction of access to training at all levels before the introduction of ISCP. For example, an increasing number of surgical registrars feel unready to take CCST, because they have had insufficient access to practice during their postgraduate training. 2.The Evidence Base for this report (pp7-8) Data collected by the ISCP website, the centre for formal communications between trainers and trainees. Three surveys from different agencies: a JCST Quality Assurance survey in November 2007, a PMETB survey at the beginning of 2008, and a small survey of a group of London trainees doing Anatomy courses at RCS. 1

Four research studies commissioned by ISCP from different deaneries: 1. Warwick focussed on continuity of care and factors impacting on surgical training and patient outcomes. Their evidence was collected from 12 focus groups between October 2007 and May 2008. 2. Kent, Surrey, & Sussex (KSS) focussed on the nature and value of the support for trainers, especially the role of Trust-based faculty groups from January 2007 to March 2008. 3. The Wales Postgraduate Deanery focussed on the impact of the ISCP website and the assessments it carried from October 2007 to March 2008 4. Southampton focussed on the most critical features for a good placement between October 2006 and October 2007.

Six elite interviews with senior surgeons.

3. Early Implementation of the ISCP (pp16-26) ISCP brought in three new features to improve the focus and management of the learning support system. The first change was a new role, that of an Assigned Education Supervisor (AES), whose first job was to negotiate the second new feature, that of a Learning Agreements (LAs) between the AES and their assigned trainees. The third innovation was the development of a learner led Website, which held the curriculum and through which communications were expected to be conducted. The website design was being improved throughout the year, and trainees were more able to use it than some trainers. Although most trainers and nearly all trainees appreciated the goals of ISCP, its implementation was challenging. In May 2008 78%of validated trainees had an AES and 51% had a Learning Agreements; and there were large variations across both deaneries and specialties. This makes it very difficult to interpret the data. The evidence reported by the PMETB survey in February 2008 suggested that AES and LA functions were being pursued off the website by a substantial number of trainees. 95% of general surgeons said they had an educational supervisor, who was being responsible for their appraisal, and 78% said they had a learning agreement. This was confirmed by the Wales study; but the actual use of the LA may have been more limited, because two questions in Table 2 (page 17) showed that formal meetings with a supervisor to discuss their progress and formal assessments of their performance in their current post had not yet occurred for almost half the sample. The evidence on issues relevant to supervision appears to be reasonable, but our small survey of London trainees in May 2008 discovered a huge variation in the hours spent working with more senior colleagues. Their estimates for an average week were that 31% spent from 0 to 12 hours, 31% spent from 13 to 24 hours and 38% spent 25 hours or more. With this type of distribution the use of averages can be profoundly misleading, and the JCST use of dissatisfaction indicators becomes very important.

Table 7 summarises data from the same London survey, which looks at the main sources of support that trainees recognise and their relative frequencies. SpRs come first, followed by other consultant surgeons, then educational supervisors. However all seven categories make significant contributions to a significant number of trainees. Table 7: From whom do trainees get their support, and how much? Frequency Rows add to 100% AES Other consultant surgeons Staff grade surgeons SpRs Surgeon peers Other consultants Other health professionals A lot 16 14 14 38 14 6 4 Quite a bit 32 47 20 40 27 17 22 A little 42 32 32 17 44 39 36 None 9 4 17 4 9 20 25 No reply 1 3 8 1 5 9 13

The Quality Assurance survey in November 2007 suggests that the issues causing trainees the greatest concern at that time were as follows: No induction to Training Programme Scheduling meetings with their AES Their Supervisors familiarity with ISCP curriculum Guidance on Personal Development Planning Deanery response to their concerns Trust response to their concerns Rota/shift patterns Service demands of their post 4. Learning Contexts and Access to Learning (pp 27-40) Table 11: Distribution of trainee time across patient contexts Ward % ST1/FTSTA1 Mean Standard deviation ST2/FTSTA2 Mean Standard deviation 47.1 30.3 47.1 27.5 Clinic % 15.7 14.0 14.5 13.7 Theatre % 26.7 10.5 30.1 16.8 Other % 10.5 11.8 8.3 16.5

This table from the London survey illustrates both the distribution of junior trainees time and the very large variations in those distributions in May 2008. Another question revealed that, in their view, only ward work was allocated too much time, and theatre and clinics received too little. 5% spent no time in theatre and 18% no time in clinics. In

addition to time spent and the balance between different settings, the qualitative data brings out other important factors: the quality of relationships in any particular setting, the appropriateness of the work allocated, the quality and timing of advice and feedback (undermining feedback is still quite common), opportunities for enhancing their understanding of surgery and sustaining a sense of purpose and progression. Experienced surgical trainers, both in interviews and informally, also refer to the main problems in these settings. Elective surgery gets most attention from ST3 onwards, but is retarded by the lack of trainee access to operational experience and the loss of training lists to meet urgent targets. Even under the old system trainees are increasingly unready to become consultants at the usual time. Emergency Surgery and Trauma are allocated considerable time from higher level trainees, but without the support they need to learn how best to handle the complex and time critical cases they receive. This is a much neglected problem in need of urgent attention. Clinics no longer allow time for trainers and trainees to see patients together, although they can consult with each other about patients examined by the trainee. One consequence is that FRCS examiners are finding that trainees are becoming increasingly weak diagnosticians. Although ward work take up a great deal of the time of junior trainees, the approach to their learning is surprisingly laissez faire. The answer to the current difference of opinion between trainers and trainees over the value of ward work must surely be that some of it is acknowledged as valuable by both groups, some of it would probably be acknowledged as having little learning value by both groups and some of it could be made valuable by giving appropriate advice and support. This needs to be explored on a wider scale, rather than leave it to every individual trainer to work it out for themselves. Hence the chapter concludes with sub-sections on When is Service Work a Learning Opportunity? and Apprenticeship and Coaching. 5. Competence and Assessment (pp 41-45) The current assessment advice is both impractical and confused. It neglects the time required and the difficulty in finding assessors; and it assumes that trainees will suddenly treat what looks like a test as being formative rather than summative, even when they have been reared in a culture of competition. Given the great variation in posts and circumstances and the ISCP claim to be competence based, it makes no sense for the three main assessment instruments for junior trainees to be normative rather than criterion based like the PBA. Nor will most of the available assessors have sufficient experience to make normative judgements in rapidly changing contexts, a new MMC trajectory and a new surgical curriculum. Trust-backed processes are required that integrate assessments with the individual trainees ongoing learning and supervision. 6. Teamwork and Relationships (pp 46-50) There is an increasing recognition of the importance of teamwork both within and across professions, just as the shift systems are making it more difficult. The key issues are continuity of patient care and reduction of risk, and improved modes of communication need to be turned into communicative practices. It is now very clear that the same issue is

affecting trainees; it is not only patient handovers that need regular attention, but also trainee handovers for one trainer to another. While there are many positive accounts of teamwork, these often involve either additional effort which cannot last for long or limitations caused by rapid changes in team membership (and exacerbated by the shorter 4 month placements). This raises the question of whether more attention should be given to developing different organisational practices rather than expecting individuals to continuously adjust to frantic changes within the status quo. 7. Training and Support for Trainers (pp51-54) Much work has been done in the last decade on provision of training and support for trainers, but the role of trainers has become increasingly difficult; and there are endless comments about the lack of time in job plans for teaching. Generally, it seems that consultants are expected to teach during the time allocated for non-clinical work, but consultants who take a major role in teaching often have no more time allocated than those who take little or no role. Some key individuals in Trusts have started to discuss the idea of withdrawing funding from those not teaching. Whilst many felt that everyone should be involved, they also recognised that some of those who did not wish to make a significant educational contribution might be poor teachers. Generally, there was much cynicism from surgeons about getting any support from trust management for the new educational roles, and this was acknowledged as a problem in the ISCP Pilot workshops. There was no evidence of any financial audit of the manpower gain received by Trusts through the work done by surgical trainees or indeed of the clinical governance implications for the Trust of not taking the quality of training seriously. 8. Distributed Apprenticeship and the Organisational Dimension (pp55-58) There appear to be four possible areas of response to this challenging range of problems, and all of them are important: 1) A risk analysis of the current situation to increase the collective understanding of all the stakeholders of the impact on training and service 2) Piloting new approaches to the organisation of surgical training within hospitals, with appropriate backing from the NHS 3) Training of individual surgical trainers 4) Training of surgical teams in all the settings discussed above The main conclusion arising from the evidence collected for this evaluation is that surgical education cannot achieve its current goals without significant changes in its current state. There has been a major reduction in the training time per annum of both trainers and trainees, and opportunities for trainees and their main trainers to meet together have been drastically reduced by the new shift systems. The results are that: 1) Trainees for elective surgery will remain safe but fail to reach CCT at the expected time 2) Training in clinics has become problematic, because joint outpatient lists have completely disappeared in most Trusts

3) Training in trauma has been virtually non-existent, and is far from meeting an acceptable standard. Unless the organisation and funding of training is properly planned, surgical education will decline in quantity faster than any conceivable improvements in quality. The example of The Royal London Trusts reorganisation of their Trauma service shows that it is both possible and extremely important to conduct ongoing risk analyses of surgical activities, and to use them for learning by all those involved; because this provides a crucial direct link between patient outcomes and educational provision. When connected with the concerns about surgical education revealed by the evidence gathered for this report, a second conclusion also emerges: that if the current organisational practices affecting surgical education cannot implement the changes required for improving patient safety and other patient outcomes, then the structure of surgical education will have to be reorganised. The other major issue is the training of surgical trainers and surgical teams. In addition to the problem of continuity between team members from different professions, there is a rapidly increasing problem of continuity between surgical trainees and their trainers. The key question to be addressed is that of how far it is possible for surgical trainers to develop continuity of training for their trainees. Not only are trainers meeting their trainees less often, but they know very little about what their trainees may have done with other consultant colleagues between their own meetings. The first problem is to decide when communication between trainees and their consultant trainees is needed for sustaining trainees continuity of learning; and the second problem is that of developing a meaningful discourse for the mutual understanding of those communications. This ambitious but very important endeavour could be supported by mediating artefacts such as recordings or still pictures around which meaningful discussions could take place. For example, still pictures taken at intervals would enable those present at an operation (not necessarily only surgeons) to add separate short commentaries on each picture about what they were thinking about at the time they were taken and their later, more reflective, thoughts. These commentaries do not have to be accurate representations of on-the-spot thinking, that would be impossible; but subsequent discussions of these commentaries should help to improve communication and mutual understanding between those concerned about their respective views of surgical events. The trust engendered by these initial joint activities should create the interpersonal relationships needed for addressing the development of teams who can begin to collectively improve the quality of their service to patients and trainees. Over time this should help to develop the common discourse, which will be needed if surgical training is to progress from its original apprenticeship system to a more transparent and reflective system of distributed apprenticeship, in which a group of trainers supports one or two trainees and offers them the continuity of training that now appears to be essential for making progress in the next few years.

Introduction In August 2007, the first cohort to undertake a new Inter-collegiate Surgical Curriculum Programme (ISCP) began their voyage. This followed 5 years in Medical School and two Foundation years, in which they were able to engage with six 4 month placements in different medical environments. Work on this new curriculum, which owed much of its conceptual framework to the Canadian CanMEDS programme, started in 2002. Two years of development were followed by a Pre-Pilot year, during which the prototype curriculum was explored with current trainers and trainees, and the design of a web-based delivery system was initiated. Then a 30 month Pilot programme from April 2005 to August 2007 (1) reached agreement on further detail with the Specialty Advisory Committees (SACs) and the ISCP Assessment Committee, (2) received the approval of PMETB, and (3) began to prepare surgeons for its formal implementation. My evaluation role in the Pilot Phase was rather different from that in the Pre-Pilot Phase (Eraut 2005). My time was still very limited, and I had to rely rather more on evidence gathered by members and employees of the Colleges. The meetings in the Pilot Phase were concerned only with implementation issues and did not collect any evaluation data. However, some funds for data collection were allocated to the ISCP Evaluation Task Group, of which I was a member; and I both contributed to and endorsed its plans. The data sources used in this evaluation are listed below. Data sources for the Pilot Evaluation 1. Website data indicating the number and type of website registrations in December 2007 and May 2008, and the use of the web facilities essential for those with special roles. 2. Quantitative data from an ISCP Quality Assurance (QA) survey investigating new trainee attitudes in November 2007 (the fourth and last month of their first ST1 posts) toward a range of ISCP features intended to support their learning. This survey provides useful data on the learning of ST1 trainees that suggests where improvements are needed. However, it was not intended to be part of an evaluation of ISCP and does not have any baseline data that would enable comparisons that might indicate whether particular aspects of surgical training had improved or deteriorated. Nor could it attribute outcomes to changes beyond the new use of a website, a new support role and the use of some new assessments to ISCP, rather than other factors. The survey was based on 350 responses (89%), and only specialties with more than 10 trainees have been used in this report. 3. Quantitative data from the PMETB website relevant to supervision, feedback, learning environments, critical events and safety, handovers, confidential support and learning agreements, portfolios and logs. This data was collected over three months from December 4th 2007 to February 28th 2008, and has been restricted by the evaluator to General Surgery and Trauma & Orthopaedics (c230 per specialty).

4. A Trainee Questionnaire in London (n=70) focused on who gives most frequent support to trainees, and the perceived balance between service work and training. 5. Four research studies commissioned from different deaneries on key issues identified in the pre-pilot evaluation: Warwick focussed on continuity of care and factors impacting on surgical training and patient outcomes. Their evidence was collected from 12 focus groups between October 2007 and May 2008. The participants were 12 consultants, 12 specialist registrars, 10 ST1s, 11 nurses and 6 physiotherapists in the West Midlands and Severn deaneries. All came from General Surgery or Orthopaedics. Kent, Surrey, & Sussex (KSS) focussed on the nature and value of the support for trainers, especially the role of Trust-based faculty groups from January 2007 to March 2008. Their evidence came from documents, including minutes of meetings not attended by the researchers; 28 interviews with 23 interviewees from 11 Trusts and personnel from the Deanery and RCS; and attendance at 10 Faculty Group meetings. The Wales postgraduate deanery focussed on the impact of the ISCP website and the assessments it carried. Its evidence came from interviews with 18 trainees from different levels and backgrounds and 14 supervisors from a range of specialties, during the period from November 2007 to March 2008. The trainees in this sample included ten FTSTA1s and two FTSTA3s. Southampton focussed on the most critical features for a good placement. Their evidence came from trainee logbooks between October 2004 and October 2006 (20 from 16 trainees) and interviews with 9 trainers and 6 trainees between October 2006 and October 2007. 6. Four long and two short interviews conducted by the evaluator with surgeons known for their work in surgical education on progression and surgical pedagogy; and one interview with a current trainee with a background in surgical research. The first three sources were not under the jurisdiction of the ISCP Evaluation Task Group, but the QA and PMETB surveys may have affected the later decision of the Evaluation Committee to abandon its planned survey for April 2008, when it became clear that it would get only a small response rate from its survey weary constituents. The biggest challenge for this evaluation was that the changes introduced by ISCP had to be situated in a context where changes in surgical education outside the remit of ISCP were threatening to swamp the changes being introduced by ISCP itself. Hence, as in the Pre-Pilot Phase (Eraut 2005), it starts with a review of this rapidly changing context before discussing the impact of ISCP itself.

Changes in the Surgical Education Context outside ISCP The Impact of MMC and EWTD One of the goals of Modernising Medical Careers (MMC) was to reduce the number of years it took to progress from Medical School to becoming a Consultant. This replaced the previous positions of PRHO (Pre-registration House Officer, Senior House Officer (SHO), and Registrar by two Foundation Years, F1 and F2, followed by six years as a Specialist Trainee (ST1 to ST6). Although two further trainee years were restored in 2007, the start of ISCP was temporarily eclipsed by the selection process to become the first group of ST1s. The pioneering F2 cohort had to apply for run through specialist training jobs in February 2007 in order to get onto a specialist training track in August 2007. This hurdle incorporated several new features and circumstances, whose impact was not fully anticipated: 1. A much earlier decision on trainees preferred specialty than previously. 2. A much higher number of applications for specialist training than previously, for which F2s had to compete with a significant number of SHOs and, for the first time, Non Consultant Career grades (NCCGs) and IMGs (Tooke et al, 2008). This replaced a gradual reduction in the cohort size by a single selection event, which rejected a much higher proportion of candidates. 3. A different application system using different criteria for the first time. The first cohort of ST1s started in August 2007, so the successful surgical applicants became the first cohort of the new ISCP curriculum. However, the new method of selection was an unmitigated public relations disaster, because of the high, but apparently unanticipated, number of rejections from a previously untested selection system. This debacle ran in the newspapers throughout the early summer; and its effect on the mood of surgeons engaged in Deanery meetings preparing for ISCP was very negative indeed. The loss of time created by MMC might been seen as a possible goal, if surgeons thought that training could be improved; but the influence of new government targets for elective surgery was already beginning to diminish the amount of effective training time. Government targets for hospitals have been high on the political agenda for several years, and its response to very long waiting lists for elective surgery is understandable. However, one of its side effects has been a reduction in the time spent on training, and the disappearance of training lists from many hospitals to improve their productivity. While this target system may eventually change, in theory, the European Working Time Directive (EWTD) is irreversible. This measure was agreed by the European Union some time ago; and the maximum number of hours per week for trainees has been gradually reduced over several years to reach its final figure of 48 hours a week in 2009. This is very much lower than previous custom and practice, and has significantly reduced the time that trainees can spend in both formal and informal learning environments. This situation has been further exacerbated by the growth of sub-specialties, which limit the range of consultants domain of expertise. The combination of both these factors has led to the dissolution of the firm structure and the introduction of shift working for both

trainers and trainees in several branches of medicine. In surgery, the overall work patterns of trainers and trainees in a shift system cannot be matched; so the time they can be in the same place together has been significantly reduced. These changes, which predate the development of ISCP, have been reducing access to training for some time. The Warwick report discusses earlier studies by Barden et al (2002), Bollschweiler (2001), Gagnon et al, 2006) and Henry et al (2005); and our new evidence has confirmed that these problems are growing in the following aspects of surgical learning. Less time for training for both trainees and trainers This problem developed long before the implementation of ISCP for its first cohort of ST1 trainees. We collected evidence of this from a wide range of sources, for example a newly appointed consultant reported a huge reduction in access to practice: I know from my own personal experience ... at Hospital X ... where I worked for a year... I had 118 colorectal operating sessions in that year. When the shift pattern came in, just as I left I calculated what my successor would be doing and it was down to 18 operating sessions per year. A specialist registrar noted limited experience of complex cases: We are nowhere near as capable as the generation above us in surgery because technically we have skills. We [may] know how to do something, [but] we just havent got the exposure to it. Perhaps what well never be as good at is dealing with complications; we havent had to think on our feet. An experienced trainer reported how final year registrars no longer felt ready to become consultants: Trainees increasingly opt to take a six month to one year fellowship, often overseas, to get more experience before applying for consultant posts. Over the last few years in one Deanery (personal experience of 2002 to 2006) approximately 70% of orthopaedic trainees have been opting to take fellowships, compared with an average of 20% from other areas of surgery. This was confirmed by the Head of a School of Surgery at the other end of the country, who was also considering how best to respond to this problem: There are a significant, rather frightening, number of people who do have significant problems in their first 3 years after CCT. The profession, if they think about it, will introduce mentoring for the first three years in a new job. There would also be some wisdom in being paired with a more experienced person, who will shepherd you through. It sounds like a good thing in theory, but I am not sure that it will ever be feasible in practice. This problem started at every level at about the same time. For example the same Head of School reported that:

10

The ST 2s have one year to do hernias, varicose veins and maybe bowel anastomoses. It used to be do-able, but now you wouldnt have done more than 20 hernias, 20 veins and 10 bowels: not big numbers. You might do small parts of the first 5, then a bit more then start to do the whole operation. You wouldnt have done more than 5 complete operations. He has confirmed this by looking at the trainees logbooks. This exposure is much less than we think it should be, and theres not much opportunity [to practice], its very worrying. The Warwick research group found that all surgical grades perceived a trend towards the learning of certain skills being deferred to later stages of training, and this compounded the problem. Thus a consultant reported that people, who had done enough years to be ready to apply for registrar jobs, had done perhaps 15 appendectomies, while he had done about 200 by that stage. This was confirmed by an SHO: I think its just become more and more expected that ... more is ... deferred to a higher year in training. I think that the expectation is dont worry about that, youll do it next year. Thats a recurring phrase. The view of nearly all surgeons seems to be that patient outcomes are better as there is more Consultant involvement at both an operative level and ward level: The patients get a better deal now because theyre getting more Consultant time ... in the old days you relied on your registrar or senior registrar to run the ward ... (now)....you do it yourself. I guess in a way patients do benefit from that becausethe most experienced person ... (is) ... doing the work. There is also a perception that increased Consultant involvement at an operative level is beneficial for surgical training: One positive thing that has come out of this ... is the fact that the Consultant is present in theatre a hell of a lot more than five years ago and you are taught to do things properly ... we may not be getting the quantity but the quality is better than it was ... (SpR) The increased involvement of Consultants is however also perceived to be potentially damaging by SpRs and SHOs, in that there is less opportunity for trainees to act independently; so trainees will not be sufficiently experienced at the end of their training to become autonomous Consultants: I think it will hurt the Consultants in the future because theyll be called in at 3am to do something that only a few years ago a registrar would be feeling very confident to do (SHO) Yeah, its only going to get worse and worse because the more its a Consultant led service the more theres going to be less of us making decisions. Yes, if we get taken through it time after time we get to perfect the techniques but if we havent got the decision making ability, [that will be a serious problem]. (SpR)

11

Providing continuity of patient care with a more fragmented workforce This issue was highlighted in my evaluation of the pre-pilot, and was the main focus of the Warwick project, who noted that the concept of the patient journey is seen as different by different groups in different settings. For example, the more senior SHOs, SpRs, nursing staff and consultants consider the whole patient journey, right from first clinic appointment through to follow-up appointments, whereas the more junior SHOs and nursing staff are more likely to think of the patient journey in terms of only the period spent in hospital. This is even more marked for emergency admissions. Surgeons of all grades perceive that trainees are more involved in the initial stages of the management and decision making process, but the necessity of handing emergency patients over to the team on-call means that trainees do not see the whole of the emergency patient journey: The patients you see in emergency, youre not necessarily going to know whats happened to them a day, two days a week later... Im sure it used to be that when you were on call and your team was on call, you knew everyone who came in on call-they were yours, they stayed yours. But now thats not the case. (SHO) In the elective setting, more senior trainees and consultants perceive that trainees are often not involved until the post-operative stages of the patient journey, particularly at SHO level, and that most of the management decisions will come from the consultant. However, during the post-operative stay in hospital, it is generally perceived by all surgeons that trainees are able to follow the patient journey more effectively than in the emergency setting, but the start and end of the pathway are still missing: Were talking about the routine patients who are booked in for major surgery or routine surgery, the juniors do tend to follow those patients through, and monitor those patients on the ward and see how theyre getting on. They wont however have seen them in outpatients prior to them being booked in. And they dont take part in follow up clinics to see how theyve got on afterwards. (Consultant) Thus surgeons at all levels and nursing staff all perceive that surgical training is adversely affected through junior staff not seeing patients at a number of stages. Theres a pressure now bringing in patients for elective procedures on the day of surgery and ... there isnt any space or time to see that patient until you join your Consultant in the operating room who is already with the patient and ready to make the incision, which isnt the way it used to be ... (SHO) Moreover, the pre-op assessment clinic is often nurse led; so the post-op ward may be the first interaction that the (junior) doctor and patient have with each other. The majority of problems reported relating to patient outcome and not following the patient journey are due to problems with handovers. Handovers between doctors were unanimously perceived to be a problem, and for a number of reasons: With the permanent shift system you can often be handing over three or four times a day ... every handover has the potential to miss off an important aspect, because youve got one individual trying to get information from another individual and theyre not

12

always going to get to grips with the whole concept of whats going on with that patient. If thats happening a couple of times day you can get important things missed or things done that arent necessary. (SHO) I think handovers a vulnerable time because I dont think its done with any sort of system. Its just done ad-hoc and it depends on the people on-call as to how good a handover they give. And if its very busy then things tend to overrun. Things get missed, you dont handover crucial details and then a few hours later the ward can contact the night SHO and theyre quite surprised at the new information. (SHO) There is no dedicated time in which to formally hand over information to colleagues; and this new handover culture leads to a diffusion of responsibility amongst trainees, which itself has knock-on effects for the patient and for training. This is a view held in particular by consultants: The whole working ethos has now changed because of the shift system where you feel that once youve finished your shift, your responsibility is over, you hand over to somebody else. Its that persons responsibility. Whereas previously, we would have come the next day to find out what would have happened to that patient. I think that whole ethos has changed. (Consultant) This current model fosters a culture of not actively dealing with problems. So if you have a slightly difficult patient there is a tendency for the juniors not really to actively manage it but to pass it onto the next team. (Consultant) Providing continuity of training when trainees and their trainers are only intermittently scheduled to be in the same place at the same time. The main response to these changes has been the emergence of a consultant led service, which supports the governments aim of improving the quality of care. However, this has now reached the stage where the only continuity of care provided is through the patients consultant; and this means that there is only a weak back up if the consultant is called to cope with an emergency elsewhere. The introduction of the shift system and the collapse of the firm have meant that consultants have less continuity of contact with their trainees. In addition to continuity of care, there is also a need for: continuity of contact between the trainee and the named trainer because of the seemingly unavoidable necessity to have shift patterns that arent synchronised. It means that very often trainers are working with a trainee [with whom they are not very familiar], which means that nearly every training opportunity has to go back to base one and start again. (Consultant) I think consultants tend to be a little bit more protective of their patients than perhaps they used to be in the past, because theres so many shift changes and often a consultant will be on call with a Registrar that theyve never met before, [and whom] they dont know anything about. And there are consultants that are not happy to trust the Registrar and what they say in terms of yes I can do this. (SpR) The handover of trainees is yet another handover problem to be negotiated.

13

Organisational factors One of the recommendations of the Pre-pilot Phase was that each Deanery should have a School of Surgery to provide a focus for surgical education in its region. This has now been implemented and contributed greatly to the Pilot Phase and the implementation of the first year of ISCP, as well as the recruitment of the first ST1 cohort by the Deanery. This makes it easier to create situations that allow trainees to live in the same place for 2 years by planning rotations across hospitals that are reasonably close. However, no comparable change has occurred at Trust level. Surgery is strongly controlled by government targets and outcomes data; but surgical education is left without any such focus. Hence, one Head of School argued that the target culture, which most surgical educators perceived as squeezing out training time in theatre, could only be resisted by making training itself a target. This influence of targets on training occurs in three main ways. 1. Pressure on consultants, some of whom are hugely overworked and devote less time to trainees: I think the Government directives to reduce waiting times are a big pressure because were always getting interrupted when were in the coffee room between cases, by the waiting list co-ordinator with her big black book saying this patients about to breach, we need to add this patient on. I see the waiting list coordinator probably once or twice a day with her big black book. One consultant said that the waiting list pressure was so strong that he was now working 96 hours a week, twice the new EWTD limit; while another described his balance between orthopaedics and trauma as 80% on both. Another factor is that consultants now have a lot more reporting duties, which take up yet more of their time, but no longer get any support for the increasing amount of administration. 2. Pressure on training opportunities, with Training Lists becoming quite rare in many Trusts. Training lists, i.e. operations for the trainee to do are clearly not possible in the context of service targets driven by the Department of Health. (Consultant) Sometimes even small bits of surgery are frowned upon: Ive been in theatre and Ive had staff tell me you need to get your registrar in to do this, because it takes too long. I was taking perhaps a couple of minutes longer to close a laparotomy than the registrar would have done, and I was doing it on my own. (SHO) 3. The 4 hour A & E target is criticized by surgical trainees and nurses, because it removes carefully negotiated opportunities to operate and some patients end up in inappropriate wards before being properly assessed:

14

I think the biggest impact since weve been here is the amount of moves a patient has ... (later in focus group) ... continuity of care to me is somebody being admitted somewhere and hopefully being discharged from there. (Staff nurse) A similar problem arises when there are split sites. Even when the patients stay in the same beds, their doctors may move from one site to the other, thus weakening the continuity of medical care. Adverse effects were recorded and trainees concerned about not being able to follow their patients: We have an elective hospital and an emergency hospital ... so Im there for a 12 hour period, admit a patient with appendicitis, do an operation on them and then I never see them again, which is frustrating and probably not great for patient care. (SpR) MMC effects on the organisation of training also contribute to some of these problems. The Warwick study found that the reduction from 3 to 4 years as an SHO to 2 years as a ST1-2 appears to have created a shortage of junior doctors, some of whose traditional work now has to be done by someone else. Some of this work is now being handled by nurses, and this provides a safe consistent service; but it does remove some opportunities for junior doctor learning, for example in pre-op. More worrying is the tendency for higher level doctors to be given some of these former SHO jobs at the expense of activities more appropriate for their role. One SpR described this as acting down, and it clearly contributes to the slower progress of their training. The fact that the SHOs arent there is impacting acutely, in that we are acting down, I may not have an SHO during the day, and at night theres no SHO so I cant go home.

The use of 4 month placements has received almost universal disapproval. One problem concerns trainees settling in time, and another is the lack of a long enough stretch of practice to reach a satisfactory level of competence, and having to pick it up elsewhere. If youre doing an orthopaedic post, you just get around to doing that DHS, youre learning how to do it and then youre off. So you never get to do that DHS from start to scratch. Its the same with appendectomy. Coming from the old BST I did six month jobs. By month three of four I was going solo with the appendices and the DHSs and then I had another two or three months to build on that experience. And you develop so much quicker once you get to that certain point but if youre cut down ... you dont even get to that three month stage. Trainers have a similar problem, because they need to spend time with new trainees in order to assess what they can do, whether they can trust them and whether their trainees self evaluation is adequate for both safe practice and new learning challenges. An ENT consultant suggested it took six weeks for trainees to find their feet and that a training gain only occurred after the fourth month when independent operating practice became viable. Many consultants claimed that four months posts were a major handicap.

15

Early implementation of the ISCP curriculum Educational Supervisors and Learning Agreements ISCP brought in three new features to improve the focus and management of the learning support system. The first change was a new role, that of an Assigned Education Supervisor (AES), whose first job was to negotiate the second new feature, that of a Learning Agreements (LAs) between the AES and their assigned trainees. The third innovation was the development of a learner led Website, through which communications were expected to be conducted. The general progress of ISCP can be traced through data from the website itself, although access to personal data is very strictly confined to a limited number of people with the appropriate passwords. The first step for a new trainee is to contact the Programme Director for their Specialty in their Deanery, who then validates their position. 50% of new trainees were validated by mid-September 2007 (their second month). This reached 75% in December 2007 and 92% in May 2008. Clinical Supervisors had to be approved for the AES role by the Programme Director, and their first duty in this new role was to develop a Learning Agreement with each of their trainees. The progress of AES appointments and agreed LAs is summarised in Table 1 below, which also shows a wide range of uptake by both deaneries and specialties. This demonstrates that some deaneries and some specialties have been much more effective than others in implementing these crucial aspects of ISCP. Table 1: % of validated trainees with an AES and a Learning Agreement Website data % of validated trainees with an Assigned Educational Supervisor Overall Mean 44% % of validated trainees with Learning Agreements Overall Mean 35%
Range of Range of Deanery Means Specialty Means 8-65 18-51

December 2007

Range of % across Range of Range of deaneries and Deanery Means Specialty Means specialties 24-63 23-54

May 2008

Overall Mean 78%

Overall Mean 51%

Range of Range of Deanery Means Specialty Means 30-75 42-58

Range of % across Range of Range of deaneries and Deanery Means Specialty Means specialties 57-96 72-92

Not surprisingly, the JCST Quality Assurance survey in November 2007 (using 1-5 scales where 1 expressed greatest dissatisfaction) found that many trainees found it very difficult to schedule meetings with their AES. Trainees from three of the seven larger specialties gave more negative than positive ratings for scheduling meetings with their AES (average 3.0). The most negative response of all concerned establishing and managing an online Learning Agreement (2.6, see Table 8). This helps to explain why the response rate dropped to one third, when the same sample was asked about the extent to which they were able to reach all the learning objectives in their Learning Agreement. Moreover, 20 % of this reduced sample felt unable to reach the agreed standard in 4 or more of the 12 areas of competence.

16

Unless the sample was very biased, the higher figures reported by the PMETB survey, which finished at the end of February 2008, suggest that the AES and LA functions were being pursued off the website by a substantial number of trainees and trainers. 95% of general surgeons said they had an educational supervisor, who was being responsible for their appraisal, and 78% said they had a learning agreement; and the figures for Trauma and Orthopaedics were almost identical. This was confirmed by the Cardiff study; but the actual use of the LA may have been more limited, because two questions in Table 2 below showed that key meetings had not yet occurred for almost half the sample: Table 2: Meetings about progress and formal assessment in General Surgery Have you had a formal Have you had a formal meeting with your supervisor assessment of your to discuss your progress? performance in this post? 52 42 5 8 30 37 13 14

PMETB survey Yes and it was useful Yes, but it wasnt useful No, but this will happen No, but I would like to

One consultant recognised the teething problems caused by the new AES role, and reported that he and his colleagues would know better next time round. Now that the ISCP curriculum is explicit, we can see when trainees have not had good continuity of learning, partly because they have not engaged with their AES. Neither the AES nor the clinical supervisor understood their roles at that time; but we hope it will be different this year. They were accustomed to looking after 2 posts in their hospital, and had embraced the MMC concept of a run-through training programme; so it was a challenge to develop a structure that provided educational supervision for 17 trainees. The guidance from ISCP is that the AES should not also be a Clinical Supervisor; but he thought it would be better if each AES became a clinical supervisor for at least one trainee. Otherwise there would be no incentive to do the AES job. Nobody is purely altruistic. A lot of people are interested in clinical supervision where there is direct interaction with trainees and you can see them develop. Nobody is interested in remote advising, which is very dull and time-consuming, and not intrinsically rewarding. One advantage of distinguishing the role of AES was that it recognised the need for more than one trainer in many posts. The Wales study focused on the new Learning Agreements (LAs) and the website; and their data was gathered between November 2007 and March 2008 (months 4-8 of the first cohort of STs) from a sample of trainees and a sample of Clinical Supervisors, who

17

supported the trainees on a regular basis both before and after ISCP was launched. Both parties found these new LAs helpful for guiding their subsequent work. Unless they were absent from work, most trainees and clinical supervisors attempted jointly to access the ISCP website in the first weeks of a post. Where the supervisor or trainee had some facility with the website, learning agreements were generally followed through as intended and benefits to learning appeared to result. However, widespread delays in completing the learning agreement were caused by administrative and websiterelated difficulties. Some agreements were signed off so late as to render them ineffectual, and mid-point meetings were often ignored, particularly in 4 month posts. The small but recent London study in May 2008 showed that about a third of trainees first discussions with supervisors did not occur in the first four weeks of their posts, and one eighth of them never took place. Despite these inauspicious beginnings, most trainees and clinical supervisors agreed informal plans for training posts, recorded them on paper and followed through with ongoing interactions (while awaiting the removal of obstacles in the system). However, some participants reversion to pre-ISCP approaches to specialty training inevitably diluted the impact of the ISCP learning agreement There was a general assumption that trainees should be proactive in preparing for and setting up the learning agreement. Some trainees carried forward an awareness of their learning needs from previous posts; others such as ST1s were dependent on a prescription of generic objectives by the supervisor. The supervisors role at the initial meeting was to mediate between the trainee and the particular type of post on offer: clarifying learning needs, setting learning objectives and identifying related practice opportunities in a training post. Most trainees accepted that the learning agreement was the outcome of negotiation with their supervisor of global objectives, personal learning objectives and the realities of the post. Trainees were predominantly satisfied with the balance of objectives achieved in their agreements and with the negotiation process. One FTSTA3 trainee managed to agree objectives that met his interests in vascular and general surgery, including the kind of operative experience he was seeking; but another found that her need for advanced experience in theatres was limited by competition from a specialist vascular registrar. She also felt that the levels set for her in the ISCP learning agreement had been too low: The expectation from the programme at my level is just to be able to clinically manage, recognise patients with problems and know what to do to formulate a management plan, which I have achieved already. (FTSTA3) Trainers in the Southampton study felt they needed more formal support to keep up to date with all the aspects of being an educational supervisor. For example:

18

There is little training and support on how to deal with a difficult trainee and I need to develop skills to tell a trainee when they are not doing well, as it is often easier not to do it. One trainer in the Wales study, who used the ISCP criteria to highlight gaps in her trainees core surgical skills and knowledge, was very positive about the new system: The learning agreement) does focus your mind as a traineras to what is expected of a trainee at different levels and I think it is very objective, so I am actually enjoying it because I can feel Right, this is what I have to be doing and make sure that I have at least arranged for the minimum of that to be examined. (General Surgeon) However, learning agreements required protected time if they were to provide a continuing focus for supervisor and trainee. In a service-oriented post, annual leave of trainer and trainee, on call, night shifts, and a trainer sometimes away from the base hospital caused a mid-point meeting to be held back until the end of the post, when an Orthopaedic Surgeon felt: it was less than satisfactory than what we had initially set out to do. Two orthopaedic surgeons also admitted to having underestimated the time required to complete ISCP supervisory tasks; while a general surgeon found it impossible to set a time: to sit down and reflect, because with their rotas and shift system I never knew when (the trainee was) going to be around.

The ISCP Website The implementation of the ISCP was totally dependent on the website, because it was the central channel for communication between the many different role holders and the trainees. Indeed, the whole training process became learner-driven through the website, and could not easily function properly without it. This website was a major focus for the Wales research, which identified 24 different problems from their interviews with 18 trainees and 17 from their interviews with 14 trainers; and its prominence in the KSS research reflected the considerable attention given to it during Faculty Group meetings in the first quarter of 2008. Administrative delays and an initial lack of facility with the ISCP website highlighted the need for a full and timely induction to ISCP. Some trainees were concerned that they had only received a brief announcement of the inauguration of a web-based system, but those who had received face-to-face guidance on the website reported an immediate boost to their confidence and ability to engage with ISCP. Site design, with 15 trainee problems and 7 trainer problems in the Wales study and a cumbersome to use verdict in KSS, was the most prominent type of complaint. Several people compared it unfavourably with other similar sites in terms of its user-friendliness. I find it difficult to use, and I'm committed. I find it not a pleasurable site to use, especially at the end of a busy day. (Surgical Tutor, November 07)

19

There were major difficulties transferring records from logbooks on other sites which some trainees were using. People complained both about too much unnecessary information and missing information, such as insufficient sub-specialty-related assessments. The Wales study uncovered 7 types of wrong information, and KSS found difficulties in correcting errors. There were also concerns about confidentiality, losing data (which did occur) and possible undesirable uses of the facility such as unthinking tick boxing. Other problems included difficulties in registering (including not receiving a password even when individuals had pre-registered), problems accessing the site from various computers and relevant people not having access to the site. More seriously still, some Welsh Trusts blocked websites and three of our four commissioned studies reported that some consultants refused to use them. One SpR, for example, explained why he had had to change his supervisor: If youre a supervisor you have to be logged on ... and there are many Consultants ... who are just flatly refusing to go onto the system. Therefore, they [cannot be an] official supervisor; so people are going elsewhere to get their supervisors I wrote to the ISCP and my programme director requesting a change [of supervisor] because there was no way that he was ever going to get online and sort it out ... I changed my supervisor to another Consultant and weve been online doing all the things properly; but there are many Consultants who cant be bothered with it. Many of the problems were relatively minor, but nevertheless time consuming and irksome to people who were not in any case convinced of its benefit. The concept of a web site being a central part of the curriculum was very new to everyone. In general, trainees were more accepting of the idea than consultants and liked having an explicit curriculum. They were generally quite positive (but not to the extent of being enthusiastic) about having a record of what they'd achieved as they felt it was a more transparent system. In KSS there appeared to be a division between STs/FTSTAs who had come through the Foundation Programme and those who hadn't. The former group were familiar and comfortable with the concept of a web-based system, and saw advantages in it. The latter group found it more of a learning curve and were more preoccupied with the practical aspects. Many trainees effectively taught their consultants how to use the site, or organised it so consultants had minimal engagement. Some trainees and surgical tutors encountered problems persuading educational supervisors to use the site and felt that there had been insufficient training. However in some trusts only a minority of consultants attended the training that was provided. It was noticeable that at both the mock-ARCP1 meetings observed, the majority of trainees had few or no assessments recorded on line, despite advance notice of the meetings. By the end of the research, both groups were only beginning to understand how the web site worked, and it is therefore too early to assess its value. The juniors are taking it up more and more, consultants less so but getting better. (Surgical Tutor)
1

The Annual Review of Competence Progression which replaced RITA

20

The Wales study concluded that continuing feedback from users had been sought and acted upon to overcome administrative and website-related delays, and to refine a rigid, over-laden website; and recommended that: Efforts to ease navigation difficulties and improve the flexibility of the ISCP website should continue, and a full and early induction of new entrants to ISCP should be provided. Initial response to ISCP The teething problems of the website notwithstanding, many trainees in the Cardiff study could appreciate the continuity between ISCP and the Foundation; and experienced trainees, comparing ISCP with the previous system, recognised its potential to improve their learning: What used to happen is that you could fly along and not be sort of reviewed or assessed in any formal way and you come to the end of your 6 months in the job and sit down and have a relatively informal sort of assessment. And as long as you kept your head above water, then youd be fine to progress to the next stage. With (ISCP) in place, it provides a more formalised, a more structured approach and I do feel that when all participants are embracing it, then it works very well. (BST3) For most trainees, the ISCP clarified what they could expect in a post and gave milestones to measure progress over time, thereby reinforcing their growing sense of confidence and efficacy in conducting procedures. However, one trainee found the ISCP curriculum to be something of a mystery: There is definitely a curriculum for Foundation programme. I dont know if there is for the specialty training.I suspect its buried in the ISCP website under the general objectives. They want to produce doctors who are competent at acute management of the patient. That keeps coming up time and time again. But we dont all get exposure to those things. I know that in the run-through medical training they have to be assessed on three central line insertions. They never do central line insertions, but its in the list to have been done. (FTSTA1) Trainees recognised that their access to operative experiences was determined by factors in posts that ISCP could do nothing about: Theres been a job where theres just been the consultant and me in the hospital and Ive done lots and lots of things myself. Whereas when youve got a lot of other junior doctors and various degrees of seniority between you and a consultant, you tend to end up being a house officer and not getting to do that much. (FTSTA1) The JCST Quality Assurance survey of November 2007 provides a useful picture of the first few months of ISCP and three sections of it are presented and discussed in this subsection: Supervision, Feedback and Support, and Organisational Issues. These will be accompanied by some evidence and a smaller London survey in May 2008. Table 3

21

below presents the QA data on supervision, and is followed by some later data from the PMETB survey in February 2008. Table 3: QA survey data on Supervision (1-5 scale) Issue Educational supervision Clinical supervision Out of hours experience Elective experience Emergency experience Consultant led ward rounds Gaining consent for procedures Mean 3.5 3.8 3.6 3.6 3.5 3.6 3.9 % 1 or 2 20.7 11.6 14.4 16.9 18.9 24.0 24.0 Specialties below 3

One 2.8 One 2.7

Whereas clinical supervision was a familiar role, the concept of educational supervision was more recent in origin and had not previously been defined. It only became formalised with the introduction of the role of an AES and the use of Learning Agreements; so it is not surprising that it gave rise to a higher level of dissatisfaction than the familiar role of clinical supervision. However, the differences between General Surgery and Trauma & Orthopaedics, confirmed by the later data from PMETB in Table 4 indicate that other factors might also be involved in accessing clinical supervision that are absent from informal feedback. Table 4: PMETB data on clinical supervision, informal feedback and ethics Do you know who provides clinical supervision when you are working? General Surgery Trauma & Ortho Yes and they are accessible 81 67 Yes, but they are not easy to access 8 18 No, but there is usually someone I can contact 8 12 No, I have been left without help at times 2 4 Frequency of informal feedback from a senior clinician Daily Weekly Monthly Rarely Never

4 24 34 30 8

4 22 33 33 10

There are simple reasons why T&O clinicians might be less accessible, and this does not affect the pattern of informal feedback from senior clinicians. However, the amount of informal feedback appears to be remarkably small. This was confirmed by the Wales study, which found that preoccupation with work in the wards starved some trainees of operative experience

and was a common cause of dissatisfaction. Other than in the initial meeting, they obtained very little contact with their supervisors during their posts and little or no feedback from their seniors. Factors such as a shortage of SHOs, an overstretched

22

consultant and competition from a surgical assistant were cited as contributing to unfavourable experiences in training posts. These trainees still accessed the ISCP website and defined learning objectives to enhance their self-directed learning, but their isolation gave rise to undue caution and diminished ambition in specialty training: I think for effective learning you shouldnt expect too much, you shouldnt exert yourself too much. You should take things that are within your grasp. (If you) expect too much from that four months. I think you will have a nervous breakdown. (FTSTA1) Taking a particular interpretation of self directed, some trainees were inclined to blame themselves if a training post fell short of their expectations. Very few identified inadequacies in the Trust organisation and teaching: the formal teaching sessions, the use made of X-ray meetings and post-take ward rounds and the follow through on the care of individual patients. We also need to consider the huge variation in trainee experiences. The small London study for example, based on trainees attending anatomy courses at RCS, discovered a huge variation in the hours spent working with more senior colleagues. Their estimates for an average week were as follows: 31% spent from 0 to 12 hours 31% spent from 13 to 24 hours 38% spent 25 hours or more With this type of distribution the use of averages can be profoundly misleading, and the JCST use of dissatisfaction indicators becomes very important. Table 5 below gives some of the QA data on appraisal and feedback. Table 5: QA data on Appraisal and Feedback (scale 1-5) Issue Fairness of appraisal process Frequency of appraisal process Quality of feedback received Mean 3.6 3.3 3.5 % 1 or 2 7.8 17.4 11.4 Specialties below 3 One 2.9

The small London survey in May 2008 asked about the quality of feedback rather differently. Respondents were asked to note their position on a continuum from 100% constructive feedback to 100% undermining feedback. Although the balance is generally positive, only 75% of trainees received more constructive comments than undermining comments. This difference could be due to the small size of the London sample, the later timing of the survey or a combination of these two factors. Table 6: The balance between constructive and undermining feedback % feedback constructive 100 12 80/90 42 60/70 21 50 9 30/40 5 20 10 10 1

23

Table 7 below summarises data from another London finding, which looks at all the sources of support that trainees recognise and their relative frequencies. SPRs come first, followed by other consultant surgeons, then educational supervisors. However all seven categories make significant contributions to a significant number of trainees. Table 7: From whom do trainees get their support, and how much? Frequency Rows add to 100% AES Other consultant surgeons Staff grade surgeons SpRs Surgeon peers Other consultants Other health professionals A lot 16 14 14 38 14 6 4 Quite a bit 32 47 20 40 27 17 22 A little 42 32 32 17 44 39 36 None 9 4 17 4 9 20 25 No reply 1 3 8 1 5 9 13

On this the QA data showed that a sixth of the trainees were also concerned about their supervisors familiarity with the ISCP curriculum and over a quarter doubted their ability to provide guidance on the Personal Development Planning which underpinned it. Table 8: QA data (1-5 scale) on Induction Issue Induction to Hospital/Trust Induction to Department Induction to Training Program Scheduling meetings with AES Establishing & managing an online Learning Agreement Supervisor familiarity with ISCP curriculum Guidance on Personal Development Planning Mean 3.5 3.7 2.6 2.8 2.6 3.3 2.8 % 1 or 2 0.82 17.43 31.1 24.3 30.6 16.4 28.7 Specialties below 3

All 2.5-2.9 One 2.7 Two 2.8 All 2.4-2.9 Two 2.9 One 2.5, One 2.7, Two 2.9

Trainees tendency to blame themselves for all their problems may have also been confirmed (1) by the findings of the PMETB survey that 45% of trainees had not been told who to talk to in confidence if they had personal or educational concerns and/or (2) by the difficulties encountered by up to a quarter of them when they tried to seek help from their Deanery or their Trust, the two organisations responsible for their well being (see Table 9 below). Neither was particularly responsive and their trainers also felt that there was little they could do about decisions and practices at an organisational level, often because of the wider issues raised in the previous chapter. However, this should not
2 3

Yes/no question Yes/no question

24

become an excuse for just ignoring individual concerns, especially in relation to rotas or shift patterns (26.8% dissatisfied in Table 10 below) and the service demands of their post (22.4 %). For example a trainer, who contrasted the relative enthusiasm of two trainees, also admitted that a shortage of ST1s and ST2s created a service workload that deflected his trainee from making the most of his opportunities.

Table 9: QA data (1-5 scale) on Response to Trainees Concerns Issue Opportunities to provide feedback on Training Confidentiality of feedback Deanery response to concerns Trust response to concerns Mean 3.5 3.3 3.0 2.9 % 1 or 2 20.5 9.8 19.2 25.0 Specialties below 3 One 2.9 One 2.9 One 2.8, Three 2.9 Three 2.7, One 2.8

Table 10: QA data (1-5 scale) on Organisational Issues Issue Rota/shift patterns Administrative Support Access to IT & Internet near ward/theatre Access to Mandatory Courses Service Demands of Post Mean 3.2 3.5 3.6 3.4 3.3 % 1 or 2 26.8 16.9 16.9 16.2 22.4 Specialties below 3

One 2.9

Another consultant said that ISCP improved the professional dialogue between supervisors and trainees by encouraging focused discussion of cases: A chap came in and was diagnosed renal colic, but he was youngish and had a ruptured aneurism. That actually focused a very good training session, to sit down and actually talk about it, getting the trainees point of view as to how he managed the case and the problems that he had. (General Surgeon) This differed from his former practice of simply telling the trainee to think about ruptured aneurisms in connection with renal colic. This type of case discussion can readily promote unplanned and reflective learning by trainees, for example, on issues arising from a complication. A general surgeon noted:

25

My ST3 missed a couple of X-ray diagnoses in a short period of time that were quite important and quite serious, and he realised that. I realised hed missed one of them but I didnt realise it was two, so we talked about it and discussed it. So it was reflection from his point of view. He realised hed missed something as hed been to an audit meeting where names are not mentioned but people are criticised and (they) say How could you miss that? We talked about it and arranged for one of our radiology colleagues to meet with the ST3. (General Surgeon) This consultant felt that, for supervisors who customarily were providing this type of support, ISCPs formalised approach would mainly act as a recording mechanism. For those who were not, however, full implementation of the ISCP learning agreement and assessments would constitute a fundamental change in supervisory practice. This issue will be further discussed in the next chapter which focuses on trainees access to learning in a wide range of contexts.

26

Learning Contexts and Access to Learning This simple heading captures the heart of trainees experiences and aspirations and the extent to which they overlap. The four main contexts are Outpatient Clinics, Wards, Elective Surgery and Trauma Surgery; and the key questions are the time spent in each setting, the quality of relationships within that setting, the balance between settings, the appropriateness of the work they are allocated, the quality and timing of the advice and feedback they receive, their opportunities for enhancing their understanding of surgery, and sustaining a sense of purpose and progression. Their answers to these questions vary considerably, and providing what they want is easier in some contexts than others and in some hospitals than others. Our small London study asked first year trainees about their estimated distribution of time between working contexts. Nearly half the time of both cohorts was spent on the Table 11: Distribution of trainee time across patient contexts Ward % ST1/FTSTA1 Mean Standard deviation ST2/FTSTA2 Mean Standard deviation 47.1 30.3 47.1 27.5 Clinic % 15.7 14.0 14.5 13.7 Theatre % 26.7 10.5 30.1 16.8 Other % 10.5 11.8 8.3 16.5

Figure 1: Trainees perceptions of the balance of their time

How much time do you spend working with patients in each of these settings?

Too much

Enough Wards Too little Theatre Clinic None 0% 10% 20% 30% 40% 50% 60% 70%

% response (n=77)

27

wards, but Table 11 indicates that the standard deviation was very large. Figure 1 shows that the time in Theatre was much less than they wanted, but Table 11 indicates that it was far from negligible. The time in clinics was also perceived as too low, and its importance for learning diagnosis, decision making and understanding the patients journey was recognised. This concern was confirmed by the QA data summarised in Table 12 below, which indicates that a sixth to a third of trainees were dissatisfied with 7 of the 9 listed learning opportunities. Table 12: QA data (1-5 scale) on Issues relating to Learning in the Workplace Issue Sufficient operating sessions Practical and surgical skills Prior discussion of operating list with consultant Balance of elective and emergency work Opportunities to receive clinical teaching & training in theatre Clinical teaching in emergency work Management of critically ill surgical patients Clinical teaching at bedside Clinical teaching in outpatients Mean 3.2 3.3 3.6 3.3 3.6 3.5 3.7 3.3 3.5 % 1 or 2 31.6 23.5 13.1 21.0 17.2 16.9 12.1 27.8 19.2 One 2.8, One 2.7, One 2.9 Specialties below 3 One 2.3, One 2.9 One 2.8, One 2.9

One 2.9 One 2.8

Elective surgery Given the concerns about surgical learning at every level discussed in Section 3, this section is not confined to new entrants to specialty training; because many of the newer problems of delivering surgical education can be found at all levels. Even with more experienced trainees, trainers cannot focus only on their year of training nor, given the great variety of training posts, can they predict what knowledge and experience each trainee will bring with them. ISCP should help them to find out more about what a trainee has done, but this may tell them as much about their learning opportunities as their future potential. The Southampton study found the trainees operative logbook to be a useful start; and that most trainers would use this to ask what trainees felt comfortable doing and where they required help. The majority of the trainers wanted to have access to the most recent Record of In Training Assessment (RITA) outcomes and the previous educational supervisors report as they felt this would help in gauging what to allow trainees to do and how closely they would need to supervise various aspects of surgical training; but this confidential document was not available to them. Hence all the trainers tended to start with the trainee assisting them in theatre and vice versa to get a feel for the level of operative competence, but in addition they would watch closely a trainees decision

28

making and clinical management skills in the ward and outpatient clinic setting as it was felt that this was a good marker of competence and awareness of ones limitations. The professional attitude of trainees was seen as perhaps the most powerful indicator of what to allow trainees to do, particularly in the operating theatre. Trainees aware of their limitations and with insight and knowledge of when to ask for help, were viewed as trustworthy and were most likely to be allowed to get on with things. Behavioural traits, demeanour and approachability, reliability and punctuality were identified as important aspects, because trainers saw them as strong predictors of how trainees were likely to behave in theatre and clinic settings. Youve got two levels; Level one is recognising the problem and Level two is actually having the ability to ask for help appropriately. Once a trainee has these two, you can let them fly, even if they are at a very junior level, because you know you will get called if theres a problem. Trainers also felt that providing good training opportunities was dependent on the development of a good working relationship with their trainee. This was influenced by the professionalism of the trainee and also the amount of continuity that trainers got with their trainees. The shift system meant that trainees were often off on night shifts, days off after night shifts, and annual or study leave. Consequently trainers saw significantly less of their trainees and had much more disjointed working relationships with them than previously. Trainees noticed that trainers who were generally supportive, respectful and interested in supervision generally offered good training experiences. So they were wary of being either micro-managed or left to swim without appropriate support. They also recognised that the mode of supervision dictated how much they actually got to do operatively and how well they were taught to do it. Ive worked in places where they had the old style apprenticeship style training and you are left to thrash on and I learnt a lot in those positions, but I have to say I never got trained to do a case. Similarly I have been in environments where the minute the boss puts his gloves on, he takes over and does the case. (Trainee) All trainees recognised that what you put in, you get out and that the experience is more positive and productive if you are prepared to work hard and contribute to the service delivery of the team. Most of them also understood the necessary balance between being proactive and keeping patients safe. I think when you are doing a case you have to push yourself and maybe feel slightly uncomfortable because if you dont do that a little bit, you dont progress. You have to be very careful doing that however, that you dont step over the boundary of moving into a situation that you dont have control of. Learning where that boundary is about recognising danger and your own confidence in your own abilities without being over confident in your own ability. (Trainee)

29

There are very wide differences in how trainee learning is supported in theatre time, but very little discussion, evaluation or research into different ways of using theatre time effectively for patients, trainees and targets. Organisational and individual approaches to Trust based surgical education need to be urgently developed, and trainers given some support in learning to use approaches that work well for all the stakeholders. Appendices A, B and C provide accounts from individual surgeons with excellent reputations as surgical trainers. These focus on the issue of progression and how they deal with it, as well as voicing views on current problems. Emergency Surgery and Trauma The ISCP curriculum indicates that the early years of training should focus mainly on minor injuries, because that gives them early access to practical surgery and ST1s and ST2s are very interested in basic emergency cases. Thats also where many registrars get their first experience of training, and that is good for them. However, very little consultant time is given to emergency work, and most emergency work beyond minor injuries is done by trainees without consultant support. Hence trainees come to see elective work as their main learning focus, and emergency work as detracting from it. This creates a terrible paradox. At least 50% of the current registrars time is spent on emergency work and trauma, where there is minimal consultant-led training and no formal assessment; because very few consultants have time dedicated to emergency surgery in their job plans. But the amount of time trainees spend operating is much greater in emergency work than in elective surgery, and the patients there are much sicker. In most places, people are on call when doing elective work; but when trainees are called to the Accident & Emergency department their consultants rarely go with them. Only a few hospitals have arranged for consultants to be on call and take junior trainees with them, so they can train them in emergency work; because this would require a larger number of consultants. Trainees in most hospitals feel very uncomfortable in emergency settings. Their lack of confidence leads to them deferring decision making by doing more tests and getting a CT scan; but delaying the operation may not be in the best interests of patients, who potentially have a problem for which time is critical. Trainees are also very reluctant to call their boss, which is a major problem. It may be due to a macho culture, or just because they dont know what they dont know. What is clear is that they never get trained, they just get on with it. Emergency patients do not get reviewed like elective patients. They dont have discussion meetings. Data is not collected on emergency cases, unlike cancer, which has rigid decision making protocols. If trainees dont know that they need to get the patient to theatre quickly, they will get a CT scan and then they will call someone, and only then get ready to operate. Two days later the patient gets multiple organ failure, because of that earlier time delay. This never gets picked up. This is a real problem. Beginning trainees see trauma as the most exciting work. Then its interest fades when their consultants show no interest. There they are in the middle of the night beyond their comfort zone, and they dont enjoy it. It becomes a chore. Its hard to know what works. (Trauma consultant)

30

Without this support, trainees treat emergency surgery like elective surgery where the expectation is that they work it up first, then present a full story, explaining everything theyve done and backed by a package of information. But this delay reduces the patients chance of recovery. If you have a complicated case with an elective patient, you take it to a multi-disciplinary meeting. You present the X rays, have a discussion with all the specialists and come up with a plan. Whereas, if youve got a trauma patient in front of you whos just spilt thee litres of blood onto the floor, and youre not sure whether hes got something in his abdomen or his chest, you dont have that luxury. You have to be quick and make a best guess decision. You need to be prepared for a range of possible problems; and if youve got it wrong, you need to be prepared to do something else. Its a totally different context. Its not very simple medicine, and its not easy to understand what youve been doing. Suddenly switching from elective to emergency is very difficult, especially if you get little training or support. (Trauma consultant) A few hospitals have a strong group of trauma surgeons, but most hospitals do not. So they pick up a case and give it to a Year 2 trainee, who rings up the nearest trauma centre and asks if they can transfer the patient. They dont present a clear enough brief to forewarn the consultant, even though the evidence shows that the whole process is much better when there is a consultant at the end of the phone. The people who meet the Trauma Team are usually the registrars. They will have done the basic course in Advanced Trauma Line Support (ATLS), which gives them a basic understanding of priorities; but they dont get much more training after that. They may get some training when a consultant is called to a case and they discuss it afterwards, but it is unlikely that the consultant would discuss the case on arrival, and give feedback on what they had already done. So they could very easily be learning the wrong things. Clinics for outpatients Nearly all the surgeons who contributed to this evaluation stressed the importance of decision making and clinical judgement. These were seen as being even more important than technical proficiency and probably less easily taught. But earlier practices involving joint decision making between trainer and trainee are becoming both less frequent and less likely to involve any joint examination. Clinics involve interacting with outpatients and coming up with a diagnosis and a management plan. Trainers are more likely to meet an unfamiliar problem there than in theatre. One trainer described it as follows: In clinics you assign some patients to your trainee, and they usually come back saying what they think it is and summarise their reasons. You say OK or Lets talk it through. Trainees say that they are not being taught in clinics, just providing a service. I would say that they are still learning, but they are not watching someone else do it and often there isnt much discussion; so its not what you could call a learning event. If you are very busy, you may have to ask them to give the patient an X ray and arrange to see them next time. Youre given very little time with outpatients.

31

Another trainer, who is also a FRCS examiner, now thinks that clinical examination and diagnosis is the greatest problem, because now there is so little training. Trainees are assumed to have learned these clinical skills at an early stage in their training and the pressures of time in the out-patient clinic may make it difficult to observe a trainee examining patients except on an occasional or irregular basis. There are frequently no clinics when patients can be routinely examined jointly and it is far too late to discover this lack of clinical skills in an FRCS (Trauma & Orthopaedics) examination. In many cases the trainer relies on the trainee to pick up those cases that need to be discussed further in the clinic. Better practices prevail in clinics where surgeons are under less pressure; but he gains an impression from examining for the FRCS that, in many cases, it is the basic clinical examination skills that cause problems for candidates. This suggests that the problem of teaching in the clinics is relatively widespread. Ward work Table 11 confirmed the findings of three of our commissioned studies that ward work takes up nearly a half of the working time of junior trainees, and that they perceive this burden as too much. Even staff in other professions perceived that that the number of ward work commitments expected from junior trainees had a detrimental effect on their other potential training opportunities. Theres so much to do on the wards ... and it takes away sometimes from the time theyve actually got ... with the registrars or ... in theatre so they dont actually have that much time to learn, I would imagine. (Physiotherapist) However, Nursing staff and Physiotherapists did perceive that, despite needing to improve further, doctors communication skills and holistic care were getting better: I think communications a little bit better of the junior doctorsThey might not be communicating enough within their team, but with nurses and patients, I feel its a lot, lot better. Thats in my unit. (Ward Manager) They see the patient as a patient and not just an appendix. (Ward Manager) However, this lack of balance may also lead to junior trainees underestimating the learning opportunities available through ward work. The research studies and interviews provided very little information on ward-based learning, except to argue that it was not pure service work, but also offered learning opportunities (see below). The London survey showed that support for leaning was most likely to come from a registrar, who spent more time on the wards, than from a consultant (Table 7); and this might account for the non-recognition of apprentice-type learning from registrars. One consultant interviewed did choose to include his contribution to ward-based learning and described some of his common practices as follows: Post-op ward rounds are a bit easier. One way is for the registrar to do the ward round and report back, so the consultant can decide which patients she wants to look at. The pressure is much less and there is more time for discussion with any junior doctors. Or

32

the trainer might go round with the registrar and junior doctor and stay in the background to see how they do it. If there was a complication, the registrar would be the first to notice, because they see the patients every day. He will still remember the operation, so he can link his memory of the case to any later complications. The skill lies in recognising when there might be a problem. That comes from experience and knowing how to react to it, and whether to ask for help. Simulators, Models, Endoscopy and Recordings Practice work on simulators helps trainees to develop hand-eye coordination. If more work is needed in this area, then the supervisor can ask the trainee to do more work on the simulator. Trainees are able to make time for that practice. A number of the trainers in the Southampton study also expressed the need for additional resources to aid training within the work environment, particularly given the constraints of time for training. Such resources included teaching materials, simulator environments and maybe even clinical environments that were set up purely for teaching and training. One trainee found that, even though people talked about hernias as simple, it took quite a bit of time to understand the anatomy. This was confirmed when she conducted a study looking at trainees of different experience levels working on a model of a hernia. The most novice participants (who had never done a hernia repair under supervision in theatre) couldnt quite understand what the model represented, and despite detailed written instructions on how to carry out a hernia repair, they couldnt work on the model without some guidance from her. In contrast, slightly more experienced participants who had performed only a few hernia repairs themselves, were able to practice their skills independently on the model, which helped to consolidate what they had seen. So, if used that way, the practice on the model could complement training time in theatre. Endoscopic surgery allows everyone to see what is going on. Deficiencies are more obvious and it is easier for trainers to teach and assess the trainees. However, the use of recordings for training purposes has not been developed so far. A trainee who did some laparoscopies in her last job found that having the visual element was an added advantage when training. In general, watching recordings of operations was helpful, especially if there was a guide to take you through it. This could be used a lot more. The consultants who are most helpful as teachers are those who can verbalise what they are thinking. But sometimes it is helpful to figure it out for yourself. We could be doing a lot more learning about operations outside the theatre. For example, if I watched 25 hemi-colectomies, I might get a feel for what I might encounter. Next time I would know what I was aiming for. Sometimes I feel that looking at lots of examples would be better than watching the same one over and over again. You see whats typical and what looks doable. Further use of recordings in the sharing of expertise is discussed later in this section in the context of coaching and distributed apprenticeship.

33

Formal Education Activities Our small sample of London trainees provided the data in Table 13, when asked about participation in the education programmes offered to them by their Trusts or Deanery. Table 13: Availability and Quality of Formal Educational Opportunities Response (%) No response Not available Not at all Too little Enough More than enough Lectures 4 25 6 32 31 1 Journal Club 3 32 14 25 25 1 Audit 5 4 4 21 57 9 Skills Training 4 35 22 32 6 0

These indicate that some formal education programmes are available, and some significant demand for more of them; but one could not assume that more time would necessarily lead to more attendance in a context where time is very scarce. The QA Survey covered formal teaching sessions and audit, and Table 14 indicates that these three questions had the highest dissatisfaction of any group. The means are not the lowest, but the range of trainee expectations and/or preferences appears to be more diverse than for other topics. Table 14: QA data (1-5 scale) Formal Teaching and Audit Issue Regular formal teaching sessions Ability to attend these sessions Participation in Clinical Audits Mean 3.0 3.3 3.5 % 1 or 2 35.9 30.3 28.2 had none Specialties below 3 Three 2.9

The PMETB survey also asked trainees about the way in which critical events and near misses were reported in their work environment, an important indicator of the informal culture of audit; and their response is given in Table 15 below. Table 15: The Informal Audit of critical events and near misses Action or inaction Reporting is encouraged and followed up Reporting is hazardous and not followed up Staff are reluctant to support due to a blame culture Gen Surg % 58 35 7 T&O% 61 33 6

34

When is Service Work a learning opportunity? The Warwick study asked both trainers and trainees about service provision and training opportunities, and found a marked dichotomy of perception amongst surgeons as to what constitutes service provision and what constitutes training. Consultants and most SpRs (particularly the most senior SpRs), generally perceived that there are opportunities for training in every daily activity: I still am learning from virtually every patient I see ... were all life-long learners and the opportunities are there. This thing of oh thats service provision, its not training is a complete fallacy. (Consultant) The problem is that learning from service work is not recognised as learning. He still learns everyday though his service-work. This used to be part of the tradition; but learning is now presented as something different. (Consultant) At the other end of the spectrum, junior trainees generally seemed to perceive that service provision and training opportunities were virtually separate entities: Theres definitely a conflict of interest. Ive found it in the post Im doing at the moment between providing service and training opportunities. The emphasis seems to be on service and seeing patients, which is obviously important but has a knock on effect on training opportunities. (ST1) I think theyve got to highlight in every post what your training opportunities are, different from your service provision. If they dont have enough training provision, then I dont think it should be an ST job. (ST1) The JCST Quality Assurance survey found this to be a major source of dissatisfaction. Table 10 showed that the mean was 3.3 and the percentage of 4 and 5 responses was 22.4%. One example of this was picked up by the London surveys question about the amount of time spent by junior trainees performing tasks involving the collection of clinical information, which could often have been devolved to others: None Too little Enough Too much 2% 4% 48 % 45 %

There is some form of middle ground however, in that more junior trainees and some SpRs feel that the current environment is more geared towards service provision but that there are opportunities within these activities for training: Theyre all service environments and if you can squeeze in training as a bonus, thats how I see most of these opportunities really. (ST1) In so far as training opportunities are recognised by junior doctors, they are also seen as part of the same service-dominated hierarchy:

35

Your first priority is the ward patients. The second priority is theatre and the third priority, if you can, is clinic. But its such a pressured environment because everythings service provision. (ST1) One consultants explanation of the service-training dichotomy was that trainees see the job of a surgeon as exclusively concerned with doing surgery. For them training is time in theatre. Hence they do not value other skills. Indeed there is a tradition in surgery of valuing surgical skills the most. He then continued to take a more conciliatory position: Lots of what we include as training isnt seen as training by the trainees. However, there are some activities that trainees are asked to do that do not contribute to training, probably because nobody has seen how they might be made relevant. For example if a trainee is called to help in a clinic because the clinic is very busy, they are unlikely to be observed when they are examining patients; so there is no obvious educational component. You have to work out how you can train these doctors in that environment. There are no quick fixes. You have to use opportunistic examples rather than formal planning. For example, children with a spinal tumour are traditionally seen as a nightmare; but this week a trainee said it was one of the best educational activities he had experienced. He only had one patient to present, but he observed several others. He said there were lots of things he had not thought about, and it hadnt taken very long. The London survey also posed junior trainees a direct question on the learning opportunities associated with their ward work. Some of the variation depicted in Figure 2 Figure 2: Learning from Unsupervised Work
To what extent does your unsupervised work...

40% 35% 30% consolidate your skills 25% 20% 15% 10% 5% 0% Not at all Not much A little A lot help you learn about variations of the same condition

re

sp on se

36

may reflect different practices and some may reflect different trainees perceptions of the same practices. Discussing these issues more specifically and looking for examples of good practice might be a good way to progress from the current situation. Beneath this difference of opinion lies the tension between two different theories of learning. The medical tradition has been that of apprenticeship, while medical educators have tended to emphasise more focussed teaching as being more effective; and this has, not surprisingly, been picked up by government. The irony is that recent research into work based learning outside medicine has shown that most of the learning discovered by researchers is regarded as working rather than learning; and therefore not even thought about as learning. This has led to a reappraisal of the concept of apprenticeship in professions from which it has disappeared or was never used. There is now an increasing recognition that both approaches are needed, and that finding the right balance is important. Apprenticeship, Informal Learning and Coaching The concept of apprenticeship suffers from peoples failure to recognise that learning is often embedded in working and cannot be separated from it. My own research, involving longitudinal research into the post qualification learning of accountants, nurses and engineers found that this hidden learning could be detected by changes in a persons practice over time. The majority of learning events occurred when learners described themselves as working; but they did not consider events of this kind as involving learning, until the interviewer asked about how their current practice differed from that on some previous date; and even then they might need to be prompted by questions about different or more challenging work. For example, a surgeons experience of many cases enhances their ability to differentiate cases and treat them accordingly, without any overt changes in their formal knowledge; and trainee surgeons may encounter variations in patients, which gradually enable them to recognise when things are going to plan or need immediate attention, without recognising that they may have learned something. It is just a taken for granted aspect of their job. Nevertheless, my research found that this informal learning could be greatly enhanced by the quality of relationships in the workplace, appropriate levels of challenging work and opportunities to work alongside a range of other people with different or more advanced expertise (Eraut 2007; Eraut & Hirsh 2007). The London survey investigated the extent to which different groups of surgeons and other professionals engaged with ST1 and ST2 trainees through: On the spot discussions of clinical/professional episodes (Table 16) Discussions about the trainees work (Table 17) Discussions about the trainees general progress (Table 18)

Not surprisingly, on the spot discussions of clinical/professional episodes were the most frequent for every professional group, discussions about the trainees work were considerably less frequent and discussions of a trainees general progress were the least frequent.

37

Table 16: Frequency of on the spot discussions of clinical/professional episodes Frequency Rows add to 100% AES Other consultant surgeons Staff grade surgeons SpRs Surgeon peers Other consultants Other health professionals Daily 6% 13 17 56 39 3 18 Weekly 13% 35 17 22 18 6 8 Monthly 26% 19 6 8 6 12 4 Less often 40% 13 12 6 12 27 16 None 10% 10 25 4 9 22 22 No reply 4% 9 23 4 16 30 32

Table 16 above suggests that informal on the spot discussions may already be more important than we recognise, with 56% of SpRs discussing clinical/professional episodes daily and a further 22% being engaged in such discussions at least once a week. 39% of surgeon peers engage in on the spot discussions daily and a further 18% discuss clinical/professional episodes with their colleagues at least weekly. The data on discussions with consultants and other health professionals needs to be read with care. One cannot infer that trainees engage with other consultant surgeons more than their AES (designated supervisor), only that those they do see engage in on the spot discussions more frequently. Nevertheless the frequency of discussions with other consultant surgeons (48% at least weekly) is much higher than that of the AES (19% at least weekly), which suggests that most trainees have more on the spot discussions with other surgical consultants (perhaps more than one) than with their AES. Given that these trainees spend an average of 47% of their time on the ward, the level of trainees discussions with other health professionals about clinical/professional episodes is disappointingly low.

Table 17: Frequency of discussions about a trainees work Frequency Rows add to 100% AES Other consultant surgeons Staff grade surgeons SpRs Surgeon peers Other consultants Other health professionals. Daily 1% 1 6 13 8 3 0 Weekly 8% 12 8 25 25 3 5 Monthly 30% 25 6 16 3 4 5 Less often 48% 31 16 21 17 14 10 None 9% 19 39 17 27 53 52 No reply 4% 12 25 9 21 23 27

38

Tables 17 and 18 show that discussions about a trainees work are much less frequent than discussions about clinical/professional episodes, and discussions about a trainees general progress are even less frequent. SpRs are still the most likely group to engage in more frequent discussions, both about a trainees work and about their progress. 38% of then discuss work at least weekly, but only 8% discuss general progress that often. Just under 40% of both AES and other consultant surgeons engage in discussions about a trainees work at least monthly, but AES (26% at least monthly) are rather more involved than other consultant surgeons (19% at least monthly) in discussions about trainees general progress. As might be expected, discussions about general progress are rarely as often as weekly, so Table 18 has a reduced the number of columns to enable better comparisons of this very important activity. The role of staff grade surgeons is more prominent in Table 18, and not insignificant in Table 16; but the data does not indicate their frequency. However, Table 16 does indicate that at least a half of the trainees met staff grade surgeons in their workplace. Table 18: Frequency of discussions about a trainees general progress Frequency Rows add to 100% AES Other consultant surgeons Staff grade surgeons SpRs Surgeon peers Other consultants Other health professionals. at least monthly 26% 19 22 47 9 3 5 less often 60% 39 19 23 27 9 19 none or no reply 14% 42 58 30 65 89 76

There are elements of coaching in the traditional relationship between surgical trainers and their trainees; but there was little incentive in traditional apprenticeship for them to try and verbalise their expertise. The prevailing assumption of apprenticeship was that of learning by participation and the occasional explanation, rather than learning by being taught. Research in many occupations has shown that even experts whose work is primarily verbal cannot describe their own expertise. So it is hardly surprising that surgeons, whose expertise has strong visual and physical dimensions, do not try to translate their experiences and practices through words. However, their communications with trainees can now be considerably enhanced by using recorded material to discuss groups of cases, including some seen by the trainees themselves, to explore the similarities and differences across a group of cases. Trainees could preview and review these cases both before and after the main discussion and then return with any remaining questions after that. The inclusion of cases the trainee has recently encountered is important for three reasons: motivation, the possible inclusion of more of the patient

39

pathway than the trainee experienced first hand, and making up for the normally 2 dimensional vision of recordings and the absence of physical interaction with the patient. These extra facilities would enhance the scope for trainers to engage more fully in coaching activities, because discussions around images and other forms of evidence make it possible to communicate without struggling to develop an over-precise discourse, which can never fully convey the relevant information. Coaching is increasingly used in management development in a wide range of organisations in relation to the improvement of more complex or more sophisticated skills. It normally includes sharing diagnoses of managers current performances at periodic intervals, an assessment of their underlying strengths and weaknesses, drawing upon a range of progression models for suggesting an appropriate learning trajectory, and discerning different aspects of complex performances in a range of possible situations. Thus coaching goes beyond a master-apprentice relationship that relies on demonstration rather than analysis, by seeking to discuss, with the aid of mediating artefacts such as pictures, recordings or pieces of evidence, just how the learner might usefully progress. Two key assets of coaching are that it aims both to develop excellent performance and to speed up the learning process, and this is just what surgical education needs to do.

40

Competence and Assessment Despite the North American tradition of equating competence with individual capability, I prefer to define competence in social terms, as meeting other peoples expectations. Precisely whose expectations are to count will depend on the local situation and its micro-politics. The importance of this definition is that it recognises the everyday role of the notion of competence, both in the workplace and as a mediating concept between (1) professionals and technicians and (2) their clients and the general public. Although lists of competencies carry general understanding within an occupational sector, judgements of competence are still very situation-specific. Not only does this specificity derive from the context of the performance, but it also covers the expectations of each individual performer. Irrespective of any relevant qualifications, expectations will differ according to the performers experience, and supervisors and managers have a duty to ensure that their workers do not get assigned to tasks beyond their competence. The ideal pattern of work for trainees allows them to consolidate their competence through further practice, while also expanding their competence through a combination of peripheral participation and coaching. However, even for experienced workers, what counts as competence will change over time as practices change and the speed and quality of their work improves. Thus, from a learning viewpoint, competence is a moving target. The nine Specialty Advisory Committees have identified what trainees should know, and be able to do at each stage of surgical training, by specifying a range of key topics and conditions (those that are common or critical) that trainees would be expected to encounter and be able to manage. In practice there are learning trajectories within each of these, as trainees engage with more complex cases, as well as the overall learning trajectory of procedures based on difficulty or rarity, which is made explicit by the allocation of procedures to particular stages of training. The current system of trainees navigating their way through a long series of posts, whose contributions are deemed to be equivalent, is based on the pattern used in formal education, even though these posts significantly differ both in the work they do and in the extent to which trainees can access learning opportunities through participating in that work. However, a persons position on any trajectory will depend on access to practice and the availability and quality of teaching, feedback and support as well as individual confidence, commitment and talent. Hence we are bound to get different profiles for trainees in the same cohort. This means that we need to plot trajectories over time to ensure continuity of learning both within and across placements and on into consultant work. Many trainers instinctively think this way, but the formal structure just assumes that different placements are equivalent. The ISCP system of assessment emphasises participation by giving trainees the responsibility for initiating formal but formative assessments; then fails to consider the cultural realities of the surgical workplace. It uses three types of assessment for ST1s,

41

whose approaches use similar normative judgements (i.e. are judged according to subjective norms of performance) for different purposes: 1) Case-Based Discussion (CbD) to assess clinical judgement, decision making and the application of medical knowledge in relation to patient care 2) Mini-Clinical Exercise (Mini-CEX) to assess skills essential to the provision of good clinical care 3) Direct Observation of Procedural Skills (DOPS), which is procedure specific and intended for the formative assessment of the simpler procedures. These together cover a wide range of 25 competencies but, unlike most competency based assessments, they are normative rather than criterion referenced. They are used with available patients, and only involve making a few notes while the patient is present; although they need more time later to fill in the forms. The system asks for a variety of assessors in order to reduce bias, but otherwise pays no attention to the wider context in which the assessments are used. First, the request for a range of assessors makes it difficult to find them and often involves assessors, who are not aware of the trainees learning trajectory. Thus the assessment record becomes an uncoordinated series of snapshot judgements of single events by a range of different assessors, who may have little stake in the result. In practice their supervisors and senior registrars may act as assessors, but their time with the trainee is scarce and many feel that they would be better engaged in giving them informal feedback and advice in much greater detail. Although these assessment tools are intended to be formative, all the evidence collected in the evaluation suggests that trainees treat them as summative, and do not use the tools until near the end of their posts. One reason for this is cultural. After a strong competition to gain entrance to medical school and get a good degree, followed by another competition to get a training post as a surgical trainee, it is not surprising that trainees regard that all assessments are competitive and will help them to gain access to the more highly valued placements. So they do not do many assessments until near the end of their posts, so that nothing appears on their record except their best results. Some trainees value these assessments more than others; but our evidence suggests that those who favour assessments do it to get feedback, while those who feel they get sufficient feedback from their supervisor, have no incentive to take the assessments. This is not surprising, because the assessment forms are generic and give little scope for any feedback related to any modes of progression that might be used in a learning trajectory. This summative interpretation by trainees worries trainers who have strong educational values, but careful inspection shows that it is reinforced by the design of the assessment tools themselves. While the range of competencies is fine, the judgements used to describe them are not criterion-based, as one might expect when competencies are used, but normative. For each competency, the assessor is asked to judge the trainees performance on a 6 point scale from Below expectations for level of training to Above expectations for level of training (see Appendix D). This assumes that the assessor has prior experience of observing trainees working on similar cases and that they have had similar learning opportunities. Since a wide range of people are eligible to be assessors,

42

most of whom do not have this experience, the use of normative judgements of this kind is extraordinary. We encountered very few trainers who trusted these assessments, and most were exacerbated by their consumption of precious training time. Surgeons are practical people who can judge what people actually do, but to go beyond that into the uncertainties of comparison against a rapidly changing trainee population is neither easy nor necessary, as long as the full range of competencies are properly used. The JCST Quality Assurance data on Assessment is presented in Table 19 below. One feature of this survey is that respondents were asked to rate the Importance of each question as well as their Satisfaction with that aspect of their work. This was not reported earlier, since all the ratings were rated 4 or above. Only five out of 49 issues were rated less than 4 and four of them were in the area of assessment; the fifth was the effect of cancelled admissions to theatre! More detailed evidence on trainers and trainees response to this new assessment regime can be found in the four commissioned reports. Table 19: QA data (1-5 scale) on Assessment Issue Mean Import Satisfact ance ion 4.2 2.8 % 1 or 2 Specialties below 3

Guidance on scheduling assessments and identifying assessors Demonstrate competences in: Learning Agreement Surgical DOPS CBD CEX PBA Mini PAT

41.8

One 2.4, One 2.5, Two 2.8, One 2.9

4.1 4.0 3.8 3.7 3.8 3.7

3.2 3.4 3.3 3.3 3.1 3.2

19.2 15.9 18.2 19.4 21.7 18.4

One 2.8, One 2.9

There are two more positive aspects to the assessment process. The Mini PAT (Mini Peer Assessment Tool) is quite a different form of assessment, based on the concept of 360 degree feedback and including other professionals views of a trainee. This was very novel when first introduced, but is now being much more favourably received by trainers and by at least some trainees. It gives a different kind of feedback, based on nontechnical aspects of the surgeons role, and it is proving very useful in the early detection of people who might turn out to have personality problems that render them inappropriate for becoming surgeons. However, it is quite time consuming for trainees because they have to nominate their respondents; and like many other assessments the number of suggested uses per post is quite impractical. The Procedure-based Assessment (PBA), which serves the same purpose as DOPS for more complex procedures, has a very different criterion-referenced design, which so far surgeons are finding much more useful. Its appropriateness is currently being investigated by Professor Beard in Sheffield.

43

Portfolios and logbooks The PMETB survey collected some quantitative data on learning portfolios and logbooks, which is presented in Table 18 below; and the Wales study collected some qualitative data. Table 20: Trainee use of Portfolios and Logbooks General Surgery 83% 92% Trauma and Orthopaedics 90% 83%

Using a portfolio Using a logbook

The Wales study concluded that there was an expectation that the ISCP portfolio should do more than document basic competence in surgical training; but it had yet to match the range of content and flexibility of format of the Foundation Curriculums portfolio. In particular trainees needed places on the portfolio to present reflective learning and wider evidence of progression and development, as well as guidance on these aspects. There were also signs that ISCP is viewed as a collection of disconnected tasks to be completed as and when convenient - a form of training by checklist. Hence participants need professional development around the educational aims, content and processes of ISCP to counterbalance a preoccupation with the mechanics of the website. Finally, the ISCP portfolio is a potential vehicle for specialty trainees to integrate formal teaching and supervision, self-directed learning, assessments, informal feedback, and other components of ISCP. Trust-backed processes are required that integrate assessments with the individual trainees ongoing learning and supervision. The quality of discussion of the portfolio in the ARCP may determine whether the regulatory aspect or development of a coherent educational approach is prioritised by trainees in their surgical posts. Annual Review of Competence Progression (ARCP, formerly RITA) One consultant interviewed had just spent 3 days on RITAs and ARCPs, and the sessions focused on the new ST1 and ST2 grades replacing the former SHOs. The purposes of this meeting were: 1) To support the annual assessment of competencies, and 2) To make decisions about run-through trainees. This was the first time that formal assessments had been made at this stage. Previously, with SHO rotations, you just went from one 6 month job to another with little, if any, thought about progression. Often, but not always, this rotation was within the same hospital. If the SHO was pretty awful, some action would be taken; otherwise there was little formal discussion of progress. Then the hurdles were examinations, including MRCS, and competitive interviews for a SpR job (now ST3). Now they have to decide (a) about ST1s going to be ST2s, where if ST1s are judged to be weak, they can be given

44

remedial support as ST2s or even have to repeat the ST1 year; and (b) about ST2s going to be ST3s, where this is not possible. Thus, in June 08, they took on the role of the previous Appointments Committees and will do so again in June 2009 for the final cohort of run through trainees. They had a large committee of about 8 people, and we distributed each trainees portfolio across the group, with each document going to only one person. They relied almost exclusively on paper documentation, mainly ISCP documentation. The most valuable document came from the Trauma and Orthopaedics Specialty Advisory Committee (T&O SAC), whose 4-5 page document of each trainees progress comprised a list of objectives for the ST1 and 2 years and a grid listing the evidence of each trainees achievements for each competence. There were also copies of the assessment tools (Mini CEX, case-based discussions, etc.) and a list of extra activities, such as MRCS, the Basic Surgical Skills course and the Advanced Trauma Life Support course (ATLS). They could also interrogate the website, but they rarely did. They looked at Learning Agreements, where the supervisors free text comments were a very rich source. It wasnt just the achievements, but the way in which the STs engaged in the process that gave you an idea of their capability as a learner. They did not see a single document in which any domain was graded less than satisfactory. Trainees were not supposed to repeat or dispose of poor assessments, but they may well have done this. They were a very good group of run-through trainees in spite of all the MTAS fuss. Their portfolios were wider ranging, and good documents could be used as tools for educational planning. Many former FTSTAs were not as good; but some were good and had applied for another post next year. Some had obtained ST2 jobs and were going to, in effect, repeat the year. The ARCP group were pleasantly surprised at being able to make decisions based on paper-based evidence alone, although some of the summary documentation was poor.

45

Teamwork and relationships Good relationships are critical for surgeons in clinics, theatre and wards; and they are especially important for trainees who have to take more responsibility for their own learning. The demise of firms makes organised communication between doctors more important, if it is to be more effective than ships passing in the night. Junior doctors need the support of the whole surgical team if they are to reach the registrar stage in two years, rather than three or four years; and this includes all the relevant health professionals, especially nurses and physiotherapists who were included in the Warwick study When they arrive, we give them an appropriate challenge and need a period of time to see them in action. We have got used to spending a month on this, but we dont have that time so we will just have to shorten it. This will only be possible if we have sufficient contact with them. Given the increased risk when new cohorts arrive, it would be good for both safety and trainee learning to limit trainer absences over that early period, and maximise the time that trainees have with their new trainers. All trainees in the Southampton sample recognised the importance and implications of relationships: If you get on well with the trainers and have a good relationship with them then the training opportunities somehow become easier and better. (Trainee) I think if you respect them, and they respect you and if you are relatively good at what you do and put the graft in, they are always willing to support and teach you (Trainee)

One FTSTA3 trainee in Wales argued that specialty training remained fundamentally rooted in apprenticeship: I think (ISCP is) a useful tool, but. I still think that like they did in the old days, (training is) basically an apprenticeship. Its the direct contact that gives you all the feedback you need. Its the immediate feedback you get to improve yourself and know what youve done right or wrong on a specific occasion and build on that; and then all this accumulates and your experience and your skills develop, which will eventually show up in the ISCP assessments. (FTSTA3) Intricately linked with the issue of following the patient journey is the issue of feedback, which surgeons of all grades perceive to be very important in optimising surgical training, but somewhat lacking in practice, because of shift working and handing patients over to other teams: I also try to feed back to trainees....whove done shifts on emergency cover with me, but thats not easy. (Consultant)

46

Weve introduced a shift system where you dont know what youve done is right or wrong. (Junior doctor) Behind these problems lies the greater difficulty of forming relationships in a shift-based system. More attention could be given to a general induction of trainees over their first month and not expecting them to take the initiative in potential learning situations quite so quickly. Proactive behaviour is very risky without good relationships and a culture of mutual acceptance. Consultants and some nursing staff perceive that junior trainees do not utilise all available training opportunities, and attribute this mainly to shift working issues, which cannot be easily changed. Whereas encouraging them to build relationships and to ask for advice or check out their decisions with other team members until they have found their feet would help them to acquire the confidence they need to take up more challenging activities. Nursing staff cite that junior trainees do not use the knowledge of the nursing staff as effectively as possible, while consultants cite instances such as operating lists and clinics, where trainees might be able to observe and learn: For me I think that the opportunitys there. Some good trainee will utilise those opportunities. For some very good ones they can turn things around and go to your extra theatre sessions. Sad to say the majority of trainees - Im not sure what they are doing really. (Consultant) One Trust checks up on progress by seeing all their trainees in December to get a handle on them. They then encourage them to reuse the material they provide for that meeting as part of their entry to their next rotation, when they have their first meeting with their new clinical supervisor. This encourages trainees to develop some insight into both what is required and their own current abilities. This helps them to focus on their learning priorities while they still have some time, instead of leaving it to the end of the year when ARCP is looming close. Consultants and SpRs were particularly concerned about decision making. Their perception was that one of the most important aspects of surgical training is how to make decisions, and that this was more important than being able to operate. They felt that most junior trainees did not recognise this, and attributed this change to trainees having less opportunity to make decisions in a Consultant-led service: I think thats very true because in surgery there are two things. The first thing is the technique, the second thing which I think is more important is making decisions ... (once) ... youve had to make that decision on your own, then you come out on top ... then youve learned it properly. (SpR) I worry about our current juniors, how on earth are they going to be able to be able to make decisions because the current emphasis on training is that everything has to be partnered by a Consultant; and it strikes me as well that theyre not actually interested in that thinking process. (Consultant) A BST3 trainee in the Wales group described a colorectal post as the best six months of my training to date. Self-directed learning was still at the heart of this post, but in this case the supervisor stimulated the BST3 with discussions of clinical cases and with

47

guidance on further learning such as conducting a literature search. Adequate academic teaching combined with surgical experience to support this trainees efforts to integrate theory and practice. Consultant supervision was complemented by a registrar, and indeed by the entire surgical team: If Im having difficulty with sort of finding something or coming to terms with something, (the registrar) will always step and in and help out. I just think the dynamics of the team at the moment are really healthy and consequently its really easy to bounce things off and air that. (BST3) However, his seniors support for learning in surgical practice was largely informal, and the trainee admitted to completing the formal ISCP components retrospectively. On reflection, he did not feel that the ISCP had made any difference to his training experiences: In the absence of ISCP, if Im honest my experience of the job and my training wouldnt necessarily have been different, but ISCP probably augments it or stands side by side with it and provides some kind of structure to it. (BST3) A Wessex trainee also spoke very positively of teamwork: Being part of the team and feeling like a valued member of the team is crucial and you want to feel that you are part of the team that is functioning well, as well as feeling that you are working for someone. (Trainee) But many trainees made comments about teamwork, which related to its absence rather than its presence (see pages 5 -6). We can help each other a lot I think, more than we do. So I think it depends on the individual. A lot just arent really interested and they want to do the exciting stuff. But I think, especially the experienced nurses, know so much more than the doctors when they first start. They could get a lot from us. (Staff Nurse) Nursing staff and Physiotherapists, when asked about the training of surgeons, perceived that there was an inter-professional lack of awareness about roles and training, which affected both surgical training and patient outcomes. They felt that surgical trainees were unaware of the role of a nurse or a physiotherapist in the care of a surgical patient, and they were unaware of the full role of a surgical trainee: Were not sure of the demands upon them. I dont know what their rota is. They must have to spend a certain amount of time in theatre, in clinics, in A&E, on-call ... I dont know. (Staff Nurse) Another sort of problem with the training of doctors is that theyre not totally aware of what our roles are as physios and what we can do. (Physiotherapist)

48

Nurses and physiotherapists also perceive that they would be able to help in the training of junior surgeons if they had some knowledge of what their training pathway entailed, and what surgical trainees were expected to achieve in any given placement. I dont know if anybody has actually been given a structure of what we are supposed to show the junior doctors. Are we supposed to be instructing them? Because thats something weve never been told. I dont know what a junior doctor is supposed to achieve ... I personally havent got a clue. (Ward Manager) We just dont know what theyre supposed to be doing, so were not assisting. (Ward Manager) This theme relates to another of our secondary outcomes. Nursing staff and Physiotherapists perceive that, as they become more senior, they are able to guide the more junior trainees, especially now that they are less experienced than in previous years. However, they also feel that junior trainees have a much heavier workload than previously: The more experience I get, the more confident I am about being able to answer questions, whereas when I first qualified I would barely speak to a Doctor. (Staff Nurse) They dont like admitting they are out of their depth or that theyre not happy with something ... and you can see them walk onto a ward and see a senior member of staff and look vaguely relieved that theres someone they can ask. Handover of patients Consultants and SpRs perceive that more junior trainees have a reduced sense of responsibility for their patients. It is perceived that this is mainly due to shift working and handing patients over: The junior doctors at the moment are in this hand-over, diffusion of responsibility, atmosphere which theyre going to carry over to registrar level. Which means its going to get worse. (SpR) The most commonly raised issue was one of incomplete or poor documentation, which was raised by members of all professional groups, because of its adverse effects upon patient outcomes: Discharge summaries are very important in continuity of care ... if you leave it to ... most registrars and most SHOs they go through fifty discharge summaries and do the minimum possible, the minimum needed and youre left with an incomplete document. So thats bad for continuity of care. (Consultant) Unless theyre writing clear plans in for that patient, particularly at the weekends ... so that, if they get seen out of hours by someone else, they can see exactly what they wanted to happen. (Ward Manager)

49

Some doctors notes vary from being very comprehensive to being very minimal. When its minimal, thats when youve got problems. (Physiotherapist) Nursing staff and physiotherapists also perceive that communication between themselves and doctors needs to improve for the benefit of patient outcome: There isnt this structure of a ward round, you dont always know when a patient hasnt been seen ... a doctor has seen a patient and gone and very often youll get the instructions from the patient. (Staff Nurse) A lot of the Doctors will say one thing to the patient whereas well say another thing; and I think thats where you need to have the bridge really. (Physiotherapist) The colorectal people would hand over all the gallbladders and the upper GI guys would hand over all the diverticulars and there was an awful lot of handing over. And the vasculars would get all the vascular, so actually continuity of care was going to be changed by specialisation, even without the shift system. (Consultant) The data from the PMETB survey of trainees also suggests that the handover problem is not confined to trainees; but needs to be properly organised and implemented by professional teams, not left to junior trainees alone, Table 21: Handover Arrangements Nature of arrangements An organised meeting of doctors and other professionals An organised meeting of doctors Phone or email communication Informal None Before Night Duty Gen Surgery T&O 15 % 13% 53 0 32 0 44 0 44 0 After Night Duty Gen Surgery T&O 15% 32% 58 1 25 1 54 0 13 2

50

Support for Trainers Hitherto, we have focused on how trainers and others support trainees, but given little attention to support for the trainers themselves. The Kent, Surrey and Sussex (KSS) Deanery was commissioned to research the support for trainers in their deanery during the period when ISCP was being introduced. KSS has the largest Postgraduate Medical Education Department in the UK, comprising four full time education advisors, one full and one part time careers advisor, and a further nine part-time education advisors employed on a consultancy basis. They hold the Education Contract for the training of junior doctors with each trust, and offer a wide range of workshops and certificated programmes. At the start of the ISCP implementation process, 187 surgeons within KSS held the Deanerys Certificate in Teaching, which is assessed through a minimum of three observed teaching sessions including clinical teaching on ward rounds, in theatre or outpatients. From the outset of their research it was evident that the KSS Deanery and the KSS Regional Co-ordinator for the RCS Eng. had a clear and agreed strategy to facilitate the introduction of the ISCP, based around a model of local, trust-based Surgical Faculty Groups. The rationale for this model was that it would facilitate ownership of the curriculum, by involving staff in mutual adaptation of the curriculum to local circumstances. This approach seeks both to adapt the more detailed aspects of a curriculum without upsetting its key principles and to adapt the new context to enable it to gain the intended benefits of the new curriculum. The research focus was on observing the meetings of the Surgical Faculty Groups and on interviewing surgeons about both the role and impact of the meetings and their experiences of introducing ISCP. The research was conducted during seven months prior to and eight months after the implementation of ISCP in August 2007. This included two training workshops in each Trust between March and August 2007 to induct staff into ISCP, followed by a series of Faculty Group meetings from July 2007 onwards. The timing of the Induction workshops reflected a belief that surgeons would be more interested immediately prior to the ISCP starting, rather than several months ahead; but it also coincided with the period when many surgeons had just been involved in the new and time consuming recruitment process (see page 9). Between July 1st 2007 and March 31st 2008, one group met monthly and one bimonthly. The others were less frequent: they all had one meeting before November 30, but four groups had no more meetings before Easter; 4 of the others had 3 meetings over the nine months; and two had two meetings. Only 23 of these 32 meetings had records, from which the frequency of the topics discussed could be discerned (details are in Appendix E). Many of these topics are also discussed in the other commissioned research reports, so they have been incorporated into the appropriate parts of this main report. While the introduction of Faculty Groups began as a response to ISCP, the wide range of problems discussed both in the KSS Report and this Main Report suggest that a great deal more local discussion is needed to find ways of addressing the issues that threaten the quality of surgical education. Faculty Groups could have an important role to play in

51

facing what all my respondents have judged to be a major challenge to the future of surgery. Hence the KSS conclusions on this particular innovation are repeated here to give a starting point for further discussion. The KSS research concluded that the Faculty Group Model could provide the following benefits: A mechanism for curriculum implementation and management A forum for education debate in which doctors exchange views on current or future developments, and in the process improve their educational knowledge and understanding An opportunity for trainers to ask questions and clarify uncertainties. This should help them to fulfil their role and promote consistency in approach Support for educational supervisors, particularly when experiencing problems with a trainee An opportunity for consultation and joint planning, for example about the content and timing of teaching programmes. This gives relevant people greater involvement in decision making than if the surgical tutor acted alone Highlighting actual and potential problems and fostering discussion of ways to resolve them. This might prevent some problems from occurring, and would increase trainers' range of strategies for tackling difficulties. Stimulate new links between surgeons in different sub-specialties, or on different sites An opportunity to share and compare current practice Standard setting, such as defining minimum requirements of ST posts and checking timetables to ensure implementation Inviting senior colleagues, e.g. Director of Medical Education, senior managers, Board members, to meet the surgeons at least once a year. Attention was also drawn to the following examples of good practice in Surgical Faculty Groups: Having two representatives per sub-specialty to ensure they are represented at each meeting Including trainee representatives from different sites and years, whilst still including a closed part of the meeting to discuss trainees progress Organising open meetings for all trainees and supervisors to give and receive feedback Holding formal meetings to check trainees' progress mid way through the first placement Publicising all available weekly teaching in sub-specialty areas Having members with expert knowledge or relevant experience

52

The degree to which the groups were effective appeared to be influenced by a number of factors: The number and consistency of people attending The degree of knowledge and ownership of the curriculum (the Trust which had piloted the curriculum was well ahead of the others for whom it was new) Effective leadership, influenced by the educational experience, curriculum knowledge and commitment of the surgical tutor The existing ethos of the hospital in relation to education (possibly virtual/vicious circles related to the social deprivation of the area, the quality of doctors recruited and quality of training offered) The degree of cohesion amongst the surgeons involved (those which were already a tight knit group prior to implementation got off to a quicker start). The degree to which a core of committed people attended the group, The ability to influence relevant areas within the Trust affecting education. The extent of the educational knowledge within the group. Another significant issue raised by the KSS study was the position of the Surgical Tutor. The role of the Surgical Tutor is to organise the teaching programme for trainees; but traditionally, the role has been given to the newest consultant in a department rather than being appointed through a selection process, and there has been no requirement for the tutor to have expertise in education. Hence, it came as no surprise that the majority of surgical tutors were ill informed about the curriculum until shortly before and, in some cases, even after it commenced. Nearly half were new to the role and lacked appropriate knowledge or experience. The RCSEng has been developing a new job description for the post which recognises that the role has become much more demanding and requires someone with more educational expertise as well as enthusiasm and leadership qualities. They also want to introduce competitive selection, despite surgeons' view that there would normally only be one applicant in any trust! To attract more interest, the post needs greater status something that may take time as it requires a cultural shift. However, appropriate funding for the post itself and for administrative support would help; and the increasing prevalence of teaching courses and qualifications means that there will be a small but increasing pool of surgeons who can demonstrate educational expertise. A further step for the Surgical Tutor might be to become the reporter of the Faculty Group and a communication channel between the group and the trainees. In most Trusts, the educational structure is less than a skeleton; and there are endless comments about the lack of time in job plans for teaching. Generally, it seems that consultants were expected to teach during PAs allocated for non-clinical work, but consultants who took a major role in teaching often had no more time allocated than those who took little or no role. Some key individuals in Trusts have started to discuss the idea of withdrawing funding from those not teaching; and at least one Head of School has also signalled this intent. Whilst many felt that everyone should be involved, they also recognised that some of those who did not wish to make a significant educational contribution might be poor teachers. Towards the end of the research period the Deanery issued guidance on the roles and responsibilities of educational supervisors, and recommended the time that should be allocated for the various educational roles. Whilst

53

it was felt unlikely that this would be taken up by Trusts, it was nevertheless seen as a positive development. One strange aspect of this situation is the lack of any financial audit of the manpower gain received by Trusts through the work done by surgical trainees; or indeed of the clinical governance implications for the Trust of not taking the quality of training seriously. The KSS study found that Surgical Tutors varied in their contract, but generally had little, if any, allocated time for the role. Administrative support also varied, but was generally little. Some tutors devoted a lot of unpaid time to their role. For example the tutor at one trust had completely revised the trainee rotas in a way which required major changes in practice and negotiation with colleagues, and set up a training programme with two neighbouring trusts. This had been done without any allocated time or payment. Another tutor ran all his Faculty Group meetings out of hours, and was expected to pay back any time taken for Deanery meetings, shortlisting or interviewing by running extra clinics. This was despite a letter from the Dean Director to Chief Executives and Medical Directors requesting the support of trusts to ensure recruitment activities were undertaken and time given. The tutor described how education had always been seen as important role but enthusiasm for it was being 'squeezed out by lack of appreciation and support.' On the other hand, one trust was reported to have paid consultants an extra session for their shortlisting activities, so there was variation between trusts. Generally, there was an enormous cynicism from surgeons about getting any support from trust management for the new educational roles, and this was acknowledged as a problem in the initial workshops. In one, for example, the RCSEng representative described 'awful problems in getting buy-in from chief executives'. Suggestions that management representatives might sit on Surgical Faculty Groups tended to be met with scepticism, sometimes bordering on derision. There was a frequently expressed perception that management drives on the clinical side were detrimental to the training of junior doctors. It was suggested several times that, with all the requirements which the ISCP put on consultants, some hospitals might prefer to lose training grades and use staff grades instead.

54

Distributed Apprenticeship and the Organisational Dimension This report has clearly shown that the implementation of ISCP depends on many factors outside its remit. Some changes may be desirable, especially in assessment, but the major problems that have developed in surgical education since the start of this century cannot be addressed by surgeons alone. Unless these problems are tackled, ISCP will not achieve its mission. Nevertheless ISCP itself, and the many conversations and consultative meetings during its development and implementation, have clarified these problems and confirmed their significance throughout the UK. There is little to be gained by reviewing past decisions made in good faith, because too many contextual factors have combined to create the current problems. Moreover, in order to avoid accusations of special pleading, it will be important to engage with other stakeholders in order to reach some collective understanding of the current situation before attempting to develop some possible options for developing a safer future. There appear to be four important areas of response to the challenging range of problems described in this report, and all of them are important: 1) A risk analysis of the current situation to increase the collective understanding of all the stakeholders of the impact on training and service 2) Piloting new approaches to the organisation of surgical training within hospitals, with appropriate backing from the NHS 3) Training of individual surgical trainers 4) Training of surgical teams in all the settings discussed above These four areas of response will be discussed two at a time Risk analysis and new organisational approaches to surgical education The main conclusion arising from the evidence collected for this evaluation is that surgical education cannot achieve its current goals without significant changes in its current state. There has been a major reduction in the training time per annum of both trainers and trainees, and opportunities for trainees and their main trainers to meet together have been drastically reduced by the new shift systems. The results are that: The current trend suggests that trainees performance in elective surgery will remain safe, but they will take longer to reach CCT at the expected time Training in clinics has become problematic because, in most Trusts, trainers and trainees have no time to see any patients together Training in trauma has been very limited, and is far from meeting an acceptable standard.

The KSS study of support for trainers during the introduction of ISCP made it clear that, unless surgical trainers in Trusts are better supported, it may become very difficult indeed to recruit them and train them properly for their training roles.

55

It has not been part of our remit to search for good models for organising surgical education, and by implication future surgical performance, at Trust level; but the evidence we have collected suggests that, even if they do exist, they are not being actively disseminated. Hence a risk analysis of future surgical capability is urgent, and it seems unlikely that any serious attention will be paid to this at national level without that kind of evidence. Both Trusts and surgical departments create the frameworks within which both teams and individuals work and learn; and these frameworks, both formal and informal, can either constrain or enhance the quality of work. Moreover, this is the level at which awareness of risk is a critical consideration, which cannot just be devolved to lower levels. One interesting example of this is the ongoing work on Trauma services at the Royal London Hospital, which started its risk analysis with a review of all their deaths by categorising them as non preventable, possibly preventable, probably preventable or errors of care. All deaths are reviewed in an open morbidity and mortality forum every month. So cases are discussed with other consultants and trainees (from all surgical disciplines and emergency medicine, ICU, intensive care, pathology etc). Important learning occurs at this level, and any errors in care are then passed up to the consultant peer-review committee to decide on remedial actions, e.g. training, protocol development, changes in the organisation of resources, etc. Most of the changes sanctioned by this committee are either educational or organisational. Over three years this has reduced errors from 9% to 1%. Nobody else collects that kind of data, but now they have shown that it is possible, and others can compare themselves through the national trauma database. At the same time they reorganised their services to create: A consultant led trauma service Dedicated wards for trauma patients and A review of how they handle every single aspect of a patients care.

Three Trauma Surgeon consultants rotate the role of Trauma surgeon of the week with responsibility for all trauma patients in the hospital. This involves doing initial operations, any further operations deemed necessary and ward rounds, and on top of that theres a lot of administrative work. However, there is an ongoing problem on the education side. They dont yet provide a robust education system for everyone involved with trauma patients; and they still have huge problems with staff turnover, partly because the 4 month and 6 month posts dont start at the same time, and nobody responds to the protocols. They need to put checks into the education system with senior consultants leading it, but training is difficult to timetable. We can conclude from this case study that it is both possible and extremely important to conduct ongoing risk analyses of surgical activities, and to use them for learning by all those involved; because this provides a crucial direct link between patient outcomes and educational provision. When connected with the concerns about surgical education revealed by the evidence gathered for this report, a second conclusion also emerges: that

56

if the current organisational practices affecting surgical education cannot implement the changes required for improving patient safety and other patient outcomes, then the structure of surgical education will have to be reorganised. The training of surgical trainers and surgical teams Currently, there is provision for the training of trainers at Deanery level, some universities and from the Royal Colleges. However, this does not yet give much attention to accelerating trainees progress or developing teamwork. The advantages of coaching and greater use of recordings in training were discussed earlier (page 24); so in this final section I will focus only on the fourth area, that of training surgical teams. Research across several professions has shown that team performance is improved by continuity of membership; and this applies to trainees even more than experienced surgeons. This challenge is not only of great importance, but is becoming increasingly difficult to meet. In addition to the problem of continuity between team members from different professions, there is a rapidly increasing problem of continuity between surgical trainees and their trainers. The key question to be addressed is that of how far it is possible for surgical trainers to develop continuity of training for their trainees. Not only are trainers meeting their trainees less often, but they know very little about what their trainees may have done with other consultant colleagues between their own meetings. The handover of trainees from one consultant to another is more difficult then the handover of patients, because there is no shared discourse available for this kind of mutual communication; but without such a discourse trainers have to ignore trainees experiences that they have not personally witnessed. I suggest that the best starting point for this exploration of what is possible would be to address situations where individual trainees may suffer from discontinuity of support, and their individual voices could contribute to the discussion. The first problem is to decide when communication between trainees and their consultant trainees (or experienced registrars?) is needed for sustaining trainees continuity of learning; and the second problem is that of developing a meaningful discourse for the mutual understanding of those communications. There is no point in asking for more communication until the problem of a shared discourse can be overcome; and the problem of developing an appropriate common discourse cannot be underestimated. Tacit knowledge and subtle differences in surgical approach are difficult to share; and will probably require visual communication at some stage. Moreover, thoughts about a trainees progress may take the form of concerns or uncertainties, rather than prophecies or hard advice; and can only be shared if they are treated as highly provisional. This is why trust in colleagues and confidentiality about uncertain communications are essential. This ambitious but very important endeavour could be supported by mediating artefacts such as recordings or still pictures around which meaningful discussions could take place. In this context the knowledge being developed and shared includes both the artefacts and the discussions which reveal other both other peoples interpretations as well as ones own. For example, still pictures taken at intervals would enable those present at an operation (not necessarily only surgeons) to add separate short commentaries on each picture about what they were thinking about at the time they were taken and their later,

57

more reflective, thoughts. These commentaries do not have to be accurate representations of on-the-spot thinking, that would be impossible; but subsequent discussions of these commentaries should help to improve communication and mutual understanding between those concerned about their respective views of surgical events. The trust engendered by these initial joint activities should create the interpersonal relationships needed for addressing the development of teams who can begin to collectively improve the quality of their service to patients and trainees. Over time this should help to develop the common discourse, which will be needed if surgical training is to progress from its original apprenticeship system to a more transparent and reflective system of distributed apprenticeship, in which a group of trainers supports one or two trainees and offers them the continuity of training that now appears to be essential for making progress in the next few years. References Barden, C.B., Specht, M.C., McCarter, M.D., Daly, J.M. and Fahey TJ (2002), Effects of limited work hours on surgical training. Journal of the American College of Surgeons, 195 (4), 531-8 Bollschweiler, E., Krings, A., Fuchs, K., Pistorius, G., Bein, T., Otto, U., Muhl, E., Backes-Gellner, U. and Hlscher, A.H. (2001), Alternative shift models and the quality of patient care An empirical study in surgical intensive care units, Langenbecks Archives of Surgery, 386, 104-9 Eraut M. (2005) Evaluation of Phase 2 of the Intercollegiate Curriculum Project, ISCP Eraut M. (2007) Learning from Other People in the Workplace, Oxford Review of Education, 33 (4), 403-422 Eraut M. & Hirsh W. (2007) The Significance of Workplace Learning for Individuals, Groups and Organisations, SKOPE Monograph 9, Oxford. Gagnon, J., Melck, A., Kamal, D., Al-Assiri, M., Chen, J., and Sidhu, R.S. (2006) Continuity of care experience of residents in an academic vascular department: Are trainees learning complete surgical care?, Journal of Vascular Surgery, 43, 999-1003 Henry, M.C.W., Silverman, B.L. and Moss R.L. (2005), The impact of the 80-hour work week on pediatric surgical training: an Association of Pediatric Surgery Program Directorssponsored study, Journal of Pediatric Surgery, 40, 60-68 Tooke, J., Ashtiany, S., Carter, C., Cole, A., Michael, J., Rashid, A., Smith, P.C., Tomlinson, S. and Petty-Saphon, K. (2008), Aspiring to Excellence Findings and Final Recommendations of the Independent Inquiry into Modernising Medical Careers.

58

Recommendations 1. The level of concern about trainees progress is very high. A significant number of trainees, now due to become consultants, have received less practical experience than their predecessors and feel less confident as a result. This group began their specialist training before the introduction of EWTD and the governments Waiting List Targets initiative. So we can now reasonably predict that both the practical experience and the training experience of each successive cohort will decline every year for the next six to eight years. The impact of this process needs to be modelled, so that the risks for the quantity and quality of future surgical consultants can be better predicted and contingency plans can be developed for plugging the major gaps in their expertise. 2. The risks associated with the current use of surgical trainees in Emergency and Trauma surgery need particularly urgent attention. 3. The training that does occur is less effective than previously because of the limited continuity of trainers. When trainers and trainees meet less often and trainees have several trainers, there may be little or no communication between the trainers involved with the same trainee. This important problem is far from simple, but needs to be urgently addressed. Developing a parsimonious but effective mode of discourse between trainers, and between trainers and trainees, could be enhanced by initiatives such as using still pictures and short audio commentaries by trainee and trainer. 4. Trainees describe service work as devoid of learning, while trainers argue that most aspects of service work provide good learning opportunities. This issue could be addressed by giving methodological attention to groups of cases of the same condition, focussing on (a) their similarities and differences, and (b) patient pathways from clinic to aftercare and audit. The latter could be usefully enhanced by contributions from nurses, physiotherapists, specialists in u imaging and pathology, etc. 5. There must be provision for more occasions where trainer and trainee see the same patients in clinics and follow some cases through from there. 6. The organisation of surgical work depends on the particular circumstances of individual Hospital Trusts. However, more attention to the training dimension of Training Posts is urgently needed. Examples of successful trainee-friendly organisational arrangements need to be developed and evaluated, so as to give a range of effective choices to Trusts.

59

Appendix A Progression is affected by (1) the general difficulty of the type of operation, which is taken care of by the curriculum level to which it is allocated; (2) anticipated variations between cases; and (3) unanticipated difficulties which only appear after the operation has started. The normal practice is to teach the trainees one stage at a time, but the trainee should still be learning by observation about the stages they have not yet begun to practice. The first stage would be making an entry, and that would be followed by dissection. We do move on to the second stage before the first stage is fully up to speed, and we dont mind taking time over it. But trying to accelerate a trainees progress is greatly handicapped by not seeing them very often; so no single surgeon is closely aware of their ongoing learning. A two week gap is a strong disincentive to trainers who might otherwise feel able to follow a challenging progression pathway. However, the root of all our difficulties is that we know that, if we take one patient off the list, we can give the trainee some very good training; but then we will get the management on our back! We have done a great job in getting the waiting lists down, but it has been at the expense of training the next generation. In 10 years time new consultants wont be able to do the work that we do now. Trainers are losing their motivation, because they are not getting the satisfaction they used to get from the developing competence of their anticipated successors. We would expect an ST3 to be doing hernias essentially unsupervised, and youd be worried if they had only done 10 before. Starting alone would depend on the trainers confidence, the trainees confidence and the trainees innate ability. The problem is that they might be starting a new post, and it could take time for their new trainer to get to know them. You dont go to a new trainer with a green ticket. You would probably do 510 with the next trainer before going solo. Then it becomes part of your regular work. After going solo, you still meet cases beyond your expertise. My experience is that this is one or two in every 20. They still learn from every case, its a process of gaining experience. By the time they have become a consultant, they will have specialised; but they will still be able to do the intermediate stuff. In your last year you would be doing more complex stuff, but assisted by your trainer. Its a progression. You wouldnt need help with the more basic advanced stuff such as laparoscopic colectomy or gall bladder. In more complex rectal work you would be part of a team of surgeons but in the pilot seat for most of it. Finally you would do it with another team which you direct. The concept of apprenticeship uniquely captures his views of the most effective approach to surgical learning. This involved continuity of training from two consultant trainers and much more time in theatre. [One trainer would also be good, but they have too many duties today for that to be possible.] This enabled trainees to become fully competent during a six month rotation. Trainers did not see themselves as trainers, they do training because they have trainees with them. Trainees learned an inexorable amount of stuff just by being there. The key principle is that To learn your craft, there is no substitute for practising your craft as much as possible.

60

Appendix B A specialist trainer of experienced registrars reported wide differences in the capabilities of the trainees he has received, and found getting all of them to the right level to be a considerable challenge. Many of them came with only six months of exposure to lower limb joint replacements, an effect of greater sub-specialisation and the consequent need to rotate through different subspecialty posts to gain exposure to the broad spectrum of Orthopaedics. If they miss part of that they are left with a big gap in their experience. The current norm is for trainees in their last two years to spend four six months attachments, which will to a large extent (depending on demand and availability, concentrate on their chosen subspecialty area (or possibly two such areas). It is sometimes necessary for a trainee to be held back for six months or a year to gain sufficient experience and competence before becoming a consultant. He also discussed his experience of new consultants and not so new consultants who declined to be involved in training. His tentative explanations were that some felt they needed to learn more themselves and gain further experience at consultant level, while others did not feel confident about letting a trainee surgeon do the operation. Both are understandable, because there are some very new challenges to being a successful trainer in this specialty. First, you have to look at the operation from the opposite side of the patient from the operating surgeon, and this makes it very difficult to give advice on cutting angles etc. Second, cutting by an experienced surgeon is instinctive, and more effective when you dont have to stop and think about it. It can be very difficult to deconstruct what you do in order to advise a trainee. A relatively unusual feature of his training practice is that, despite their varied levels of experience, he starts his first operation with a final year trainee almost immediately (usually within the first week) with the trainee at the helm, while he takes the role of an assistant, albeit a very helpful assistant. It should be noted however that if the trainee is very slow at operating it may become necessary for the trainer to do the second arthroplasty himself as there may be insufficient remaining operating time for the trainee to do both operations when there are two arthroplasties on the list. With more complex operations a greater proportion of the procedure will be performed by him; but the trainee is advised to expect to perform at least part of all operations, and most come prepared to do so. He will show how he does specific aspects of the surgery but this is done as they perform the surgery. There is quite a bit of transfer between the hip and knee operations, as well as generic operating skills. He normally has approximately four arthroplasty operations a week available to trainees and his trainee doesnt miss many of them, even though there are other duties, including trauma work. So they could do 60 or so over the six months. He acknowledges, however, that many surgeons would feel uncomfortable with this approach.

61

Appendix C There are differences among the consultants on what registrars should do in theatre. Two, perhaps three of us, are happy to be primarily assistants to the trainees for nearly all cases; and believe that nearly every case should be a training case, once we have got to know them. If someone came at ST5 level from another vascular clinic, we would expect them to lead from the beginning. The other 3 consultants are more cautious, and wouldnt let registrars do carotid surgery but would let them lead in some other cases. My informant thought that some consultants reluctance to train was not about confidence in trainees, but about confidence in themselves. In particular: 1. The trainer has to feel that he can get a trainee out of trouble, or use his finger to prevent an inappropriate cut. 2. Outcomes of operations are now published, and these include the outcomes when trainees were operating. Hence trainers do the most complex cases. Perhaps thats right anyway? 3. Those who overbook their lists are not expecting to train anyway. These consultants dont feel confident when juniors are operating, they are always wanting to take over, even if the junior is just being a bit hesitant. Their ploys include That stitch is easier from my side and Let me just show you this, after which they continue operating until the end. Unless you have planned who does what in advance, it is better to start by physically standing in the position of the assistant, so that the trainee knows that they are expected to operate. If theres time pressure, you do some bits and try to delegate the bits on which the trainee most needs to get more practice. In a complex operation, this is often better for the trainee because it is difficult to pick up all the learning points from a whole operation. For example, when teaching a carotid to a registrar on a six month post, you might spend the first two months on dissection, the second two months on cleaning out, and the last two months doing both. An ST 2 might just do the start and the stitching, focusing on the stitching technique. There would be no point in going further if they were a prospective liver surgeon, but the stitching is generic. Getting the logbook entries right is important for trainees progression. For example, terms like performed or supervised mean very little to another trainer, unless they are accompanied by details about which parts were done by the trainee in which roles. Trainees need to be involved in some way in a wide range of operations. In elective vascular surgery, this might include carotids, abdominal aneurisms, lower leg reconstruction, etc. His last ST3 had 180 procedures across this range. His current ST5 is very competent, and is doing almost all the operating. He is now focusing his learning around decision making: thinking through the steps in the case for himself, dealing with any complications he might cause, e.g. cutting a vein.

62

Appendix D Front page of DOPS to be inserted

63

Appendix E Table 22: Frequency of topics discussed at KSS Faculty Group meetings Topic
Number of times discussed at Faculty Group meetings from July 07 - March '08

On-line registration/learning agreements Teaching programme for trainees setting up Faculty Group remit, composition, links Workplace Based Assessments Composition of Rotations Monitoring/evaluation of programme Other teaching for trainees Resourcing for ISCP, e.g. computer facilities, admin Recruitment/selection issues Other changes affecting ISCP Job Planning PAs for teaching Staff development for trainers Educational responsibilities roles of individuals/groups FTSTA posts/ career issues Concerns about or achievements of trainees Feedback from CTSTC RITA/ARCP assessments Tooke Report Other issues

15 11 10 10 10 10 7 7 7 6 5 5 4 4 3 3 3 1 1

64