The American Journal of Surgery 185 (2003) 146 149

Surgical education

Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills
Teodor P. Grantcharov, M.D.a,b,c,*, Linda Bardram, M.D., D.Sc.b, Peter Funch-Jensen, M.D., D.Sc.a, Jacob Rosenberg, M.D., D.Sc.b
Department of Surgical Gastroenterology L, Aarhus University, Kommunehospitalet, Aarhus, Denmark Department of Surgical Gastroenterology 435, University of Copenhagen, Hvidovre Hospital, Hvidovre, Denmark c Department of Surgical Gastroenterology D 16, Copenhagen University, Glostrup Hospital, Ndr. Ringvej 29-67, DK-2600 Glostrup, Denmark
b a

Manuscript received March 5, 2002; revised manuscript August 12, 2002

Abstract Background: The study was carried out to analyze the learning rate for laparoscopic skills on a virtual reality training system and to establish whether the simulator was able to differentiate between surgeons with different laparoscopic experience. Methods: Forty-one surgeons were divided into three groups according to their experience in laparoscopic surgery: masters (group 1, performed more than 100 cholecystectomies), intermediates (group 2, between 15 and 80 cholecystectomies), and beginners (group 3, fewer than 10 cholecystectomies) were included in the study. The participants were tested on the Minimally Invasive Surgical TrainerVirtual Reality (MIST-VR) 10 consecutive times within a 1-month period. Assessment of laparoscopic skills included time, errors, and economy of hand movement, measured by the simulator. Results: The learning curves regarding time reached plateau after the second repetition for group 1, the fth repetition for group 2, and the seventh repetition for group 3 (Friedmans tests P 0.05). Experienced surgeons did not improve their error or economy of movement scores (Friedmans tests, P 0.2) indicating the absence of a learning curve for these parameters. Group 2 error scores reached plateau after the rst repetition, and group 3 after the fth repetition. Group 2 improved their economy of movement score up to the third repetition and group 3 up to the sixth repetition (Friedmans tests, P 0.05). Experienced surgeons (group 1) demonstrated best performance parameters, followed by group 2 and group 3 (Mann-Whitney test P 0.05). Conclusions: Different learning curves existed for surgeons with different laparoscopic background. The familiarization rate on the simulator was proportional to the operative experience of the surgeons. Experienced surgeons demonstrated best laparoscopic performance on the simulator, followed by those with intermediate experience and the beginners. These differences indicate that the scoring system of MIST-VR is sensitive and specic to measuring skills relevant for laparoscopic surgery. 2003 Excerpta Medica Inc. All rights reserved.
Keywords: Surgical performance; Laparoscopic surgery; Virtual reality; Training; Assessment; Learning curve

Minimally invasive surgery has developed enormously during the past decade. The method is today the golden standard for cholecystectomy and antireux surgery and is beginning to demonstrate advantages compared with conventional open methods in a number of other procedures. However, a major limitation for laparoscopic surgery is training. There is consensus that the educational activities in minimally invasive surgery should be intensied and assessment of surgeons skills introduced in order to ensure
* Corresponding author. Tel.: 45-2826-0934; fax: 45-4673-4616. E-mail address: ttgrant@dadlnet.dk

good quality of treatment. Virtual reality simulators are gaining territory as means of training and objective assessment of psychomotor performance [13]. These systems allow repeated practice of standardized tasks and provide unbiased and objective measurements of laparoscopic performance, but their wide application in the surgical training programs is not yet generally accepted. The Minimally Invasive Surgical TrainerVirtual Reality ([MIST-VR] Mentice Medical Simulation, Gothenburg, Sweden) is a simulator that has been in use for several years, but the familiarization rate on the system has not been investigated. This study aimed to analyze the learning curve

0002-9610/03/$ see front matter 2003 Excerpta Medica Inc. All rights reserved. doi:10.1016/S0002-9610(02)01213-8

T.P. Grantcharov et al. / The American Journal of Surgery 185 (2003) 146 149

147

patterns for surgeons with different laparoscopic experience and to provide evidence on the construct validity of the computer system by comparing the performance scores of the three groups of participants.

calculated by summing the scores for each hand into a single value. Statistical analysis Data analysis was performed using the scores from task six, which includes elements from most of the other tasks, is of highest complexity and requires the highest level of concentration and coordination. Previous studies have shown that performance scores on this task correlate best with surgical skills measured during an in-vivo laparoscopic procedure [3]. Nonparametric analysis (Friedmans test) was carried out in order to examine the difference between the performance score values from each attempt and the values from the consecutive attempts. The difference in performance scores between the three groups of surgeons was analyzed by use of the Kruskal-Wallis and MannWhitney tests. Values are given as median (range) if not stated otherwise.

Subjects and methods Setting The study was carried out in two gastroenterological surgical units of teaching hospitals. None of the participants in the study had had previous contact with the MIST-VR. Procedures Forty-one surgeons (30 male) were tested on the MIST-VR 10 consecutive times within a 1-month period. The participants were divided into three groups according to their experience in laparoscopic surgery: masters (group 1, 8 subjects who had performed more than 100 cholecystectomies); intermediates (group 2, 8 subjects who had performed between 15 and 80 cholecystectomies); and beginners (group 3, 25 subjects who had performed fewer than 10 cholecystectomies). Assessment system Laparoscopic skills were objectively measured by performing the tasks on the MIST-VR system. The system is based on a PC, linked to a jig containing two laparoscopic instruments and a diathermy pedal. Movement of the instruments is translated as a real-time graphical display. MIST-VR has six tasks of progressive difculty, based on abstract graphics, which simulate the technique of manipulations during laparoscopic cholecystectomy. All tasks begin with bilateral movements to touch a virtual sphere with the virtual instruments tips. For task one the trainee is required to grasp a virtual sphere and place it in a virtual box. As with all tasks this is repeated two times for each hand. In the second task the virtual sphere is grasped, transferred between instruments, and then placed in the box. Task three consists of grasping alternately the segments of a virtual pipe. Task four requires the trainee to grasp the virtual sphere, touch it with the tip of the other instrument, withdraw and reinsert this instrument, and once more touch the sphere. In the fth task, once the virtual sphere has been grasped, three plates appear on the surface of the sphere, 90 degrees apart; these are then touched by the other instrument and, using the pedal, removed using virtual diathermy. Task six combines the actions of tasks four and ve with the aim of diathermying the plates while holding the sphere in the virtual box. Errors, economy of movement for each hand (actual path length/ideal path length), and time are registered. For the analysis, a total economy of motion score was

Results The learning curve patterns for the three groups of surgeons can be seen on Fig. 1. The curves regarding time reached plateau after the second repetition for group 1, the fth repetition for group 2, and the seventh repetition for group 3 (Friedmans tests, P 0.05). Experienced surgeons did not improve their error- or economy of movement scores (Friedmans tests, P 0.2) indicating the absence of a learning curve for these parameters. Group 2 error scores reached plateau after the rst repetition, and group 3, after the fth repetition. Group 2 improved their economy of movement score up to the third repetition and group 3 up to the sixth repetition (Friedmans tests, P 0.05). Signicant differences in the performance scores of surgeons with different operative experience were observed (Kruskal-Wallis tests, P 0.05). Experienced surgeons (group 1) demonstrated best performance in all parameters, followed by group 2 and group 3 (Mann-Whitney test, P 0.05). The values are presented in Fig. 2.

Comments The present study provides data on the rate of acquisition of laparoscopic psychomotor skills in a virtual environment as well as demonstrates the construct validity of the MIST system. Clinical experience has shown that there is a signicant learning curve for each surgeon and for each new laparoscopic procedure, such as fundoplication, cholecystectomy, and appendectomy [4]. This learning phase includes 10 to 30 patients and results in longer operating room time, higher complication rates, and higher conversion rates to open laparotomy [5], all contributing to higher costs. For surgeons and surgical residents laparoscopic training with a

148

T.P. Grantcharov et al. / The American Journal of Surgery 185 (2003) 146 149

Fig. 1. Learning curves for masters (lled triangles), intermediates (lled squares), and beginners (lled circles). (a) Time to complete the task (sec), (b) error scores, and (c) number of unnecessary movements. For details of statistical analyzes, see text.

VR simulator would be more efcient than training on patients and may replace some of the learning curve typical of new laparoscopic procedures. Previous work has attempted to identify the familiarization rate on the MIST-VR comparing the curves for surgeons against nonsurgeons based on six repetitions of each task [6]. The study showed a signicant improvement in the performance scores up to the third repetition in all tasks and did not demonstrate difference in the familiarization curves between the two groups. Our study compared the learning curves for surgeons of three experience levels who per-

Fig. 2. Comparison of the performance scores between the three groups. Horizontal bands indicate medians, boxes indicate 25th and 75th percentiles, and whisker lines indicate highest and lowest values. (a) Time to complete the task (sec), (b) error scores, and (c) number of unnecessary movements. Levels of experience: group 1 (performed more than 100 cholecystectomies), group 2 (15 to 80 cholecystectomies), and group 3 (fewer than 10 cholecystectomies).

T.P. Grantcharov et al. / The American Journal of Surgery 185 (2003) 146 149

149

formed 10 repetitions of the MIST-VR tasks. We found signicant differences in the familiarization curves on the simulator among the three groups. Experienced surgeons had a very rapid learning curve regarding time and made no signicant improvements in their error and economy of motion scores. This absence of initial familiarization rate indicates both the high quality of the human-computer interface and the relevance of the performance parameters measured by the simulator. Surgeons with intermediate experience made quick improvements in their performance scores, while the beginners required more time and repetitions in order to reach their maximum score. These results suggest that experienced surgeons will not benet from training on this simulator while surgeons with moderate experience as well as beginners could probably gain significant improvement of their psychomotor skills by training in a virtual environment. However, we cannot state at this point whether improvements of surgical skills on a computer simulator can be transferred into real surgery with improved clinical outcome. Another important nding of this study is the demonstration of the construct validity of the MIST-VR scoring system. It has already been shown that MIST-VR can distinguish between surgeons and nonsurgeons [7]. However, the groups were very disparate in their surgical capabilities and demonstrating difference in performance values between these groups does not condently prove the construct validity of the simulator (as other factors than surgical skills could inuence the performance on the simulator). Our study compared three groups of surgeons with different levels of experience, and in this way provides strong evidence that the computer system measures technical skills. Experienced surgeons demonstrated best laparoscopic performance on the simulator, followed by those with intermediate experience and the beginners. Further, group 1 showed less intersubject variability, ie, they were all performing well. These differences indicate that the scoring system of MIST-VR is reliable, sensitive, and specic to measuring skills relevant for laparoscopic surgery. Signicant difference in performance scores between the three groups was present in the beginning of MIST-VR training (attempt 1), but not at the end (attempt 10) indicating that the basic manual skills for the performance of laparoscopic surgery can be acquired after 10 repetitions of the six MIST-VR tasks. This nding can have impact on the construction of the training programs in minimally invasive surgery for residents (for example by requiring the performance of minimum 10 repetitions of all virtual tasks on the

MIST-VR, before starting supervised procedures on patients). Previous studies have demonstrated the validity of the MIST-VR scoring system by showing good correlation between performance scores in a live animal operation and scores registered by the simulator [3]. The present study provided additional evidence that MIST-VR can precisely differentiate among three groups of surgeons with different levels of experience. Further, the simulator system showed excellent interface with the tested surgeons and had the ability to teach skills in a virtual environment. This objective and valid assessment provided by MIST-VR has implications for the future evaluation of manual dexterity and hand-eye coordination in the context of laparoscopic skills for surgical trainees. Analysis of the participants learning curves allows us to determine how many examination sessions are necessary to evaluate an individuals motor skills reliably and should be considered when designing surgical assessment programs in the future.

Acknowledgments This study was supported by Sygekassernes Helsefond, Copenhagen, Denmark.

References
[1] Jordan JA, Gallagher AG, McGuigan J, et al. A comparison between randomly alternating imaging, normal laparoscopic imaging, and virtual reality training in laparoscopic psychomotor skill acquisition. Am J Surg 2000;180:208 11. [2] Prystowsky JB, Regehr G, Rogers DA, et al. A virtual reality module for intravenous catheter placement. Am J Surg 1999;177:1715. [3] Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen PM. Virtual reality computer simulationan objective method for evaluation of laparoscopic surgical skills. Surg Endosc 2001;15:242 4. [4] Watson DI, Baigrie RJ, Jamieson GG. A learning curve for laparoscopic fundoplication. Denable, avoidable, or a waste of time? Ann Surg 1996;224:98 203. [5] MacFadyen BV, Vecchio R, Ricardo AE, Mathis CR. Bile duct injury after laparoscopic cholecystectomy. The United States Experience. Surg Endosc 1998;12:31521. [6] Chaudhry A, Sutton C, Wood J, et al. Learning rate for laparoscopic surgical skills on MIST-VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl 1999;81:281 6. [7] Tafnder N, Sutton C, Fishwick RJ, et al. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomized controlled studies using the MIST-VR laparoscopic simulator. Stud Health Technol Inform 1998;50:124 30.