Objective Evaluation of a Laparoscopic Surgical Skill Program for Residents and Senior Surgeons FREE

THIS ARTICLE reports on the Yale Laparoscopic Skills and Suturing Program, a training method^1- 3 for objective evaluation of laparoscopic surgical skills including intracorporeal suturing. Since the introduction of laparoscopic general surgery, many courses to retrain established surgeons have emerged. However, objective parameters are not commonly used to assess the outcome of courses.^4- 6 The safe and efficient inclusion of laparoscopic surgery as an integral part of general surgery demands dissemination of laparoscopic surgical skills among trained surgeons and residents with a common platform. An effective standardized skill-acquisition course with objective evaluations can facilitate skill transfer for the retraining of senior surgeons and establishing initial surgical training in residents.^7- 9

Our aim was to evaluate the effectiveness of a method of teaching basic laparoscopic skills and suturing to trained surgeons and residents. A database with percentile graphs was created to allow valid comparison of the skills of the participants (Figure 1).

All 291 trained surgeons were board eligible or board certified or had equivalent qualifications and experience. All were previously certified to perform laparoscopic surgery in their hospitals. They attended a standardized training program for 2 12 days, as described previously.^1- 3 All the trained surgeon participants performed 3 types of standardized drills (rope pass drill, triangle transfer drill, and cup drop drill) 10 times each. They were also trained in a standardized intracorporeal suturing technique. The instructor-trainee ratio was 1:4. Data were collected by the instructors, who ensured the competency and timing of each drill and suturing technique. All exercises, including standardized intracorporeal anastomosis (8 cm long, 2 layers), were performed within the a laparoscopic trainer (Surgi-Trainer, US Surgical Corporation, Norwalk, Conn) using a Sony monitor (Trinitron-model PVM-1943MD, Sony, Tokyo, Japan) and a 0°, 10-mm laparoscope (Stryker Endoscopy, Santa Clara, Calif) and a 3-chip medical video camera (Stryker model 777).

The participants performed 3 types of drills to improve their dexterity, depth perception, instrument targeting accuracy, visual and spatial abilities, and hand-eye coordination. Use of the nondominant hand was emphasized. These are generalized skill requirements, applicable not only to laparoscopic surgery but also to other types of endoscopic surgical techniques. The drills are subset tasks relevant to increasing efficiency in laparoscopic suturing. The steps involved in intracorporeal suturing are described in detail elsewhere.^1,6 A detailed suturing algorithm was developed and strictly followed. The process was broken into 3 phases: needle positioning, tissue penetration, and knotting. Only 1 practice drill was allowed, and thereafter all the exercises were timed in a standardized fashion. The participants performed the drills before the suturing exercises on each day of the 212 -day course. The data collection for trained surgeons began in 1991 and ended in 1994.

The same course was conducted for residents at Yale University School of Medicine over a 3-year period (from 1994 to 1996). The residents attended the course in batches of 15 to 20 participants. The courses were conducted during a weekend and were an essential requirement of the residency training program. At the end of the course, each resident was given an analysis indicating the resident's percentile performance compared with that of the data bank of trained surgeons.

The time required to perform each dexterity drill or suturing exercise was documented in seconds. The 291 trained surgeons who performed 10 of each drill or suturing exercise were included in the final analysis. The total time required to perform the first 10 drills and suturing exercises was calculated for each participant and used for comparison with all the calculations. This method yielded 1 value in seconds per participant per drill or suturing exercise. Cumulative percentile graphs were prepared for all the drills and suturing exercises.^10- 11 The comparisons of residents with trained surgeons and of male with female participants were made with unpaired, 2-tailed t tests.

The total time in seconds required by 291 trained surgeons (268 male and 23 female) and 99 residents (78 male and 21 female) to perform 10 drills and 10 suturing exercises of each type is shown in Table 1. The resident group included 61 general surgical residents and 19 gynecology residents across the spectrum from introduction to chief residency, and 19 preresidency participants. The senior surgeons were older than the residents (mean ± SEM, 44±0.55 years vs 31±1.15 years, P <.001), as expected. The container used to perform the cup drop drill was changed to a porcelain cup with an aperture, as it was difficult to obtain the former type of cup.This alteration, however, increased the difficulty in performing the cup drop drill, thus requiring a longer time to complete it. Separate analysis was, therefore, performed for both old and new cup drop drills. The drill with the larger sample size was evaluated unless the amalgamation of data did not influence the outcome of the analysis. A comparison of time requirements for male and female residents and for male and female trained surgeons is shown in Table 2 and Table 3, respectively. Each resident's performance in drills and suturing exercises was compared with the percentile graphs of trained surgeons, and an accurate percentile rank for each resident was thus determied.

The time required to perform 10 drills and suturing exercises for the residents as a group and for the trained surgeons are compared in Table 1. Data of 3 categories of residents are given separately, so that the performance of these subgroups can be appreciated on their own merit. Data of all the residents and data of their subgroups were compared separately with the performance of the trained surgeons; P values in Table 1 refer to these comparisons. There was no significant difference in the time requirement of residents and trained surgeons in performing the rope pass drill and suturing exercises. Furthermore, residents took significantly less time to perform the triangle transfer drill. They required more time than the trained surgeons, however, to perform the old cup drop drill, new cup drop drill, and an 8-cm, 2-layer anastamosis. Data given in Table 1 for subgroups of residents are informative for evaluating subgroup behavior in comparison with that of all of the residents as a whole.

The time requirements of male and female residents and male and female trained surgeons are shown in Table 2 and Table 3, respectively. No significant difference was noted in the performance of male and female residents. However, the trained female surgeons required more time than trained males to perform the rope pass drill and suturing exercises. No significant difference was found between the 2 groups in the time required to perform the old cup drop drill and the triangle transfer drill.

The times required to perform 10 drills and 10 suturing exercises were tested for correlation in the total study population of 392 that included all the trained surgeons and all the residents. A significant correlation was noted between the time required for the suturing exercises and the time recorded in all 3 drills (rope pass drill, n=352, r=0.51, P <.001, Figure 2; triangle transfer drill, n=354, r =0.53, P<.001, Figure 3; cup drop drill, n=354, r =0.51, P<.001, Figure 4). The residents were younger than the trained surgeons (mean ± SEM, 30.79±1.15 years vs 43.94±0.55 years, P<.001), as expected. The age of all the participants (41.46±0.58 years) including trained surgeons and residents had very low correlation with the time required to perform 10 drills and 10 suturing exercises (rope pass drill, n=274, r =0.17, P<.005; triangle transfer drill, n=277, r=0.23, P <.005; cup drop drill, n=278, r =0.07, P<.0001; suturing exercises, n=244, r=0.27, P <.001).

Traditional educational strategies in surgery need to be altered to cope with recent developments in minimally invasive surgery. Current training systems are mainly dependent on subjective evaluation of trainee surgeons by the surgical preceptors, a loosely organized curriculum, and cognitive testing through multiple-choice questions. Very few training programs use objective tests of dexterity and surgical skills in their usual surgical training and evaluation methods. This is probably justified because the relationship between surgical skills and manual dexterity has been questioned. For example, some investigators did not find any difference in manual dexterity between physicians and surgeons.¹² In addition, others¹³ did not find any correlation between surgical clinical proficiency and psychomotor skills. These studies, however, were not conducted in a laparoscopic environment and results cannot be extrapolated to laparoscopic surgery. It is agreed that performance in an objective evaluation should not be taken as the sole factor in judging the clinical or technical competency of a surgeon. Cognitive tests and objective evaluation of dexterity and psychomotor skills assess different characteristics of a surgeon. While tests of cognitive knowledge represent a valid measure of the retention of factual material, they cannot be used as the sole basis for assessing clinical competence in a residency program.¹⁴ Objective evaluations should be used in conjunction with subjective and cognitive tests and should not replace them.

Correlation can be observed between the teaching of gross motor skill development by physical educators and the teaching of fine manual motor skills associated with surgical skill development.¹⁵ The need for a reliable method of skill acquisition was documented by a survey of 152 program directors in the United States,¹⁶ 50% of whom felt that there was a need for improving the establishment of technical skill. No single method has been accepted as the best by any group, but many are being explored.¹⁷ The concept of skill labs is widely accepted,¹⁸ but curriculum content must dictate the effectiveness of these labs to affect skill transference.

Minimal-access surgery demands high-quality assurance standards in order for patients to reap maximum benefits.¹⁹ Whatever transpires from the emerging technology, we must not overlook surgical principles governing operative intervention, which apply regardless of the approach and instrumentation. High performance standards with widespread availability cannot become a reality without the estalishment of skills. Some programs have tried to introduce skill development more systematically than others.^19- 20 They did not, however, use objective parameters of laparoscopic skill, but only recorded an increase in the total number of laparoscopic cholecystectomies performed by the trainees in their hospitals. Coppola and Merrell²¹ discussed outcome reference points in their study on the effect of a laparoscopic skill-development regimen on a resident training program. After residents participated in this program, the academic level of the residents able to perform a laparoscopic cholecystectomy significantly decreased without extended operative times or increased complication rates.

The objective evaluation of dexterity and skill can only become a reality if a large database is established. There are no such data banks in conventional surgery, but this database for laparoscopic surgery could prove very helpful in the development of a training system for laparoscopic surgery. A database provides a way for participants to know how successful their efforts are in attaining a level of competence. It gives them a performance reference point—the skill levels of trained surgeons—to try to achieve and to pass. It also helps to establish effective training countermeasures for participants who have difficulty. The relationship between dexterity drills and suturing techniques that was noted in our initial reports^1- 3 was also found in the current study (Figure 2, Figure 3, Figure 4). This strongly supports correlation between dexterity skills and a meaningful surgical task.

It has been a commonly held assumption that younger surgeons have a natural advantage in the development of laparoscopic skills. Some authors reported that age influenced pure motor skills but neither age nor pure motor skills are necessarily important for operative skills. Age did not appear to play a dominant role in the outcome of our courses. All the participants who attended our courses, regardless of age, sex, and previous training, learned intracorporeal suturing and performed anastamosis. There was no participant whose performance did not improve. The correlation between age and the time required to perform the dexterity drills and intracorporeal suturing was very low. All the residents as a group took marginally longer to complete an anastamosis, although their suturing time was not significantly different from that of the trained surgeons. This difference may be due to a lack of experience in performing anastamosis rather than in suturing. Residents required more time than the trained surgeons to perform the cup drop drill, which requires considerable depth-perception compensation to the 2-dimensional environment. However, residents performed significantly better than the trained surgeons in the triangle transfer drill. It is possible that age and experience might influence some types of dexterity drills, but overall they do not seem to play a dominant role.

Laparoscopic intracorporeal suturing is probably the most difficult exercise to master in the minimally invasive environment. The drills in this program were associated with mastery of the suturing process. Both the residents and the trained surgeons reached a comparable efficiency with the same educational program. It is notable that the residents were effectively taught intracorporeal suturing in the same training program as were the attending surgeons. The residents did not have the same experience as senior surgeons in constructing an intestinal anastomosis in the open environment. Although their anastomosis completion times were longer than those of the trained surgeons, they developed a foundation on which to competently build (Table 1). There was no significant difference (P<.001), however, between the intracorporeal suturing times of the residents as a group vs the trained surgeons. The residents' performance is all the more noteworthy considering that 70% of board eligible, trained surgeons, before taking our course, could not complete a knot in 5 minutes.¹

Some authors reported that female residents scored better in cognitive tasks but scored less well in visual-motor tasks in comparison with male residents. In our study there does not appear to be a dominant influence of sex on laparoscopic skill acquisition. No difference was noted in the performance of male and female residents in performing drills and suturing exercises. The finding that trained females took a longer time to complete suturing exercises and rope pass drills was only marginally significant (P<0.5). Hayward et al²² compared the abilities of male and female residents in 6 areas: ethics, judgment, technical skills, knowledge, interpersonal skills, and work habits. A 5-point scale was used for evaluation. They did not find any difference between male and female residents. This finding was also consistent with our current study.

The Yale Laparoscopic Skills and Suturing Program is effective in establishing laparoscopic and suturing skills in surgeons and residents regardless of age or sex in a short time, resulting in greater teaching efficiency. Because the database generated by this study is applicable to a multilevel spectrum of participants, comparison of skill levels of residents against the percentile graphs derived from the performance of trained surgeons is possible. Participants who finish this standardized curriculum can compare their performance with that of trained surgeons, giving them an initiative to practice and improve.

Reprints: James C. Rosser, MD, Department of Surgery, Yale University School of Medicine, 40 Temple St, Suite 3A, New Haven, CT 06510.