0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Paper |

General Surgery Resident Remediation and Attrition:  A Multi-institutional Study FREE

Arezou Yaghoubian, MD; Joseph Galante, MD; Amy Kaji, MD, PhD; Mark Reeves, MD, PhD; Marc Melcher, MD, PhD; Ali Salim, MD; Matthew Dolich, MD; Christian de Virgilio, MD
[+] Author Affiliations

Author Affiliations: Departments of Surgery (Drs Yaghoubian and de Virgilio) and Emergency Medicine (Dr Kaji), and Los Angeles Biomedical Institute (Drs Kaji and de Virgilio), Harbor-UCLA Medical Center; Department of Surgery, University of California Davis Medical Center, Davis (Dr Galante); Department of Surgery, Loma Linda University, Loma Linda (Dr Reeves); Department of Surgery, Stanford University, Stanford (Dr Melcher); Department of Surgery, Cedars Sinai Medical Center, Los Angeles (Dr Salim); and Department of Surgery, University of California, Irvine Medical Center, Irvine (Dr Dolich), California.


Arch Surg. 2012;147(9):829-833. doi:10.1001/archsurg.2012.1676.
Text Size: A A A
Published online

Objective To determine the rates and predictors of remediation and attrition among general surgery residents.

Design, Setting, and Participants Eleven-year retrospective analysis of 348 categorical general surgery residents at 6 West Coast programs.

Main Outcome Measures Rates and predictors of remediation and attrition.

Results Three hundred forty-eight categorical general surgery residents were included. One hundred seven residents (31%) required remediation, of which 27 were remediated more than once. Fifty-five residents (15.8%) left their programs, although only 2 were owing to failed remediation. Remediation was not a predictor of attrition (20% attrition for those remediated vs 15% who were not [P = .40]). Remediation was most frequently initiated owing to a deficiency in medical knowledge (74%). Remediation consisted of monthly meetings with faculty (79%), reading assignments (72%), required conferences (27%), therapy (12%), and repeating a clinical year (6.5%). On univariate analysis, predictors of remediation included receiving honors in the third-year surgery clerkship, United States Medical Licensing Examination (USMLE) step 1 and/or step 2, and American Board of Surgery In-Training Examination scores at postgraduate years 1 through 4. On multivariable regression analysis, remediation was associated with receiving honors in surgery (odds ratio, 1.9; P = .01) and USMLE step 1 score (odds ratio, 0.9; P = .02). On univariate analysis, the only predictor of attrition was the American Board of Surgery In-Training Examination score at the postgraduate year 3 level (P = .04).

Conclusions Almost one third of categorical general surgery residents required remediation during residency, which was most often owing to medical knowledge deficits. Lower USMLE step 1 scores were predictors of the need for remediation. Most remediated residents successfully completed the program. Given the high rates of remediation and the increased educational burden on clinical faculty, medical schools need to focus on better preparing students to enter surgical residency.

In 1999, the Accreditation Council for Graduate Medical Education (ACGME) residency review and institutional review committees announced a new model to measure resident performance. Six general competencies were endorsed including patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and system-based practice. The impetus behind this shift was to create an effective means of educating and evaluating physicians, with the ultimate goal of creating well-trained, educated, ethical, and compassionate physicians.

To our knowledge, to date, there is little data available regarding how successful surgical residency programs have been at achieving these competencies. With the advent of an 80-hour work week restriction and now the 16-hour work limit for interns, there is heightened concern among surgical educators that future surgical residents will be inadequately trained. One measure of the adequacy of the residents' education is whether they require any form of remediation during residency. Thus, the purposes of our study were to determine the frequency of resident remediation (ie, which of the 6 ACGME competencies most commonly needed remediation) and to identify factors predictive of the need for remediation. This may provide insight into how to more effectively modify the surgical curriculum in this new era of limited hours. Another purpose of the study was to determine the rate of attrition by surgical residents.

The study was approved by the human subjects committee of the Los Angeles Biomedical Research Institute at Harbor-University of California, Los Angeles Medical Center. Program directors from 6 residency programs in the Western United States were contacted and surveyed. Data were collected by program directors and/or program coordinators without using any resident identifiers. Variables for graduates from 1999 to 2010 were collected including sex; medical education (US vs non-US medical school); United States Medical Licensing Examination (USMLE) steps 1, 2, and 3 scores; and the American Board of Surgery In-Training Examination (ABSITE) scores (reported as national percentiles) for each postgraduate year (PGY). Remediation was defined as a specific intervention that was in addition to standard resident requirements and that was initiated by the program director owing to an identified deficiency in 1 or more of the 6 ACGME competencies (patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism, and/or system-based practice). A deficiency in an ACGME competency was identified based on a review of each resident's file, including formal evaluations, biannual reviews, and/or any adverse reports or letters. For residents who required remediation, the program director was asked to identify which competency was deficient. The PGY level at which remediation was initiated, the format/type of remediation, and whether the resident required repeat remediation was recorded. Resident attrition was also collected as was whether the resident left the program voluntarily owing to a career change or involuntarily owing to performance issues.

Resident data were collected in an Excel database (Microsoft Corporation) and translated into a native SAS format using DBMS/Copy (Dataflux Corporation). Analyses were conducted using SAS version 9.1 (SAS Institute). Descriptive statistics were calculated for all variables. When appropriate, numerical variables were compared using the nonparametric Wilcoxon rank sum test and are reported as medians with interquartile ranges. Categorical or nominal variables were compared using the χ2 test or Fisher exact test, as appropriate. P < .05 was considered significant. Univariate analysis was performed to determine factors predictive of the need for remediation and predictive of attrition. Factors found to be significant at a level less than .05 were entered into a multivariable analysis.

There were 348 categorical general surgery residents included in the study. Of these, 220 were men (63.2%) and 128 were women (36.8%). United States Medical Licensing Examination scores, ABSITE scores, and third-year medical school surgery clerkship performance are displayed in Table 1. Overall, 107 residents (31%) required remediation. The attrition rate was 15.8% (55 residents).

RESIDENT REMEDIATION

One hundred seven residents (31%) required remediation. The remediation was initiated at the PGY 1 level in 27 residents (25%), 37 (35%) at the PGY 2 level, 22 (21%) at the PGY 3 level, 17 (16%) at the PGY 4 level, and 4 (4%) at the PGY 5 level. In addition, 27 residents required remediation more than once. Remediation was owing to deficiencies in the following ACGME competencies: medical knowledge in 80 residents (74%), interpersonal and communication skills in 26 (24%), patient care in 24 (22%), professionalism in 23 (18%), system-based practice in 15 (14%), and practice-based learning in 9 (8%). Residents were remediated by monthly meetings with faculty (79%), specific reading assignments (72%), required attendance at review courses and/or conferences (27%), and evaluation by a therapist/psychologist/psychiatrist (12%). Seven of the residents who were remediated were required to repeat a year. On univariate analysis, predictors of needing remediation included receiving honors in the third-year surgery clerkship, USMLE step 1 and/or step 2, and ABSITE scores at PGYs 1 through 4 (Table 2). On multivariable regression analysis, factors predictive of remediation included receiving honors in surgery (odds ratio, 1.9; 95% CI, 1.13-3.00; P = .01) and USMLE step 1 score (odds ratio, 0.900; 95% CI, 0.985-0.998; P = .02).

Table Graphic Jump LocationTable 2. Univariate Predictors of Resident Remediation
RESIDENT ATTRITION

Fifty-five residents (15.8%) left their surgical programs. In 53 of these instances, the attrition was voluntary. Only 2 residents (1.1%) were forced to leave their programs owing to failed remediation. On univariate analysis, the only predictor of attrition was the ABSITE score at the PGY 3 level (Table 3). The need for remediation was not a predictor of attrition (20% attrition for those who required remediation vs 15% for those who did not [P = .40]).

Table Graphic Jump LocationTable 3. Univariate Predictors of Resident Attrition

Our 11-year study of 348 categorical general surgery residents from 6 academic general surgery residency programs in California reports that 31% of residents required remediation owing to a deficiency in 1 of the 6 ACGME competencies at some point during residency. Remediation was by far most frequently initiated owing to a deficiency in medical knowledge competency (74% of those who required remediation). On multivariable analysis, the USMLE step 1 score was predictive of remediation. Although counterintuitive, having received honors in the third-year surgery clerkship was also predictive of the need for remediation.

The high rate of remediation observed in our study is concerning on several fronts. Resident remediation places additional strain on the program director and entire clinical faculty. As pointed out by Williams et al,1 problem residents may also complicate patient care, as they may increase the workload of other health care providers in the form of duplication of care and an increase in communication demands. They also increase the need for faculty supervision. In the 80-hour resident work week era, as reported by Coverdill and colleagues,2 teaching faculty have increased workloads, higher stress levels, and less satisfaction with work. The high remediation level noted is also surprising given that the resident cohort was a relatively high-achieving group as judged by the mean USMLE step 1 (229) and step 2 (230) scores, both of which were greater than the US national average. Thus, one would not have anticipated such a significant need for remediation.

The high rate of remediation reported in our study begs the question of whether we are falling short in the education of surgical residents. One thought is that medical students start residency underprepared for the rigors of surgical residency. The final year of medical school is often liberally structured, allowing students considerable leeway in choosing electives. Numerous surgical educators have advocated a restructuring of the final year of medical school. Naylor and colleagues3 instituted an integrated cognitive and proficiency-based skills curriculum based on American College of Surgeons Graduate Medical Education Committee competencies. The course included cadaver dissections, didactic sessions, team training, and training in clinical and technical skills. Similarly Esterl and colleagues4 instituted a 4-week senior medical student boot camp. We believe that more needs to be done in the final year of medical school to better prepare future general surgeons and that we as surgeons need to take the initiative.

Another potential explanation is that we as surgical educators are inadequately training our residents, a problem that is now possibly heightened by the 80-hour work week limit. To address this issue, national initiatives are underway to reform surgical education. Representatives from the American Surgical Association, the American College of Surgeons, the American Board of Surgery, the Residency Review Committee for Surgery, the Association of Program Directors in Surgery, and the Association for Surgical Education created a national consortium called the Surgical Council on Resident Education.5,6 It is still too early to determine what effect these efforts will have.

Although the duty-hour limits have reduced resident work hours in the hospital, the 80-hour work week limit may have resulted in other, less desirable effects. General surgery residency now requires more efficiency and a higher-functioning resident to complete more work within a shorter timeframe. This in turn may give the faculty a sense that the resident is not adequately performing. Additionally, although the work hour reduction has resulted in an improvement in the resident lifestyle, it may not have translated into more time for self-study at home.7

Several studies have evaluated remediation in surgical residency. Williams and colleagues1 noted in a single-institution study a 22% rate of serious performance problems among surgery residents. Although they did not categorize the problems by ACGME competency, they reported that the problems fell into 3 broad categories of relations with health care workers, including interpersonal conflict, insufficient knowledge, and failure to communicate effectively. A sobering finding of the study was that virtually all of the problems persisted throughout residency, and all problems appeared to be refractory to remediation attempts. In addition, 41% of the problem residents failed to achieve board certification compared with 100% board certification for residents who did not have problems. Bergen et al8 also identified in a single-institution study that 20.8% of surgical residents were high-risk or problem residents. Most were found to have deficiencies in interpersonal behavior, followed by cognitive deficiencies. Approximately one fifth of those problem residents withdrew from the program, and most did so involuntarily. We similarly found that the most common ACGME competency deficiency was in medical knowledge, followed by interpersonal and communication skills as well as inadequate patient care. Other studies have focused specifically on remediation of low ABSITE scores.

There is no uniformity on the best process for surgical resident remediation. In a national survey of surgery programs, Torbeck and Canal9 found that only 52% to 75% of programs had a specific policy for ACGME competency remediation. In about half of the programs, the remediation plan was developed by the program director alone. In nearly 75%, the program director alone was responsible for supervising and monitoring the program. For medical knowledge deficits, a study program was most often instituted, as in our study. For both patient care and professionalism as well as interpersonal communication skills deficiencies, most programs simply recommended an increase in direct resident observation. For system-based practice and practice-based learning and improvement, tutorial meetings were the most frequent remediation method. Hauer et al10 proposed a 4-step model for remediation applicable to both medical students and residents across all specialties. The model begins with competence assessment using validated tools, followed by diagnosis of deficiencies and development of individualized learning plans; instruction/remediation with deliberate practice, feedback, and reflection; and finally focused reassessment and certification of competence. Other medical specialties have created a national task force to address remediation.11 Given the significant need for remediation in our study and the variability in remediation methods, we feel that national, surgery-specific remediation guidelines should be developed.

Our study found a resident attrition rate of 15.8%, with most being voluntary. Remediation was not a predictor of attrition. Attrition within general surgery has been well studied and is an ongoing concern. A 20-year retrospective study at Yale University found that 30 of 99 surgery residents (30%) failed to complete the program. Of these, 21 (70%) withdrew, 5 (17%) transferred, and 4 (13%) were dismissed. Attrition occurred before entering the third clinical year in 23 of 30 (77%). The attrition rate was 40% (12 of 30) since the 2000 academic year.12 Similarly, a national study was performed by Yeo and colleagues13 of a survey performed of residents taking the ABSITE from 2007 to 2008. They found an overall attrition rate of 19.5%, highest in the PGY 1 and 2 levels. Women were no more likely to leave programs than men (2.1% vs 1.9%).13 Other researchers have demonstrated that the 80-hour work week has not resulted in a reduction in surgical resident attrition.14 Attrition has been attributed to several causes, including the desire for a less-stressful environment; marital, family, and personal issues; economic pressures necessitating rapid entry into a higher-wage workforce; health-related issues; and the perception that a resident is incompetent accompanied by the belief that firing is imminent. Interestingly, this latter cause may apply to our residents, as the only predictor of resident attrition in our study was a low ABSITE score at the PGY 3 level. Bergen et al8 found that deterioration of monthly evaluations, a counseling event, or a letter of complaint trigger close scrutiny and may predict residents at risk for attrition.

There are several limitations to this study. This was a retrospective study, and we were not able to fully evaluate the reasons why residents left surgical residency and the specific circumstances. Furthermore, we did not separate the attrition rate by year and consequently cannot determine whether the attrition rate has improved following the institution of the 80-hour work week. We also did not collect information on the American Board of Surgery board pass rates. Ultimately, the success of a program is measured by residents' success of the American Board of Surgery certifying and qualifying examinations. It is unclear whether remediated residents have a higher or lower board pass rate.

In conclusion, we identified a relatively high remediation rate among residents of 6 academic surgical residencies in California. A lower USMLE 1 score was predictive of the need for remediation. Most remediated residents successfully completed the surgical residency program, and remediation was not a predictor of attrition. The high rate of remediation should give surgical educators pause as we should closely examine the potential sources of these deficiencies. Remediation places an increased educational burden on clinical faculty. Surgical societies should take the initiative to encourage the restructuring of medical school education, such that future surgeons are better prepared to enter surgical residency. At the same time, residency programs need to determine whether current educational methods are adequate to prepare future surgeons.

Correspondence: Christian de Virgilio, MD, Harbor-UCLA Medical Center, 1000 W Carson St, Box 25, Bldg 1-E, Torrance, CA 90509 (cdevirgilio@labiomed.org).

Accepted for Publication: April 30, 2012.

Author Contributions:Study concept and design: Yaghoubian, Galante, and de Virgilio. Acquisition of data: Galante, Reeves, Melcher, Salim, Dolich, and de Virgilio. Analysis and interpretation of data: Galante, Kaji, Reeves, and de Virgilio. Drafting of the manuscript: Yaghoubian and de Virgilio. Critical revision of the manuscript for important intellectual content: Yaghoubian, Galante, Kaji, Reeves, Melcher, Salim, Dolich, and de Virgilio. Statistical analysis: Kaji. Administrative, technical, and material support: Reeves, Salim, and Dolich. Study supervision: Yaghoubian and de Virgilio.

Financial Disclosure: None reported.

Previous Presentation: This paper was presented at the 83rd Annual Meeting of the Pacific Coast Surgical Association; February 20, 2012; Napa Valley, California, and is published after peer review and revision.

Williams RG, Roberts NK, Schwind CJ, Dunnington GL. The nature of general surgery resident performance problems.  Surgery. 2009;145(6):651-658
PubMed   |  Link to Article
Coverdill JE, Finlay W, Adrales GL,  et al.  Duty-hour restrictions and the work of surgical faculty: results of a multi-institutional study.  Acad Med. 2006;81(1):50-56
PubMed   |  Link to Article
Naylor RA, Hollett LA, Castellvi A, Valentine RJ, Scott DJ. Preparing medical students to enter surgery residencies.  Am J Surg. 2010;199(1):105-109
PubMed   |  Link to Article
Esterl RM Jr, Henzi DL, Cohn SM. Senior medical student “boot camp”: can result in increased self-confidence before starting surgery internships.  Curr Surg. 2006;63(4):264-268
PubMed   |  Link to Article
Sachdeva AK, Bell RH Jr, Britt LD, Tarpley JL, Blair PG, Tarpley MJ. National efforts to reform residency education in surgery.  Acad Med. 2007;82(12):1200-1210
PubMed   |  Link to Article
Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education.  J Am Coll Surg. 2007;204(3):341-346
PubMed   |  Link to Article
Leibrandt TJ, Pezzi CM, Fassler SA, Reilly EF, Morris JB. Has the 80-hour work week had an impact on voluntary attrition in general surgery residency programs?  J Am Coll Surg. 2006;202(2):340-344
PubMed   |  Link to Article
Bergen PC, Littlefield JH, O’Keefe GE,  et al.  Identification of high-risk residents.  J Surg Res. 2000;92(2):239-244
PubMed   |  Link to Article
Torbeck L, Canal DF. Remediation practices for surgery residents.  Am J Surg. 2009;197(3):397-402
PubMed   |  Link to Article
Hauer KE, Ciccone A, Henzel TR,  et al.  Remediation of the deficiencies of physicians across the continuum from medical school to practice: a thematic review of the literature.  Acad Med. 2009;84(12):1822-1832
PubMed   |  Link to Article
Katz ED, Dahms R, Sadosty AT, Stahmer SA, Goyal D.CORD-EM Remediation Task Force.  Guiding principles for resident remediation: recommendations of the CORD remediation task force.  Acad Emerg Med. 2010;17:(suppl 2)  S95-S103
PubMed   |  Link to Article
Longo WE, Seashore J, Duffy A, Udelsman R. Attrition of categoric general surgery residents: results of a 20-year audit.  Am J Surg. 2009;197(6):774-778, discussion 779-780
PubMed   |  Link to Article
Yeo H, Bucholz E, Ann Sosa J,  et al.  A national study of attrition in general surgery training: which residents leave and where do they go?  Ann Surg. 2010;252(3):529-534, discussion 534-536
PubMed
Sneider EB, Larkin AC, Shah SA. Has the 80-hour workweek improved surgical resident education in New England?  J Surg Educ. 2009;66(3):140-145
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable 2. Univariate Predictors of Resident Remediation
Table Graphic Jump LocationTable 3. Univariate Predictors of Resident Attrition

References

Williams RG, Roberts NK, Schwind CJ, Dunnington GL. The nature of general surgery resident performance problems.  Surgery. 2009;145(6):651-658
PubMed   |  Link to Article
Coverdill JE, Finlay W, Adrales GL,  et al.  Duty-hour restrictions and the work of surgical faculty: results of a multi-institutional study.  Acad Med. 2006;81(1):50-56
PubMed   |  Link to Article
Naylor RA, Hollett LA, Castellvi A, Valentine RJ, Scott DJ. Preparing medical students to enter surgery residencies.  Am J Surg. 2010;199(1):105-109
PubMed   |  Link to Article
Esterl RM Jr, Henzi DL, Cohn SM. Senior medical student “boot camp”: can result in increased self-confidence before starting surgery internships.  Curr Surg. 2006;63(4):264-268
PubMed   |  Link to Article
Sachdeva AK, Bell RH Jr, Britt LD, Tarpley JL, Blair PG, Tarpley MJ. National efforts to reform residency education in surgery.  Acad Med. 2007;82(12):1200-1210
PubMed   |  Link to Article
Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education.  J Am Coll Surg. 2007;204(3):341-346
PubMed   |  Link to Article
Leibrandt TJ, Pezzi CM, Fassler SA, Reilly EF, Morris JB. Has the 80-hour work week had an impact on voluntary attrition in general surgery residency programs?  J Am Coll Surg. 2006;202(2):340-344
PubMed   |  Link to Article
Bergen PC, Littlefield JH, O’Keefe GE,  et al.  Identification of high-risk residents.  J Surg Res. 2000;92(2):239-244
PubMed   |  Link to Article
Torbeck L, Canal DF. Remediation practices for surgery residents.  Am J Surg. 2009;197(3):397-402
PubMed   |  Link to Article
Hauer KE, Ciccone A, Henzel TR,  et al.  Remediation of the deficiencies of physicians across the continuum from medical school to practice: a thematic review of the literature.  Acad Med. 2009;84(12):1822-1832
PubMed   |  Link to Article
Katz ED, Dahms R, Sadosty AT, Stahmer SA, Goyal D.CORD-EM Remediation Task Force.  Guiding principles for resident remediation: recommendations of the CORD remediation task force.  Acad Emerg Med. 2010;17:(suppl 2)  S95-S103
PubMed   |  Link to Article
Longo WE, Seashore J, Duffy A, Udelsman R. Attrition of categoric general surgery residents: results of a 20-year audit.  Am J Surg. 2009;197(6):774-778, discussion 779-780
PubMed   |  Link to Article
Yeo H, Bucholz E, Ann Sosa J,  et al.  A national study of attrition in general surgery training: which residents leave and where do they go?  Ann Surg. 2010;252(3):529-534, discussion 534-536
PubMed
Sneider EB, Larkin AC, Shah SA. Has the 80-hour workweek improved surgical resident education in New England?  J Surg Educ. 2009;66(3):140-145
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 3

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles