0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation | Association of VA Surgeons

Association Between American Board of Surgery In-Training Examination Scores and Resident Performance

Juliet J. Ray, MD1; Joshua A. Sznol, BS2; Laura F. Teisch, BS1; Jonathan P. Meizoso, MD1; Casey J. Allen, MD1; Nicholas Namias, MD, MBA1; Louis R. Pizano, MD, MBA1; Danny Sleeman, MD1; Seth A. Spector, MD1,3; Carl I. Schulman, MD, PhD, MSPH1
[+] Author Affiliations
1DeWitt Daughtry Family Department of Surgery, University of Miami Miller School of Medicine, Miami, Florida
2Department of Public Health Sciences, University of Miami Miller School of Medicine, Miami, Florida
3Department of Surgery, Bruce W. Carter Department of Veterans Affairs Medical Center, Miami, Florida
JAMA Surg. 2016;151(1):26-31. doi:10.1001/jamasurg.2015.3088.
Text Size: A A A
Published online

Importance  The American Board of Surgery In-Training Examination (ABSITE) is designed to measure progress, applied medical knowledge, and clinical management; results may determine promotion and fellowship candidacy for general surgery residents. Evaluations are mandated by the Accreditation Council for Graduate Medical Education but are administered at the discretion of individual institutions and are not standardized. It is unclear whether the ABSITE and evaluations form a reasonable assessment of resident performance.

Objective  To determine whether favorable evaluations are associated with ABSITE performance.

Design, Setting, and Participants  Cross-sectional analysis of preliminary and categorical residents in postgraduate years (PGYs) 1 through 5 training in a single university-based general surgery program from July 1, 2011, through June 30, 2014, who took the ABSITE.

Exposures  Evaluation overall performance and subset evaluation performance in the following categories: patient care, technical skills, problem-based learning, interpersonal and communication skills, professionalism, systems-based practice, and medical knowledge.

Main Outcomes and Measures  Passing the ABSITE (≥30th percentile) and ranking in the top 30% of scores at our institution.

Results  The study population comprised residents in PGY 1 (n = 44), PGY 2 (n = 31), PGY 3 (n = 26), PGY 4 (n = 25), and PGY 5 (n = 24) during the 4-year study period (N = 150). Evaluations had less variation than the ABSITE percentile (SD = 5.06 vs 28.82, respectively). Neither annual nor subset evaluation scores were significantly associated with passing the ABSITE (n = 102; for annual evaluation, odds ratio = 0.949; 95% CI, 0.884-1.019; P = .15) or receiving a top 30% score (n = 45; for annual evaluation, odds ratio = 1.036; 95% CI, 0.964-1.113; P = .33). There was no difference in mean evaluation score between those who passed vs failed the ABSITE (mean [SD] evaluation score, 91.77 [5.10] vs 93.04 [4.80], respectively; P = .14) or between those who received a top 30% score vs those who did not (mean [SD] evaluation score, 92.78 [4.83] vs 91.92 [5.11], respectively; P = .33). There was no correlation between annual evaluation score and ABSITE percentile (r2 = 0.014; P = .15), percentage correct unadjusted for PGY level (r2 = 0.019; P = .09), or percentage correct adjusted for PGY level (r2 = 0.429; P = .91).

Conclusions and Relevance  Favorable evaluations do not correlate with ABSITE scores, nor do they predict passing. Evaluations do not show much discriminatory ability. It is unclear whether individual resident evaluations and ABSITE scores fully assess competency in residents or allow comparisons to be made across programs. Creation of a uniform evaluation system that encompasses the necessary subjective feedback from faculty with the objective measure of the ABSITE is warranted.

Figures in this Article

Figures

Place holder to copy figure label and caption
Figure 1.
Histograms of Distributions of Resident Annual Evaluation, American Board of Surgery In-Training Examination (ABSITE) Percentile, and ABSITE Percentage Correct Scores

Histograms of distribution of resident annual evaluation scores (A), ABSITE percentile (B), and ABSITE percentage correct (C).

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Regression Models of American Board of Surgery In-Training Examination (ABSITE) Percentile and Evaluation Scores by Postgraduate Year (PGY) and of ABSITE Percentage Correct and Evaluation Scores by PGY

Regression models of ABSITE percentile and annual (A) and medical knowledge (B) evaluation scores by PGY, and regression models of ABSITE percentage correct and annual (C) and medical knowledge (D) evaluation scores by PGY.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 3.
Comparison of Mean Evaluation Scores Between Residents Who Passed and Failed the American Board of Surgery In-Training Examination (ABSITE)

Diamond indicates mean; horizontal line in center of box, median; top and bottom borders of box, upper and lower quartiles, respectively; error bars, maximum and minimum values; and notch, median 95% confidence interval.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 4.
Comparison of Mean Evaluation Scores Between Residents Who Scored in the Top 30% of All American Board of Surgery In-Training Examination (ABSITE) Scorers and Those Who Did Not

Diamond indicates mean; horizontal line in center of box, median; top and bottom borders of box, upper and lower quartiles, respectively; error bars, maximum and minimum values; and notch, median 95% confidence interval.

Graphic Jump Location

Tables

References

Correspondence

CME


You need to register in order to view this quiz.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

1,974 Views
0 Citations
×

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles
Jobs
JAMAevidence.com

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Experience of Operator

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
How Should This Procedure Be Taught or Learned?

brightcove.createExperiences();