Timothy M Farrell, MD, Michael O Meyers, MD, Lindsee E McPhail, MD, Geoffrey P Kohn, MD, Mark J Koruda, MD, Anthony A Meyer, MD PhD. University of North Carolina at Chapel Hill
Objective: Surgery resident knowledge is assessed subjectively and objectively throughout training. Subjective evaluation depends upon faculty assessment. Objective measurement derives from the American Board of Surgery In-Training Examination (ABSITE) or similar tests. Few efforts have been made to characterize correlation between subjective and objective measures of knowledge, or to assess longitudinal trends.
Methods: From 2003-2008, resident knowledge was rated subjectively by the faculty on a scale of 3-15 after each rotation, and annual averages were calculated. In parallel, resident knowledge was evaluated objectively by annual ABSITE testing, and percentile scores were assigned based on postgraduate year (PGY)-stratified national normative data. Overall, 167 contemporaneous subjective and objective knowledge assessments were identified: 47 PGY-1, 38 PGY-2, 27 PGY-3, 28 PGY-4, and 27 PGY-5. Paired subjective and objective knowledge scores were compared overall and for each PGY level, and Pearson Product-Moment Correlation coefficients were generated (Table).
Results: Overall, a statistically significant (weak) positive association exists between subjective assessment of resident knowledge and ABSITE performance. Year-by-year analysis shows faculty ratings statistically correlate with resident ABSITE performance only during PGY-1 and PGY-5. There is no correlation in the middle years of residency, with a negative Pearson’s r-value in PGY-3.
| PGY Year | Pearson Correlation (r) | p-value |
|---|---|---|
| 1 | 0.4578 | 0.0007 |
| 2 | 0.2377 | 0.1506 |
| 3 | -0.0794 | 0.6198 |
| 4 | 0.3696 | 0.0528 |
| 5 | 0.5397 | 0.0003 |
Conclusions: Faculty assessment of resident knowledge correlates only weakly with ABSITE performance, with closest correspondence in PGY-1 and PGY-5 years. These variations may be explained by the higher proportion of specialty and off-site rotations that mark the PGY-2 and PGY-3 years in our program, and by the tendency for increased resident scrutiny during high-stakes entry and exit years. However, it is clear additional objective assessment tools are needed.
Session: Poster
Program Number: P201
View Poster