Matthew Lineberry, PhD1, E Matthew Ritter, MD2. 1Zamierowski Institute for Experiential Learning, University of Kansas Medical Center and Hospital, Kansas City, KS, 2Uniformed Services University/Walter Reed National Military Medical Center, Department of Surgery, Division of General Surgery & National Capital Region Simulation Consortium, Bethesda MD
Introduction: The Fundamentals of Endoscopic Surgery (FES) certification program became part of the American Board of Surgery’s required Flexible Endoscopy Curriculum in 2014. The goal of this study was to evaluate the performance of the FES manual skills examination during its initial offering.
Methods: De-identified data from the initial FES skills examination was reviewed and 346 unique participants were identified. Each FES task was evaluated for difficulty and internal consistency. The effects of simulator used, physical factors, clinical experience, and upper endoscopy (UE) and lower endoscopy (LE) experience on pass rates and task performance were examined.
Results: Scores on each task were distributed skew-normally except for Loop Reduction, with a large mode at 0 points. Tasks’ score distributions differed significantly (p < .001), ordering tasks from easiest to most difficult as Tool Targeting, Retroflexion, Navigation, Mucosal Inspection, and Loop Reduction. Correlations between each task and the average of the other tasks ranged from 0.27-0.54, suggesting considerable task specificity, especially for Loop Reduction and Mucosal Inspection. Pass rates were not statistically discernible for Mentor Express vs. GI Mentor but showed an effect size that may be practically significant (83% vs. 75%; c2 = 2.68, p = 0.10). Median scores varied by degree of experience (p < .01); post-hoc tests show fellows with higher scores than PGY5s but no other discernible differences. For both Navigation and Mucosal Inspection, scores varied by LE experience but not UE experience (p < .05); post-hoc analyses revealed that those with the most experience did not necessarily have the highest scores. For Tool Targeting, neither UE nor LE experience predicted scores. Analyses of the effects of experience on Loop Reduction and Retroflexion showed discernible differences by UE experience for both, but differences by LE experience only for Retroflexion. UE and LE experience both related to pass rate (p < .05 and p < .001), with pass rates generally higher for those with more experience.
Conclusions: Overall, task performance on the FES Skills exam is as expected for a certification exam, with normal distributions slightly skewed toward higher scores. Loop Reduction continues to be the most difficult task, with unique psychometric properties; test developers should examine this task carefully, though these findings are not necessarily evidence of invalidity. Possible differences by simulator used also warrant continued attention. Pass rates and task performance relate to clinical experience, especially endoscopic experience.
Presented at the SAGES 2017 Annual Meeting in Houston, TX.
Abstract ID: 80442
Program Number: S060
Presentation Session: Education, Simulation and Assessment
Presentation Type: Podium