Rachel H Mathis, MD, Amber W Trickey, PhD, Erica Emery, MS, Gordon Hafner, MD, Jonathan Dort, MD. Inova Fairfax Medical Center
INTRODUCTION: Given the rising need for quantifiable methods of characterizing resident technical skills, we developed a Short Messaging Service (SMS)-based rating system using the Zwisch 4-point scale for autonomy, to evaluate resident performance of laparoscopic cholecystectomies. We assess whether metrics derived from simulation-based skills practice reflects resident autonomy in the operating room (OR), as perceived by faculty evaluation.
METHODS: Over 12 months, 14 post graduate year (PGY) 1-3 general surgery residents were rated for autonomy by general surgery faculty in laparoscopic cholecystectomies. During the same period, 15 residents completed 10 laparoscopic cholecystectomies on the Symbionix LapMentor simulator. Performance metrics were recorded and averaged over all completed simulator cases per resident for analysis. Each resident’s OR autonomy was defined as the percentage of high scores (Zwisch level 3-4) among all evaluated OR procedures. Simulator performance was evaluated for relationships with resident OR autonomy and PGY level using Spearman rank correlations.
RESULTS: Resident autonomy correlated with PGY (rho=0.83, p<0.001). Among 22 simulator metrics evaluated, 12 demonstrated some relationship to OR autonomy or PGY level (p<0.15). The majority (75%) of these related metrics demonstrate stronger correlations with resident autonomy than PGY. Metrics significantly associated with autonomy are included in the Table.
CONCLUSIONS: Our SMS-based rating system is a simple and effective way to obtain quantifiable feedback on resident technical skills. The correlation noted between simulator technical metrics and autonomy level suggests skills may be better measured by resident autonomy level than PGY. This suggests more effective and efficient learning could be tailored to autonomy level rather than PGY.
| Autonomy Correlation | PGY Correlation | PGY1 | PGY2 | PGY3 | |||
| Rho | P | Rho | P | Mean | Mean | Mean | |
| Left instrument movements(No.) | -0.63 | 0.017* | -0.49 | 0.063 | 359 | 165 | 193 |
| Time cautery with adhesion contact(%) | 0.67 | 0.009* | 0.60 | 0.017* | 2.5% | 4.5% | 6.2% |
| Time cautery<15mm of clip(%) | 0.53 | 0.049* | 0.42 | 0.12 | 2.2% | 5.9% | 5.0% |
| Time cautery(%) | 0.66 | 0.010* | 0.62 | 0.013* | 6% | 12% | 14% |
| Total left instrument path length(sec) | -0.62 | 0.018* | -0.53 | 0.043* | 689 | 317 | 341 |
Table. Technical metrics signficantly associated with autonomy (p<0.05)
Presented at the SAGES 2017 Annual Meeting in Houston, TX.
Abstract ID: 79615
Program Number: P300
Presentation Session: Poster (Non CME)
Presentation Type: Poster
