Mazen R Al-mansour, MD, Kelly M Tyler, MD, Ziad N Kutayli, MD, Angelica C Belo, MD, Paul F Visintainer, PhD, Neal E Seymour, MD. Baystate Medical Center
Introduction: The aim of the current study is to characterize concurrent performance of colonoscopy on a virtual reality (VR) simulator and in clinical endoscopy, and to determine if skills assessment results correlate in these two settings.
Methods: Participants in the study were PGY 1-5 surgical residents (RES) and expert (EXP) endoscopists. Each participant performed a VR colonoscopy case on the Simbionix GI mentor II endoscopy simulator and an actual colonoscopy during a 2-week period. Built-in software-based measurements in the VR endoscopy simulator were used to evaluate performance on the simulator. The GAGES-C scale (Vassiliou, et al.) was used to evaluate clinical performance. Separate GAGES-C assessments were done by supervising attendings and by the resident performing the procedure (self-assessment). Data comparisons were performed by unmatched T-test. Cronbach’s alpha and intraclass correlation (ICC) were used to assess internal consistency and interrater reliability (IRR) of GAGES-C results, respectively. Pearson correlation was used to evaluate correlation of performance in VR and clinical colonoscopy.
Results: The RES group (n=9) consisted of 4 males and 5 females, mean age 28 ± 2 years, with prior endoscopy experience ranging between 0-35 upper endoscopies and 0-60 colonoscopies. EXP group (n=6) consisted of 4 gastroenterologists and 2 colorectal surgeons, 4 male and 2 female, mean age 42 ± 7.0, with prior endoscopy experience ranging between 25-10,000 upper endoscopies and 300-5,000 colonoscopies. Mean GAGES-C scores for RES group performance on attending evaluation (2.5±1.0) did not differ significantly from resident self-assessment (2.8±1.2; p=0.59). IRR was good with ICC = 0.78 (95%CI: 0.08-0.95) and internal consistency was excellent (Cronbach’s alpha = 0.94). Mean GAGES-C scores for residents were significantly lower than for experts for both EXP assessment and RES self-assessment (2.8±1.2 versus 5±0.0; p<0.005). On VR endoscopy mean time to reach the cecum was significantly less for EXP than RES group (4.12±1.27 vs. 12.5±10.25 minutes; p= 0.04), as was calculated mean sum of errors (7.9±6.6 vs. 1.8±2.0, p=0.03). Efficiency of screening and time with clear view were not significantly different between groups. No significant correlation between VR endoscopy measures and GAGES-C scores could be demonstrated.
Conclusion: Mean GAGES-C scores for measurement of performance in clinical colonoscopy demonstrated good IRR and excellent internal consistency. Expert endoscopists performed better than residents in concurrently performed VR and actual colonoscopies, confirming construct validity of the assessment systems employed. However, correlation of VR measures and GAGES-C scores could not be shown. We suspect that small sample size and vagaries of the nonstandardized clinical task versus highly standardized VR task are likely contributors to this finding.
Session Number: Poster – Poster Presentations
Program Number: P171
View Poster