Y Watanabe, MD, E Bilgic, E M Ritter, MD, S Schwaitzberg, MD, P A Kaneva, MSc, K I Hoffman, PhD, J R Korndorffer, MD, D J Scott, MD, A Okrainec, MD, M T Odonnell, MD, L S Feldman, MD, G M Fried, MD, M C Vassiliou, MD. McGill University Health Centre, Tulane University, Uniformed Services University/Walter Reed Medical Simulation Center, University of Southern California, University of Texas Southwestern Medical Center, University of Toronto.
Introduction
The Fundamentals of Laparoscopic Surgery (FLS) manual skills program consists of 5 tasks. Experts identified camera navigation and cannulation as important skills that are neither taught nor assessed with FLS. The purpose of this multicenter study was to provide validity evidence for camera navigation and cannulation tasks and to explore the value of adding these new tasks to the FLS program.
Methods
Participants from 5 North American centers completed the camera navigation (N), cannulation (C), and five FLS tasks. They also completed a questionnaire about the educational value of the new skills. Two raters scored the new tasks at each institution, and an FLS certified proctor scored the other tasks. For N, participants were required to acquire and take a snapshot of 6 targets using a 30 degree simulated scope; accuracy was assessed at a single site based on photos. For the C task, participants were timed while introducing a catheter into a simulated duct from 3 different positions. Validity was assessed by comparing performance between Novice (PGY1 and 2) and Experienced (PGY3 and higher) participants and correlating scores on the new tasks with total FLS scores. Multiple regression analysis was conducted to evaluate the predictive validity of surgical experience. Internal consistency was assessed using Cronbach’s alpha. Intra class correlation coefficients (ICCs) were calculated to assess inter-rater reliability.
Results
Sixty subjects (45 surgical residents, 4 fellows, 11 attending surgeons) participated. Fifty-five (92%) participants had experience using a laparoscope and 32 (53%) participants had never performed intraoperative cholangiogram (IOC) in practice. Novice and Experienced participants scored 74±17.8 vs 85±8.3 (p<.01) and 21±17.3 vs 39±20.1 (p<.01) on N and C tasks respectively. Correlation with FLS scores for the N and C tasks were .39, p<.01 and .53, p<.01 respectively. Regression analysis revealed that FLS score alone predicted 45% of the training level variance (R2 =.45, p<.01) which rose by 2% when the 2 new tasks were added. Internal consistency of the five FLS tasks was .802; internal consistency adding N was .783, adding C was .796 and adding both new tasks was .785. Inter-rater reliability for both N and C was .99. Of 55 participants with scope experience, 51% reported N to be similar in difficulty to reality, and 67% of participants thought N added value to the FLS program. Of 28 participants who performed IOC, 25% reported C as similar in difficulty to practice, and around 65% thought it added value to FLS program.
Conclusions
This study provides validity evidence for the metrics of both camera navigation and cannulation. Participants found the new tasks to add educational value to the FLS program. They measure important clinical skills that are not currently represented in FLS and have excellent inter-rater reliability. However, the benefit of adding these tasks to the FLS manual skills assessment is marginal in terms of predicting experience level.
sages_adbutler_leaderboard