Proficiency-Based Training for Robotic Surgery: Construct Validity and Workload for Nine Inanimate Exercises

Genevieve Dulan, MD, Robert V Rege, MD, Deborah C Hogg, BS, Kristine K Gilberg-Fisher, RN BSN, Nabeel A Arain, MD MBA, Seifu T Tesfay, RN MS, Daniel J Scott, MD. University of Texas, Southwestern

Introduction: We previously developed 9 inanimate training exercises as part of a comprehensive, proficiency-based robotic training curriculum that addressed 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to evaluate construct validity and workload of the 9 exercises.

Methods: Expert robotic surgeons (n = 8, fellows and faculty) and novice trainees (n = 4, medical students) each performed 3-5 consecutive repetitions of the 9 standardized exercises. 5 used FLS models with or without modifications, including Peg Transfer, Clutch/Camera Peg Transfer, Pattern Cut, and Interrupted and Running Suture; 4 used other commercially available and custom made components, including Rubber Band Transfer, Stair Rubber Band Transfer, Clutch/Camera Navigation, and Running/Cutting Rubber Band. Each task was scored for time and accuracy using modified FLS metrics; task scores were normalized to a previously established proficiency level and a composite score equaled the sum of the 9 normalized task scores. Questionnaires were administered regarding prior experience. After each exercise, participants completed a validated NASA-TLX Workload Scale to rate the mental, physical, temporal, performance, effort and frustration levels of each task. Comparisons used Mann-Whitney tests; p<0.05 was considered significant.

Results: Experts had performed 151 (range 15-600) robotic operations; novices had observed ≤ 1 robotic operation. For all 9 tasks (Table) and the composite score, experts achieved significantly better performance than novices: 932 ± 67 vs. 618 ± 111 (p<0.001), respectively. No significant differences were detected between experts and novices for the 9 tasks and the overall workload (scale 1-10, 10 = high workload) ratings: 4.8 ± 2.1 vs. 5.3 ± 0.6 (n.s), respectively. Importantly, frustration ratings were relatively low for both groups (4.0 ± 0.7 vs. 3.8 ± 1.6, n.s.).

Conclusion: Using objective performance metrics, all 9 exercises demonstrated construct validity. Workload was similar between experts and novices and frustration was low for both groups. These data suggest that the 9 structured exercises are suitable for proficiency-based robotic training.