Genevieve Dulan, MD, Robert V Rege, MD, Deborah C Hogg, BS, Kristine K Gilberg-Fisher, RN BSN, Nabeel A Arain, MD MBA, Seifu T Tesfay, RN MS, Daniel J Scott, MD. University of Texas, Southwestern
Introduction: We previously developed 9 inanimate training exercises as part of a comprehensive, proficiency-based robotic training curriculum that addressed 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to evaluate construct validity and workload of the 9 exercises.
Methods: Expert robotic surgeons (n = 8, fellows and faculty) and novice trainees (n = 4, medical students) each performed 3-5 consecutive repetitions of the 9 standardized exercises. 5 used FLS models with or without modifications, including Peg Transfer, Clutch/Camera Peg Transfer, Pattern Cut, and Interrupted and Running Suture; 4 used other commercially available and custom made components, including Rubber Band Transfer, Stair Rubber Band Transfer, Clutch/Camera Navigation, and Running/Cutting Rubber Band. Each task was scored for time and accuracy using modified FLS metrics; task scores were normalized to a previously established proficiency level and a composite score equaled the sum of the 9 normalized task scores. Questionnaires were administered regarding prior experience. After each exercise, participants completed a validated NASA-TLX Workload Scale to rate the mental, physical, temporal, performance, effort and frustration levels of each task. Comparisons used Mann-Whitney tests; p<0.05 was considered significant.
Results: Experts had performed 151 (range 15-600) robotic operations; novices had observed ≤ 1 robotic operation. For all 9 tasks (Table) and the composite score, experts achieved significantly better performance than novices: 932 ± 67 vs. 618 ± 111 (p<0.001), respectively. No significant differences were detected between experts and novices for the 9 tasks and the overall workload (scale 1-10, 10 = high workload) ratings: 4.8 ± 2.1 vs. 5.3 ± 0.6 (n.s), respectively. Importantly, frustration ratings were relatively low for both groups (4.0 ± 0.7 vs. 3.8 ± 1.6, n.s.).
| Task 1 | Task 2 | Task 3 | Task 4 | Task 5 | Task 6 | Task 7 | Task 8 | Task 9 | |
| Experts | 97±5 | 104±7 | 94±9 | 100±5 | 106±5 | 104±13 | 110±9 | 101±31 | 134±20 |
| Novice | 80±16 | 94±9 | 66±19 | 71±16 | 88±7 | 89±11 | 66±24 | 30±36 | 26±31 |
| p value | 0.002 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
Conclusion: Using objective performance metrics, all 9 exercises demonstrated construct validity. Workload was similar between experts and novices and frustration was low for both groups. These data suggest that the 9 structured exercises are suitable for proficiency-based robotic training.
Session: SS16
Program Number: S090