Alexandra J Berges1, S. Swaroop Vedula, MBBS, PhD2, Edward J Tanner, MD3, Amanda N Fader, MD3, Stacey Scheib, MD3, Betty Chou, MD3, Gregory D Hager, PhD2, C. C. Grace Chen, MD, MHS3, Anand Malpani, PhD2. 1Johns Hopkins University School of Medicine, 2Malone Center for Engineering in Healthcare, 3Johns Hopkins Medicine Department of Gynecology and Obstetrics
INTRODUCTION: The objective in this study was to determine agreement between attending and trainee surgeons on important aspects of what happened in the operating room during robot-assisted laparoscopic hysterectomy (RALH) procedures, which are essential for teaching and assessment, e.g., case participation for accurate case logs, intraoperative skill assessment for performance evaluation and feedback.
METHODS AND PROCEDURES: Attending and trainee surgeons performing 56 RALH procedures responded to questionnaires postoperatively concerning: 1) who performed each of 11 steps (L/R excise infundibulopelvic ligament, excise utero-ovarian ligament, skeletonize uterine vessel, electrodessicate uterine vessels, dissect bladder, colpotomy or amputate uterus, close vaginal cuff or cervical stump), 2) vaginal cuff closure technique (VCCT) and suture device (SD) used, 3) trainees’ self-assessed skill and attending assessment of trainees’ skill using GEARS, and difficulty of surgery. For bladder dissection, colpotomy, and vaginal cuff closure, we asked what percentage (0%, 25%, 50%, 75%, or 100%) was performed by the trainee. We analyzed agreement between attending and trainee surgeons using Cohen’s kappa for categorical variables and Krippendorff’s alpha (KA) for ordinal variables, as well as Spearman’s correlation (rho).
RESULTS: Kappa was greater than 0.6 for 8 surgical steps (kappa = 0.71, 0.62, 0.82, 0.85, 0.71, 0.78, 0.86, 0.86 respectively), suggesting strong agreement between attending and trainee surgeons. We observed kappa values of 0.79 and 0.79 for VCCT and SD, indicating strong agreement.
There was weak agreement for depth perception and bimanual dexterity criteria of the GEARS scale (rho = 0.21, 0.29 and KA = 0.158, 0.007, respectively). Skill assessments by attending and trainee surgeons on the domains of efficiency, force sensitivity, autonomy, and r
obotic control exhibited moderate correlation and agreement (rho = 0.468, 0.534, 0.440, 0.495; KA = 0.473, 0.506, 0.412, 0.484, respectively)
There was a moderate level of agreement (kappa = 0.484) on estimating difficulty of surgical case.
CONCLUSIONS: Our findings show that there is imperfect agreement between attending and trainee surgeons even with who performed parts of the procedure, which intuitively, should be unambiguous. They also disagreed on case difficulty, and technical skill. This incongruency on critical aspects of what happened in the operating room poses a significant challenge to teach and learn hysterectomy.
Presented at the SAGES 2017 Annual Meeting in Houston, TX.
Abstract ID: 95639
Program Number: P678
Presentation Session: Poster Session (Non CME)
Presentation Type: Poster