Development of a formative feedback tool for advanced laparoscopic suturing

Katherine M McKendy, MD1, Yusuke Watanabe, MD1, Elif Bilgic2, Ghada Enani, MBBS2, Mo Yu Lanny Li2, Talla Raja2, Munshi Amani, MD1, Rajesh Aggarwal, MBBS, MA, PhD1, Liane S Feldman, MD1, Gerald M Fried, MD1, Melina C Vassiliou, MD, MEd1. 1McGill University Health Centre, 2McGill University

INTRODUCTION

Commonly used metrics to assess performance of simulated laparoscopic skills in a box trainer focus on time and accuracy. These measures, however, do not provide meaningful information to trainees about how to improve their skills. The aim of this mixed methods study was to develop a tool that could be used to provide residents with specific, formative feedback about their laparoscopic suturing skills.

METHODS AND PROCEDURES

We videotaped senior surgical residents (PGY3-5) and MIS-trained surgeons performing 3 advanced laparoscopic suturing tasks: needle handling (NH), suturing under tension (UT), and continuous suturing (CS). After completing the tasks, the surgeons were asked to comment on the key technical aspects that were important for the completion of each task, and to provide feedback on the videotaped performance of a novice completing the tasks. Interviews were performed until saturation was reached. They were then transcribed, and common themes regarding advanced suturing skills and decision-making were extracted to generate a feedback rubric for each of the tasks. The feedback tool (FT) was then used to assess the videotaped performances of residents and MIS-trained surgeons by two blinded independent raters. A two-way random effects model was used to calculate the single-rater intra-class correlation coefficient (ICC), expressed as: ICC (95% confidence interval). The correlation between the combined score for time and accuracy and the FT score was calculated for each of the tasks. Median and interquartile ranges (IQR) are expressed as: Medians[IQR], p-value.

RESULTS

Seven MIS surgeons were interviewed. From these interviews, a feedback rubric was generated for each of the tasks; NH consisted of 10 items, UT 18, and CS 20. Each item was classified according to one of 7 key principles that emerged from the interview data: depth perception, safety, bimanual dexterity, exposure, tissue handling, instrument manipulation, and forward planning. Performance was graded on a 3-point Likert scale (“does well”, “needs some improvement”, an “does poorly”) and scores were calculated as an equally weighted sum of the points. The FT was then used to assess the video performance of residents and experts on all of the tasks: 16 resident and 11 expert videos were assessed for NH, 12 residents and 11 experts for UT, and 16 resident and 11 expert for CS. ICCs for NH, UT, and CS were 0.90 (0.79-0.95), 0.87 (0.73-0.95) and 0.90 (0.76-0.95) respectively. Scores were found to correlate strongly with combined time and accuracy measurements for UT (0.82, p<0.01) and CS (0.81, p<0.01), and moderately for NH (0.65, p<0.01). MIS surgeons also performed significantly better than senior residents on UT (experts: 29.5[26.5-33] vs. novices: 23.75[19.6-29.1], p=0.02) and CS (experts: 34.5[30-36.5] vs. novices 26[21.4-29], p=0.05), while scores on NH were similar (experts: 13.5[11-15.5] vs. novices 12.8[9.5-15.5], p=0.57).

CONCLUSIONS

We developed a specific, comprehensive tool for providing feedback about advanced laparoscopic suturing skills. The FT demonstrates evidence for validity as a measure of suturing skills and experience, and also provides meaningful guidance to trainees about how to improve their skills and engage in more deliberate and efficient practice.

« Return to SAGES 2015 abstract archive