Satoshi Endo, MD, Marylise Boutros, Gabriela Ghitulescu, Elif Bilgic, Madoka Takao, Nancy Morin, Carol-Ann Vasilevsky, A. Sender Liberman, Patrick Charlebois, Barry Stein, Gerald M Fried. Mcgill University
Background: Laparoscopic anterior resection is technically challenging and the learning curve is long. Well-designed formative assessments can provide trainees effective and constructive feedback, an important element in efficient learning. Previously reported assessments for laparoscopic colorectal procedures were developed for summative assessment. We aimed to develop a formative assessment tool to evaluate competence and provide trainees with effective feedback in laparoscopic anterior resection.
Methods: The assessment tool was developed by an expert panel from McGill University affiliated hospitals. The procedure was deconstructed into a series of sequential steps including general domains, surgical principles, injury prevention and technical skills specific to laparoscopic anterior resection. The tool contains 12 discrete items with global rating scales for each step of the operation; each domain was scored using a 5-point Likert scale, with anchors for scores of 1, 3 and 5. Each operation was assessed through direct observation in the operating-room by the attending, a trained observer, and trainees themselves.
Intraclass correlation coefficients (ICCs) were calculated to estimate interrater reliability for (1) attending surgeon and trained observer, (2) attending surgeon and self-assessment, and (3) trained observer and self-assessment. Internal consistency was measured using Cronbach’s alpha. Comparison between training levels was done using Mann–Whitney U-test. The Global Operative Assessment of Laparoscopic Skills (GOALS) was also used to assess trainees’ general laproscopic skills. Spearman’s correlation was used to determine association between GOALS and this procedure-specific tool. Overall usefulness of this tool was evaluated using a 10 cm Visual Analog Scale.
Results: In this pilot study, fourteen operations, performed by 5 experienced surgeons and 5 trainees were assessed. The ICC between (1) attending surgeon and observer was 0.77 (95% CI 0.26 to 0.93) (2) observer and self-assessment was 0.74(95% CI 0.30 to 0.92), and (3) attending surgeon and self-assessment was 0.43(95% CI -0.11 to 0.79).
The internal consistency of the items was excellent (Cronbach's α=0.93). There was a significant difference in median total score between experienced surgeons and trainees (87.2 ± 9.4 vs. 68.8 ± 9.3; p=0.016). There was strong correlation (r=0.884) between GOALS and this procedure-specific score. Overall usefulness of this assessment tool was rated as 7.4±1.7. All assessments were completed in about 5 minutes.
Conclusions: We present a new procedure-specific formative assessment tool for laparoscopic anterior resection and provide preliminary evidence of its reliability and validity. This formative assessment tool could be used for constructive feedback and tracking performance in competency-based surgical training.
Presented at the SAGES 2017 Annual Meeting in Houston, TX.
Abstract ID: 85117
Program Number: P305
Presentation Session: iPoster Session (Non CME)
Presentation Type: Poster