Development of A Novel Video-Based Script Concordance Test for Laparoscopic Suturing

Ghada Enani, MBBS, MEd, Yusuke Watanabe, MD, PhD, Amani Munshi, MD, Melina Vassiliou, MD, MEd, FRCSC, FACS. McGill

Introduction: Laparoscopic intracorporeal suturing is a challenging skill to acquire. Current surgical training for laparoscopic intracorporeal suturing is partly based on simulation training, which focuses more on the technical aspects of suturing, and less on the cognitive and decision-making elements, which tend to vary considerably among experts.

The script concordance test (SCT) is a tool based on cognitive psychology script theory that can be used to measure how an individual evaluates an ill-defined clinical problem.  The goal of this study was to develop a video-based assessment of cognitive and decision-making skills for laparoscopic suturing and a scoring system based on a modified SCT methodology, and to provide preliminary validity evidence.

Methods: To define the skill set, a cognitive task analysis (CTA) was performed by having 10 MIS experts review selected videos on intraoperative suturing scenarios. Using FluidSurveys (, an online assessment using videos of a Nissen Fundoplication and Paraesophageal hernia repair were developed based on the CTA findings.  Five-point scales with anchoring descriptors from -2 to +2 were used (Figure 1). To calibrate the scoring system, 13 separate MIS expert panelists, completed the survey. Based on the variability of answers of the panel members, a maximum score of one is given for the response chosen by most experts, other responses are given partial credit and responses not selected receive zero scores. The SCT was then administered to participants for test optimization using post hoc elimination of questions with a negative item-total correlation or a correlation of less than 0.05. Internal consistency of test items was estimated using Cronbach’s alpha. Wilcoxon rank test was used to compare scores between fellows and residents.

Results: Eighteen participants (6 MIS fellows, 3 PGY5, 3 PGY4, and 6 PGY3) completed the 64-item/4 video test.  Of a total of 64 questions for 4 videos, 47 questions were selected based on item-total correlation.  The internal consistency of test items was 0.91. The median and interquartile range (IQR) score of the panel was 77.4 [72.3-81.0]. There was a significant difference between fellows and senior residents: 67.8  [62-81.3] versus 56.6  [31.7-63.3], p=0.02.

Conclusion: This video-based online SCT for laparoscopic suturing is a novel way to assess clinical reasoning while capturing the complexity and variability of the clinical environment. This study provides preliminary validity evidence for this tool as a measure of clinical decision making for laparoscopic suturing. 

Figure 1. An example of the clinical vignette and questions. 

« Return to SAGES 2016 abstract archive

Reset A Lost Password