Ian Choy, BEng MD MEd, Andras Fesco, BSc MD, Josephine Kwong, BScH MPA, Timothy Jackson, MD MPH, Allan Okrainec, MD MHPE. Department of Surgery, University of Toronto, Temerty/Chang Telesimulation Centre, University Health Network, Toronto, Ontario, Canada
Telesimulation and telementoring have emerged as important tools in improving access to advanced surgical techniques amongst surgeons globally. With the continuing improvement in telecommunication infrastructure in the developing world, as well as support from programs such as the SAGES Go Global Initiative, telesimulation and telementoring have the potential to play a significant role in the global development of laparoscopic surgery.
In order for these tools to be used to provide surgical training in remote settings, an objective and reproducible assessment tool must be available to provide feedback to trainees. The objective of this study was to determine the feasibility of using the Global Operative Assessment of Laparoscopic Skills (GOALS) assessment tool to assess laparoscopic skills remotely via the same telecommunication technology used in telesimulation and telementoring. A second objective was to identify how factors unique to remote assessment such as bandwidth and image quality influence the reliability of the the GOALS assessment tool.
Methods and Procedures
This study used an opportunity sample of 34 surgical trainees and general surgeons with varying experience levels. Study participants were evaluated for their technical performance during laparoscopic cholecystectomies (LC) by four expert observers. One observer was located in the operating room and observed the subject directly while the other three observers were located in remote locations within the same hospital. These remote observers viewed the operation using SkypeTM videoconferencing software on a laptop that received the direct video output from the laparoscope. The remote observers were randomly assigned, and blinded to, one of three bandwidth restrictions (unlimited, 256 kbits/s, and 64.4 kbits/s). All observers used the GOALS assessment tool to evaluate the trainee’s technical skills. The Maryland Visual Comfort Scale (MVCS) was used by the remote observers to evaluate the video quality of their respective connections.
Results
The intraclass correlation coefficient (ICC) for the total GOALS score between local observers and all remote observers was 0.45 (95% confidence interval (CI) 0.23 to 0.69). The ICCs between local observers and high-bandwidth remote observers was 0.62 (95% CI 0.23 to 0.82), between local observers and medium-bandwidth remote observers was 0.52 (95% CI 0.19 to 0.75), and between local observers and low-bandwidth remote observers was 0.49 (95% CI 0.11 to 0.74). There was a statistically significant difference between the MVCS scores of the high and low-bandwidth observers (p=0.0004) and the medium and low-bandwidth observers (p<0.0001).
Conclusions
The results of this study demonstrated a positive correlation between GOALS scores of local observers and remote observers, with increasing reliability seen as the internet bandwidth increased. These findings support the use of the GOALS assessment tool in remotely evaluating laparoscopic surgical skills. Further research is needed to assess how this current system of remote assessment can be improved upon to obtain greater correlation for specific GOALS domains and how these tools affect surgical outcomes in the remote communities in which they are applied.
Session Number: SS05 – Education
Program Number: S025