Introduction: Standard skill evaluation methods in the operating room (OR) include direct observation and checklists; both are time consuming and subject to bias. Objective assessment systems have been proposed for simulators using dexterity measures. However, motor performance in simulators can differ from that in the OR, so monitoring motor performance in the OR is necessary to supplement evaluations of performance on simulators. The purpose of this study was to evaluate the feasibility of using quantitative measures acquired in the OR to distinguish between levels of laparoscopic skill development.
Methods: We observed two sets of subjects representing two stages of training: 3 residents (4th year), and 3 experts (attending surgeons) performing 3 Laparoscopic Cholecystectomies. Electromagnetic position sensors were attached by the surgeons to a Maryland dissector and an atraumatic grasper. From the tools’ positions, we extracted measures related to time, kinematics and movement transitions. Various methods such as the Kolmogorov-Smirnov (K-S) statistic and the Jensen-Shanon Divergence (JSD) were used to provide intuitive dimensionless difference measures ranging from 0 to 1. These scores were used to compare residents and expert surgeons executing two key surgical tasks: exposing Calot’s Triangle and dissecting the cystic duct and artery (CD/CA).
Results: The triangle exposure task separated the two groups more clearly than the dissection task. We found significantly larger between-group than within-subject variability (ANOVA p < 0.05). We assessed procedure-to-procedure variability using K-S values for kinematics and time profiles and JSD values for movement transitions and found that kinematic measures most strongly differentiated between surgical levels.
Conclusions: We used time, kinematic and movement transition measures as our intraoperative performance measures. Kinematics proved to be the most effective performance measure to use to differentiate between skill levels. Moreover, the degree of discrimination depended on the type of surgical task used for comparison. We conclude that objective assessment of surgical motor behaviour is likely feasible in the OR context.
Session: Podium Presentation
Program Number: S030