Richard Castillo, MD1, Juan Alvarado, MD1, Cristobal Maiz, MD1, Billeke Pablo, MD, PhD2, Andrea Vega, RN1, Nicolas Jarufe, MD1, Camilo Boza, MD1. 1Escuela de Medicina, Pontificia Universidad Catolica de Chile, 2Universidad del Desarrollo de Chile
Our objective was to assess reliability and validity of a visual-spatial secondary task (VSST) as a method to measure automaticity on basic simulated laparoscopic skill model.
In motor skill acquisition, expertise is defined by automaticity. The highest level of performance with less cognitive and attentional resources characterizes this stage, allowing experts to perform multiple tasks.
Conventional validated parameters, as operative time, objective assessment skills scales (OSATS) and movement economy, are insufficient.
There is literature about using a VSST as an attention indicator that correlates with the automaticity level.
Novices with approved FLS (n=11) and laparoscopy experts (n=4) were enrolled for an experimental study and measured under dual tasks conditions.
Each participant performed the test giving priority to the primary task while at the same time they responded to a VSST. The primary task consisted of four laparoscopic stitches (LS) on a bench-model. The VSST was a screen that showed different patters that the surgeon had to recognize and press a pedal while doing the stitches (PsychoPsy software, Python, MacOS).
Novices were overtrained on LS until reach at least 100 repetitions and were retested.
Participants were video recorded and then assessed by two blinded evaluators who measured operative time and OSATS. These scores were considered indicators of quality for the primary task. The VSST performance was measured by the detectability index (DI), which is a correlation between correct and wrong detections.
A reliable evaluation was defined as two measures of DI with less than 10% of difference, maintaining the performance on the primary task (operative time <110seg and OSATS >17points).
Novice achieved reliable measure of the test after 2(2-5) repetitions on the pre-evaluation and 3.75(2-5) on the post-evaluation (p=0.04); while laparoscopy experts did it after 3.5(3-4) repetitions.
Proficiency scores of primary task (defined previously) were accomplished on every measure for novices (pre-post overtraining) and experts.
Expert performance on VSST was DI 0.78(0.69-0.87). Novice performance was significantly better on post-evaluation [DI-pre 0.48(0.06-0.71) vs DI-post 0.78(0.48-0.95), p=0.003]. Overtraining consisted on 140(100-210) repetitions of LS for all novices, made on 8 hours (3-15).
By categorizing DI based on expert performance, novices with DI-post>0.65 achieve better OSATS score and less operative time than novices with DI-post<0.65 (p=0.007 y p=0.089, respectively)
Measuring of automaticity is feasible by using a VSST. This instrument is reliable and has a face, content and construct validity. A detectability index over 0.65 may be a cut-off point correlated with high standard performance on the primary task.
This instrument measured performance on laparoscopic skills, and along with conventional indicators, better define advance levels of expertise. More studies are required applying this VSST to achieve external validity by reproducing our results.