Construct Validity of Assessing Technical Skills of Trainees After Repetition in Simulated Laparoscopic Surgery

Aims
Assessing and appraising laparoscopic technical skills is a topical issue. To date there has been little attempt to comprehensively assess both human or computer based assessment tools after repetition of a task/procedure. In this study we aim to compare these different assessment tools in repeated simulated laparoscopic cholecystectomy.

Methods
Three groups of trainees were prospectively recruited: surgical medical students, junior residents and senior residents. Validated human based specific likert scales were used to assess specific technical skills. Two surgical observers assessed each full length laparoscopic cholecystectomy independently. Computer based assessment tools (total number of movements and total path length) were concurrently used to assess technical skills. These trainees performed the laparoscopic cholecystectomy again after a minimum of 2 weeks and their technical skills were assessed again with both sets of tools.

Results
60 simulated laparoscopic cholecystectomies were performed by 10 medical students, 10 junior residents and 10 senior residents. Inter-rater reliability Cronbach alpha for the two observers was 0.85 and 0.89, p = < 0.05, for technical skills using the human assessment tool for the first and second operation respectively. Construct validity using ANOVA for each group of trainees were significant, p = 0.000 and p = 0.000 for the first and second operations using the human assessment tool. Construct validity was significant, p = 0.002 for total instrument movements and for total path length, p = 0.000 for the first operation. However for the second operation validity was not conclusive with p = 0.08 and p = 0.015 for the instrument movements and path lengths respectively.

Conclusion
This human based assessment tool seems to have predictive and construct validities compared to computer based assessment parameters after repetition of the same operation. It seems that the human based tool does not suffer from the plateau effect as that of the computer parameters. This tool could be a better assessment tool of technical skill performance over a longer period of time for trainees.

Session: Poster

Program Number: P365