An Information-Rich Technology for Comprehensive Evaluation of the Human-Machine Interaction in Robotic Surgery

Konstantin Umanskiy, MD, Marie Ziesat, MD, William Hrdina, Saeed Richardson, Stephen Small, MD. University of Chicago Departments of Surgery and Anesthesia. University of Chicago Simulation Center

Fig. 1

Objective of the technology
The currently available surgical robot (da Vinci, Intutive Surgical, Sunnyvale, CA) is an electromechanical manipulator that is fully controlled by a human operator. In contrast to conventional or laparoscopic surgery, robotic surgery is performed via remote console that eliminates direct interaction between the surgeon and the patient. Thus, the surgeon only interacts directly with the robot, which in turn interacts with the patient. An additional level of complexity is added through control of the camera and position of the masters-manipulators via a series of foot pedals. All communication with the surgical team (anesthesia, assistant, etc) occurs via robotic microphone and speakers. These complex interactions have not been studied. Conventional training techniques used in laparoscopic surgery focus on observation or measurements of task accuracy, force used, and time to completion. Rapid adaptation of robotic technology increases the need to develop a comprehensive technique to capture a surgeon’s performance to be used in training, assessment, and possibly credentialing process. We developed a novel technique to record and evaluate all aspects of robotic surgery by combining video and audio information of human-machine and machine-patient interaction during a live surgical operation.

Description of the technology and method of its application
We record video feed from four sources as well as audio communication simultaneously during a live robotic surgical operation. Two camcorders are used to record the movements of the surgeon’s right and left hands, while a webcam mounted underneath a hand rest captures foot controls. The robotic camera itself captures a separate video. All video and audio feeds are combined on a single screen during post-recording processing to provide a complete representation of surgeon’s operative performance (Fig. 1).

Conclusions/Future directions
In this pilot study we present a technique for simultaneous acquisition of the information combined in a single screen format for a comprehensive assessment of the surgeon’s performance during the live surgical operation. This approach could be used for capturing the expert surgeon’s performance that can then serve as a part of a cognitive task analysis to elucidate the specific aspects of the advanced robotic surgical technique that are easily overlooked if only robotic camera recording is used. The information obtained from the expert performance can be used in designing training curriculum, assessing the trainee’s progress, or for credentialing of the surgeons to perform robotic surgical operations.