A novel natural user interface for image-guided minimally invasive surgery

Francisco M Sanchez-Margallo, PhD, Juan A Sanchez-Margallo, PhD, Jose L Moyano Cuevas, MsC, Juan Maestre Antequera. Minimally Invasive Surgery Centre

Objective: Image-guided surgical (IGS) systems allow surgeons to plan and guide the intervention providing additional information on the status of the anatomic structures or position of the surgical instruments. In extirpative surgeries such as hepatectomy or partial nephrectomy for tumors, these systems provide the location of the structures involved in the resection as well as the optimal surgical margins for preserving much healthy tissue as possible. However, surgical environments require special aseptic conditions that hamper the use of traditional input devices such as keyboard and mouse for interacting with the IGS systems. The main objective of this work is to test the feasibility of using a touchless interface for image-guided minimally invasive surgery.

Technology and Methods: For interacting with the preoperative imaging during the surgical intervention the TedCube (TedCas Medical Systems, Spain) system has been used. This technology allows surgeon interaction in a completely sterile way by means of touch-free natural user interfaces based on optoelectronic devices. Different gestural control sensors for user interaction have been used in this study such as Microsoft Kinect (Microsoft, USA), Leap Motion (Leap Motion, Inc., USA), MYO (Thalmic Labs Inc., Canada) and voice control (Figure 1).

Figure 1. Use of TedCube system during a minimally invasive intervention.

Different minimally invasive surgeries have been performed such as hepatectomy or partial nephrectomy. For the partial nephrectomy an experimental porcine model of a renal pseudotumor was created. The location and extent of pseudotumor was conducted by CT and ultrasound imaging. For the hepatectomy, the identification of the hepatic vascular anatomy was obtained by CT imaging with the use of contrast medium. This preoperative information was provided to the surgeon during the development of both surgical procedures for surgical support and interaction through the IGS system.

A preliminary validation of the system was performed. For this purpose a questionnaire (Likert scale-5, 1 as minimum value and 5 as maximum) was designed to evaluate the usability and usefulness of the system. This questionnaire was complete after performing the surgeries with the support of the IGS system.

Results: All procedures were successfully completed without complications. The surgeon was able to interact properly with the IGS system during the course of all surgical procedures. Some of the preoperative images used during the IGS interventions are shown in Figure 2 and Figure 3. The surgeon showed great satisfaction in the usability of the IGS system and its interaction during the performance of hepatectomy and nephrectomy interventions, considering it a useful tool for the location of the risk structures during surgery such as the tumors and vascular tree of the organs.

Figure 2. (a) 3D reconstruction of pseudotumor in left kidney. (b) Ultrasound image of pseudotumor in the left kidney.

Figure 3. Maximum intensity projection image of the vascularization of the liver's portal venous system.

Conclusions: Image-guided minimally invasive surgeries based on natural interfaces allow surgeons to interact with preoperative patient information in a more intuitive and sterile manner, and therefore safer for the patient.