In-Vivo 3D Surface Reconstruction and CT Registration for Minimally Invasive Surgery

Jaime E Sanchez, MD, Bingxiong Lin, Adrian Johnson, Yu Sun, PhD, Xiaoning Qian, PhD

University of South Florida Center for Advanced Medical Learning and Simulation

The aim of this technology is to provide a registered, augmented reality, visual overlay of patient specific radiologic data in minimally invasive surgery. We have developed a means to digitally reconstruct the 3D surfaces of in-vivo tissues and organs and then use the dense 3D surface data to register the live laparoscopic video scene to the 3D volume data from CT scans. With registration, the pre-identified
anatomical structures of interest from the CT data can be rendered and overlaid on the live surgery video to clearly identify specific structures of interest.

To reconstruct the 3D surfaces of in-vivo intraabdominal anatomy, we exploit the inherent geometric property of the anatomical surfaces to improve feature-matching performance by using a Thin Plate Spline (TPS) model. The feature correspondences and pixel intensity information in images from stereo cameras allows for the estimation of an accurate 3D surface of a small area from one frame of the
stereo camera. To reconstruct the entire surgical area, we register the stereo images from different frames using simulated images from the generated small 3D surface with reprojection. The descriptors of the feature points under different view angles are stored to ensure that the proposed method can tolerate a large range of view angle change.

We have evaluated the proposed 3D surface reconstruction approach with silicon phantoms and in-vivo images. In addition, we have applied the iterative closest point (ICP) approach to register the dense 3D surface reconstruction of intraabdominal structure areas to the 3D volume data from CT scans.

This method allows mapping and identification of hidden surgical anatomy during minimally invasive surgery through a visual overlay of rendered CT data that is registered to the surgical video using 3D surface anatomy. Ultimately this technology may be used to provide image guided surgery, map vascular anatomy and avoid injury to deep structures such as those located in the retroperitoneum when operating in the abdomen and pelvis.

Session: Podium Presentation

Program Number: ET005