Helena M Mentis, PhD, Christopher Wong, MS, Yuanyuan Feng, MS, Jordan Ramsey, Meredith Evans. University of Maryland Baltimore County
OBJECTIVE OF THE TECHNOLOGY
Senior surgeons guide residents in laparoscopic procedures by pointing at anatomy on the display or inside the body with available tools1. The frequency of this form of target identification and target fixation is significantly higher than instrument guidance2.
DESCRIPTION OF TECHNOLOGY
We developed a Kinect-based image annotation system for surgeons to provide reference and annotation as an overlay to the real-time endoscopic video. This is on a secondary display so there is no interference with the primary display (Figure 1). This system provides a surgeon from either side of the table with the ability to overlay (1) a reticle that remains in the center of the screen (Figure 2), (2) a pointer to move around the screen, or (3) a drawing tool to create freehand markings (Figure 3). Voice control is used to activate these referencing tools and to clear the screen. Once a tool is activated, gestures are used for moving the pointer or drawing.
Figure 1. Example OR Room Setup
Figure 2. Close up Image of Reticle Overlay
Figure 3. Screenshot with Freehand markings
PRELIMINARY RESULTS
Two laparoscopic surgeons conducted a preliminary evaluation of the system in a simulated environment with a prerecorded video feed. Our aim with this evaluation was to elicit feedback on the functionality and further design requirements. The surgeons’ overall feedback was positive towards the usefulness of the system for resident training, but there was a concern with more senior surgeon acceptance. There was a request to freeze the laparoscopic video image or be able to replay captured video for further telestration – i.e. freeze the image in order to point out specific anatomy without worrying about the camera moving. Along the same lines, there was a request for the system to record the video in a buffer so the surgeon could rewind the video and telestrate over that recorded video. It was also suggested that we allow surgeons to select the color of the overlays instead of having it predefined. For instance, one participant explained, “I’m thinking, arthoscopy is going to be a lot of red and white, lapcholes are red and yellow…”.
CONCLUSIONS & FUTURE DIRECTIONS
There is a need for laparoscopic telestration and low-cost developmental sensor kits such as the Kinect show great promise as the touchless mechanism to make this a reality. Our preliminary evaluation has uncovered further design directions for the system – namely freezing or rewinding the video and an option for overlay color selection. As we continue to develop the system, we will integrate these refinements and elicit further requirements from surgeons.
REFERENCES
1. Mentis HM, Chellali A, Schwaitzberg S. Learning to See the Body?: Supporting Instruction al Practices in Laparoscopic Surgical Procedures. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems.; 2014:2113-2122. doi:10.1145/2556288.2557387.
2. Feng Y, Wong C, Park A, Mentis H. Taxonomy of instructions given to residents in laparoscopic cholecystectomy. Surg Endosc. July 2015. doi:10.1007/s00464-015-4300-0.
« Return to SAGES 2016 abstract archive
