Detecting Abnormalities in Endoscopic Examinations By Combining Information From Multiple Images

Qian Zhao, Gerard E Mullin, MD, Max Q-h Meng, PhD, Themistocles Dassopoulos, MD, Rajesh Kumar, PhD. Johns Hopkins University, Chinese University of Hong Kong, Washington University School of Medicine

INTRODUCTION: Endoscopic, and more recently wireless capsule endoscopic (CE) examinations produce large number of images (e.g. over 50,000 images for CE), not all of which are interpreted with equal emphasis by the reviewing clinician. It is a time-consuming and tedious task to review such large amounts of data for clinicians, and traditionally only selected images are used for clinical diagnosis. Such analysis focus on abnormality detection on an individual image, by treating images with best view of the abnormality as the only, individual, and independent observations. As a result, relevant information available in neighboring images is often discarded, or at least does not contribute to the clinical diagnosis. We present methods of combining information from multiple images containing the same abnormalities, permitting stronger inferences, more automatically. These methods can be used for decision support during endoscopy (including CE), or during any endoscopic surgery.

METHODS AND PROCEDURES: We have developed supervised statistical classification methods based on a Hidden Markov Model (HMM) framework. Using color, edge, and texture information extracted from each images in a sequence where an abnormality might be visible, a Support Vector Machine (SVM) classifier performs an individual binary normal/abnormal classification. The HMM framework then combines the classifier output from individual images into a sequence classification. We tested such classification on CE images containing bile, air bubble, extraneous matter, lesions, normal lumen and polyps. Additional analysis of CE and endoscopic images is ongoing.
 

RESULTS: The proposed method was evaluated using a CE image database containing 47 studies collected using a Johns Hopkins Institutional Review Board (IRB) protocol. Image sequences of varying lengths were extracted from these studies and abnormalities were annotated by a clinical expert.  Table I shows the accuracy, precision and recall (sensitivity) comparison for individual image classification using the same SVM binary classifier and image sequence classification. The accuracy for all classes increase when classification uses multiple images.
 

CONCLUSIONS: The proposed framework shows promising performance and has the potential to reduce the reading time for clinicians for offline data, and real-time performance for providing decision support during endoscopic procedures. It can also be used to summarize the studies for generation of a synopsis containing relevant images. Further work towards development of a semi-automated computer aided diagnosis system using these methods is ongoing.
 

View Poster