Colon capsule endoscopy position estimation based on image classification

Abstract Colon capsule endoscopy (CCE) is a non-invasive investigation method where a camera capsule travels through and records a video of the large bowel. Accurate localization of findings such as polyps is important for planning and performing follow-up colonoscopies. However, no method has proven reliable or clinically viable for estimating the position of a CCE capsule as a function of time. We present an algorithm that predicts which colon segment a CCE capsule is in at each time point. A ResNet50 deep neural network was trained to classify individual images as small bowel, ascending colon, transverse colon, or descending colon. To improve temporal consistency, a sliding window approach combined with a matrix of transition weights is applied on top of the per-image predictions. The study uses 1,668 videos and clinical reports from the Danish CareForColon2015 trial, from which training images are automatically assigned a report-derived (weak) colon segment label based on landmark timestamps in the reports. Per-image classification accuracy on an independent test set extracted from 150 videos is 0.792. Evaluating polyp colon segment prediction against clinical report annotations across 818 held-out test videos yields an accuracy of 0.779. The algorithm gives a position estimate for all time points, as wall-facing images and images of low colon cleaning quality are filtered out and the sliding window uses the remaining images. The method can potentially help clinicians to locate polyps before a follow-up colonoscopy, but this has to be evaluated in a prospective clinical study.