INTRODUCTION
Colorectal cancer (CRC) is a major cause of cancer mortality worldwide, with more than one million new cases diagnosed annually[1]. CRC screening reduces CRC incidence and mortality[2-4]. Currently, colonoscopy is widely accepted as the most accurate method of screening the colon for CRC[5-7]. Although there is good evidence for its positive impact in the reduction of CRC, it is also recognized that its effectiveness is dependent on the quality of the procedure[8] which, may be highly variable in clinical practice. Several studies have highlighted important limitations in the accuracy of colonoscopy. Ineffective bowel preparation, an inability to consistently intubate the caecum, and rapid withdrawal times are significant contributors to missed lesions during colonoscopy[9-11].
It has been suggested that a high-quality examination that ensures the detection and removal of all neoplastic lesions is key to screening efficacy[12]. In response, professional societies have proposed the use of various quality-assessment indicators. Crucial quality indicators for colonoscopy were published in 2002 by the American Society for Gastrointestinal Endoscopy/American College of Gastroenterology[13,14]. The assurance that colonoscopy procedures are completed with adherence to these quality standards has become important[15,16] in the quest for high quality and effective CRC screening. Current quality indicators for colonoscopy such as mean withdrawal times and bowel preparation are endoscopist-based[17,18]. Therefore, the quality of a colonoscopy for an individual patient may vary even when performed by an experienced endoscopist[19,20]. The effectiveness of colonoscopy depends on adequate visualization of the entire colon and diligence during examination of the mucosa. Many colon segments can have low diagnostic yield caused by transiently rapid withdrawal, for example, when passing acute angulations of the organ[21]. A quality indicator that is only based on the total withdrawal time cannot recover information about colon segments which have been poorly visualized due to rapid withdrawal. In addition, the effectiveness of any colorectal cancer screening program is critically dependent on adequate bowel preparation[22]. Poor bowel preparation contributes to inadequate visualization of the colonic mucosa. A universal, automated, real-time, feedback-based, and non-operator dependent method which could provide information about the dynamics and the level of the bowel preparation of the procedure would be particularly useful in this regard.
We propose a colometer system[23]; a software-based, automated image analysis tool to improve the quality of the screening colonoscopy thus providing three major outputs: (1) real-time visual feedback indication of image changing velocity and image blurriness to the endoscopist; (2) automated summative statistics report provided immediately following the colonoscopy, including withdrawal time, % time of adequate visualization, and a novel graph of dynamics over time; and (3) automated stool coverage analysis for the documentation of bowel preparation. All of these outputs can be obtained automatically. Hence, this method could allow future colonoscopy quality control in the day-to-day medical practice setting.
The goal of our study was to comparatively validate the performance of the real-time feedback system for screening colonoscopy against expert opinion in clinical practice.
MATERIALS AND METHODS
System overview
The conceptual block diagram of the entire colometer system is shown in Figure 1. Initially, a high-speed analog video acquisition device (VCE-Express, Imperx, Boca Raton, FL, United States) was connected to an EPX-1 colonoscope (Pentax Canada, Mississauga, Ontario) through its S-video port before a standard colonoscopic procedure to enable a real-time video acquisition. A high-performance laptop workstation (Thinkpad W520, Lenovo Canada, North York, Ontario) equipped with Matlab 7.14 (The Mathworks, Inc., MA, United States) was employed to interface with the video acquisition unit and to allow a real-time video processing of the colonoscopy video stream at sampling frequency of 30 frames per second.
Figure 1 Conceptual block diagram of the proposed colometer system.
A real-time visual feedback of the colonoscopy image changing velocity and image blurriness was embedded within this video utilizing Matlab (Image Processing Toolbox™). A custom-design user control interface was developed on the workstation using Matlab (MathWorks, Natick, MA, United States) to allow the user to control the real-time feedback indicator and to provide an offline summative report.
Real-time visual feedback system
In this study, a total of 14 screening colonoscopy videos from a single endoscopy unit (Foothills Medical Campus, Calgary Zone, Alberta Health Services), Calgary, Canada) were collected, processed, and rated in a real-time fashion. The colonoscopy videos were completely anonymized videos without any patient or endoscopist information on the images, filenames, or in the file metadata. The videos were acquired for a quality assurance exercise (practice audit), for maintenance of certification purposes (for Andrews CN) and was thus exempt from consent requirements. Three other experienced gastroenterologists reviewed these videos in a blinded fashion and rated the withdrawal velocity, image quality, and colon preparation of each (scored 1-5; 1, poor; 3, average; 5, excellent) as well as overall quality on the same scale.. The automated quality ratings were compared to the averaged endoscopist quality ratings.
Image changing velocity measurement: In general, the dynamics of moving objects in a video sequence can be assessed by frame differencing technique. If the temporal changes of pixel intensity have changed in a successive sequence of frames, it had to be due to the changing dynamics of objects within the image[24]. Most frame differencing methods offer low computational complexity, which was the main constraint when designing the system to operate in real-time[25]. Since the frame differencing technique is contrast-based and the colonoscopic image contrast can be manually adjusted according to endoscopist’s preference, a normalized approach was chosen to eliminate this problem.
Initially, an overall minimum and maximum values of velocity changes for all 14 videos were determined in real time with 30 ms time delay. Subsequently, two different threshold ranges between these values were selected corresponding to an adequate and rapid withdrawal speeds. These ranges were further optimized to achieve agreement with the average rating from the gastroenterologists (Figure 2).
Figure 2 Relative withdrawal speed over time.
Red line is the speed threshold configured by the endoscopists.
Image blurriness measurement: Colonoscopy videos also contain many blurry (out-of-focus) frames due to frequent shifts of camera positions while moving along the colon. Current endoscopes are equipped with a single, wide-angle lens that cannot be focused[26]. Sharpness, brightness and contrast of the image are optimized using the endoscopist’s skills. In addition, the tip of the colonoscope during the procedure can be temporally buried in mucosa or closely face the colonic wall, which also results in blurry images. It is estimated that the average number of blurry frames in a colonoscopy video is 37%. However, that number can range as high as 60%, depending on endoscopists, patients and colon preparation[27]. Numerous methods have been proposed to assess blurriness in colonoscopy videos. However, a real-time algorithm for blurriness measurement for the colonoscopy video has not yet been implemented. A variance metric calculated as the variance of the whole image was utilized to measure the blurriness of the video frames in our real-time application[28]. During blurry video sequence, information about the image changing velocity is not important and thus was not calculated. Figure 3 shows comparison between focused and blurred snapshots from a colonoscopy withdrawal video.
Figure 3 A sample frame with focused (left) and poor image clarity.
Text indicators “adequate speed”, “rapid withdrawal”, “blurry”, were embedded into the video in a real-time to provide a visual feedback to the endoscopist.
Stool coverage analysis: The core factor to be detected in the stool coverage analysis is the level of cleanliness in the colon. A previously reported color recognition algorithm was used for measuring this feature based on recorded colonoscopy videos[23]. In this study, the stool coverage analysis was successfully integrated within the withdrawal and blurriness algorithm in real-time with reasonable correlation to Ottawa stool scores[29,30]; these results are reported elsewhere[31].
Statistical analysis
Based on these thresholds, a summative statistics report following each colonoscopy could be provided, including withdrawal time, % time with adequate visualization, and a novel graph of image changing dynamics over time. This graph may allow endoscopists to see through which portion of the colon the withdrawal was too fast. An example of a summative colometer report is given as an example in Figure 4. Moving average filter was used to smooth the velocity data for every 20 s. The peaks in the image changing dynamics graph indicate the period with frequent abrupt image changes during scoping. The slope of the line on this dynamics-time graph reveals useful information about the acceleration of the scoping sequences. Using this information missed colonic segments could be estimated. Comparisons of continuous variables used Student’s t test and correlations of ordinal data used Spearman coefficient. Free-margin multirater Kappa statistics were calculated for inter-rater variability[32]. A level of P < 0.05 was used to determine statistical significance.
Figure 4 An automatically generated chart providing % of time of adequate speed in a colonoscopy procedure (left) and chart showing filtered velocity over time (right).
RESULTS
Proof of concept
For a colonoscopy procedure, it is widely known that the visualization quality during withdrawal is higher than the visualization quality during insertion process. Based on this assumption, a comparative study was performed to validate the functionality of the proposed algorithm by comparing the system outputs (% of time of adequate speed in a colonoscopy) between the insertion and withdrawal for 10 colonoscopy procedures. The 10 videos, which were used for validation of the functionality of the proposed algorithm, were different from the videos used to validate the automated scores against expert endoscopist. The average percentage for the colonoscope withdrawals was 79.3% ± 4.96% and it was 50.3% ± 13.95% for the insertion procedures. In a total of 10 collected videos, there was a significant difference of 29.1% ± 11% (P < 0.01) between both procedures. Comparative results are shown in Figure 5.
Figure 5 Comparative results (withdrawal vs insertion) for 10 colonoscopy procedures.
The output mean values are marked with a cross.
Correlation between the system output and the subjective endoscopist evaluation
Subsequently, an overall colonoscopy quality rating was computed based on the percentage of the withdrawal time with adequate visualization (scored 1-5; 1, when the percentage was 1%-20%; 2, when the percentage was 21%-40%, etc.). Adenomatous polyps were detected in 4/14 (29%) of the collected colonoscopy video samples. There were large differences in the withdrawal times during which no polyp was removed (range: 4-12 min). The percentage time with adequate visualization in the videos (i.e., not blurry and not over the velocity threshold) ranged from 54% to 81% (mean 68% ± 2%). The median quality rating from the automated system and the reviewers was 3.45 [interquartile range (IQR), 3.10-3.68] and 2.67 (IQR, 2.33-3.00), respectively, for all colonoscopy video samples. However, there was significant variability in the endoscopist ratings (free-margin kappa statistic = 0.20).
The automated overall quality rating revealed a strong correlation with the reviewers overall quality rating (Spearman r coefficient = 0.65, P = 0.01) as can be seen in Figure 6A. Similarly, there was good correlation of the automated overall quality rating and the mean endoscopist withdrawal speed rating (Spearman r coefficient = 0.59, P = 0.03) (Figure 6B). There was no correlation of automated overall quality rating with mean endoscopists image quality rating (Spearman r coefficient = 0.41, P = 0.15) (Figure 6C). There was also no correlation of the automated blurriness or excessive velocity metrics and mean endoscopists ratings (data not shown).
Figure 6 Correlation between the overall quality ratings from the Colometer system.
A: Mean endoscopist overall quality rating (ρ = 0.65, P < 0.01); B: Mean endoscopist withdrawal speed rating (ρ = 0.59, P < 0.01); C: Mean endoscopist image quality rating (ρ = 0.41, P = 0.15).
Comparative analysis between the withdrawal standard and the proposed algorithm
The mean withdrawal time of the videos was 5.8 min (± 0.4 min SE, range 4-10 min). Out of the 14 colonoscopy withdrawal videos, 2 videos with 6 min withdrawal time were rated as poor quality with low diagnostic yield by 3 endoscopists and confirmed with the colometer. There was no correlation of automated overall quality rating and withdrawal time (Spearman r coefficient= 0.11, P = 0.70).
DISCUSSION
Colonoscopy is widely used for CRC screening[33-35] and its miss rate for advanced adenomas, neoplastic lesions or adenomas remains a concern[36,37]. Obviously, the rate of adenoma detection is closely related to the quality of the colonoscopy[38]. Quality assurance initiatives have been adopted by most national gastroenterology societies, with the mean withdrawal time for colonoscopy strongly correlating with adenoma detection rate. It has been accepted that endoscopists who use withdrawal times longer than 6 min detected significantly more adenomas. In this study, we present a novel, real-time colonoscopy video quality indication system for evaluating the adequacy of image clarity, the withdrawal velocity, and the bowel preparation quality.
The colometer system is a user-friendly method for evaluating the quality of individual screening colonoscopies. This is in contrast to many endoscopist-based quality indicators, such as mean withdrawal time[39]. For example if an endoscopist has a 6 min mean withdrawal rate, many of that endoscopist’s colonoscopies will presumably have a withdrawal time below 6 min[40] and may have lower adenoma detection rates. The individuals with the more rapid withdrawals will not likely be aware of this fact. Further, calculation of mean withdrawal rates is labor-intensive, and even if published, will not likely affect patient choice of endoscopist. Thus a simple and immediate objective quality summary that can be integrated into the patient’s colonoscopy report would be ideal. This is especially so in areas where reimbursement may be tied to procedure quality, as has been proposed in the United States[41].
In our study, widely recommended criteria for colonoscopy procedures were analyzed, including withdrawal time, quality of the bowel preparation and the video quality. Quality indicators for colonoscopy have been selected to establish competence in performing colonoscopy procedures and to help define areas for continuous quality improvement[42,43]. The Colometer quality measurements appear to correlate well with these outcomes in this small study.
This new computer-based method is based on image processing analysis of the live video feed from the colonoscope processor, and just as easily can be run on digitized video files for retrospective review. The method uses widely available video processing technology and can be retrofitted onto any endoscope processor with digital video output. No remote processing of video is required, which will prevent any privacy breach risks. Output of summative metrics is immediate, and could be added to individual endoscopy reports, as well as database recording for simple collection of quality control data in the practice setting on a large scale.
This study was designed to show the proof of the Colometer concept and compare it to subjective quality assessments based on retrospective video recordings of colonoscopy. The software however is designed to provide real-time visual feedback to the endoscopist during the withdrawal itself. We did not evaluate the actual feedback to endoscopists during the colonoscopy to determine whether this type of output (like a speedometer for the scope driver) affects withdrawal speed or other quality measures, but studies in this regard are underway. Further uses could be used in the education and (re)credentialing fields.
This study has some limitations. First, it was a small pilot-study looking at mostly good quality colonoscopies. The performance characteristics of Colometer may change with a broader spectrum of colonoscopy withdrawals. Second, overall quality of a screening colonoscopy is a multi-dimensional concept, which may not be completely captured by a small number of quality metrics. However, the established metrics are certainly not perfect (as shown by the fact that 2 of the videos with a 6 min withdrawal time were in fact of poor quality). The advantage of an automated system however removes the variability and subjectivity of current quality metrics. Third, the small numbers in this study may have led to underpowering of the correlations. Additionally, the inter-rater variability was high. This is due to a number of factors, including the fact that the rating scale used was unvalidated, despite its simplicity and face validity. Moreover, the raters were not trained prior to the study to “anchor” their responses, which we admit is a limitation of this study. Interestingly, the rater with the most experience in colonoscopy quality assessments had outstanding correlation with colometer (data not shown), and thus inter-rater correlation will likely be tighter in future anticipated studies. Finally, some endoscopists may resist the concept of real-time constant scrutiny of their screening colonoscopies. However, it is clear that high-quality screening colonoscopy requires skill and attention, in addition to expensive infrastructure and has measurable (although small) patient risk. We therefore should strive for continuous quality improvement of this procedure, and if heightened, non-punitive, scrutiny improves it (i.e., the Hawthorne effect) this is likely a positive direction for patients.
In conclusion, a new real-time feedback system for screening colonoscopy procedures was proposed. The results from this system based on the withdrawal dynamics and image quality strongly agreed with the endoscopists’ quality ratings. The colometer system could facilitate a real-time colonoscopy quality feedback for clinical practice, with easily accessible, in depth colonoscopy quality assessment for individual patients. Such a device could also be a potential training tool for new endoscopists. Further study is required in order to better define the optimal quality thresholds and to validate this approach in a larger clinical trial.
COMMENTS
Background
Colonoscopy is widely accepted as the most accurate method of screening the colon for colorectal cancer (CRC) however its effectiveness is dependent on the quality of the procedure which may be highly variable in clinical practice. Image quality and withdrawal time or velocity are important determinants of quality. A novel automated computer-based assessment tool was described and tested.
Research frontiers
It has become clear that colonoscopy quality is a major determinant of the detection of polyps in CRC screening, and there has been significant recent interest in the creation and evaluation of quality metrics for colonoscopy.
Innovations and breakthroughs
To date, there has been limited research on computerized colonoscopy video quality assessment in the published literature. The Colometer software represents an advance by using simple algorithms to characterize a complex procedure, with the added advantages of offering a real-time feedback tool to the endoscopist, immediately available patient-level quality metrics, and with minimal hardware or software requirements.
Applications
The colometer could be potentially applied to clinical screening colonoscopy practice to assess and ultimately improve colonoscopy quality.
Terminology
Colonoscopy quality: A multi-dimensional concept that affects the ability to identify and remove pre-cancerous polyps from the colon in screening colonoscopy for CRC. It incorporates many factors, including the effectiveness of the bowel preparation, the withdrawal speed of the scope, and the skill and care taken to thoroughly assess all surfaces of the colon.
Peer review
The manuscript is well written and has useful information that would be of interest to the readers.
