Computer-aided polyp characterization in colonoscopy: sufficient performance or not?
Article information
Abstract
Computer-assisted polyp characterization (computer-aided diagnosis, CADx) facilitates optical diagnosis during colonoscopy. Several studies have demonstrated high sensitivity and specificity of CADx tools in identifying neoplastic changes in colorectal polyps. To implement CADx tools in colonoscopy, there is a need to confirm whether these tools satisfy the threshold levels that are required to introduce optical diagnosis strategies such as “diagnose-and-leave,” “resect-and-discard” or “DISCARD-lite.” In this article, we review the available data from prospective trials regarding the effect of multiple CADx tools and discuss whether they meet these thresholds.
INTRODUCTION
Colorectal cancer (CRC) is the third most common cancer worldwide and a major cause of cancer-related deaths. Approximately 85% of CRCs arise from adenomas via the adenoma-carcinoma pathway.1 CRC screening has been implemented in several countries because early detection of CRC and removal of adenomas are considered to reduce the incidence and mortality of CRC. Therefore, removal of all adenomas is generally recommended for colonoscopy. However, most non-neoplastic polyps (e.g., hyperplastic polyps) of the colon do not develop into cancer. These polyps do not require removal; however, some or most are removed in clinical practice because the differentiation of neoplastic changes using endoscopy (i.e., optical diagnosis) is considered challenging. This leads to considerable costs and consumption of healthcare resources.2,3
Recently, computer-assisted polyp characterization (computer-aided diagnosis, CADx) has become a possible tool for implementing optical diagnosis by increasing confidence in diagnosis and decreasing the rate of removal of non-neoplastic lesions (Fig. 1). However, CADx can be performed using different tools produced by a variety of manufacturers, and not all CADx tools have sufficient accuracy to be implemented in healthcare according to the current standards. In this review, we examine the results of prospective trials and evaluate the possibility of using CADx tools in clinical practice.
Example of computer-assisted polyp characterization (CADx) with EndoBRAIN (Cybernet Systems Corp.). NBI, narrow-band imaging.
OPTICAL DIAGNOSIS: GOALS AND CHALLENGES
In the previous 30 years, efforts have been made to improve the quality and endoscopists’ confidence in optical diagnosis. The application of indigo carmine and methylene blue to help evaluate surface structure and pit patterns could be an attractive option for this purpose,4 while recent virtual chromoendoscopy technologies, such as narrow-band imaging (NBI), allow more user-friendly optical diagnosis.5 However, optical diagnosis has not been widely disseminated, except in a limited number of countries and centers. Several factors may have affected this situation, including the fear of causing harm. Possible harm includes incorrect recommendations of surveillance intervals, risk of leaving malignant lesions due to incorrect evaluations, and possible liability issues caused by optical diagnosis-driven decision-making.
To overcome these barriers, academic societies have proposed several “standards” that endoscopists must follow when introducing optical diagnosis in colonoscopy. These standards include the preservation and incorporation of valuable endoscopic innovations (PIVI 1 and 2) criteria proposed by the American Society for Gastrointestinal Endoscopy (ASGE) and the Simple Optical Diagnosis Accuracy (SODA 1 and 2) criteria proposed by the European Society of Gastrointestinal Endoscopy (ESGE) (Table 1).6,7 Examples of these polyp handling strategies for optical diagnosis include the following: (1) “Leave-in-situ strategy”7: Diminutive polyps (≤5 mm) in the rectosigmoid predicted as non-neoplastic with high confidence are left in situ, while the other polyps are removed and assessed histologically. A negative predictive value (NPV) of >90% for identifying neoplastic changes is required to implement this strategy. (2) “Resect-and-discard”7: Diminutive polyps predicted as neoplastic with high confidence are resected without being sent for histopathology. Optical diagnosis information is used to predict the surveillance intervals. Endoscopists must provide >90% agreement in surveillance interval recommendations between histology-based determination and optical diagnosis-based prediction. (3) “DISCARD”-lite8: This strategy is a modification of the resect-and-discard strategy considering that most of the diminutive polyps on the right side should be either adenomas or sessile serrated lesions. All diminutive polyps in the proximal colon (between the cecum and descending colon) are assumed to be neoplastic and thus removed and discarded without pathological assessment. In addition, diminutive polyps in the rectosigmoid region, predicted to be non-neoplastic with high confidence, are left in situ.
Although these optical diagnostic strategies have specific threshold levels, many endoscopists still lack confidence in their ability to implement optical diagnosis.9 To overcome this barrier, the use of artificial intelligence in the form of CADx has attracted considerable attention, owing to its potential to provide endoscopists with confidence in optical diagnosis.10
ARTIFICIAL INTELLIGENCE FOR OPTICAL DIAGNOSIS
CADx tools in endoscopy utilize machine learning methods to classify images into specific categories, such as neoplastic versus non-neoplastic, which facilitates the endoscopist’s optical diagnostic process.11 While several preclinical studies have been published in this field, clinical studies in which CADx tools have been used and evaluated in real-time are limited. Table 2 shows eight representative prospective studies that evaluated the performance of CADx in clinical colonoscopy.10,12-19 To date, there have been no published randomized trials in this academic field, and only two well-designed comparative prospective studies have been conducted. In this review, we elaborate on these two comparative studies because they provide dedicated knowledge on how the use of CADx affects standard optical diagnosis procedures.10,12
Prospective studies using CADx in diminutive lesions
ITALIAN SINGLE CENTER PROSPECTIVE TRIAL
Blue light imaging (BLI; Fujifilm Corp.) is an image-enhanced technology similar to NBI that emphasizes the vascular and structural patterns of polyp surfaces. Fujifilm Corp. recently introduced a CADx tool designed to interpret BLI images in the market in Europe, Japan, and several other areas of the world (CAD EYE; Fujifilm Corp.). CAD EYE provides a binary prediction of polyp histology (neoplastic or non-neoplastic).
Rondonotti et al.12 conducted an observational clinical trial to assess whether BLI with CADx software was useful in colonoscopy by comparing the optical diagnostic performance of the following three groups: (1) endoscopists alone, (2) CADx alone, and (3) endoscopists using CADx. The primary endpoint was whether CADx-assisted optical diagnosis had ≥90% NPV for adenomatous histology, with histopathology as the reference point. This NPV threshold is one of the standards for optical diagnosis proposed by the ASGE (Table 1). Secondary endpoints were whether the endoscopists alone or CADx alone managed to reach this standard and the threshold level for the resect-and-discard strategy proposed by the ASGE (Table 1).
A total of 389 patients were included in the study. These patients had 596 diminutive polyps in the rectosigmoid that were subject to analysis, of which 259 were neoplastic and 337 were non-neoplastic. The NPV, sensitivity, and specificity were 90.9% (95% confidence interval [CI], 86.8 –93.7), 88.6% (95% CI, 83.6–92.2), and 88.8% (95% CI, 84.5–91.9), respectively, in group 1. However, the diagnostic performances of group 2 were 86.7 % (95% CI, 82.3–90.1), 81.9 % (95% CI, 76.2–86.5), and 88.7% (95% CI, 84.4 –91.9), respectively. Group 3 achieved 91.0% (95% CI, 87.1–93.9), 88.6% (95% CI, 83.7–91.4), and 88.1% (83.9–91.4), respectively. Agreement with post-polypectomy surveillance intervals according to the US recommendations was 92.6% (95% CI, 90.0–95.2), 92.1% (95% CI, 89.4–94.8), and 92.6% (95% CI, 90.0–95.2) in groups 1, 2, and 3, respectively20; in contrast, that for the European recommendations was 97.1% (95% CI, 95.4–98.8), 96.8% (95% CI, 95.0–98.6), and 97.4% (95% CI, 95.7–98.9), respectively.21
The study showed that CADx-assisted optical diagnosis outperformed the threshold levels proposed by the ASGE and ESGE. However, this study did not provide convincing data on the added value of using CADx compared with optical diagnosis by endoscopists alone.
AN INTERNATIONAL MULTICENTER PROSPECTIVE TRIAL
The endocytoscope, a high-resolution magnification colonoscope (CF-H290ECI; Olympus Corp.), can provide 520-fold magnification of a lesion that enables the evaluation of microvascular morphology. CADx software with this tool is commercially available in Japan and several Asian countries (EndoBRAIN; Cybernet Systems Corp.). EndoBRAIN predicts binary histology, namely neoplastic vs. non-neoplastic.
Barua et al.10 conducted a clinical study to assess whether the endocytoscope with CADx software could positively affect optical diagnosis compared with optical diagnosis by endoscopists alone. The study was performed in two sequential steps: (1) endoscopists alone and (2) endoscopists performing CADx. The primary endpoint was to compare the sensitivity of identifying diminutive adenomas in the rectosigmoid region between optical diagnoses with and without CADx. The results from optical diagnosis were then compared to the gold standard, namely histopathological diagnosis, with neoplastic lesions being evaluated as either adenomas or sessile serrated adenomas in primary analyses and non-neoplastic lesions as hyperplastic or other benign tissues.
In total, 518 patients were included in the final analysis. A total of 892 diminutive polyps in the rectosigmoid region were analyzed, of which 359 were neoplastic and 533 were non-neoplastic. The NPV, sensitivity, and specificity of optical diagnosis by endoscopists alone were 91.5% (95% CI, 88.5–93.8), 88.4% (95% CI, 84.3–91.5), and 83.1% (95% CI, 79.2–86.4), respectively. However, the endoscopists using CADx achieved 92.8% (95% CI, 90.1–94.9), 90.4% (95% CI, 86.8–93.1), and 85.9% (95% CI, 82.3–88.8), respectively. This study showed no significant increase in sensitivity when endoscopists used CADx for optical diagnosis. In contrast, the use of CADx significantly increased the confidence level in optical diagnosis from 74.2% (95% CI, 70.9–77.3) to 92.6% (95% CI, 90.6–94.3), which may contribute to the reduction of healthcare costs, given that optical diagnosis is usually performed only with high-confidence prediction.
DISCUSSION
Two comparative studies showed that endoscopists alone and endoscopists using CADx outperformed most threshold standards for optical diagnosis. These studies have recently brought additional knowledge to this academic field.
First, these two studies highlighted the importance of confidence levels in optical diagnosis. High-confidence prediction is mandatory for optical diagnosis according to the ASGE/ESGE guidelines. However, previous studies evaluating CADx tools have not focused on this value. Barua et al.10 showed that CADx improved the confidence in performing optical diagnosis, which may affect the number of unnecessary polypectomies and histopathological assessments, as well as the cost of colonoscopy. However, an objective definition of high-confidence prediction is extremely difficult. This may depend on the personalities or cultures of the endoscopists. We expect that future studies will clarify the value of high-confidence diagnoses considering cultural backgrounds.
Second, the potential role of CADx tools in training inexperienced endoscopists should be further discussed.22 Rondonotti et al.12 found that CADx software provided non-experts with a rapid learning curve. However, the contribution of CADx to optical diagnosis training remains uncertain, given that it was not evaluated as a primary outcome measure in this study. Given that CADx helps in the interpretation of the polyp image but not the acquisition of a high-quality polyp image (which is a prerequisite for accurate optical diagnosis), further studies are needed. In addition, endoscopists’ reliance on CADx may hamper the development of optical diagnostic skills of endoscopists.
Furthermore, meeting the threshold levels proposed by ASGE/ESGE is only part of the CADx contribution: there is a need to comprehensively evaluate this innovative technology with a focus on the balance between its potential benefits and harms. This includes an analysis of the cost-effectiveness of CADx in several healthcare settings. Understanding the balance of using CADx (which is usually subtle in medicine) will guide the implementation of this innovative technology.
CONCLUSIONS
CADx for colonoscopy is expected to optimize optical diagnosis for the assessment of small polyps, eventually leading to reduction of unnecessary polypectomies and relevant cost. However, the currently available, most reliable, prospective studies casted a question against its contribution to clinical practice. Further improvement of the artificial intelligence models together with convincing clinical testing is of great need.
Notes
Conflicts of Interest
Yuichi Mori has conflicts of interest with Olympus Corp. (consultancy and equipment on loan) and Cybernet System Corp. (loyalty fee). Natalie Halvorsen has no potential conflicts of interest.
Funding
None.
Author Contributions
Conceptualization: all authors; Data curation: all authors; Formal analysis: all authors; Supervision: YM; Writing–original draft: all authors; Writing–review & editing: all authors.