Artificial intelligence in gastrointestinal endoscopy: a comprehensive review

AI in GI endoscopy

The past decade has witnessed significant advances in the application of AI in GI endoscopy. Early AI systems primarily used computer-aided diagnosis (CADx) and computer-aided detection (CADe) algorithms. Their role was to identify, characterize and differentiate suspicious lesions, such as polyps, tumors, ulcers, and areas of dysplasia [8]. These AI systems deployed predetermined algorithms to pinpoint deviations from “normal” tissue, presenting them to the interpreter. In GI endoscopy, the processing speed of CAD algorithms and their visual output, i.e., graphics, are paramount—initially, computational power was CAD’s main limitation. But the rise of faster, more affordable, portable and reliable digital platforms has improved CAD’s efficiency [9]. Simultaneously, there have been notable advances in machine learning and deep learning (DL) methods, paving the way for more refined AI models. These AI/machine learning models are instrumental in the prevention, diagnosis, management and prognosis of various GI lesions, encompassing both premalignant and malignant types.

Machine learning allows AI algorithms to function like the human brain, refining its performance with every data interaction without being explicitly programmed to do so. DL, emerging as one of the most promising machine learning subsets, empowers AI to reason, discern intricate patterns, establish connections, and make decisions using raw data. Its structure resembles the human neural network, with an input layer, multiple hidden layers, and an output layer. Thus, DL offers significantly enhanced capabilities compared to its AI predecessors [10]. In supervised learning, techniques such as support vector machines, regression analysis, and random forests prevail.

Conversely, unsupervised learning adopts principal component and cluster analysis methods to detect recurring patterns within datasets. Based on these learning paradigms, algorithms emulate the human brain’s function, leading to artificial neural networks under the machine learning umbrella. DL, a machine learning subset, harnesses multiple artificial neural networks to analyze datasets and predict outcomes directly. Convolutional neural networks (CNN) are another prominent neural network in medical imaging. Their interconnected layers reflect the workings of the human visual cortex [11].

In gastroenterology, a spectrum of imaging modalities is utilized to evaluate the digestive system and pinpoint GI tumors. These include histologic sections of GI specimens, radiography, and endoscopy. While machine learning is a go-to choice for the automated analysis of GI images, gastroenterology is open to exploring other AI branches, such as natural language processing [12,13]. There may often be a synergy between radiomic techniques and machine learning/DL algorithms. The term ‘radiomics’, introduced in 2012, denotes the computational, objective analysis of features within radiologic images, shedding light on insights typically elusive or challenging to quantify. A fundamental component of radiomic evaluations revolves around the analysis of image texture attributes [14]. Radiomics and AI share a symbiotic relationship, especially since the frequent deployment of machine learning strategies in radiomics for pattern identification in extensive datasets (Fig. 1) [15].

Figure 1 Overview of AI applications in gastrointestinal disorders

AI, artificial intelligence; H. pylori, Helicobacter pylori

Esophageal disorders

AI has emerged as a transformative tool for investigating Barrett’s disease (BE) and esophageal cancer. A recent study by Dumoulin et al has demonstrated the potential of AI systems in enhancing the screening and surveillance of BE and early esophageal adenocarcinoma, providing a nuanced approach to high-quality endoscopy and addressing the challenges of esophageal cancer in western societies [16]. Concurrently, advances in imaging technology, integrated with AI, are revolutionizing the detection and characterization of esophageal neoplastic lesions and diagnosing early esophageal squamous-cell carcinoma and precancerous lesions [17]. Innovations in surveillance methods, as explored by Iyer et al, including AI-powered approaches, are making strides in identifying and predicting dysplasia/adenocarcinoma in BE, with emergent innovations spanning efficient sampling methods, advanced imaging tools and molecular marker-powered approaches [18]. As discussed by Vulpoi et al, DL models show promise in diagnosing and managing upper digestive tract diseases, including BE. They may also help diagnose and manage gastroesophageal reflux disease [19]. Furthermore, the establishment of AI models like YOLOv5l, as reported by Wang et al, is assisting in diagnosing esophageal squamous-cell carcinoma and precancerous lesions, thereby enhancing diagnostic accuracy and reducing missed cases, which can assist junior endoscopists in improving diagnosis [17].

Gastric precancerous lesions and H. pylori infection

Gastric precancerous lesions and H. pylori infection are recognized risk factors for gastric cancer (GC). In addressing these concerns, the rise of AI, backed by its computational prowess and learning capacities, has been marked as a potent asset.

AI algorithms have notably boosted diagnostic precision for GI conditions such as early-stage GC. A study by Tang et al is has highlighted AI’s diagnostic supremacy over endoscopists, particularly in detecting early GC [20]. This emphasizes how the judicious deployment of AI in GI endoscopy can equip clinicians with the means to identify lesions at earlier stages, elevate diagnostic accuracy, and personalize treatments—all converging towards optimized therapeutic results for the patient.

Diving into the predictive abilities of AI in the realm of endoscopy, Huang et al probed its capability to anticipate H. pylori gastritis [21]. Their findings charted a sensitivity and specificity of 85.4% and 90.9%, respectively, for H. pylori detection. Furthermore, this study shed light on AI’s commendable accuracy in predicting gastric atrophy, intestinal metaplasia, and the intensity of H. pylori-linked gastritis. Most observations hover above 80% accuracy, although no specific study provided an exact figure. The evolution of this methodology could trim down the resources expended on needless biopsies for H. pylori diagnosis. Supporting this perspective, a meta-analysis by Bang et al, consolidating 8 studies and data from 1719 patients, reported that AI attained a sensitivity of 0.87 (95% confidence interval [CI] 0.72-0.94) and a specificity of 0.86 (95%CI 0.77-0.92) in detecting H. pylori infections [22]. Additionally, AI exhibited an 82% accuracy in differentiating non-infected gastric mucosa images from their post-eradication counterparts [22].

Exploring this domain further, a distinct study encompassing school children in Ethiopia employed machine learning to discern risk factors and predict H. pylori infection on the basis of those risks [23]. The outcomes spotlighted the efficiency of machine learning tactics, such as the XGBoost classifier, in forecasting H. pylori infection status—a performance that surpassed traditional statistical models.

GC

Previous studies have reported that the miss rates for GC detection by endoscopists ranged from 4.6-25.8% [24]. Misses in GC detection during endoscopy are influenced by tumor characteristics and the endoscopists’ level of experience. A definitive diagnosis of GC currently relies on visual examination of whole slide imaging (WSI) pathological images, which pose challenges for pathologists, necessitating long-term concentration and tedious efforts. While advanced image-enhanced endoscopy techniques show potential in improving GC detection [25], their widespread implementation is hindered by the need for additional training and expertise. AI has the potential to assist in automatic, precise, and rapid endoscopic detection, as well as histopathological examination, addressing these challenges. In GC detection, AI-based image classification has achieved a sensitivity of 92.2% in classifying endoscopic images [26].

In the field of GC diagnosis, traditional machine-learning methods have typically relied on the extraction of handcrafted features from images. For instance, Miyaki et al [27] employed densely sampled scale-invariant feature transform descriptors as local features, and applied a hierarchical k-means clustering algorithm to obtain a bag-of-features representation. They combined this with a support vector machine-based classifier to achieve a cancer detection sensitivity of 84.8% and specificity of 87.0%, using a cutoff value of 0.59 [28]. However, these approaches faced limitations, including reliance on human-selected cutoff values and small data sets. DL has emerged as a powerful tool to address these challenges, enabling end-to-end training that automates feature extraction and classification. Hirasawa et al [26] explored the use of a CNN model, specifically a single-shot multi-box detector, for detecting GC lesions. The CNN model achieved a sensitivity of 92.2% for automatic GC diagnosis, outperforming traditional machine-learning methods. Luo et al [29] developed a GI AI diagnostic system (GRAIDS) based on DL techniques, using a large dataset of endoscopic images. GRAIDS demonstrated a performance similar to expert endoscopists and showed potential in improving the detection of gastric tumors, particularly for trainee endoscopists and low-volume hospitals.

DL approaches have been utilized in GC detection on WSI. Most successful approaches extract and use small image patches instead of the whole image as input. Patch selection is a key research area, and existing approaches can be categorized based on whether they employ patch- or slide-level annotations. Li et al [30] proposed GastricNet, a GC detection model on WSI. The model combines shallow and deep layers to extract multi-scale features and achieve slice-level prediction by averaging the scores of the top 10 patches with the highest cancer probability. Wang et al [31] developed a method called recalibrated multi-instance DL (RMDL) for detecting GC in WSIs. They used a 2-stage framework, where a ResNet-based network was used to generate discriminative features and abnormal probabilities for each patch instance, and then a local-global feature fusion method and attention-based approach were used to aggregate the instance features. The RMDL method outperformed other multi-instance learning algorithms in classifying gastric slices into cancer, dysplasia and normal states. Machine learning has successfully diagnosed and guided treatment decisions for patients with GI stromal tumors, as well using integrated radiomics analysis [32-37].

Colorectal polyps and cancer

AI-assisted computer vision applications, particularly CADe and CADx, have emerged as critical tools for prevention of colorectal cancer (CRC) through colonoscopy screening and monitoring [30]. These technological advances are not without their limitations, as adenoma miss rates (AMR) range between 6% and 28% [38], mainly because of challenges in lesion detection influenced by operator expertise. The pressing need to overcome such obstacles has led to a surge in the validation and development of these tools, notably for AI-assisted polyp detection studies. Here, outcome measures such as adenoma detection rate, adenomas per colonoscopy, and AMR serve as pivotal assessment parameters.

Although most studies are still in their preliminary stages, they exhibit promising clinical relevance. One recent study showcased a DL-based AI system’s significant improvement in real-time polyp detection rates [39]. Supporting this, a systematic review and a meta-analysis found that CADe usage during colonoscopy boosted the detection rates of adenomas and polyps compared to a control cohort [40]. In a randomized controlled trial, CADe was found to reduce AMR and miss rates for sessile-serrated lesions, and increase the count of adenomas per colonoscopy during the initial sweep [41].

Recent advances in CADe, particularly demonstrated by a systematic review and meta-analysis by Hassan et al, encompassed 21 randomized trials involving 18,232 patients. This study showed that the adenoma detection rate was notably higher in the CADe group (44.0%) compared to the standard colonoscopy group (35.9%), indicating a 55% relative reduction in the AMR [42]. Another systematic review and a meta-analysis found that CADe usage during colonoscopy boosted the detection rates of adenomas and polyps compared to a control cohort [43]. In a randomized controlled trial, CADe was found to reduce AMR and miss rates of sessile-serrated lesions and increase the count of adenomas per colonoscopy [44].

AI-driven CNN models could refine bowel preparation quality and enhance polyp detection [45]. Moreover, real-time polyp identification systems rooted in DL algorithms can potentially improve polyp detection and localization during colonoscopy sessions [46,47]. Notably, incorporating AI into routine colonoscopy can halve the AMR and augment tumor detection, even for those under 10 mm, as highlighted in a recent global study [48].

On the other hand, in tumor disease diagnosis and radiation treatment planning, manual tumor boundary delineation is time-consuming, requires expertise, and suffers from interobserver variability. However, fully convolutional networks, such as SegNet, U-Net, and DenseVNet, have proven to be efficient frameworks for semantic image segmentation. U-Net, a specialized CNN, and DenseVNet, a fully convolutional network, enable precise and quick biomedical image segmentation [49-51]. T2-weighted, dynamic contrast enhanced and multiparametric magnetic resonance imaging, and computed tomography-based multi-organ segmentation methods, often paired with multi-atlas label fusion techniques, yield more reproducible treatment area measurements, leading to more consistent radiation doses [52]. Some notable trials of AI models in colonic polyps and CRC detection are described in Table 1 [45-62].

Table 1 Studies demonstrating use of AI in colon cancer and polyp detection

Supplementing the above illustrations, AI’s potential to significantly refine endoscopy spans various avenues:

Real-time support: AI stands ready to offer instantaneous aid to endoscopists, parsing live endoscopic imagery with a velocity that outstrips human cognition. It can highlight dubious territories, guiding pivotal decision making [63].

Elevated clinician efficacy: By automating tasks like image captures, data extractions, and report generation, AI amplifies clinician efficiency during endoscopies [64].

Superior image clarity: AI enhances endoscopic image fidelity by bolstering contrast, attenuating ambient noise, and discerning nuanced alterations [65].

Inflammatory bowel disease (IBD)

AI has found its use in diagnosing and managing IBD. A recent systematic review highlighted the promising technical results of AI-assisted endoscopy in IBD, emphasizing its growth as a research field and its additional benefits in experimental clinical scenarios [66]. The integration of AI with capsule endoscopy, as illustrated by Mascarenhas et al, is enhancing the future of IBD management, particularly in the investigation of obscure hemorrhagic lesions [67]. The current advancement of AI in diagnosing IBD from imaging data highlights its importance in the coming decades [68]. The feasibility of accurately predicting adverse outcomes using complex and novel AI models on large longitudinal data sets of patients with IBD has been demonstrated by Zand et al, suggesting potential applications for risk stratification and implementation of preemptive measures [69]. Furthermore, an overview by Takenaka et al provides insights into how AI can improve clinical practice in endoscopy for IBD, with some components already beginning to shape our understanding [70]. The proposed 2-tier IBDN classification, as evaluated by Ang et al, helps detect the “CRC/high-grade dysplasia” group for consideration of proctocolectomy. However, it does not differentiate low-grade dysplasia or sporadic adenomas from normal mucosa [71]. A recent study has taken a significant step toward enhancing dysplasia detection in patients with IBD. The researchers developed the first known model for CADe of colorectal lesions in patients with IBD. This model was retrained using high-definition white-light endoscopy (HDWLE) images and dye-based chromoendoscopy images of IBD-associated colorectal lesions. The retrained IBD-CADe model demonstrated impressive performance metrics, including a sensitivity of 95.1% and an accuracy of 96.8% for HDWLE. The study concluded that this model represents the first step toward developing AI-based endoscopic tools to enhance the detection of polypoid and non-polypoid dysplasia, potentially reducing CRC rates in patients with IBD. The ability to detect lesions as small as 5 mm with 93% sensitivity further emphasizes the potential of this AI model in early detection and intervention [72].

Pancreato-biliary diseases

Diagnosing IDBS and pancreatic cancer poses significant clinical challenges. The differentiation between malignant and benign biliary strictures remains difficult, despite advances in endoscopic techniques. Conventional ERCP techniques, involving cholangiogram appearance and brush cytology, often fail to differentiate reliably between malignant and benign biliary strictures because of their limited sensitivity [73,74]. Probe-based confocal laser endomicroscopy (pCLE) represents a novel imaging technique offering “optical biopsies” and providing in vivo cellular-level architectural information. However, its specificity can be compromised by previous endoscopic interventions. Cholangioscopy, especially with the advent of single-operator systems, has significantly advanced the diagnostic capabilities for biliary strictures. Endoscopic-ultrasound-guided fine-needle aspiration (EUS-FNA) has been shown to be more effective than conventional ERCP techniques in diagnosing malignant biliary strictures, with higher sensitivity and accuracy [73,74]. However, it is crucial to note that negative results from these techniques do not completely exclude malignancy, given their low negative predictive value. AI can play a transformative role in this context. AI algorithms can assist in the analysis of complex imaging data obtained from these advanced endoscopic techniques. By integrating AI with pCLE, cholangioscopy and EUS-FNA data, it may be possible to enhance the diagnostic accuracy for IDBS and pancreatic cancer. This integration could lead to more reliable differentiation between malignant and benign strictures, better informing treatment decisions, and improving patient outcomes [73,74].

Cholangiocarcinoma is notoriously difficult to diagnose early, since traditional techniques like ERCP have limited sensitivity [73]. With its ability to process and analyze complex image data at an unprecedented scale, AI presents a viable solution to this challenge. Recent studies have shown that CNNs, when used in conjunction with cholangioscopy or EUS, can significantly improve the accuracy of diagnosing malignant biliary strictures and cholangiocarcinoma. For instance, CNN with EUS imaging has demonstrated superior clinical performance, potentially streamlining the diagnostic process and providing real-time, reliable feedback during endoscopic procedures. However, the application of AI in this context is not without its challenges [73]. Issues such as data quality, inconsistency, and the risk of overfitting need to be carefully managed to ensure the reliability of AI algorithms in clinical settings. Moreover, while the initial results are promising, further comparative studies and external validations are essential to firmly establish the role of AI in the routine diagnostic process. The potential of AI to reduce procedure length, enhance diagnostic accuracy and possibly negate the need for invasive testing, such as biopsies, is significant, particularly for reducing procedure-associated adverse events [73]. Some recent trials using AI in pancreato-biliary disorders are described in Table 2 [74-77].

Table 2 Recent trials of use of AI in the diagnosis of malignant biliary strictures and cholangiocarcinoma

Limitations

The field of medical diagnostics has a great deal of potential for AI, but there are restrictions on how it can be used. One of the major limitations is that we cannot completely rely on AI; clinical judgment is still essential, as demonstrated by inability of algorithms to take contextual information into account. Computational errors may also be made worse by the issue of automation bias, which occurs when clinicians accept the AI’s conclusions even when they are incorrect. Another obstacle is the need for more clarity in the data collection process. All the training and test datasets were derived from a single source, restricting diversity and potentially introducing bias in the AI’s diagnosis capabilities. Additionally, the AI’s ability to generalize its learned knowledge to new, unseen data is challenging, necessitating continuous and stringent external validation using various imaging and endoscopy sources. Furthermore, there is a need to expand the use of CNN algorithms to include currently available models, such as GoogLeNet and a few ResNet, to improve the effectiveness of the AI. Additional limitations are described in Table 3. These limitations underscore the need for cautious optimism and continuous development in this promising field.

Table 3 Overview of key limitations of AI in digestive endoscopy