Enhancing gastroenterology with multimodal learning: the role of large language model chatbots in digestive endoscopy

Развитие гастроэнтерологии с помощью мультимодального обучения: роль чат-ботов на основе больших языковых моделей в диагностической эндоскопии органов пищеварения
Yuanyuan Qin, Jianming Chang, Li Li, Mianhua Wu
2025-05-21

digestive endoscopydomain-adaptive learninglarge language model chatbotslesion detectionmultimodal learning
Introduction: Advancements in artificial intelligence (AI) and large language models (LLMs) have the potential to revolutionize digestive endoscopy by enhancing diagnostic accuracy, improving procedural efficiency, and supporting clinical decision-making. Traditional AI-assisted endoscopic systems often rely on single-modal image analysis, which lacks contextual understanding and adaptability to complex gastrointestinal (GI) conditions. Moreover, existing methods struggle with domain shifts, data heterogeneity, and interpretability, limiting their clinical applicability. Methods: To address these challenges, we propose a multimodal learning framework that integrates LLM-powered chatbots with endoscopic imaging and patient-specific medical data. Our approach employs self-supervised learning to extract clinically relevant patterns from heterogeneous sources, enabling real-time guidance and AI-assisted report generation. We introduce a domain-adaptive learning strategy to enhance model generalization across diverse patient populations and imaging conditions. Results and discussion: Experimental results on multiple GI datasets demonstrate that our method significantly improves lesion detection, reduces diagnostic variability, and enhances physician-AI collaboration. This study highlights the potential of multimodal LLM-based systems in advancing gastroenterology by providing interpretable, context-aware, and adaptable AI support in digestive endoscopy.
1
A multimodal framework integrates LLM-powered chatbots, endoscopic images, and patient-specific medical data for digestive endoscopy support.
2
Domain-adaptive learning improves model generalization across diverse patient populations and imaging conditions.
3
Experiments on multiple gastrointestinal datasets show improved lesion detection, reduced diagnostic variability, and enhanced physician–AI collaboration.
4
Self-supervised learning extracts clinically relevant patterns from heterogeneous data sources, enabling real-time guidance and AI-assisted report generation.
5
The proposed multimodal systems provide interpretable, context-aware, and adaptable support for clinical decision-making in gastroenterology.

multimodal LLM-based systems integrating digestive endoscopic imaging and patient-specific medical data

context-aware and adaptable AI support in digestive endoscopy, including lesion detection, diagnostic variability reduction, real-time guidance, and AI-assisted report generation

Publication Details
Publication Date
2025-05-21
Journal
Publisher
ISSN
Access Type
Author Information
Authors
Yuanyuan Qin
Jianming Chang
Li Li
Mianhua Wu
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%