Development and prospective implementation of a large language model based system for early sepsis prediction
Разработка и проспективное внедрение системы на основе большой языковой модели для раннего прогнозирования сепсиса
2025-05-16
SCID: 54.1/59qp4ym2
Discuss with AI
COMPOSER-LLMearly sepsis predictionelectronic health recordslarge language modelunstructured clinical notes
Figures from the paper
Abstract (AI)
Sepsis is a dysregulated host response to infection with high mortality and morbidity. Early detection and intervention have been shown to improve patient outcomes, but existing computational models relying on structured electronic health record data often miss contextual information from unstructured clinical notes. This study introduces COMPOSER-LLM, an open-source large language model (LLM) integrated with the COMPOSER model to enhance early sepsis prediction. For high-uncertainty predictions, the LLM extracts additional context to assess sepsis-mimics, improving accuracy. Evaluated on 2,500 patient encounters, COMPOSER-LLM achieved a sensitivity of 72.1%, positive predictive value of 52.9%, F-1 score of 61.0%, and 0.0087 false alarms per patient hour, outperforming the standalone COMPOSER model. Prospective validation yielded similar results. Manual chart review found 62% of false positives had bacterial infections, demonstrating potential clinical utility. Our findings suggest that integrating LLMs with traditional models can enhance predictive performance by leveraging unstructured data, representing a significant advance in healthcare analytics.
Key Findings
1
Across 2,500 patient encounters, COMPOSER-LLM achieved 72.1% sensitivity, 52.9% positive predictive value, a 61.0% F1 score, and 0.0087 false alarms per patient hour.
2
COMPOSER-LLM integrates an open-source large language model with COMPOSER to incorporate contextual information from unstructured clinical notes for early sepsis prediction.
3
For high-uncertainty cases, the LLM extracts additional context and evaluates sepsis mimics, improving prediction accuracy over standalone COMPOSER.
4
Manual chart review found bacterial infections in 62% of false-positive predictions, indicating that many apparent false alarms may have clinical relevance.
5
Prospective validation produced results similar to the retrospective evaluation, supporting the system’s potential for real-world clinical implementation.
Research Object
patients with suspected sepsis represented by structured electronic health records and unstructured clinical notes
Research Subject
early sepsis prediction performance, including detection of sepsis mimics and false-alarm reduction, using contextual information from clinical notes
Publication Details
Publication Date
2025-05-16
Journal
Publisher
ISSN
Cited by
39
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai7
The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3)2016
Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study2020
Large language models encode clinical knowledge2023
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks2020
Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum2023
Benchmarking Large Language Models in Retrieval-Augmented Generation2024
Direct Preference Optimization: Your Language Model is Secretly a Reward Model2023