Evaluating Retrieval-Augmented Generation Variants for Clinical Decision Support: Hallucination Mitigation and Secure On-Premises Deployment
Оценка вариантов генерации с дополненным поиском для поддержки принятия клинических решений: снижение галлюцинаций и безопасное развертывание в локальной инфраструктуре
2025-10-29
SCID: 54.1/eb7cgb3u
Discuss with AI
clinical decision supporthallucination mitigationhybrid information retrievalretrieval-augmented generationsecure on-premises deployment
Figures from the paper
Abstract (AI)
For clinical decision support to work, medical knowledge needs to be easy to find quickly and accurately. Retrieval-Augmented Generation (RAG) systems use big language models and document retrieval to help with diagnostic reasoning, but they could cause hallucinations and have strict privacy rules in healthcare. We tested twelve different types of RAG, such as dense, sparse, hybrid, graph-based, multimodal, self-reflective, adaptive, and security-focused pipelines, on 250 de-identified patient vignettes. We used Precision@5, Mean Reciprocal Rank, nDCG@10, hallucination rate, and latency to see how well the system worked. The best retrieval accuracy (P@5 ≥ 0.68, nDCG@10 ≥ 0.67) was achieved by a Haystack pipeline (DPR + BM25 + cross-encoder) and hybrid fusion (RRF). Self-reflective RAG, on the other hand, lowered hallucinations to 5.8%. Sparse retrieval gave the fastest response (120 ms), but it was not as accurate. We also suggest a single framework for reducing hallucinations that includes retrieval confidence thresholds, chain-of-thought verification, and outside fact-checking. Our findings emphasize pragmatic protocols for the secure implementation of RAG on premises, incorporating encryption, provenance tagging, and audit trails. Future directions encompass the incorporation of clinician feedback and the expansion of multimodal inputs to genomics and proteomics for precision medicine.
Key Findings
1
A proposed hallucination-mitigation framework combines retrieval-confidence thresholds, chain-of-thought verification, and external fact-checking, while secure on-premises deployment incorporates encryption, provenance tagging, and audit trails.
2
Self-reflective RAG reduced the hallucination rate to 5.8%, outperforming the other evaluated approaches on hallucination mitigation.
3
Sparse retrieval delivered the lowest latency at 120 ms but sacrificed retrieval accuracy relative to higher-performing methods.
4
The Haystack DPR+BM25+cross-encoder pipeline and hybrid reciprocal-rank fusion achieved the highest retrieval performance, with P@5 ≥ 0.68 and nDCG@10 ≥ 0.67.
5
Twelve RAG variants were evaluated on 250 de-identified clinical vignettes using retrieval accuracy, hallucination rate, and latency metrics.
Research Object
Retrieval-Augmented Generation (RAG) systems for clinical decision support, evaluated on de-identified patient vignettes
Research Subject
Retrieval accuracy, hallucination mitigation, response latency, and secure on-premises deployment of RAG pipelines
Publication Details
Publication Date
2025-10-29
Journal
Publisher
ISSN
Cited by
20
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest