Evaluating Retrieval-Augmented Generation Variants for Clinical Decision Support: Hallucination Mitigation and Secure On-Premises Deployment

Оценка вариантов генерации с дополненным поиском для поддержки принятия клинических решений: снижение галлюцинаций и безопасное развертывание в локальной инфраструктуре
Krzysztof Wołk
2025-10-29

clinical decision supporthallucination mitigationhybrid information retrievalretrieval-augmented generationsecure on-premises deployment
For clinical decision support to work, medical knowledge needs to be easy to find quickly and accurately. Retrieval-Augmented Generation (RAG) systems use big language models and document retrieval to help with diagnostic reasoning, but they could cause hallucinations and have strict privacy rules in healthcare. We tested twelve different types of RAG, such as dense, sparse, hybrid, graph-based, multimodal, self-reflective, adaptive, and security-focused pipelines, on 250 de-identified patient vignettes. We used Precision@5, Mean Reciprocal Rank, nDCG@10, hallucination rate, and latency to see how well the system worked. The best retrieval accuracy (P@5 ≥ 0.68, nDCG@10 ≥ 0.67) was achieved by a Haystack pipeline (DPR + BM25 + cross-encoder) and hybrid fusion (RRF). Self-reflective RAG, on the other hand, lowered hallucinations to 5.8%. Sparse retrieval gave the fastest response (120 ms), but it was not as accurate. We also suggest a single framework for reducing hallucinations that includes retrieval confidence thresholds, chain-of-thought verification, and outside fact-checking. Our findings emphasize pragmatic protocols for the secure implementation of RAG on premises, incorporating encryption, provenance tagging, and audit trails. Future directions encompass the incorporation of clinician feedback and the expansion of multimodal inputs to genomics and proteomics for precision medicine.
1
A proposed hallucination-mitigation framework combines retrieval-confidence thresholds, chain-of-thought verification, and external fact-checking, while secure on-premises deployment incorporates encryption, provenance tagging, and audit trails.
2
Self-reflective RAG reduced the hallucination rate to 5.8%, outperforming the other evaluated approaches on hallucination mitigation.
3
Sparse retrieval delivered the lowest latency at 120 ms but sacrificed retrieval accuracy relative to higher-performing methods.
4
The Haystack DPR+B​​M25+cross-encoder pipeline and hybrid reciprocal-rank fusion achieved the highest retrieval performance, with P@5 ≥ 0.68 and nDCG@10 ≥ 0.67.
5
Twelve RAG variants were evaluated on 250 de-identified clinical vignettes using retrieval accuracy, hallucination rate, and latency metrics.

Retrieval-Augmented Generation (RAG) systems for clinical decision support, evaluated on de-identified patient vignettes

Retrieval accuracy, hallucination mitigation, response latency, and secure on-premises deployment of RAG pipelines

Publication Details
Publication Date
2025-10-29
Journal
Publisher
ISSN
Cited by
20
Access Type
Author Information
Authors
Krzysztof Wołk
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%