Key challenges for delivering clinical impact with artificial intelligence
Ключевые проблемы обеспечения клинического эффекта искусственного интеллекта
2019-10-29
SCID: 54.1/umbkbqnp
Discuss with AI
AI translation to clinical practicealgorithmic biasclinical artificial intelligencedataset shiftrandomized controlled trials
Figures from the paper
Abstract (AI)
BACKGROUND: Artificial intelligence (AI) research in healthcare is accelerating rapidly, with potential applications being demonstrated across various domains of medicine. However, there are currently limited examples of such techniques being successfully deployed into clinical practice. This article explores the main challenges and limitations of AI in healthcare, and considers the steps required to translate these potentially transformative technologies from research to clinical practice. MAIN BODY: Key challenges for the translation of AI systems in healthcare include those intrinsic to the science of machine learning, logistical difficulties in implementation, and consideration of the barriers to adoption as well as of the necessary sociocultural or pathway changes. Robust peer-reviewed clinical evaluation as part of randomised controlled trials should be viewed as the gold standard for evidence generation, but conducting these in practice may not always be appropriate or feasible. Performance metrics should aim to capture real clinical applicability and be understandable to intended users. Regulation that balances the pace of innovation with the potential for harm, alongside thoughtful post-market surveillance, is required to ensure that patients are not exposed to dangerous interventions nor deprived of access to beneficial innovations. Mechanisms to enable direct comparisons of AI systems must be developed, including the use of independent, local and representative test sets. Developers of AI algorithms must be vigilant to potential dangers, including dataset shift, accidental fitting of confounders, unintended discriminatory bias, the challenges of generalisation to new populations, and the unintended negative consequences of new algorithms on health outcomes. CONCLUSION: The safe and timely translation of AI research into clinically validated and appropriately regulated systems that can benefit everyone is challenging. Robust clinical evaluation, using metrics that are intuitive to clinicians and ideally go beyond measures of technical accuracy to include quality of care and patient outcomes, is essential. Further work is required (1) to identify themes of algorithmic bias and unfairness while developing mitigations to address these, (2) to reduce brittleness and improve generalisability, and (3) to develop methods for improved interpretability of machine learning predictions. If these goals can be achieved, the benefits for patients are likely to be transformational.
Key Findings
1
AI performance metrics should reflect real clinical applicability and remain understandable to intended users.
2
Despite rapid advances and promising healthcare applications, few AI techniques have been successfully deployed into clinical practice.
3
Developers must address dataset shift, confounder fitting, discriminatory bias, poor generalization, and unintended adverse effects on health outcomes.
4
Randomized controlled trials are considered the gold standard for clinical evidence, although they may be impractical or infeasible for some AI systems.
5
Safe adoption requires balanced regulation, post-market surveillance, and independent, local, representative test sets enabling direct comparison of AI systems.
6
Translating healthcare AI requires addressing machine-learning limitations, implementation logistics, adoption barriers, and necessary sociocultural or care-pathway changes.
Research Object
Clinical AI systems (artificial intelligence systems intended for clinical use)
Research Subject
Challenges and requirements for safely translating, clinically evaluating, regulating, implementing, and adopting AI systems in clinical practice
Publication Details
Publication Date
2019-10-29
Journal
Publisher
ISSN
Cited by
3136
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by14
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI2020
Artificial intelligence in healthcare: transforming the practice of medicine2021
Influence of artificial intelligence (AI) on firm performance: the business value of AI-based transformation projects2020
A Review of the Role of Artificial Intelligence in Healthcare2023
Deep learning in cancer diagnosis, prognosis and treatment selection2021
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis2021
FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare2025
Ethics and governance of trustworthy medical artificial intelligence2023
Interpretability of machine learning‐based prediction models in healthcare2020
Explainable, trustworthy, and ethical machine learning for healthcare: A survey2022
Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks2023
Deep Learning for Accurate Diagnosis of Liver Tumor Based on Magnetic Resonance Imaging and Clinical Data2020
Artificial intelligence in the COVID-19 pandemic: balancing benefits and ethical challenges in China’s response2025
Reducing misdiagnosis in AI-driven medical diagnostics: a multidimensional framework for technical, ethical, and policy solutions2025