Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models
Результаты ChatGPT на экзамене USMLE: потенциал применения больших языковых моделей для медицинского образования с использованием искусственного интеллекта
2023-02-09
SCID: 54.1/z37mnuje
Discuss with AI
AI-assisted medical educationChatGPTUnited States Medical Licensing Examclinical decision-makinglarge language models
Figures from the paper
Abstract (AI)
We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.
Key Findings
1
ChatGPT achieved this performance without specialized training or reinforcement specifically for the USMLE.
2
ChatGPT performed at or near the passing threshold on all three USMLE examinations: Step 1, Step 2CK, and Step 3.
3
Its explanations demonstrated high concordance with expected answers and substantial insight.
4
The findings suggest large language models could assist medical education and potentially support clinical decision-making.
Research Object
ChatGPT (a large language model) evaluated on the United States Medical Licensing Examination (USMLE) exams Step 1, Step 2CK, and Step 3
Research Subject
Exam performance relative to the passing threshold and the concordance and insight of ChatGPT’s explanations without specialized training or reinforcement
Publication Details
Publication Date
2023-02-09
Journal
Publisher
ISSN
Cited by
3841
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
Cited by20
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns2023
A Survey on Evaluation of Large Language Models2024
ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope2023
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education2023
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine2023
Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios2023
The future landscape of large language models in medicine2023
ChatGPT for Education and Research: Opportunities, Threats, and Strategies2023
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
ChatGPT and Open-AI Models: A Preliminary Review2023
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health2023
The role of ChatGPT in higher education: Benefits, challenges, and future research directions2023
A scoping review of artificial intelligence in medical education: BEME Guide No. 842024
Improving large language models for clinical named entity recognition via prompt engineering2023
A comprehensive survey of ChatGPT: Advancements, applications, prospects, and challenges2023
Large language models in medical and healthcare fields: applications, advances, and challenges2024
Prompt Engineering in Medical Education2023
The Clinicians’ Guide to Large Language Models: A General Perspective With a Focus on Hallucinations2025
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review2024
Interactive computer-aided diagnosis on medical image using large language models2024