Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models
Результаты ChatGPT на экзамене USMLE: потенциал применения больших языковых моделей для медицинского образования с использованием искусственного интеллекта
2023-02-09
SCID: 54.1/z37mnuje
Discuss with AI
AI-assisted medical educationChatGPTUnited States Medical Licensing Examclinical decision-makinglarge language models
Figures from the paper
Abstract (AI)
We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.
Key Findings
1
ChatGPT achieved this performance without specialized training or reinforcement specifically for the USMLE.
2
ChatGPT performed at or near the passing threshold on all three USMLE examinations: Step 1, Step 2CK, and Step 3.
3
Its explanations demonstrated high concordance with expected answers and substantial insight.
4
The findings suggest large language models could assist medical education and potentially support clinical decision-making.
Research Object
ChatGPT (a large language model) evaluated on the United States Medical Licensing Examination (USMLE) exams Step 1, Step 2CK, and Step 3
Research Subject
Exam performance relative to the passing threshold and the concordance and insight of ChatGPT’s explanations without specialized training or reinforcement
Publication Details
Publication Date
2023-02-09
Journal
Publisher
ISSN
Cited by
4128
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai1
Cited by20
Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy2023
A Survey on Evaluation of Large Language Models2024
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education2023
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns2023
ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope2023
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?2023
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine2023
Artificial Hallucinations in ChatGPT: Implications in Scientific Writing2023
ChatGPT for Education and Research: Opportunities, Threats, and Strategies2023
A SWOT analysis of ChatGPT: Implications for educational practice and research2023
Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers2023
Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios2023
The future landscape of large language models in medicine2023
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage2023
The role of ChatGPT in higher education: Benefits, challenges, and future research directions2023
ChatGPT and Open-AI Models: A Preliminary Review2023
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health2023
The rise of ChatGPT : Exploring its potential in medical education2023
A scoping review of artificial intelligence in medical education: BEME Guide No. 842024