Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

Результаты ChatGPT на экзамене USMLE: потенциал применения больших языковых моделей для медицинского образования с использованием искусственного интеллекта
Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, Victor Tseng, Victor Tseng
2023-02-09

AI-assisted medical educationChatGPTUnited States Medical Licensing Examclinical decision-makinglarge language models
We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.
1
ChatGPT achieved this performance without specialized training or reinforcement specifically for the USMLE.
2
ChatGPT performed at or near the passing threshold on all three USMLE examinations: Step 1, Step 2CK, and Step 3.
3
Its explanations demonstrated high concordance with expected answers and substantial insight.
4
The findings suggest large language models could assist medical education and potentially support clinical decision-making.

ChatGPT (a large language model) evaluated on the United States Medical Licensing Examination (USMLE) exams Step 1, Step 2CK, and Step 3

Exam performance relative to the passing threshold and the concordance and insight of ChatGPT’s explanations without specialized training or reinforcement

Publication Details
Publication Date
2023-02-09
Journal
Publisher
ISSN
Cited by
3841
Access Type
Author Information
Authors
Tiffany H. Kung
Morgan Cheatham
Arielle Medenilla
Czarina Sillos
Lorie De Leon
Camille Elepaño
Maria Madriaga
Rimel Aggabao
Giezel Diaz-Candido
James Maningo
Victor Tseng
Victor Tseng
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%