Evaluating Microsoft Bing with ChatGPT-4 for the assessment of abdominal computed tomography and magnetic resonance images
Оценка Microsoft Bing с использованием ChatGPT-4 для анализа изображений органов брюшной полости, полученных при компьютерной и магнитно-резонансной томографии
2024-08-19
SCID: 54.1/wcjrdkk6
Discuss with AI
ChatGPT-4Microsoft Bingabdominal CT and MRIlarge language modelsradiology image interpretation
Figures from the paper
Abstract (AI)
PURPOSE: To evaluate the performance of Microsoft Bing with ChatGPT-4 technology in analyzing abdominal computed tomography (CT) and magnetic resonance images (MRI). METHODS: A comparative and descriptive analysis was conducted using the institutional picture archiving and communication systems. A total of 80 abdominal images (44 CT, 36 MRI) that showed various entities affecting the abdominal structures were included. Microsoft Bing's interpretations were compared with the impressions of radiologists in terms of recognition of the imaging modality, identification of the imaging planes (axial, coronal, and sagittal), sequences (in the case of MRI), contrast media administration, correct identification of the anatomical region depicted in the image, and detection of abnormalities. RESULTS: Microsoft Bing detected that the images were CT scans with 95.4% accuracy (42/44) and that the images were MRI scans with 86.1% accuracy (31/36). However, it failed to detect one CT image (2.3%) and misidentified another CT image as an MRI (2.3%). On the other hand, it also misidentified four MRI as CT images (11.1%) and one as an X-ray (2.7%). Bing achieved an 83.75% success rate in correctly identifying abdominal regions, with 90% accuracy for CT scans (40/44) and 77.7% for MRI scans (28/36). Concerning the identification of imaging planes, Bing achieved a success rate of 95.4% for CT images and 83.3% for MRI. Regarding the identification of MRI sequences (T1-weighted and T2-weighted), the success rate was 68.75%. In the identification of the use of contrast media for CT scans, the success rate was 64.2%. Bing detected abnormalities in 35% of the images but achieved a correct interpretation rate of 10.7% for the definite diagnosis. CONCLUSION: While Microsoft Bing, leveraging ChatGPT-4 technology, demonstrates proficiency in basic task identification on abdominal CT and MRI, its inability to reliably interpret abnormalities highlights the need for continued refinement to enhance its clinical applicability. CLINICAL SIGNIFICANCE: The contribution of large language models (LLMs) to the diagnostic process in radiology is still being explored. However, with a comprehensive understanding of their capabilities and limitations, LLMs can significantly support radiologists during diagnosis and improve the overall efficiency of abdominal radiology practices. Acknowledging the limitations of current studies related to ChatGPT in this field, our work provides a foundation for future clinical research, paving the way for more integrated and effective diagnostic tools.
Key Findings
1
Although Bing detected abnormalities in 35% of images, it produced a correct definitive diagnosis in only 10.7%, limiting clinical reliability.
2
Bing correctly identified abdominal regions in 83.75% of images, performing better on CT scans than MRI scans.
3
Bing identified CT contrast-media administration with 64.2% accuracy.
4
Imaging-plane recognition reached 95.4% for CT and 83.3% for MRI, while MRI sequence identification achieved 68.75% accuracy.
5
Microsoft Bing with ChatGPT-4 identified abdominal CT images with 95.4% accuracy and MRI images with 86.1% accuracy.
Research Object
Abdominal computed tomography (CT) and magnetic resonance imaging (MRI) images analyzed by Microsoft Bing with ChatGPT-4
Research Subject
Performance and diagnostic capabilities of Microsoft Bing with ChatGPT-4 in recognizing imaging modalities, planes, MRI sequences, contrast-media use, anatomical regions, and abdominal abnormalities
Publication Details
Publication Date
2024-08-19
Journal
Publisher
ISSN
Cited by
11
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest