Evaluating Microsoft Bing with ChatGPT-4 for the assessment of abdominal computed tomography and magnetic resonance images

Оценка Microsoft Bing с использованием ChatGPT-4 для анализа изображений органов брюшной полости, полученных при компьютерной и магнитно-резонансной томографии
Alperen Elek, Duygu Doğa Ekizalioğlu, Ezgi Güler
2024-08-19

ChatGPT-4Microsoft Bingabdominal CT and MRIlarge language modelsradiology image interpretation
PURPOSE: To evaluate the performance of Microsoft Bing with ChatGPT-4 technology in analyzing abdominal computed tomography (CT) and magnetic resonance images (MRI). METHODS: A comparative and descriptive analysis was conducted using the institutional picture archiving and communication systems. A total of 80 abdominal images (44 CT, 36 MRI) that showed various entities affecting the abdominal structures were included. Microsoft Bing's interpretations were compared with the impressions of radiologists in terms of recognition of the imaging modality, identification of the imaging planes (axial, coronal, and sagittal), sequences (in the case of MRI), contrast media administration, correct identification of the anatomical region depicted in the image, and detection of abnormalities. RESULTS: Microsoft Bing detected that the images were CT scans with 95.4% accuracy (42/44) and that the images were MRI scans with 86.1% accuracy (31/36). However, it failed to detect one CT image (2.3%) and misidentified another CT image as an MRI (2.3%). On the other hand, it also misidentified four MRI as CT images (11.1%) and one as an X-ray (2.7%). Bing achieved an 83.75% success rate in correctly identifying abdominal regions, with 90% accuracy for CT scans (40/44) and 77.7% for MRI scans (28/36). Concerning the identification of imaging planes, Bing achieved a success rate of 95.4% for CT images and 83.3% for MRI. Regarding the identification of MRI sequences (T1-weighted and T2-weighted), the success rate was 68.75%. In the identification of the use of contrast media for CT scans, the success rate was 64.2%. Bing detected abnormalities in 35% of the images but achieved a correct interpretation rate of 10.7% for the definite diagnosis. CONCLUSION: While Microsoft Bing, leveraging ChatGPT-4 technology, demonstrates proficiency in basic task identification on abdominal CT and MRI, its inability to reliably interpret abnormalities highlights the need for continued refinement to enhance its clinical applicability. CLINICAL SIGNIFICANCE: The contribution of large language models (LLMs) to the diagnostic process in radiology is still being explored. However, with a comprehensive understanding of their capabilities and limitations, LLMs can significantly support radiologists during diagnosis and improve the overall efficiency of abdominal radiology practices. Acknowledging the limitations of current studies related to ChatGPT in this field, our work provides a foundation for future clinical research, paving the way for more integrated and effective diagnostic tools.
1
Although Bing detected abnormalities in 35% of images, it produced a correct definitive diagnosis in only 10.7%, limiting clinical reliability.
2
Bing correctly identified abdominal regions in 83.75% of images, performing better on CT scans than MRI scans.
3
Bing identified CT contrast-media administration with 64.2% accuracy.
4
Imaging-plane recognition reached 95.4% for CT and 83.3% for MRI, while MRI sequence identification achieved 68.75% accuracy.
5
Microsoft Bing with ChatGPT-4 identified abdominal CT images with 95.4% accuracy and MRI images with 86.1% accuracy.

Abdominal computed tomography (CT) and magnetic resonance imaging (MRI) images analyzed by Microsoft Bing with ChatGPT-4

Performance and diagnostic capabilities of Microsoft Bing with ChatGPT-4 in recognizing imaging modalities, planes, MRI sequences, contrast-media use, anatomical regions, and abdominal abnormalities

Publication Details
Publication Date
2024-08-19
Journal
Publisher
ISSN
Cited by
11
Access Type
Author Information
Authors
Alperen Elek
Duygu Doğa Ekizalioğlu
Ezgi Güler
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%