Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers
Сравнение научных аннотаций, сгенерированных ChatGPT, с реальными аннотациями с помощью детекторов и слепых рецензентов
2023-04-26
SCID: 54.1/5u4uygpd
Discuss with AI
AI output detectorAUROC 0.94ChatGPTGPT-2 Output Detectorblinded human reviewersplagiarism detector/iThenticatescientific abstracts
Figures from the paper
Abstract (AI)
Abstract Background Large language models such as ChatGPT can produce increasingly realistic text, with unknown information on the accuracy and integrity of using these models in scientific writing. Methods We gathered ten research abstracts from five high impact factor medical journals (n=50) and asked ChatGPT to generate research abstracts based on their titles and journals. We evaluated the abstracts using an artificial intelligence (AI) output detector, plagiarism detector, and had blinded human reviewers try to distinguish whether abstracts were original or generated. Results All ChatGPT-generated abstracts were written clearly but only 8% correctly followed the specific journal’s formatting requirements. Most generated abstracts were detected using the AI output detector, with scores (higher meaning more likely to be generated) of median [interquartile range] of 99.98% [12.73, 99.98] compared with very low probability of AI-generated output in the original abstracts of 0.02% [0.02, 0.09]. The AUROC of the AI output detector was 0.94. Generated abstracts scored very high on originality using the plagiarism detector (100% [100, 100] originality). Generated abstracts had a similar patient cohort size as original abstracts, though the exact numbers were fabricated. When given a mixture of original and general abstracts, blinded human reviewers correctly identified 68% of generated abstracts as being generated by ChatGPT, but incorrectly identified 14% of original abstracts as being generated. Reviewers indicated that it was surprisingly difficult to differentiate between the two, but that the generated abstracts were vaguer and had a formulaic feel to the writing. Conclusion ChatGPT writes believable scientific abstracts, though with completely generated data. These are original without any plagiarism detected but are often identifiable using an AI output detector and skeptical human reviewers. Abstract evaluation for journals and medical conferences must adapt policy and practice to maintain rigorous scientific standards; we suggest inclusion of AI output detectors in the editorial process and clear disclosure if these technologies are used. The boundaries of ethical and acceptable use of large language models to help scientific writing remain to be determined.
Key Findings
1
AI output detectors may be useful as editorial tools to help maintain scientific standards, but ethical boundaries and publisher policies for LLM-assisted writing vary and remain under discussion.
2
An AI output detector ('GPT-2 Output Detector') assigned median 'fake' scores of 99.98% [IQR 12.73%, 99.98%] for generated abstracts versus 0.02% [IQR 0.02%, 0.09%] for original abstracts.
3
Blinded human reviewers correctly identified 68% of generated abstracts but incorrectly labeled 14% of original abstracts as generated, reporting generated abstracts as vaguer and more formulaic.
4
ChatGPT can generate believable scientific research abstracts from titles and journal names, but the abstracts contain completely generated data.
5
Generated abstracts scored lower (i.e., had fewer matches) on plagiarism detection tools and iThenticate compared with original abstracts, indicating more novel wording.
6
The AI output detector achieved an AUROC of 0.94 for distinguishing generated from original abstracts.
Research Object
Scientific research abstracts (original vs. ChatGPT-generated) from high-impact medical journals
Research Subject
Comparative detection and discernibility of ChatGPT-generated versus original abstracts, including AI-detector scores (GPT-2 Output Detector AUROC), plagiarism/iThenticate similarity, and blinded human reviewer identification accuracy and qualitative impressions
Publication Details
Publication Date
2023-04-26
Journal
Publisher
ISSN
Cited by
1204
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Scikit-learn: Machine Learning in Python2012
SciPy 1.0: fundamental algorithms for scientific computing in Python2020
SciPy 1.0: fundamental algorithms for scientific computing in Python2019
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models2023
The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies2023
Cited by20
Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy2023
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns2023
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT2023
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?2023
Artificial Hallucinations in ChatGPT: Implications in Scientific Writing2023
The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review2024
ChatGPT for Education and Research: Opportunities, Threats, and Strategies2023
A SWOT analysis of ChatGPT: Implications for educational practice and research2023
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly2024
The future landscape of large language models in medicine2023
The role of ChatGPT in higher education: Benefits, challenges, and future research directions2023
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health2023
War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education2023
Using ChatGPT in academic writing is (not) a form of plagiarism: What does the literature say?2023
AI-generated research paper fabrication and plagiarism in the scientific community2023
Generalization bias in large language model summarization of scientific research2025
Generative AI Literacy: Twelve Defining Competencies2024
Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review2024
Using ChatGPT for human–computer interaction research: a primer2023
Evaluation of large language models as a diagnostic aid for complex medical cases2024