War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education
Война чат-ботов: Bard, Bing Chat, ChatGPT, Ernie и другие. Новая золотая лихорадка ИИ и её влияние на высшее образование
2023-04-24
SCID: 54.1/56uhejr8
Discuss with AI
BardBing ChatChatGPTErnieGPT-4assessment and teaching recommendationschatbotscomparative evaluationhigher education
Figures from the paper
Abstract (AI)
Developments in the chatbot space have been accelerating at breakneck speed since late November 2022. Every day, there appears to be a plethora of news. A war of competitor chatbots is raging amidst an AI arms race and gold rush. These rapid developments impact higher education, as millions of students and academics have started using bots like ChatGPT, Bing Chat, Bard, Ernie and others for a large variety of purposes. In this article, we select some of the most promising chatbots in the English and Chinese-language spaces and provide their corporate backgrounds and brief histories. Following an up-to-date review of the Chinese and English-language academic literature, we describe our comparative method and systematically compare selected chatbots across a multi-disciplinary test relevant to higher education. The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary. The much-vaunted AI is not yet that intelligent, it would appear. GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages. We conclude our article with four types of recommendations for key stakeholders in higher education: (1) faculty in terms of assessment and (2) teaching & learning, (3) students and (4) higher education institutions.
Key Findings
1
GPT-4 and its predecessor performed best in the comparative test, while Bing Chat and Bard performed poorly, averaging F-grade equivalents.
2
No chatbot in the tested cohort achieved A- or B-student level performance; overall intelligence remains overstated in public discourse.
3
Rapid proliferation of competitor chatbots since November 2022 has created an AI arms race impacting higher education usage by students and academics.
4
The authors systematically compared selected English and Chinese-language chatbots across a multi-disciplinary test relevant to higher education.
5
The paper offers four types of recommendations for higher education stakeholders: faculty assessment, teaching and learning, students, and institutions.
Research Object
Contemporary large-language-model chatbots (e.g., ChatGPT, Bing Chat, Bard, Ernie) as used in higher education
Research Subject
Comparative performance and limitations of these chatbots across multidisciplinary tasks relevant to higher education and their impact on assessment, teaching, learning, students, and institutions
Publication Details
Publication Date
2023-04-24
Journal
Publisher
ISSN
Cited by
419
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai17
Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics2023
LLaMA: Open and Efficient Foundation Language Models2023
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT2023
What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education2023
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?2023
A SWOT analysis of ChatGPT: Implications for educational practice and research2023
Chatbots applications in education: A systematic review2021
ChatGPT in higher education: Considerations for academic integrity and student learning2023
Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond2023
The rise of ChatGPT : Exploring its potential in medical education2023
Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI)2023
Are We There Yet? - A Systematic Literature Review on Chatbots in Education2021
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT2023
Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers2022
Will ChatGPT get you caught? Rethinking of Plagiarism Detection2023
Chatbots: History, technology, and applications2020
Chatbots for learning: A review of educational chatbots for the Facebook Messenger2020
Cited by5
Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings2023
The role of ChatGPT in higher education: Benefits, challenges, and future research directions2023
AI literacy and its implications for prompt engineering strategies2024
Exploring collaborative decision-making: A quasi-experimental study of human and Generative AI interaction2024
“When Honesty is Good, for Imitation is Bad”: Strategies for Using Generative Artificial Intelligence in Russian Higher Education Institutions2025