War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education

Война чат-ботов: Bard, Bing Chat, ChatGPT, Ernie и другие. Новая золотая лихорадка ИИ и её влияние на высшее образование
Jürgen Rudolph, Shannon Tan, Samson Tan
2023-04-24

BardBing ChatChatGPTErnieGPT-4assessment and teaching recommendationschatbotscomparative evaluationhigher education
Developments in the chatbot space have been accelerating at breakneck speed since late November 2022. Every day, there appears to be a plethora of news. A war of competitor chatbots is raging amidst an AI arms race and gold rush. These rapid developments impact higher education, as millions of students and academics have started using bots like ChatGPT, Bing Chat, Bard, Ernie and others for a large variety of purposes. In this article, we select some of the most promising chatbots in the English and Chinese-language spaces and provide their corporate backgrounds and brief histories. Following an up-to-date review of the Chinese and English-language academic literature, we describe our comparative method and systematically compare selected chatbots across a multi-disciplinary test relevant to higher education. The results of our test show that there are currently no A-students and no B-students in this bot cohort, despite all publicised and sensationalist claims to the contrary. The much-vaunted AI is not yet that intelligent, it would appear. GPT-4 and its predecessor did best, whilst Bing Chat and Bard were akin to at-risk students with F-grade averages. We conclude our article with four types of recommendations for key stakeholders in higher education: (1) faculty in terms of assessment and (2) teaching & learning, (3) students and (4) higher education institutions.
1
GPT-4 and its predecessor performed best in the comparative test, while Bing Chat and Bard performed poorly, averaging F-grade equivalents.
2
No chatbot in the tested cohort achieved A- or B-student level performance; overall intelligence remains overstated in public discourse.
3
Rapid proliferation of competitor chatbots since November 2022 has created an AI arms race impacting higher education usage by students and academics.
4
The authors systematically compared selected English and Chinese-language chatbots across a multi-disciplinary test relevant to higher education.
5
The paper offers four types of recommendations for higher education stakeholders: faculty assessment, teaching and learning, students, and institutions.

Contemporary large-language-model chatbots (e.g., ChatGPT, Bing Chat, Bard, Ernie) as used in higher education

Comparative performance and limitations of these chatbots across multidisciplinary tasks relevant to higher education and their impact on assessment, teaching, learning, students, and institutions

Publication Details
Publication Date
2023-04-24
Journal
Publisher
ISSN
Cited by
419
Access Type
Author Information
Authors
Jürgen Rudolph
Shannon Tan
Samson Tan
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat
Make a presentation
100%