Impact of AI assistance on student agency
Влияние AI-помощи на агентность студентов
2023-11-30
SCID: 54.1/kzfmfn8x
Discuss with AI
AI assistancepeer feedbackrandomised controlled experimentself-regulated learningstudent agency
Figures from the paper
Abstract (AI)
AI-powered learning technologies are increasingly being used to automate and scaffold learning activities (e.g., personalised reminders for completing tasks, automated real-time feedback for improving writing, or recommendations for when and what to study). While the prevailing view is that these technologies generally have a positive effect on student learning, their impact on students’ agency and ability to self-regulate their learning is under-explored. Do students learn from the regular, detailed and personalised feedback provided by AI systems, and will they continue to exhibit similar behaviour in the absence of assistance? Or do they instead continue to rely on AI assistance without learning from it? To contribute to filling this research gap, we conducted a randomised controlled experiment that explored the impact of AI assistance on student agency in the context of peer feedback. With 1625 students across 10 courses, an experiment was conducted using peer review. During the initial four-week period, students were guided by AI features that utilised techniques such as rule-based suggestion detection, semantic similarity, and comparison with previous comments made by the reviewer to enhance their submissions if the feedback provided was deemed insufficiently detailed or general in nature. Over the following four weeks, students were divided into four different groups: control (AI) received prompts, (NR) received no prompts, (SR) received self-monitoring checklists in place of AI prompts, and (SAI) had access to both AI prompts and self-monitoring checklists. Results of the experiment suggest that students tended to rely on rather than learn from AI assistance. If AI assistance was removed, self-regulated strategies could help in filling in the gap but were not as effective as AI assistance. Results also showed that hybrid human-AI approaches that complement AI assistance with self-regulated strategies (SAI) were not more effective than AI assistance on its own. We conclude by discussing the broader benefits, challenges and implications of relying on AI assistance in relation to student agency in a world where we learn, live and work with AI.
Key Findings
1
A randomized controlled experiment with 1,625 students across 10 courses examined AI assistance effects on student agency in peer feedback.
2
Combining AI prompts with self-monitoring checklists (hybrid SAI) was not more effective than AI assistance alone.
3
During an initial four-week period, AI features provided personalised prompts using rule-based suggestion detection, semantic similarity, and comparison to past comments to enhance insufficient feedback.
4
Students tended to rely on AI assistance rather than learn from it, showing reduced independent application of feedback when assistance was removed.
5
When AI assistance was removed, self-regulated strategies (checklists) partially filled the gap but were less effective than prior AI assistance.
Research Object
AI-powered learning technologies used to provide personalised feedback and prompts in peer-review activities with students
Research Subject
Impact of AI assistance on student agency and self-regulated learning behaviors (reliance on versus learning from AI), including effects of removing AI and of self-monitoring checklists or combined AI+self-regulation interventions
Publication Details
Publication Date
2023-11-30
Journal
Publisher
ISSN
Cited by
649
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai10
Systematic review of research on artificial intelligence applications in higher education – where are the educators?2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation2016
Practical and ethical challenges of large language models in education: A systematic scoping review2023
The Promises and Challenges of Artificial Intelligence for Teachers: a Systematic Review of Research2022
Vision, challenges, roles and research issues of Artificial Intelligence in Education2020
Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence2023
Application and theory gaps during the rise of Artificial Intelligence in Education2020
The impact of artificial intelligence on learner–instructor interaction in online learning2021
Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper)2023
Qualitative Spatial Reasoning over Questions (Short Paper)2022
Cited by5
AI literacy and its implications for prompt engineering strategies2024
Agency in the age of generative AI: a critical review of educational implications2026
Unveiling interaction patterns between students and generative AI teachable agent: Focusing on students' agency and AI agents' authority2026
When machines join the moral circle: The persona effect of generative AI agents in collaborative reasoning2026
Generative AI in Academia: How Engineering Students Perceive and Approach Ethical Challenges?2026