Desirable Characteristics for AI Teaching Assistants in Programming Education
Желательные характеристики ИИ-ассистентов преподавателя в обучении программированию
2024-07-03
SCID: 54.1/jneqjedh
Discuss with AI
AI teaching assistantslarge language modelslearning scaffoldingpersonalized feedbackprogramming education
Figures from the paper
Abstract (AI)
Providing timely and personalized feedback to large numbers of students is a long-standing challenge in programming courses. Relying on human teaching assistants (TAs) has been extensively studied, revealing a number of potential shortcomings. These include inequitable access for students with low confidence when needing support, as well as situations where TAs provide direct solutions without helping students to develop their own problem-solving skills. With the advent of powerful large language models (LLMs), digital teaching assistants configured for programming contexts have emerged as an appealing and scalable way to provide instant, equitable, round-the-clock support. Although digital TAs can provide a variety of help for programming tasks, from high-level problem solving advice to direct solution generation, the effectiveness of such tools depends on their ability to promote meaningful learning experiences. If students find the guardrails implemented in digital TAs too constraining, or if other expectations are not met, they may seek assistance in ways that do not help them learn. Thus, it is essential to identify the features that students believe make digital teaching assistants valuable. We deployed an LLM-powered digital assistant in an introductory programming course and collected student feedback ($n=813$) on the characteristics of the tool they perceived to be most important. Our results highlight that students value such tools for their ability to provide instant, engaging support, particularly during peak times such as before assessment deadlines. They also expressed a strong preference for features that enable them to retain autonomy in their learning journey, such as scaffolding that helps to guide them through problem-solving steps rather than simply being shown direct solutions.
Key Findings
1
Feedback from 813 introductory programming students showed that instant and engaging assistance is especially valued during peak periods before assessment deadlines.
2
Human teaching assistants can provide inequitable support and may sometimes give direct solutions that undermine students’ problem-solving development.
3
LLM-powered digital teaching assistants offer scalable, instant, and potentially equitable programming support, but their educational effectiveness depends on promoting meaningful learning.
4
Overly restrictive guardrails or unmet expectations may lead students to seek assistance in ways that do not support learning.
5
Students strongly preferred autonomy-preserving scaffolding that guides problem-solving steps instead of simply presenting direct solutions.
Research Object
LLM-powered digital teaching assistants for introductory programming education
Research Subject
Students’ perceived desirable characteristics and learning-support effectiveness, particularly instant engaging support and autonomy-preserving scaffolding rather than direct solutions
Publication Details
Publication Date
2024-07-03
Journal
Publisher
ISSN
Cited by
134
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest