Desirable Characteristics for AI Teaching Assistants in Programming Education

Желательные характеристики ИИ-ассистентов преподавателя в обучении программированию
Paul C. Denny, Andrew Luxton-Reilly, Jaromír Šavelka, Stephen MacNeil, Leo Porter
2024-07-03

AI teaching assistantslarge language modelslearning scaffoldingpersonalized feedbackprogramming education
Providing timely and personalized feedback to large numbers of students is a long-standing challenge in programming courses. Relying on human teaching assistants (TAs) has been extensively studied, revealing a number of potential shortcomings. These include inequitable access for students with low confidence when needing support, as well as situations where TAs provide direct solutions without helping students to develop their own problem-solving skills. With the advent of powerful large language models (LLMs), digital teaching assistants configured for programming contexts have emerged as an appealing and scalable way to provide instant, equitable, round-the-clock support. Although digital TAs can provide a variety of help for programming tasks, from high-level problem solving advice to direct solution generation, the effectiveness of such tools depends on their ability to promote meaningful learning experiences. If students find the guardrails implemented in digital TAs too constraining, or if other expectations are not met, they may seek assistance in ways that do not help them learn. Thus, it is essential to identify the features that students believe make digital teaching assistants valuable. We deployed an LLM-powered digital assistant in an introductory programming course and collected student feedback ($n=813$) on the characteristics of the tool they perceived to be most important. Our results highlight that students value such tools for their ability to provide instant, engaging support, particularly during peak times such as before assessment deadlines. They also expressed a strong preference for features that enable them to retain autonomy in their learning journey, such as scaffolding that helps to guide them through problem-solving steps rather than simply being shown direct solutions.
1
Feedback from 813 introductory programming students showed that instant and engaging assistance is especially valued during peak periods before assessment deadlines.
2
Human teaching assistants can provide inequitable support and may sometimes give direct solutions that undermine students’ problem-solving development.
3
LLM-powered digital teaching assistants offer scalable, instant, and potentially equitable programming support, but their educational effectiveness depends on promoting meaningful learning.
4
Overly restrictive guardrails or unmet expectations may lead students to seek assistance in ways that do not support learning.
5
Students strongly preferred autonomy-preserving scaffolding that guides problem-solving steps instead of simply presenting direct solutions.

LLM-powered digital teaching assistants for introductory programming education

Students’ perceived desirable characteristics and learning-support effectiveness, particularly instant engaging support and autonomy-preserving scaffolding rather than direct solutions

Publication Details
Publication Date
2024-07-03
Journal
Publisher
ISSN
Cited by
134
Access Type
Author Information
Authors
Paul C. Denny
Andrew Luxton-Reilly
Jaromír Šavelka
Stephen MacNeil
Leo Porter
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%