Integrating a ChatGPT-enhanced error feedback system with epistemic network analysis to explore students' learning programming behaviors

Abstract

With technology advancements, programming education has become increasingly important. As students often face difficulties understanding compiler error messages while learning programming, we developed a programming assistance system including a ChatGPT-enhanced error feedback (CEF) mechanism. CEF provides detailed and easily understandable error feedback, and is integrated with a LINE chatbot to offer students accessible guidance. Its impact was evaluated through a 7-week experiment in an undergraduate C/C++ programming course involving an experimental group using CEF and a control group. The integration of CEF with Epistemic Network Analysis (ENA) allowed us to quantify, visualize, and compare the structural differences in the two groups' error co-occurrence patterns, thereby linking the intervention effect directly to structural changes in students' debugging connection patterns. CEF significantly improved students' understanding of compiler error messages and the precision of their error correction, while ENA revealed that mutual influence among different error types was lessened for experimental group students. Findings support integrating generative AI tools into programming education to assist students' debugging processes, alleviate instructors' workload, and foster students' self-directed learning. Integrating CEF with ENA establishes a novel methodological framework which captures and visualizes the structural evolution of students' debugging behaviors.