Home > Publications Search > Publication Detail

Student-Perceived Cognitive Load of LLM-Generated Programming Exercises

Conference Paper/Proceedings
Student-Perceived Cognitive Load of LLM-Generated Programming Exercises
Publication Year:
2025
Publication Source:
IEEE International Conference on Data Science and Advanced Analytics (DSAA)
Author(s):
Khan, Muhammad Fawad Akbar; Feri, Ludia Eka; Nguyen, Ha; Karimi, Hamid Reza
Supporting Project(s):

Large language models are increasingly used to generate programming practice, yet their cognitive demands for CS1 learners remain underexplored. We investigate student-perceived cognitive load on GPT-generated Python exercises using a custom intelligent tutoring system that captures fine-grained keystrokes and offers in-exercise AI feedback. Nineteen undergraduates completed nine exercises spanning loops, recursion, and classes and objects, yielding 171 submissions and 83,377 keystrokes. We triangulate self-reports with keystroke dynamics, code quality metrics, and knowledge component (KC) mastery. Perceived load largely tracked labeled difficulty and was inversely related to satisfaction. Keystroke features were the strongest correlates of load, with more typing bursts and longer inter-keystroke latencies accompanying higher demand; among code metrics, the SLoC-LLoC differential showed the highest association with demand, aligning higher structural complexity with greater mental effort. Lower mastery of core knowledge components—especially loops plus string and list operations—coincided with higher load. Students rated instant AI feedback helpful for debugging and best practices, though at times redundant or inconsistent with instructions. To estimate load automatically, we trained models on observed traces. An Extra Trees regressor predicted the primary load measure Demanding with RMSE 0.81, and a three-level classifier achieved 86 % accuracy with weighted F1 0.85 on held-out students. These results suggest actionable design levers for LLM-based tutors: calibrate task complexity, scaffold high-strain concepts, and deliver concise, context-aligned feedback while using keystroke- and code-based signals for real-time load monitoring. The code and data are available: https://github.com/DSAatUSU/LLMCognitiveLoadProgramming.