Publications
This dissertation studies how artificial intelligence, especially large language models such as GPT, can help students learn introductory computer programming when the models are used inside a carefully designed learning system. Instead of focusing on AI as a standalone tool, the dissertation follows a connected story: generating learning resources, building a tutoring platform, running a user study, and then analyzing how students behave while they program. The work first uses prompt engineering to create a large collection of 11,700 Python exercises aligned with introductory computer science topics. Students and instructors then evaluate these exercises to check whether they are clear, relevant, and useful for learning. The dissertation next introduces CodeMasterAI, an intelligent tutoring system that presents these exercises in an interactive coding environment and provides AI-generated feedback without simply giving away the answer. Using CodeMasterAI, the dissertation conducts a user study with 19 undergraduate students working on nine programming exercises. The platform records keystrokes, code snapshots, feedback requests, and survey responses, which makes it possible to study learning as it happens rather than only looking at final submissions. The later chapters use these data to examine cognitive load, how students use AI feedback, and how different learner profiles can be detected from programming behavior. Overall, this work shows that large language models can be valuable tools for introductory programming education when they are paired with human evaluation, thoughtful system design, and detailed learning analytics. The dissertation provides both practical guidance for building AI-supported tutoring tools and research evidence for future personalized learning systems.
Large language models are increasingly used to generate programming practice, yet their cognitive demands for CS1 learners remain underexplored. We investigate student-perceived cognitive load on GPT-generated Python exercises using a custom intelligent tutoring system that captures fine-grained keystrokes and offers in-exercise AI feedback. Nineteen undergraduates completed nine exercises spanning loops, recursion, and classes and objects, yielding 171 submissions and 83,377 keystrokes. We triangulate self-reports with keystroke dynamics, code quality metrics, and knowledge component (KC) mastery. Perceived load largely tracked labeled difficulty and was inversely related to satisfaction. Keystroke features were the strongest correlates of load, with more typing bursts and longer inter-keystroke latencies accompanying higher demand; among code metrics, the SLoC-LLoC differential showed the highest association with demand, aligning higher structural complexity with greater mental effort. Lower mastery of core knowledge components—especially loops plus string and list operations—coincided with higher load. Students rated instant AI feedback helpful for debugging and best practices, though at times redundant or inconsistent with instructions. To estimate load automatically, we trained models on observed traces. An Extra Trees regressor predicted the primary load measure Demanding with RMSE 0.81, and a three-level classifier achieved 86 % accuracy with weighted F1 0.85 on held-out students. These results suggest actionable design levers for LLM-based tutors: calibrate task complexity, scaffold high-strain concepts, and deliver concise, context-aligned feedback while using keystroke- and code-based signals for real-time load monitoring. The code and data are available: https://github.com/DSAatUSU/LLMCognitiveLoadProgramming.
Online coding platforms (OCPs) often offer a limited selection of exercises, which can restrict the scope of Computer Science (CS) education. This study investigates the capabilities of Large Language Models (LLMs), particularly GPT-4 Turbo, in broadening this scope by autonomously generating Python programming exercises. These exercises are tailored to the CS1 curriculum-an introductory course in computer science. Utilizing curriculum-driven prompt engineering, we developed a dataset of 11,700 exercises, characterized by a variety of cate-gories, types, and difficulty levels. These exercises are distributed across 78 unique topics, which were derived from the CS1 course catalogs of leading universities and supplemented with online educational resources. To evaluate the effectiveness of GPT-4 Turbo in generating CSI Python programming exercises, we conducted a user study involving both students and instruc-tors. The study focused on several metrics: exercise quality, curriculum relevance, understandability, appropriate difficulty level, and the generation of useful hints. Our findings indicate that GPT-4 Turbo can produce high-quality, educationally effective programming exercises at scale, provided that the prompts are systematically crafted. Based on insights from the user study, adjustments to prompt design are recommended to optimize exercise generation. Our research concludes that GPT-4 Turbo can be seamlessly integrated into AI-driven OCPs, offering a scalable, cost and time-effective method to enhance CS edu-cation. This is achieved through targeted prompt engineering and thorough data preprocessing to mitigate inconsistencies. The code is available online: https://github.com/DSAatUSU/GPT_CS1400_Exercise_Generation


