
Using Fine-grained Programming Trace Data to Inform Disciplinary Models of Self-Regulated Learning in Computing Education
Effective Years: 2023-2026
Computer science (CS) is an increasingly important topic for undergraduate students; it is where they learn to program and develop software. However, it is also a challenging subject for many students to learn. A key determinant of students' success in modern CS classrooms is whether and how they use self-regulated learning (SRL) skills, such as planning, goal setting, and adapting their learning strategies to meet new challenges. The aim of this project is to help develop theory to explain how students engage in self-regulation, specifically in CS classrooms, and how this impacts their learning outcomes. This CS-specific model of SRL is critical to help scientists develop effective interventions for improving student learning (e.g., teaching students how to set effective goals) because it will help computer science educators and researchers predict how students might respond to those interventions. To advance theory, this project will develop new methods for studying SRL in CS classrooms. Currently, one of the best ways to study SRL in detail is to have students think out loud as they work and learn, so scientists can understand what SRL skills and strategies students are using and why they are using them. However, this is a time-consuming process, and it is impractical to use in real classrooms. To address this challenge, this project will pair think-aloud data with the log data students produce when learning with technology, to verify how to use this log data as a measure of students’ SRL. Being able to rely on these automatically logged data will allow scientists to study SRL in authentic CS classrooms, at scale, with large, diverse groups of learners. These studies may also help determine how SRL processes in CS courses are similar or distinct from those in other STEM courses. The results of this project will help CS education researchers to develop more effective learning interventions, especially for students who are underrepresented in CS.
This project will apply, test, and elaborate a model of self-regulated learning (SRL) in computer science (CS) education (i.e., SRCSL) by validating and analyzing fine-grained trace data from students' interactions with programming tools in authentic undergraduate classroom settings. SRL encompasses students' thoughtful pursuit of academic goals by planning, monitoring, controlling, and reflecting upon what and how they learn. These behaviors are a key determinant of students' success in many domains, including CS. Discipline-based models of SRL in CS are still nascent and largely untested in rigorous ways that reflect cutting-edge practices for data collection and validation that can scale to larger populations. The development of a SRCSL model requires large-scale collection of validated SRL traces from authentic CS classrooms. To do so, first the project will use laboratory studies to collect both digital trace data from programming tools and think-aloud protocol (TAP) data from students as they complete authentic learning activities, coding TAPs to identify SRL events. Researchers will align these two data sources, mapping digitally logged events to the verbal SRL events that they reflect, and then use the coded TAP data to validate inferences about what SRL processes the digital trace data indicate. Second, researchers will collect a large sample (N ≈ 2,000) of trace data from two CS courses and apply the validated mapping to identify SRL events, from digital trace data, as they occur in the classroom. This will afford analyses regarding the sequential, contingent, and dynamic nature of SRL, informing an empirically-supported initial model of SRCSL. The proposed model will also posit how SRL behaviors predict students' cognitive and non-cognitive course outcomes, how these relationships vary across course contexts, and how they are moderated by students' personal characteristics. This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.