Numerous studies aim to enhance learning in digital environments through emotionally-sensitive interventions. The D'Mello and Graesser (2012) model of affect dynamics hypothesizes that when a learner encounters confusion, the degree to which it is prolonged (and transitions into frustration) or resolved, significantly affects their learning outcomes in digital environments. However, studies yield inconclusive results regarding relations between confusion, frustration, and learning. More research is needed to explore how confusion and frustration manifest during learning and its relation to outcomes. We go beyond past work looking at the rate, duration, and transitions of confusion and frustration by treating these affective states as non-linear dynamical systems consisting of expressive and behavioral components. We examined the frequency and recurrence of facial expressions associated with basic emotions (as automatically labeled by AffDex, a standard tool for analyzing emotions with video data) during confused and frustrated states (as automatically labeled with BROMP-based detectors applied to students' interaction data). We compare these co-occurring patterns to learning outcomes (pre-tests, post-tests, and learning gains) within a digital learning environment, Betty ' s Brain. Results showed that the frequency and recurrence rate of basic emotions expressed during confusion and frustration are complex and remain incompletely understood. Specifically, we show that confusion and frustration have different relationships with learning outcomes, depending on which basic emotion expressions they co-occur with. Implications of this study open avenues for better understanding these emotions as complex and non-linear dynamical systems, in the long-term enabling personalized feedback and emotional support within digital learning environments that enhance learning outcomes.

In this paper, we propose a new method for selecting cases for in situ, immediate interview research: detector-driven classroom interviewing (DDCI). Published work in educational data mining and learning analytics has yielded highly scalable measures that can detect key aspects of student interaction with computer-based learning in close to real-time. These measures detect a variety of constructs and make it possible to increase the precision and time-efficiency of this form of research. We review four examples that show how the method can be used to study why students become frustrated and how they respond, how anxiety influences how students respond to frustration, how metacognition interacts with affect, and how to improve the design of an adaptive learning system. Lastly, we compare DDCI to other mixed-methods approaches and outline opportunities for detector-driven classroom interviewing in research and practice, including research opportunities, design improvement opportunities, and pedagogical opportunities for teachers.

Time management is crucial for college students' academic success and learning of computer programming. Yet the changes of time management behaviors and their associations with learning outcomes are underexplored in online learning of programming. To address the gap, this study employed an intensive longitudinal approach to examine undergraduates' time management behaviors in an online programming problem system. Specifically, we analyzed weekly indicators of academic procrastination and spaced practice derived from programming traces. We applied dynamic structural equation modeling to examine the changes in these behaviors over time and their correlations with weekly quiz performance. Academic procrastination and selfselected spaced practice showed a significant upward trend over time, while incentivized spaced practice exhibited a significant downward trend. Moreover, students with prior programming experience showed a greater growth rate in spacing behaviors. At both within- and between -person levels, procrastination predicted quiz performance significantly and negatively, while self-selected spaced practice predicted quiz performance significantly and positively. In contrast, incentivized spaced practice predicted quiz performance positively at the within -person level but negatively at the between -person level. Additionally, quiz performance in the current week predicted subsequent time management behaviors significantly. These findings contribute to the understanding of procrastination and spaced practice in online programming learning and have implications for the design of scaffolding on time management. Furthermore, this study demonstrates the significance of combining intensive longitudinal approaches and action logs in examining the temporality of learning in online environments.

Help from virtual pedagogical agents has the potential to improve student learning. Yet students often do not seek help when they need it, do not use help effectively, or ignore the agent's help altogether. This paper seeks to better understand students' patterns of accepting and seeking help in a computer-based science program called Betty's Brain. Focusing on student interactions with the mentor agent, Mr. Davis, we examine the factors associated with patterns of help acceptance and help seeking; the relationship between help acceptance and help seeking; and how each behavior is related to learning outcomes. First, we examine whether students accepted help from Mr. Davis, operationalized as whether they followed his suggestions to read specific textbook pages. We find a significant positive relationship between help acceptance and student post-test scores. Despite this, help accepters made fewer positive statements about Mr. Davis in the interviews. Second, we identify how many times students proactively sought help from Mr. Davis. Students who most frequently sought help demonstrated more confusion while learning (measured using an interaction-based ML-based detector); tended to have higher science anxiety; and made more negative statements about Mr. Davis, compared to those who made few or no requests. However, help seeking was not significantly related to post-test scores. Finally, we draw from the qualitative interviews to consider how students understand and articulate their experiences with help from Mr. Davis.

The feelings of difficulty and familiarity (FOD and FOF) are two types of metacognitive experiences. Both may influence student engagement and the application of metacognitive strategies, but these relationships are not well understood, in part because many studies have relied on self-report measures of behaviors that may not accurately reflect students' actual behaviors. In this study, FOD and FOF were related to objective measures of off-task behaviors and metacognitive strategies. These measures were extracted from 88 sixth graders' action logs within a computer-based learning environment known as Betty's Brain. Pre- and post-tests were administered to assess learning. Results reveal that high-FOD students showed more off-task behaviors and fewer strategic behaviors than low-FOD students, particularly when this difference was measured in terms of the frequency (as opposed to proportion) of strategic behaviors. FOF was not associated with off-task behaviors and metacognitive strategies but emerged as a moderator in the relationship between FOD and learning gains. Low-FOD students learned more than high-FOD students in the low-FOF group, but such a difference was not found in the high-FOF group.

BackgroundProviding adaptive scaffolds to help learners develop effective self-regulated learning (SRL) behaviours has been an important goal for intelligent learning environments. Adaptive scaffolding is especially important in open-ended learning environments (OELE), where novice learners often face difficulties in completing their learning tasks. ObjectivesThis paper presents a systematic framework for adaptive scaffolding in Betty's Brain, a learning-by-teaching OELE for middle school science, where students construct a causal model to teach a virtual agent, generically named Betty. We evaluate the adaptive scaffolding framework and discuss its implications on the development of more effective scaffolds for SRL in OELEs. MethodsWe detect key cognitive/metacognitive inflection points, that is, moments where students' behaviours and performance change during learning, often suggesting an inability to apply effective learning strategies. At inflection points, Mr. Davis (a mentor agent in Betty's Brain) or Betty (the teachable agent) provides context-specific conversational feedback, focusing on strategies to help the student become a more productive learner, or encouragement to support positive emotions. We conduct a classroom study with 98 middle schoolers to analyse the impact of adaptive scaffolds on students' learning behaviours and performance. We analyse how students with differential pre-to-post learning outcomes receive and use the scaffolds to support their subsequent learning process in Betty's Brain. Results and ConclusionsAdaptive scaffolding produced mixed results, with some scaffolds (viz., strategic hints that supported debugging and assessment of causal models) being generally more useful to students than others (viz., encouragement prompts). Additionally, there were differences in how students with high versus low learning outcomes responded to some hints, as suggested by the differences in their learning behaviours and performance in the intervals after scaffolding. Overall, our findings suggest how adaptive scaffolding in OELEs like Betty's Brain can be further improved to better support SRL behaviours and narrow the learning outcomes gap between high and low performing students. ImplicationsThis paper contributes to our understanding and impact of adaptive scaffolding in OELEs. The results of our study indicate that successful scaffolding has to combine context-sensitive inflection points with conversational feedback that is tailored to the students' current proficiency levels and needs. Also, our conceptual framework can be used to design adaptive scaffolds that help students develop and apply SRL behaviours in other computer-based learning environments.

There is a growing interest in viewing self-regulated learning as events unfolding over time, especially when students perform learning tasks in computer-based environments. Metacognitive activities are critical events in self-regulated learning. This study investigated the evolution of metacognitive strategy use in an open-ended computer-based learning environment, Betty's Brain. The data were from 93 sixth graders who used Betty's Brain to learn about climate change for four days. We extracted indicators of metacognitive strategy use from action logs. A knowledge test and self-report questionnaire were administrated before students started using Betty's Brain to assess prior domain knowledge and motivation, respectively. Results showed that metacognitive strategy use increased from the first to the second day and remained stable from the second to the fourth day of the study. The evolution of these behaviors varied across students. Task value and prior domain knowledge partially explained the individual differences in this evolution. Task value and prior domain knowledge also predicted the use of metacognitive strategies. Self-efficacy did not influence metacognitive strategy use. These results suggest the need for further investigation into the role of motivation and prior domain knowledge in the temporal evolution of metacognitive events.

The micro-level analyses of how students’ self-regulated learning (SRL) behaviors unfold over time provides a valuable framework for understanding their learning processes as they interact with computer-based learning environments. In this paper, we use log trace data to investigate how students self-regulate their learning in the Betty’s Brain environment, where they engage in three categories of open-ended problem-solving actions: information seeking, solution construction and solution assessment. We use Epistemic Network Analysis (ENA) to provide us with an overall understanding of the co-occurrences between action types both within and between the three action categories. Comparisons of epistemic networks generated for two groups of students, those with low and high performance, provided us with insights into their self-regulated behaviors. © 2021, Springer Nature Switzerland AG.

Students in computerized learning environments often direct their own learning processes, which requires metacognitive awareness of what should be learned next. We investigated a novel method of measuring verbalized metacognition by applying natural language processing (NLP) to transcripts of interviews conducted in a classroom with 99 middle school students who were using a computerized learning environment. We iteratively adapted the NLP method for the linguistic characteristics of these interviews, then applied it to study three research questions regarding the relationships between verbalized metacognition and measures of 1) learning, 2) confusion, and 3) metacognitive problem-solving strategies. Verbalized metacognition was not directly related to learning, but was related to confusion and metacognitive problem-solving strategies. Results also suggested that interviews themselves may improve learning by encouraging metacognition. We discuss implications for designing computerized environments that support self-regulated learning through metacognition. © 2021 ACM.

Confusion may benefit learning when it is resolved or partially resolved. Metacognitive strategies (MS) may help learners to resolve confusion when it occurs during learning and problem solving. This study examined the relationship between confusion and MS that students evoked in Betty's Brain, a computer-based learning-bymodelling environment where elementary and middle school students learn science by building causal maps. Participants were sixth graders. Emotion data were collected from real-time observations by trained researchers. MS and task performance information were determined by analyzing the action logs. Pre- and post-tests were used to assess learning gains. The results revealed that the use of MS was a function of the state of student confusion. However, confusion resolution was not related to MS behaviour, and MS did not moderate the effect of confusion on student task performance in Betty's Brain or on learning gains.