In this work, we investigate how two factors, metacognitive skills and motivation, would impact student learning across domains. More specifically, our primary goal is to identify the critical, yet robust, interaction patterns of these two factors that would contribute to students' performance in learning logic first and then their performance on a subsequent new domain, probability. We are concerned with two types of metacognitive skills: strategy-awareness and time-awareness, that is, which problem-solving strategy to use and when to use it. Our data were collected from 495 participants across three consecutive semesters, and our results show that the only students who consistently outperform their peers across both domains are those who are not only highly motivated but also strategy-aware and time-aware.

Learning sciences are embracing the significant role technologies can play to better detect, diagnose, and act upon self-regulated learning (SRL). The field of SRL is challenged with the measurement of SRL processes to advance our understanding of how multimodal data can unobtrusively capture learners' cognitive, metacognitive, affective, and motivational states over time, tasks, domains, and contexts. This paper introduces a self-regulated learning processes, multimodal data, and analysis (SMA) grid and maps joint and individual research of the authors (63 papers) over the last five years onto the grid. This shows how multimodal data streams were used to investigate SRL processes. The two-dimensional space on the SMA grid is helpful for visualizing the relations and possible combinations between the data streams and how the measurement of SRL processes. This overview serves as an analytical introduction to the current special issue “Advancing SRL Research with Artificial Intelligence (AI)” and we encourage to position new research and unexplored frontiers. We emphasize the need for intensive and strategic collaboration to accelerate progress using new interdisciplinary methods to develop accurate measurement of SRL in educational technologies. © 2022 The Authors

Self-regulated learning (SRL) is critical for learning across tasks, domains, and contexts. Despite its importance, research shows that not all learners are equally skilled at accurately and dynamically monitoring and regulating their self-regulatory processes. Therefore, learning technologies, such as intelligent tutoring systems (ITSs), have been designed to measure and foster SRL. This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students' SRL while they learn about the human circulatory system. MetaTutor's architecture and instructional features are designed based on models of SRL, empirical evidence on human and computerized tutoring principles of multimedia learning, Artificial Intelligence (AI) in educational systems for metacognition and SRL, and research on SRL from our team and that of other researchers. We present MetaTutor followed by a synthesis of key research findings on the effectiveness of various versions of the system (e.g., adaptive scaffolding vs. no scaffolding of self-regulatory behavior) on learning outcomes. First, we focus on findings from self-reports, learning outcomes, and multimodal data (e.g., log files, eye tracking, facial expressions of emotion, screen recordings) and their contributions to our understanding of SRL with an ITS. Second, we elaborate on the role of embedded pedagogical agents (PAs) as external regulators designed to scaffold learners' cognitive and metacognitive SRL strategy use. Third, we highlight and elaborate on the contributions of multimodal data in measuring and understanding the role of cognitive, affective, metacognitive, and motivational (CAMM) processes. Additionally, we unpack some of the challenges these data pose for designing real-time instructional interventions that scaffold SRL. Fourth, we present existing theoretical, methodological, and analytical challenges and briefly discuss lessons learned and open challenges.

[No abstract available]

Self-regulated learning (SRL) with advanced learning technologies has shown to significantly augment learners' performance across contexts. Yet studies find learners lack sufficient SRL skills to successfully implement strategies (e.g., judgments of learning, note taking, self-testing, etc.). Current research does not fully explain how and why this failure of effective strategy deployment occurs. We used principle component analysis (PCA) on process data (i.e., log files) from 190 undergraduates learning with MetaTutor, a hypermedia-based intelligent tutoring system, to explore underlying patterns in the frequency of strategy deployment occurring with and without pedagogical agent scaffolding to better understand any underlying structures of system- and learner-initiated cognitive and metacognitive SRL strategy use. Results showed that the system's underlying architecture deploys processes corresponding to both the phases of learning and type of effort allocation according to Winne's (2018) Information Processing Theory of SRL. However, learner-initiated processes for those who received scaffolding only displayed strategy deployment that corresponded to the type of effort allocation required of the processes (i.e., more effortful constructionist processes like note-taking versus short canned responses for judgements of learning). Additionally, results suggest all learners deploy strategies based on the familiarity of processes. Regression models using these principle components outperformed raw frequency models for capturing post-test learning performance across all participants.

The goal of this study was to investigate 65 students' evidence scores of emotions while they engaged in cognitive and metacognitive self-regulated learning processes as they learned about the circulatory system with MetaTutor, a hypermedia-based intelligent tutoring system. We coded for the accuracy of detecting students' cognitive and metacognitive processes, and examined how the computed scores related to mean evidence scores of emotions and overall learning. Results indicated that mean evidence score of surprise negatively predicted the accuracy of making a metacognitive judgment, and mean evidence score of frustration positively predicted the accuracy of taking notes, a cognitive learning strategy. These results have implications for understanding the beneficial role of negative emotions during learning with advanced learning technologies. Future directions include providing students with feedback about the benefits of both positive and negative emotions during learning and how to regulate specific emotions to ensure the most effective learning experience with advanced learning technologies.

To effectively process complex information within intelligent tutoring systems (ITSs), learners are required to engage in metacognitive monitoring micro-processes (content evaluations [CEs], judgments of learning [JOLs], feelings of knowing [FOKs], and monitoring progress towards goals [MPTGs]). Learners' average monitoring micro-process strategy frequencies were used to examine learning gains using a person-centered approach as they interacted with MetaTutor. Undergraduates (n = 94) engaged in self-initiated and system-facilitated self-regulated learning (SRL) strategies as they studied the human circulatory system with MetaTutor, a hypermedia-based ITS. Using hierarchical clustering, results showed a difference in learning between clusters differing in metacognitive monitoring process usage. Specifically, learners who used both CEs and FOKs for a greater proportion of monitoring strategy usage had significantly greater learning gains than learners who used MPTGs. Implications for monitoring strategy usage across different micro-processes and the development of ITSs to facilitate and scaffold learners' interactions with these micro-processes via prompting are discussed.

[No abstract available]

Learning disabilities (LDs) encompass disorders of those who have difficulty learning and using academic skills, exhibiting performance below expectations for their chronological age in the areas of reading, writing, and/or mathematics. Each of the disorders making up the LDs involve different deficits; however, some commonalities can be found within that heterogeneity, such in terms of learning self-regulation and metacognition. Unlike in early ages and later educational levels, there are hardly any evidence-based evaluation protocols for adults with LDs. LDs influence academic performance but also have serious consequences in professional, social, and family contexts. In response to this, the current work proposes a multimodal evaluation protocol focused on metacognitive, self-regulation of learning, and emotional processes, which make up the basis of the difficulties in adults with LDs. The assessment is carried out through analysis of the on-line learning process using a variety methods, techniques, and sensors (e.g., eye tracking, facial expressions of emotion, physiological responses, concurrent verbalizations, log files, screen recordings of human-machine interactions) and off-line methods (e.g., questionnaires, interviews, and self-report measures). This theoretically-driven and empirically-based guideline aims to provide an accurate assessment of LDs in adulthood in order to design effective prevention and intervention proposals.

This chapter focuses on the challenges of measuring self-regulatory processes and learning outcomes while learning from multiple representations with advanced learning technologies (ALTs). More specifically, we present challenges associated with measuring learners' processes and outcomes while using multiple representations with ALTs. ALTs are technology-based learning and training systems designed to teach learners of all ages to learn, reason, problem solve, and understand different topics and domains such as physics, math, history, biology, medicine and can include different systems such as multimedia, hypermedia, serious games, simulations, intelligent tutoring systems, immersive virtual environments and so forth. The structure of our chapter is as follows: (1) a brief overview of the major issues related to using multiple representations for learning with ALTs; (2) a description of the analytical techniques for understanding multiple representations and their goals for successful learning with ALTs; (3) a discussion of the issues associated with using multiple representations to measure and assess learning outcomes and how we can use ALTs as research tools to capture self-regulated learning; and (4) a presentation of challenges and future directions that need to be addressed by researchers.