Scientific reasoning is a critical foundational skill learners need to practice and know for increased science learning outcomes. Game-based learning environments (GBLEs) provide learners a platform for developing and practicing scientific reasoning skills but little is known about how learners should engage in scientific reasoning during game-based learning. As such, this paper aimed to understand if and how learners engaged in effective scientific reasoning activities during learning with a GBLE. This paper used analytical techniques from Complex Systems Theory to quantify learners’ scientific reasoning actions during game-based learning. High-school students (N = 170) played Crystal Island, a microbiology GBLE requiring learners to engage in scientific reasoning to successfully identify an illness infecting residents of a virtual island. Categorical auto-Recurrence Quantification Analysis was run on participants’ log files as they deployed scientific reasoning actions. This analysis revealed several metrics of complexity, including recurrence rate which is the proportion of repetitive to novel scientific reasoning actions. Results found that as time progressed, recurrence rates decreased. Successful learners (i.e., those who solved the mystery) demonstrated less repetition in their scientific reasoning activities where their recurrence rates decreased at a slower rate over time than learners who were unsuccessful in solving the mystery. Findings provide implications for adaptively scaffolding learners’ emerging complexities in their scientific reasoning processes during game-based learning to increase learners’ GBLE success. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

Stealth assessment in game-based learning analyzes gameplay behaviors to measure student competencies unobtrusively. Grounded in the theoretical premise that in-game objectives shape learning outcomes, this article investigates how goal recognition can enhance stealth assessment by using predictions about immediate gameplay objectives as evidence for modeling learning. We evaluated this approach with 119 middle school students using an educational microbiology game, examining both overall posttest performance and mastery of individual science concepts. Our deep learning architecture combines gameplay interactions, written reflections, and goal recognition to assess different learning dimensions. Results demonstrate that goal recognition significantly improves concept-level assessment across all four concepts, achieving higher accuracy and earlier prediction convergence. Incorporating goal recognition makes stealth assessment more responsive to students' intentions during gameplay, enabling concept-level scaffolding to enhance student learning.

Intelligent tutoring systems (ITSs) incorporate pedagogical agents (PAs) to scaffold learners' self-regulated learning (SRL) via prompts and feedback to promote learners' monitoring and regulation of their cognitive, affective, metacognitive and motivational processes to achieve their (sub)goals. This study examines PAs' effectiveness in scaffolding and teaching SRL during learning with MetaTutor, an ITS on the human circulatory system. Undergraduates (N = 118) were randomly assigned to a condition: Control Condition (i.e. learners could only self-initiate SRL strategies) and Prompt and Feedback Condition (i.e. PAs prompted learners to engage in SRL). Learners' log-file data captured when strategies were used, the initiator of the strategy (i.e. learner and PA), and the relevance of instructional content pages in relation to learner subgoals. While results showed that PAs were effective scaffolders of SRL in which they prompted learners to engage in SRL strategies more when content was relevant towards their subgoals and as time on page and task increased, there were mixed findings about the effectiveness of PAs as teachers of SRL. Findings show how production rules guiding PA prompts can improve their scaffolding and teaching of SRL across the learning task - through contextualizing SRL strategies to the instructional content and in relation to the relevance of the content to learners' subgoals.What is already known about this topicWhat this paper addsImplications for practice and/or policyPractitioner notesMost learners struggle to efficiently and effectively use self-regulated learning (SRL) strategies to attain goals and subgoals.There is a need for SRL to be scaffolded for learners to manage multiple goals and subgoals while learning about complex STEM topics.Intelligent tutoring systems (ITSs) typically incorporate pedagogical agents (PAs) to prompt learners to engage in SRL strategy and provide feedback.There are mixed findings on the effectiveness of PAs in scaffolding learners' SRL.We consider PAs not only scaffolders but also teachers of SRL.Results showed that while PAs encouraged the use of SRL strategies when the content was relevant to subgoals, they did not discourage the use of SRL strategies when the content was not relevant.Results for this study were mixed in their support of PAs as teachers of SRL.Learners increasingly depended on PAs to prompt SRL strategies as time on task progressed.PAs are effective scaffolders of SRL with more research needed to understand their role as teachers of SRL.PA scaffolding is more essential as time on task progresses.When deploying specific cognitive and metacognitive SRL strategies, the relevance of the content to learners' subgoals should be taken into account.

Self-regulated learning (SRL), or the ability for a learner to monitor and change their cognitive, affective, metacognitive, and motivational processes, is a critical skill to enact, especially while learning about difficult topics within an intelligent tutoring system (ITS). Learners' enactment of SRL behaviors during learning with ITSs has been extensively studied within the human-computer interaction field but few studies have examined the extent to which learners' SRL behaviors quantitatively demonstrate a functional system (i.e., equilibrium of repetitive and novel behaviors). However, current analytical approaches do not evaluate how the functionality of learners' SRL behaviors unfolds as time on task progresses. This paper reviews two analytical approaches, both based within categorical auto-recurrence quantification analysis (aRQA), for examining how learners' SRL complex behaviors emerge during learning with an ITS. The first approach, binned categorical aRQA, segments learners' SRL behaviors into bins and performs categorical aRQA on the SRL behaviors enacted within those bins to produce metrics of complexity that demonstrate how learners' functionality of their SRL systems change over time. The second approach, cumulative categorical aRQA, continuously calculates complexity metrics as learners enact SRL behaviors to identify the evolution of learners' functional SRL. These two approaches allow researchers to identify how the functionality of SRL behaviors change over time in relationship to the occurrences within the ITS environment. From this discussion, we provide actionable implications for contributing to how learners' SRL functionality can be visualized and scaffolded during learning with an ITS.

Game-based learning environments (GBLEs) supplement classroom instruction so students can demonstrate their scientific reasoning abilities and increase knowledge, providing a platform that promotes interest and engagement in science. The goal of this study was to examine the effectiveness of game mechanics for science learning. This study identifies how two types of game mechanics-learning and assessment mechanics-are used by high school participants (N = 137) as they learn about microbiology with Crystal Island, a game-based learning environment for science education. Participants' learning outcomes were evaluated in two ways: learning gains, which assessed participants' domain knowledge acquisition, and game completion, which assessed participants' ability to successfully demonstrate scientific reasoning abilities. Results from this study showed that game completion is not related to learning gains. However, as participants engaged with increasingly more assessment mechanics, learning gains decreased. Further, profiles of learners were extracted to better understand the learning process that best supports greater learning outcomes. Results showed that learners who engaged in less recurrent transitions across assessment mechanics were more likely to successfully demonstrate scientific reasoning abilities. Implications for the design of games which provide scaffolding based on process data of learners' game mechanic use are provided.

Introduction: Self-regulated learning (SRL), or learners’ ability to monitor and change their own cognitive, affective, metacognitive, and motivational processes, encompasses several operations that should be deployed during learning including Searching, Monitoring, Assembling, Rehearsing, and Translating (SMART). Scaffolds are needed within GBLEs to both increase learning outcomes and promote the accurate and efficient use of SRL SMART operations. This study aims to examine how restricted agency (i.e., control over one’s actions) can be used to scaffold learners’ SMART operations as they learn about microbiology with Crystal Island, a game-based learning environment. Methods: Undergraduate students (N = 94) were randomly assigned to one of two conditions: (1) Full Agency, where participants were able to make their own decisions about which actions they could take; and (2) Partial Agency, where participants were required to follow a pre-defined path that dictated the order in which buildings were visited, restricting one’s control. As participants played Crystal Island, participants’ multimodal data (i.e., log files, eye tracking) were collected to identify instances where participants deployed SMART operations. Results: Results from this study support restricted agency as a successful scaffold of both learning outcomes and SRL SMART operations, where learners who were scaffolded demonstrated more efficient and accurate use of SMART operations. Discussion: This study provides implications for future scaffolds to better support SRL SMART operations during learning and discussions for future directions for future studies scaffolding SRL during game-based learning. Copyright © 2023 Dever, Wiedbusch, Romero, Smith, Patel, Sonnenfeld, Lester and Azevedo.

Reflection plays a critical role in learning. Game-based learning environments have significant potential to elicit and support student reflection by prompting learners to think critically about their own learning processes and performance. Stealth assessment models, used for unobtrusively assessing student competencies from evidence of game interaction data and facilitating learning through adaptive feedback, can be enhanced by incorporating evidence from students' written reflections. We present a deep learning-based stealth assessment framework that predicts depth of student reflections and science content post-test scores during game-based learning. With the increasing adoption of AI techniques in decision-making processes, it is important to evaluate the fairness of these models. To address this concern, we investigate encoded bias in our stealth assessment model with respect to student gender and prior game-playing experience in deep learning-based stealth assessment models and examine the impact of debiasing on the models' predictive performance. We evaluate the predictive performance of the deep learning-based stealth assessment models and measure encoded bias with the Absolute Between-ROC Area (ABROCA) statistic using gameplay data from 119 students collected in a series of classroom studies with a reflection-enriched game-based learning environment for middle school microbiology, Crystal Island . The results demonstrate the effectiveness of deep learning-based stealth assessment models and multiple debiasing techniques for deriving algorithmically fair stealth assessment models.

Game-based learning environments have the distinctive capacity to promote learning experiences that are both engaging and effective. Recent advances in sensor-based technologies (e.g., facial expression analysis and eye gaze tracking) and natural language processing have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics and student modeling informed by multimodal data captured during students' interactions with game-based learning environments hold significant promise for designing effective learning environments that detect unproductive student behaviors and provide adaptive support for students during learning. Learning analytics frameworks that can accurately predict student learning outcomes early in students' interactions hold considerable promise for enabling environments to dynamically adapt to individual student needs. In this paper, we investigate a multimodal, multi-task predictive student modeling framework for game-based learning environments. The framework is evaluated on two datasets of game-based learning interactions from two student populations (n=61 and n=118) who interacted with two versions of a game-based learning environment for microbiology education. The framework leverages available multimodal data channels from the datasets to simultaneously predict student post-test performance and interest. In addition to inducing models for each dataset individually, this work investigates the ability to use information learned from one source dataset to improve models based on another target dataset (i.e., transfer learning using pre-trained models). Results from a series of ablation experiments indicate the differences in predictive capacity among a combination of modalities including gameplay, eye gaze, facial expressions, and reflection text for predicting the two target variables. In addition, multi-task models were able to improve predictive performance compared to single-task baselines for one target variable, but not both. Lastly, transfer learning showed promise in improving predictive capacity in both datasets.

Learning sciences are embracing the significant role technologies can play to better detect, diagnose, and act upon self-regulated learning (SRL). The field of SRL is challenged with the measurement of SRL processes to advance our understanding of how multimodal data can unobtrusively capture learners' cognitive, metacognitive, affective, and motivational states over time, tasks, domains, and contexts. This paper introduces a self-regulated learning processes, multimodal data, and analysis (SMA) grid and maps joint and individual research of the authors (63 papers) over the last five years onto the grid. This shows how multimodal data streams were used to investigate SRL processes. The two-dimensional space on the SMA grid is helpful for visualizing the relations and possible combinations between the data streams and how the measurement of SRL processes. This overview serves as an analytical introduction to the current special issue “Advancing SRL Research with Artificial Intelligence (AI)” and we encourage to position new research and unexplored frontiers. We emphasize the need for intensive and strategic collaboration to accelerate progress using new interdisciplinary methods to develop accurate measurement of SRL in educational technologies. © 2022 The Authors

Self-regulated learning (SRL), learners' monitoring and control of cognitive, affective, metacognitive, and motivational processes, is essential for learning. However, cognitive and metacognitive SRL strategies are not typically used accurately leading to poor learning outcomes. Intelligent tutoring systems (ITSs) attempt to address this issue by prompting and scaffolding learners to engage in SRL via using pedagogical agents. However, current literature does not examine the extent to which learners' deployed strategies are functional or dysfunctional in relation to pedagogical agent scaffolding. The current study collected 117 undergraduate students' data as they learned with MetaTutor, an ITS about the human circulatory system. Participants were randomly assigned to either the (1) Prompt and Feedback Condition where pedagogical agents scaffolded cognitive and metacognitive SRL strategies or (2) Control Condition where no prompts or feedback were provided. Results demonstrated that learners who received prompts by the pedagogical agents to engage in SRL had higher learning gains as well as greater frequencies across most strategies compared to those in the Control Condition who relied on self-initiated strategy use. While sequential transitions across all strategies were not significant between conditions, further analysis grounded in Complex Systems Theory found that learners who were prompted to engage in strategies demonstrated a significantly lower degree of repetition and balance between repetitive and novel patterns of strategy use. The findings suggest that pedagogical agents within MetaTutor successfully scaffolded the functional deployment of cognitive and metacognitive SRL strategies and are indicative of higher learning after interacting with ITSs.