Scientific reasoning is a critical foundational skill learners need to practice and know for increased science learning outcomes. Game-based learning environments (GBLEs) provide learners a platform for developing and practicing scientific reasoning skills but little is known about how learners should engage in scientific reasoning during game-based learning. As such, this paper aimed to understand if and how learners engaged in effective scientific reasoning activities during learning with a GBLE. This paper used analytical techniques from Complex Systems Theory to quantify learners’ scientific reasoning actions during game-based learning. High-school students (N = 170) played Crystal Island, a microbiology GBLE requiring learners to engage in scientific reasoning to successfully identify an illness infecting residents of a virtual island. Categorical auto-Recurrence Quantification Analysis was run on participants’ log files as they deployed scientific reasoning actions. This analysis revealed several metrics of complexity, including recurrence rate which is the proportion of repetitive to novel scientific reasoning actions. Results found that as time progressed, recurrence rates decreased. Successful learners (i.e., those who solved the mystery) demonstrated less repetition in their scientific reasoning activities where their recurrence rates decreased at a slower rate over time than learners who were unsuccessful in solving the mystery. Findings provide implications for adaptively scaffolding learners’ emerging complexities in their scientific reasoning processes during game-based learning to increase learners’ GBLE success. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

The process of setting goals and creating plans is crucial for self-regulated learning (SRL), yet students often struggle to construct efficient plans and establish goals. Adaptive learning environments hold promise for assisting students with such processes through adaptive scaffolding. Through the examination of data collected from 144 middle school students, we present a data-driven analysis of students' explicit planning activities in Crystal Island, a narrative game-based learning environment. In this game, students are provided with a planning support tool that aids them in externalizing their science-related goals and plans before putting them into action. We extracted features from their planning tool use and connected them to several SRL processes and problem-solving outcomes. We found that students who engaged with the planning support tool were more likely to successfully complete the learning scenario. To investigate the potential for adaptive support with this tool, we also constructed a student plan recognition framework aimed at predicting students' goals and planned action sequences. This framework uses student gameplay sequences as input and student interactions with the planning tool as labels for both prediction tasks. We evaluated these tasks using six machine learning models and found that all approaches improved on the majority baseline classification performance. We then investigated additional machine-learning architectures and a technique for detecting when students enact all steps in their plans as methods for improving the framework. We demonstrated performance improvement with these enhancements. Overall, results demonstrated that the planning support tool can help students engage in SRL activities and drive adaptive support in real time.

Game-based learning environments (GBLEs) supplement classroom instruction so students can demonstrate their scientific reasoning abilities and increase knowledge, providing a platform that promotes interest and engagement in science. The goal of this study was to examine the effectiveness of game mechanics for science learning. This study identifies how two types of game mechanics-learning and assessment mechanics-are used by high school participants (N = 137) as they learn about microbiology with Crystal Island, a game-based learning environment for science education. Participants' learning outcomes were evaluated in two ways: learning gains, which assessed participants' domain knowledge acquisition, and game completion, which assessed participants' ability to successfully demonstrate scientific reasoning abilities. Results from this study showed that game completion is not related to learning gains. However, as participants engaged with increasingly more assessment mechanics, learning gains decreased. Further, profiles of learners were extracted to better understand the learning process that best supports greater learning outcomes. Results showed that learners who engaged in less recurrent transitions across assessment mechanics were more likely to successfully demonstrate scientific reasoning abilities. Implications for the design of games which provide scaffolding based on process data of learners' game mechanic use are provided.

Intelligent tutoring systems (ITSs) incorporate pedagogical agents (PAs) to scaffold learners' self-regulated learning (SRL) via prompts and feedback to promote learners' monitoring and regulation of their cognitive, affective, metacognitive and motivational processes to achieve their (sub)goals. This study examines PAs' effectiveness in scaffolding and teaching SRL during learning with MetaTutor, an ITS on the human circulatory system. Undergraduates (N = 118) were randomly assigned to a condition: Control Condition (i.e. learners could only self-initiate SRL strategies) and Prompt and Feedback Condition (i.e. PAs prompted learners to engage in SRL). Learners' log-file data captured when strategies were used, the initiator of the strategy (i.e. learner and PA), and the relevance of instructional content pages in relation to learner subgoals. While results showed that PAs were effective scaffolders of SRL in which they prompted learners to engage in SRL strategies more when content was relevant towards their subgoals and as time on page and task increased, there were mixed findings about the effectiveness of PAs as teachers of SRL. Findings show how production rules guiding PA prompts can improve their scaffolding and teaching of SRL across the learning task - through contextualizing SRL strategies to the instructional content and in relation to the relevance of the content to learners' subgoals.What is already known about this topicWhat this paper addsImplications for practice and/or policyPractitioner notesMost learners struggle to efficiently and effectively use self-regulated learning (SRL) strategies to attain goals and subgoals.There is a need for SRL to be scaffolded for learners to manage multiple goals and subgoals while learning about complex STEM topics.Intelligent tutoring systems (ITSs) typically incorporate pedagogical agents (PAs) to prompt learners to engage in SRL strategy and provide feedback.There are mixed findings on the effectiveness of PAs in scaffolding learners' SRL.We consider PAs not only scaffolders but also teachers of SRL.Results showed that while PAs encouraged the use of SRL strategies when the content was relevant to subgoals, they did not discourage the use of SRL strategies when the content was not relevant.Results for this study were mixed in their support of PAs as teachers of SRL.Learners increasingly depended on PAs to prompt SRL strategies as time on task progressed.PAs are effective scaffolders of SRL with more research needed to understand their role as teachers of SRL.PA scaffolding is more essential as time on task progresses.When deploying specific cognitive and metacognitive SRL strategies, the relevance of the content to learners' subgoals should be taken into account.

Self-regulated learning (SRL), or the ability for a learner to monitor and change their cognitive, affective, metacognitive, and motivational processes, is a critical skill to enact, especially while learning about difficult topics within an intelligent tutoring system (ITS). Learners' enactment of SRL behaviors during learning with ITSs has been extensively studied within the human-computer interaction field but few studies have examined the extent to which learners' SRL behaviors quantitatively demonstrate a functional system (i.e., equilibrium of repetitive and novel behaviors). However, current analytical approaches do not evaluate how the functionality of learners' SRL behaviors unfolds as time on task progresses. This paper reviews two analytical approaches, both based within categorical auto-recurrence quantification analysis (aRQA), for examining how learners' SRL complex behaviors emerge during learning with an ITS. The first approach, binned categorical aRQA, segments learners' SRL behaviors into bins and performs categorical aRQA on the SRL behaviors enacted within those bins to produce metrics of complexity that demonstrate how learners' functionality of their SRL systems change over time. The second approach, cumulative categorical aRQA, continuously calculates complexity metrics as learners enact SRL behaviors to identify the evolution of learners' functional SRL. These two approaches allow researchers to identify how the functionality of SRL behaviors change over time in relationship to the occurrences within the ITS environment. From this discussion, we provide actionable implications for contributing to how learners' SRL functionality can be visualized and scaffolded during learning with an ITS.

A key aim for science education is the improvement of scientific reasoning through inquiry-based learning which asks students to think and act like scientists. This requires the regulation of specific skills and abilities such as identifying problems, generating hypotheses and evidence, and drawing conclusions. Goal setting and planning can help learners regulate their learning as they engage with scientific inquiry, especially within investigative exploration. Contemporary work on scaffolds for science inquiry-learning requires we (1) understand how students set goals and plans, (2) measure the quality of goals and plans, and (3) develop adaptive intelligent goal and planning scaffolds. This paper presents the development of a new planning scaffold used by 101 middle school students during interactions with CRYSTAL ISLAND, a game-based learning environment designed to teach students about microbiology and through scientific inquiry. We map student goals and plans from the scaffold to a series of epistemic scientific reasoning activities to use in conjunction with online trace data of student behaviors to analyze student goal setting and planning constructed throughout the game. This study describes the development of an analytical mapping approach between (sub)goals and (sub)activities to measure student plan quality build using a planning scaffold. We report on the use of the scaffold as well as implications for future development of an adaptive and intelligent version of the tool. This work highlights that online-trace data should be contextualized to (sub)goals, and the design of intelligent and adaptive goal and planning tools should be dynamic to account for open-ended exploration that differs across learners.

The development of large language models offers new possibilities for enhancing adaptive scaffolding of student learning in game-based learning environments. In this work, we present a novel framework for automatic plan generation that utilizes text-based representations of students' actions within a game-based learning environment, Crystal Island, to inform adaptive scaffolding of student goal setting and planning, which are critical elements of self-regulated learning. Plan generation is the task of automatically generating a set of low-level actions that contribute toward accomplishing a target goal given a sequence of student gameplay and their prior completed goals. We investigate the use of two pre-trained large language models, T5 and GPT-3.5, in the plan generation framework. The models utilize 144 middle school students gameplay data, encompassing a total of 11,610 event sequences, as input. The plans generated by the model are subsequently evaluated against plans crafted by students during gameplay utilizing an in-game planning support tool in Crystal Island. We compare automatically generated plans to students' manually generated in terms of the number of matching low-level actions, the number of actions that match when mapped to higher-level categories of actions, and the distribution of categories of actions within plans. Results indicate that automatically generated plans from both models largely align in terms of the high-level categories of actions that are included, but the generated plans feature fewer low-level actions than students' plans. Plans generated by T5 align more closely with student plans, whereas GPT-3.5, though not following student planning patterns, produces valid plans as well. These findings suggest that LLMs show significant promise for automatically generating plans that can be used to devise run-time adaptive scaffolding for student planning in game-based learning environments.

Introduction: Self-regulated learning (SRL), or learners’ ability to monitor and change their own cognitive, affective, metacognitive, and motivational processes, encompasses several operations that should be deployed during learning including Searching, Monitoring, Assembling, Rehearsing, and Translating (SMART). Scaffolds are needed within GBLEs to both increase learning outcomes and promote the accurate and efficient use of SRL SMART operations. This study aims to examine how restricted agency (i.e., control over one’s actions) can be used to scaffold learners’ SMART operations as they learn about microbiology with Crystal Island, a game-based learning environment. Methods: Undergraduate students (N = 94) were randomly assigned to one of two conditions: (1) Full Agency, where participants were able to make their own decisions about which actions they could take; and (2) Partial Agency, where participants were required to follow a pre-defined path that dictated the order in which buildings were visited, restricting one’s control. As participants played Crystal Island, participants’ multimodal data (i.e., log files, eye tracking) were collected to identify instances where participants deployed SMART operations. Results: Results from this study support restricted agency as a successful scaffold of both learning outcomes and SRL SMART operations, where learners who were scaffolded demonstrated more efficient and accurate use of SMART operations. Discussion: This study provides implications for future scaffolds to better support SRL SMART operations during learning and discussions for future directions for future studies scaffolding SRL during game-based learning. Copyright © 2023 Dever, Wiedbusch, Romero, Smith, Patel, Sonnenfeld, Lester and Azevedo.

Self-regulated learning (SRL), learners' monitoring and control of cognitive, affective, metacognitive, and motivational processes, is essential for learning. However, cognitive and metacognitive SRL strategies are not typically used accurately leading to poor learning outcomes. Intelligent tutoring systems (ITSs) attempt to address this issue by prompting and scaffolding learners to engage in SRL via using pedagogical agents. However, current literature does not examine the extent to which learners' deployed strategies are functional or dysfunctional in relation to pedagogical agent scaffolding. The current study collected 117 undergraduate students' data as they learned with MetaTutor, an ITS about the human circulatory system. Participants were randomly assigned to either the (1) Prompt and Feedback Condition where pedagogical agents scaffolded cognitive and metacognitive SRL strategies or (2) Control Condition where no prompts or feedback were provided. Results demonstrated that learners who received prompts by the pedagogical agents to engage in SRL had higher learning gains as well as greater frequencies across most strategies compared to those in the Control Condition who relied on self-initiated strategy use. While sequential transitions across all strategies were not significant between conditions, further analysis grounded in Complex Systems Theory found that learners who were prompted to engage in strategies demonstrated a significantly lower degree of repetition and balance between repetitive and novel patterns of strategy use. The findings suggest that pedagogical agents within MetaTutor successfully scaffolded the functional deployment of cognitive and metacognitive SRL strategies and are indicative of higher learning after interacting with ITSs.

Goal setting and planning are integral components of self-regulated learning. Many students struggle to set meaningful goals and build relevant plans. Adaptive learning environments show significant potential for scaffolding students' goal setting and planning processes. An important requirement for such scaffolding is the ability to perform student plan recognition, which involves recognizing students' goals and plans based upon the observations of their problemsolving actions. We introduce a novel plan recognition framework that leverages trace log data from student interactions within a game-based learning environment called CRYSTAL ISLAND, in which students use a drag-and-drop planning support tool that enables them to externalize their science problem-solving goals and plans prior to enacting them in the learning environment. We formalize student plan recognition in terms of two complementary tasks: (1) classifying students' selected problem-solving goals, and (2) classifying the sequences of actions that students indicate will achieve their goals. Utilizing trace log data from 144 middle school students' interactions with CRYSTAL ISLAND, we evaluate a range of machine learning models for student goal and plan recognition. All machine learning-based techniques outperform the majority baseline, with LSTMs outperforming other models for goal recognition and naive Bayes performing best for plan recognition. Results show the potential for automatically recognizing students' problem-solving goals and plans in game-based learning environments, which has implications for providing adaptive support for student self-regulated learning.