Reflection plays a critical role in learning by encouraging students to contemplate their knowledge and previous learning experiences to inform their future actions and higher-order thinking, such as reasoning and problem solving. Reflection is particularly important in inquiry-driven learning scenarios where students have the freedom to set goals and regulate their own learning. However, despite the importance of reflection in learning, there are significant theoretical, methodological, and analytical challenges posed by measuring, modeling, and supporting reflection. This paper presents results from a classroom study to investigate middle-school students' reflection during inquiry-driven learning with Crystal Island, a game-based learning environment for middle-school microbiology. To collect evidence of reflection during game-based learning, we used embedded reflection prompts to elicit written reflections during students' interactions with Crystal Island. Results from analysis of data from 105 students highlight relationships between features of students' reflections and learning outcomes related to both science content knowledge and problem solving. We consider implications for building adaptive support in game-based learning environments to foster deep reflection and enhance learning, and we identify key features in students' problem-solving actions and reflections that are predictive of reflection depth. These findings present a foundation for providing adaptive support for reflection during game-based learning.

The goal of this study was to investigate 65 students' evidence scores of emotions while they engaged in cognitive and metacognitive self-regulated learning processes as they learned about the circulatory system with MetaTutor, a hypermedia-based intelligent tutoring system. We coded for the accuracy of detecting students' cognitive and metacognitive processes, and examined how the computed scores related to mean evidence scores of emotions and overall learning. Results indicated that mean evidence score of surprise negatively predicted the accuracy of making a metacognitive judgment, and mean evidence score of frustration positively predicted the accuracy of taking notes, a cognitive learning strategy. These results have implications for understanding the beneficial role of negative emotions during learning with advanced learning technologies. Future directions include providing students with feedback about the benefits of both positive and negative emotions during learning and how to regulate specific emotions to ensure the most effective learning experience with advanced learning technologies.

Game-based learning environments are designed to provide effective and engaging learning experiences for students. Predictive student models use trace data extracted from students' in-game learning behaviors to unobtrusively generate early assessments of student knowledge and skills, equipping game-based learning environments with the capacity to anticipate student outcomes and proactively deliver adaptive scaffolding or notify instructors. Reflection is a key component of self-regulated learning, and it is critical in effective learning. However, there is currently limited work exploring the utility of reflection for inducing accurate predictive student models. This article presents a predictive student modeling framework that leverages natural language responses to in-game reflection prompts to predict student learning outcomes in a game-based learning environment for middle school microbiology, CRYSTAL ISLAND. With data from a pair of classroom studies involving 118 middle school students, we investigate the accuracy of early prediction models that utilize features extracted from student trace data combined with word embedding-based representations (i.e., GloVe, ELMo) of student reflection responses. We evaluate the accuracy of the predictive models over time using data from incremental segments of each student's interaction with the game-based learning environment, and we compare against models that omit student reflection features. Results reveal that models encoding students' natural language reflections with ELMo word embeddings yield significantly improved accuracy compared to other representations, with the greatest accuracy demonstrated by an ensemble of predictive models. We discuss the implications of these results for the design of game-based learning environments.

Reflection is critical for adolescents' problem solving and learning in game-based learning environments (GBLEs). Yet challenges exist in the literature because most studies lack a theoretical perspective and clear operational definition to inform how and when reflection should be scaf-folded during game-based learning. In this paper, we address these issues by studying the quantity and quality of 120 adolescents' written reflections and their relation to their learning and problem solving with Crystal Island, a GBLE. Specifically, we (1) define reflection and how it relates to skill and knowledge acquisition; (2) review studies examining reflection and its relation to problem solving and learning with emerging technologies; and (3) provide direction for building reflection prompts into GBLEs that are aligned with the learning goals built into the learning session (e.g., learn about microbiology versus successfully solve a problem) to maximize adolescents' reflection, learning, and performance. Overall, our findings emphasize how important it is to examine not only the quantity of reflection but also the depth of written reflection as it relates to specific learning goals. We discuss the implications of using game-learning analytics to guide instructional decision making in the classroom.

To effectively process complex information within intelligent tutoring systems (ITSs), learners are required to engage in metacognitive monitoring micro-processes (content evaluations [CEs], judgments of learning [JOLs], feelings of knowing [FOKs], and monitoring progress towards goals [MPTGs]). Learners' average monitoring micro-process strategy frequencies were used to examine learning gains using a person-centered approach as they interacted with MetaTutor. Undergraduates (n = 94) engaged in self-initiated and system-facilitated self-regulated learning (SRL) strategies as they studied the human circulatory system with MetaTutor, a hypermedia-based ITS. Using hierarchical clustering, results showed a difference in learning between clusters differing in metacognitive monitoring process usage. Specifically, learners who used both CEs and FOKs for a greater proportion of monitoring strategy usage had significantly greater learning gains than learners who used MPTGs. Implications for monitoring strategy usage across different micro-processes and the development of ITSs to facilitate and scaffold learners' interactions with these micro-processes via prompting are discussed.

Self-regulated learning (SRL) with advanced learning technologies has shown to significantly augment learners' performance across contexts. Yet studies find learners lack sufficient SRL skills to successfully implement strategies (e.g., judgments of learning, note taking, self-testing, etc.). Current research does not fully explain how and why this failure of effective strategy deployment occurs. We used principle component analysis (PCA) on process data (i.e., log files) from 190 undergraduates learning with MetaTutor, a hypermedia-based intelligent tutoring system, to explore underlying patterns in the frequency of strategy deployment occurring with and without pedagogical agent scaffolding to better understand any underlying structures of system- and learner-initiated cognitive and metacognitive SRL strategy use. Results showed that the system's underlying architecture deploys processes corresponding to both the phases of learning and type of effort allocation according to Winne's (2018) Information Processing Theory of SRL. However, learner-initiated processes for those who received scaffolding only displayed strategy deployment that corresponded to the type of effort allocation required of the processes (i.e., more effortful constructionist processes like note-taking versus short canned responses for judgements of learning). Additionally, results suggest all learners deploy strategies based on the familiarity of processes. Regression models using these principle components outperformed raw frequency models for capturing post-test learning performance across all participants.

This chapter focuses on the challenges of measuring self-regulatory processes and learning outcomes while learning from multiple representations with advanced learning technologies (ALTs). More specifically, we present challenges associated with measuring learners' processes and outcomes while using multiple representations with ALTs. ALTs are technology-based learning and training systems designed to teach learners of all ages to learn, reason, problem solve, and understand different topics and domains such as physics, math, history, biology, medicine and can include different systems such as multimedia, hypermedia, serious games, simulations, intelligent tutoring systems, immersive virtual environments and so forth. The structure of our chapter is as follows: (1) a brief overview of the major issues related to using multiple representations for learning with ALTs; (2) a description of the analytical techniques for understanding multiple representations and their goals for successful learning with ALTs; (3) a discussion of the issues associated with using multiple representations to measure and assess learning outcomes and how we can use ALTs as research tools to capture self-regulated learning; and (4) a presentation of challenges and future directions that need to be addressed by researchers.

[No abstract available]

Game-based learning environments are designed to foster high levels of student engagement and motivation during learning of complex topics. Game-based learning environments allow students freedom to navigate a space to interact with game elements that foster learning, i.e., agency. Agency has been studied in learning, and it has been demonstrated that increased student agency results in greater learning outcomes. However, it is unclear what is the level of agency that is required to demonstrate this effect, and whether this effect applies only to learning or to problem solving and affect during game-based learning as well. To investigate how the level of student agency impacts learning, problem solving, and affect, a study was conducted with 138 college students interacting with a game-based learning environment for microbiology, Crystal Island. This study is an extension of a previous study that examined the impact of agency on learning and problem-solving behaviors during game-based learning with Crystal Island. Students were randomly assigned to either a High Agency condition, a Low Agency condition, or a No Agency condition. It was found that students in the Low Agency condition achieved significantly higher normalized learning gain scores than students in the No Agency condition, and marginally higher normalized learning gains than the High Agency condition. Post-surveys of interest and presence indicated that students in the No Agency condition were less interested, and perceived themselves as less present in the virtual environment, than students in the other conditions. Students in the No Agency condition also experienced less frustration, confusion, and joy than the other agency conditions, indicating a less cognitively stimulating experience. Overall the results indicate that a moderate degree of agency provided to students in game-based learning environments leads to better learning outcomes without sacrificing interest and without yielding a negative emotional experience, demonstrating how even low levels of agency can positively impact learning, problem solving, and affect during game-based learning. © 2019 Elsevier Ltd

Game-based learning environments enable students to engage in authentic, inquiry-based learning. Reflective thinking serves a critical role in inquiry-based learning by encouraging students to think critically about their knowledge and experiences in order to foster deeper learning processes. Freeresponse reflection prompts can be embedded in game-based learning environments to encourage students to engage in reflection and externalize their reflection processes, but automatically assessing student reflection presents significant challenges. In this paper, we present a framework for automatically assessing students' written reflection responses during inquiry-based learning in Crystal Island, a game-based learning environment for middle school microbiology. Using data from a classroom study involving 153 middle school students, we compare the effectiveness of several computational representations of students' natural language responses to reflection prompts-GloVe, ELMo, tf-idf, unigrams-across severalmachine learning-based regression techniques (i.e., random forest, support vector machine, multi-layer perceptron) to assess the depth of student reflection responses. Results demonstrate that assessment models based on ELMo deep contextualized word representations yield more accurate predictions of students' written reflection depth than competing techniques. These findings point toward the potential of leveraging automated assessment of student reflection to inform real-time adaptive support for inquiry-based learning in game-based learning environments.