A key aim for science education is the improvement of scientific reasoning through inquiry-based learning which asks students to think and act like scientists. This requires the regulation of specific skills and abilities such as identifying problems, generating hypotheses and evidence, and drawing conclusions. Goal setting and planning can help learners regulate their learning as they engage with scientific inquiry, especially within investigative exploration. Contemporary work on scaffolds for science inquiry-learning requires we (1) understand how students set goals and plans, (2) measure the quality of goals and plans, and (3) develop adaptive intelligent goal and planning scaffolds. This paper presents the development of a new planning scaffold used by 101 middle school students during interactions with CRYSTAL ISLAND, a game-based learning environment designed to teach students about microbiology and through scientific inquiry. We map student goals and plans from the scaffold to a series of epistemic scientific reasoning activities to use in conjunction with online trace data of student behaviors to analyze student goal setting and planning constructed throughout the game. This study describes the development of an analytical mapping approach between (sub)goals and (sub)activities to measure student plan quality build using a planning scaffold. We report on the use of the scaffold as well as implications for future development of an adaptive and intelligent version of the tool. This work highlights that online-trace data should be contextualized to (sub)goals, and the design of intelligent and adaptive goal and planning tools should be dynamic to account for open-ended exploration that differs across learners.

This work investigates how tutoring discourse interacts with students' proximal knowledge to explain and predict students' learning outcomes. Our work is conducted in the context of high-dosage human tutoring where 9th-grade students (N = 1080) attended small group tutorials and individually practiced problems on an Intelligent Tutoring System (ITS). We analyzed whether tutors' talk moves and students' performance on the ITS predicted scores on math learning assessments. We trained Random Forest Classifiers (RFCs) to distinguish high and low assessment scores based on tutor talk moves, student's ITS performance metrics, and their combination. A decision tree was extracted from each RFC to yield an interpretable model. We found AUCs of 0.63 for talk moves, 0.66 for ITS, and 0.77 for their combination, suggesting interactivity among the two feature sources. Specifically, the best decision tree emerged from combining the tutor talk moves that encouraged rigorous thinking and students' ITS mastery. In essence, tutor talk that encouraged mathematical reasoning predicted achievement for students who demonstrated high mastery on the ITS, whereas tutors' revoicing of students' mathematical ideas and contributions was predictive for students with low ITS mastery. Implications for practice are discussed.

We conducted a 2 x 2 study comparing the digital learning game Decimal Point to a comparable non-game tutor with or without self-explanation prompting. We expected to replicate previous studies showing the game improved learning compared to the non-game tutor, and that self-explanation prompting would enhance learning across platforms. Additionally, prior research with Decimal Point suggested that self-explanation was driving gender differences in which girls learned more than boys. To better understand these effects, we manipulated the presence of self-explanation prompts and incorporated a multidimensional gender measure. We hypothesized that girls and students with stronger feminine-typed characteristics would learn more than boys and students with stronger masculine-typed characteristics in the game with self-explanation condition, but not in the game without self-explanation or in the non-game conditions. Results showed no advantage for the game over the non-game or for including self-explanation, but an analysis of hint usage indicated that students in the game conditions used (and abused) hints more than in the non-game conditions, which in turn was associated with worse learning outcomes. When we controlled for hint use, students in the game conditions learned more than students in the non-game tutor. We replicated a gender effect favoring boys and students with masculine-typed characteristics on the pretest, but there were no gender differences on the posttests. Finally, results indicated that the multidimensional framework explained variance in pretest performance better than a binary gender measure, adding further evidence that this framework may be a more effective, inclusive approach to understanding gender effects in game-based learning.

Since GPT-4's release it has shown novel abilities in a variety of domains. This paper explores the use of LLM-generated explanations as on-demand assistance for problems within the ASSISTments platform. In particular, we are studying whether GPT-generated explanations are better than nothing on problems that have no supports and whether GPT-generated explanations are as good as or better than teacher-authored explanations. This study contributes to existing literature since as of yet, there are no studies on the scale of ASSISTments evaluating the effectiveness of GPT support in education. Should GPT explanations prove effective then we plan to continue developing and evaluating explanations, hints, and other supports with GPT within ASSISTments.

A student-centered intelligent monitoring system collects data from students and provides insights and feedback to students about various aspects of the learning process. It often leverages data collected from educational technology systems, such as Learning Management Systems (LMS), to support students' self-regulated learning. A well-designed system needs to strike a delicate balance between the power of machine intelligence (e.g., automatic characterization and inference of students' behaviors) and the need to promote students' agency (e.g., the desire to be responsible and take control of their own behaviors). This paper presents a comprehensive qualitative analysis of anonymous student survey data collected from over 500 students with experience with a Learning Activity Monitoring System (LAMS), which has been in operation for over a decade in a public minority-serving higher education institute in the US. The study offers valuable insights into the effectiveness and user perceptions of LAMS they use in their learning context. This analysis reveals the sense-making process or lack thereof, and its implication in exploiting the potential utility of the LAMS. The findings also highlight students' varied expectations and requirements, providing critical insights for the ongoing development and refinement of the LAMS system toward an intelligent monitoring system that truly centers students' agency and promotes self-regulated learning. This study contributes to the growing body of work in hearing and understanding students' genuine voices and the sparse literature of large-scale qualitative analysis of students' feedback during the post-deployment phase on student-facing data-driven monitoring systems in an ecologically valid context in higher education.

This study investigates the effects on different racial/ethnic groups of middle school students when learning with a digital learning game, Decimal Point, and a comparable computer tutor. Using data from three classroom studies with 835 students, we compared learning outcomes and engagement among students from racial/ethnic groups that are well-represented in STEM (white and Asian) to those that are underrepresented in STEM (Black, Hispanic/Latine, Indigenous, and multiracial). Relative to students from underrepresented groups, students from well-represented groups in STEM scored higher on all tests (pre, post, and delayed, despite similar learning gains from pre-to-post and pre-to-delayed) and showed more engagement and less anxiety. The game also enhanced the experience of mastery only among students from well-represented groups. At the same time, students from underrepresented groups learned from the intervention and matched students from well-represented groups in learning efficiency. In short, we found similar learning gains from the game and tutor interventions among students from well-represented and underrepresented racial/ethnic groups, despite the lower performance and lower engagement among students from underrepresented groups. These insights highlight how students from diverse backgrounds may engage differently with educational technology, guiding future efforts in making Decimal Point - as well as digital learning tools in general - more inclusive.

Recent advancements in text-to-image (T2I) diffusion models have demonstrated their remarkable capability to generate high-quality images from textual prompts. However, increasing research indicates that these models memorize and replicate images from their training data, raising concerns about potential copyright infringement and privacy risks. In our study, we provide a novel perspective to understand this memorization phenomenon by examining its relationship with cross-attention mechanisms. We reveal that during memorization, the cross-attention tends to focus disproportionately on the embeddings of specific tokens. The diffusion model is overfitted to these token embeddings, memorizing corresponding training images. To elucidate this phenomenon, we further identify and discuss various intrinsic findings of cross-attention that contribute to memorization. Building on these insights, we introduce an innovative approach to detect and mitigate memorization in diffusion models. The advantage of our proposed method is that it will not compromise the speed of either the training or the inference processes in these models while preserving the quality of generation. Our code is available at github.com/renjie3/MemAttn.

Identifying and annotating student use of debugging strategies when solving computer programming problems can be a meaningful tool for studying and better understanding the development of debugging skills, which may lead to the design of effective pedagogical interventions. However, this process can be challenging when dealing with large datasets, especially when the strategies of interest are rare but important. This difficulty lies not only in the scale of the dataset but also in operationalizing these rare phenomena within the data. Operationalization requires annotators to first define how these rare phenomena manifest in the data and then obtain a sufficient number of positive examples to validate that this definition is reliable by accurately measuring Inter-Rater Reliability (IRR). This paper presents a method that leverages Large Language Models (LLMs) to efficiently exclude computer programming episodes that are unlikely to exhibit a specific debugging strategy. By using LLMs to filter out irrelevant programming episodes, this method focuses human annotation efforts on the most pertinent parts of the dataset, enabling experts to operationalize the coding scheme and reach IRR more efficiently. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

An emergent challenge in geriatric care is improving the quality of care, which requires insight from stakeholders. Qualitative methods offer detailed insights, but they can be biased and have limited generalizability, while quantitative methods may miss nuances. To address these limitations, network-based approaches such as Epistemic Network Analysis (ENA) can bridge the methodological gap. By leveraging the strengths of both methods, ENA provides profound insights into healthcare expert interviews. In this paper, to better understand geriatric care attitudes, we interviewed ten nursing assistants, used ENA to analyze the data, and compared their real-life daily activities with training experiences. A two-sample t-test with a large effect size (Cohen’s d = 1.63) indicated a significant difference between real-life and training activities. The findings suggested incorporating more empathetic training scenarios into the future design of our geriatric care simulation. The results have implications for human-computer interaction and effective nursing training. This is illustrated by presenting an example of using quantitative ethnography to analyze expert interviews with nursing assistants as caregivers and inform subsequent simulation and design processes. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

A typical classroom exercise in hydrogeology is to develop a conceptual model of a contaminated site, identify groundwater flow direction(s), and predict the location and mass of a contaminant plume. This requires knowledge of key hydrogeological concepts and is highly visuospatial in nature. Among multiple discrete spatial thinking skills identified by cognitive science, the combination of visual penetrative ability and working in multiple frames of reference were identified to significantly predict performance on a hydrogeology task and showed that together with hydrogeology knowledge, these spatial thinking skills account for 49% of the variability on task performance. Seventy-two hydrogeology practitioners and students with varying levels of expertise were administered multiple spatial thinking tests and an assessment of hydrogeology knowledge before completing a hydrogeology task that was developed for the study. Using spatial thinking and knowledge test scores as predictor variables, a hierarchical regression analysis was conducted with performance on the hydrogeology task as the outcome variable. The resulting model predicts that at low levels of hydrogeology knowledge, the identified spatial thinking skills account for more than a 25% difference on the hydrogeology task. This study provides empirical evidence that visual penetrative ability and working in multiple frames of reference are important skills in hydrogeology; thus, instructors are encouraged to recognize that underdeveloped spatial thinking skills could present hurdles for students and that targeted spatial thinking training may yield positive results for both weak and strong spatial thinkers.