This work investigates how tutoring discourse interacts with students' proximal knowledge to explain and predict students' learning outcomes. Our work is conducted in the context of high-dosage human tutoring where 9th-grade students (N = 1080) attended small group tutorials and individually practiced problems on an Intelligent Tutoring System (ITS). We analyzed whether tutors' talk moves and students' performance on the ITS predicted scores on math learning assessments. We trained Random Forest Classifiers (RFCs) to distinguish high and low assessment scores based on tutor talk moves, student's ITS performance metrics, and their combination. A decision tree was extracted from each RFC to yield an interpretable model. We found AUCs of 0.63 for talk moves, 0.66 for ITS, and 0.77 for their combination, suggesting interactivity among the two feature sources. Specifically, the best decision tree emerged from combining the tutor talk moves that encouraged rigorous thinking and students' ITS mastery. In essence, tutor talk that encouraged mathematical reasoning predicted achievement for students who demonstrated high mastery on the ITS, whereas tutors' revoicing of students' mathematical ideas and contributions was predictive for students with low ITS mastery. Implications for practice are discussed.

Student’s shift of attention away from a current learning task to task-unrelated thought, also called mind wandering, occurs about 30% of the time spent on education-related activities. Its frequent occurrence has a negative effect on learning outcomes across learning tasks. Automated detection of mind wandering might offer an opportunity to assess the attentional state continuously and non-intrusively over time and hence enable large-scale research on learning materials and responding to inattention with targeted interventions. To achieve this, an accessible detection approach that performs well for various systems and settings is required. In this work, we explore a new, generalizable approach to video-based mind wandering detection that can be transferred to naturalistic settings across learning tasks. Therefore, we leverage two datasets, consisting of facial videos during reading in the lab (N = 135) and lecture viewing in-the-wild (N = 15). When predicting mind wandering, deep neural networks (DNN) and long short-term memory networks (LSTMs) achieve F1 scores of 0.44 (AUC-PR = 0.40) and 0.459 (AUC-PR = 0.39), above chance level, with latent features based on transfer-learning on the lab data. When exploring generalizability by training on the lab dataset and predicting on the in-the-wild dataset, BiLSTMs on latent features perform comparably to the state-of-the-art with an F1 score of 0.352 (AUC-PR = 0.26). Moreover, we investigate the fairness of predictive models across gender and show based on post-hoc explainability methods that employed latent features mainly encode information on eye and mouth areas. We discuss the benefits of generalizability and possible applications. © The Author(s) 2024.

Eye movements have long been used to reflect ongoing cognitive processing to develop explanatory and predictive models of mental states and processes. This relationship, deemed the eye-mind link, contains underlying assumptions of the mental processes occurring in the brain, which have rarely been explicitly investigated. We propose a multimodal approach to investigate alignment of eye movements and brain activations (eye-brain alignment) and how it might be predicted by unfolding cognitive processes. We applied this method to a dataset of 76 participants who read long, connected texts while their eye movements and the hemodynamic responses in their brains were tracked using functional near-infrared spectroscopy (fNIRS). We found that reliable eye-brain alignment signals varied based on the participants’ cognitive state during reading. Implications for multimodal modeling of cognitive processes are discussed. © 2024 Copyright held by the owner/author(s).

Neuroimaging studies using functional magnetic resonance imaging (fMRI) have provided unparalleled insights into the fundamental neural mechanisms underlying human cognitive processing, such as high-level linguistic processes during reading. Here, we build upon this prior work to capture sentence reading comprehension outside the MRI scanner using functional near infra-red spectroscopy (fNIRS) in a large sample of participants (n = 82). We observed increased task-related hemodynamic responses in prefrontal and temporal cortical regions during sentence-level reading relative to the control condition (a list of non-words), replicating prior fMRI work on cortical recruitment associated with high-level linguistic processing during reading comprehension. These results lay the groundwork towards developing adaptive systems to support novice readers and language learners by targeting the underlying cognitive processes. This work also contributes to bridging the gap between laboratory findings and more real-world applications in the realm of cognitive neuroscience.

This work investigates relationships between consistent attendance -attendance rates in a group that maintains the same tutor and students across the school year- and learning in small group tutoring sessions. We analyzed data from two large urban districts consisting of 206 9th-grade student groups (3 - 6 students per group) for a total of 803 students and 75 tutors. The students attended small group tutorials approximately every other day during the school year and completed a pre and post-assessment of math skills at the start and end of the year, respectively. First, we found that the attendance rates of the group predicted individual assessment scores better than the individual attendance rates of students comprising that group. Second, we found that groups with high consistent attendance had more frequent and diverse tutor and student talk centering around rich mathematical discussions. Whereas we emphasize that changing tutors or groups might be necessary, our findings suggest that consistently attending tutorial sessions as a group with the same tutor might lead the group to implicitly learn as a team despite not being one.

Reading is one of the most common everyday activities, yet research elucidating how affective influence reading processes and outcomes is sparse with inconsistent results. To investigate this question, we randomly assigned participants (N = 136) to happiness (positive affect), sadness (negative affect), and neutral video-induction conditions prior to engaging in self-paced reading of a long, complex science text. Participants completed assessments targeting multiple levels of comprehension (e.g. recognising factual information, integrating different textual components, and open-ended responses of concepts from memory) after reading and after a week-long delay. Results indicated that the Sadness (vs. Happiness) condition had higher comprehension scores, with the largest effects emerging for assessments targeting deeper levels comprehension immediately after reading. Eye-tracking analyses revealed that such benefits may be partly driven by sustained attentional focus over the 20-minute reading session. We discuss results with respect to theories on affect, cognition, and text comprehension.

The eye-mind link postulates a relationship between overt attention and covert cognitive processes. Eye movements are the key measure used to reflect attention allocation, and have long been used to develop explanatory and predictive models of mental processing. However, there is very little research investigating the direct relationship between eye movements and the processing occurring in the brain. The proposed research aims to leverage a multimodal dataset where both eye tracking and the cortical hemodynamic responses in the brain were recorded in parallel. The goal is to develop a flexible methodology that can relate eye movements to the brain activations occurring in parallel, including state-of-the-art transformer models to extract meaningful information from the two very different time series. The goal is to support the understanding of the eye-mind link through a focus on the "mind" while also working towards multimodal modeling of cognitive processes. © 2024 Copyright held by the owner/author(s).

We examined associations between mind wandering - where attention shifts from the task at hand to task-unrelated thoughts - and learning outcomes. Our data consisted of 177 students who self-reported mind wandering while reading five long, connected texts on scientific research methods and completed learning assessments targeting multiple depths of processing (rote, inference, integration) at different timescales (during and after reading each text, after reading all texts, and after a week-long delay). We found that mind wandering negatively predicted measures of factual, text-based (explicit) information and global integration of information across multiple parts of the text, but not measures requiring a local inference on a single sentence. Further, mind wandering only predicted comprehension measures assessed during the reading session and not after a week-long delay. Our findings provide important nuances to the established negative link between mind wandering and learning outcomes, which has predominantly focused on rote comprehension assessed during the learning session itself. Implications for interventions to address mind wandering during learning are discussed. © 2023 Owner/Author.

Engagement is critical to satisfaction and performance in a number of domains but is challenging to measure and sustain. Thus, there is considerable interest in developing affective computing technologies to automatically measure and enhance engagement, especially in the wild and at scale. This article provides an accessible introduction to affective computing research on engagement detection and enhancement using educational applications as an application domain. We begin with defining engagement as a multicomponential construct (i.e., a conceptual entity) situated within a context and bounded by time and review how the past six years of research has conceptualized it. Next, we examine traditional and affective computing methods for measuring engagement and discuss their relative strengths and limitations. Then, we move to a review of proactive and reactive approaches to enhancing engagement toward improving the learning experience and outcomes. We underscore key concerns in engagement measurement and enhancement, especially in digitally enhanced learning contexts, and conclude with several open questions and promising opportunities for future work. © 1963-2012 IEEE.