Modeling Brain and Behavior to Uncover the Eye-Brain-Mind Link during Complex Learning
Effective Years: 2019-2024
The project, based at the University of Colorado, will advance fundamental knowledge on learning from complex STEM texts. This is a critical skill for success in an increasingly information-driven world and workforce, but it is also an area where students consistently struggle. Scores on standardized assessments are stubbornly stagnant, troublesome achievement gaps remain, and the U.S. continues to lag behind its international peers. This is especially relevant to the reading of complex STEM content. The main idea of the project is that learning from text is fundamentally about the coordination of vision (the eye) with thought (the mind) in a manner constrained by the content (the text), the context (the task), and the individual (the learner). Thus, there is a need to understand how these factors interact in order to advance basic knowledge and to inform effective interventions. The team will study how brain and behavioral signals can be used to understand the reading process and associated learning outcomes in a manner that is sensitive to individual differences. Anticipated near-term broader impacts include the interdisciplinary training of students, especially those from groups underrepresented in STEM fields, community outreach, and promoting scientific reasoning skills relevant to multiple STEM areas. In the longer term, the results of the project will inform the design of future interventions that aim to make learning from text more efficient, engaging, and effective. The project is funded by the EHR Core Research (ECR) program, which supports work that advances the fundamental research literature on STEM learning. The intellectual merit of the project is to gain fundamental knowledge on how the eye and brain dynamically coordinate to construct mental representations during complex learning from STEM texts. The project will address this goal by collecting eye movements, reading behaviors, noninvasive brain signals, and learning outcomes from high-school and college students in controlled lab settings and in school environments. It is expected that students will improve critical thinking and scientific reasoning skills by engaging with multiple texts on scientific research methods. The project will use behavioral (eye tracking and reading times) and non-invasive brain measures (high-density functional near-infrared spectroscopy and electroencephalography) to identify neurobehavioral markers of core processes involved in reading comprehension (e.g., focused attention vs. mind wandering, inference generation and elaboration) and the extent to which these measures can predict learning outcomes (comprehension, knowledge, and transfer) assessed immediately after learning and a week later. The project will also leverage advances in deep machine learning to build computational neurobehavioral models of the process and outcome variables in a manner that is sensitive to textual difficulty and to individual differences (e.g., prior knowledge, interest). Thus, the project will advance fundamental knowledge by integrating three research areas: theories of low-level reading processes and associated models of eye movements, models of higher-order STEM text comprehension including multiple document comprehension, and emerging research on the cognitive neuroscience of reading including co-registration with eye movements. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
25 Publications Related to This Project:
- Going Deep and Far: Gaze-Based Models Predict Multiple Depths of Comprehension During and One Week Following Reading
- Psychological Measurement in the Information Age: Machine-Learned Computational Models
- A Multimodal Understanding of the Eye-Mind Link
- Putting the "Brain" Back in the Eye-Mind Link: Aligning Eye Movements and Brain Activations During Naturalistic Reading
- Not a Team but Learning as One: The Impact of Consistent Attendance on Discourse Diversification in Math Group Modeling
- Do Associations Between Mind Wandering and Learning from Complex Texts Vary by Assessment Depth and Time?
- Evaluating Calibration-Free Webcam-Based Eye Tracking for Gaze-Based User Modeling
- Feasibility of Longitudinal Eye-Gaze Tracking in the Workplace
- Can Computers Outperform Humans in Detecting User Zone-Outs? Implications for Intelligent Interfaces
- Crossed Eyes: Domain Adaptation for Gaze-Based Mind Wandering Models
- Breaking Out of the Lab: Mitigating Mind Wandering with Gaze-Based Attention-Aware Technology in Classrooms
- Mind Wandering During Reading: An Interdisciplinary and Integrative Review of Psychological, Computing, and Intervention Research and Theory
- What Eye Movements Reveal About Later Comprehension of Long Connected Texts
- Multimodal Engagement Analysis from Facial Videos in the Classroom
- Engagement Detection and Its Applications in Learning: A Tutorial and Selective Review
- Eye-Mind Reader: An Intelligent Reading Interface that Promotes Long-Term Comprehension by Detecting and Responding to Mind Wandering
- Sadness Facilitates "Deeper" Reading Comprehension: A Behavioural and Eye Tracking Study
- Machine-Learned Computational Models Can Enhance the Study of Text and Discourse: A Case Study Using Eye Tracking to Model Reading Comprehension
- Tl;Dr: Longer Sections of Text Increase Rates of Unintentional Mind-Wandering
- Cortical Cognitive Processing During Reading Captured Using Functional-Near Infrared Spectroscopy
- The Eye-Mind Wandering Link: Identifying Gaze Indices of Mind Wandering Across Tasks
- Momentary Measures of Emotions During Technology-Enhanced Learning Prospectively Predict Standardized Test Scores in Two Large Samples
- From the Lab to the Wild: Examining Generalizability of Video-Based Mind Wandering Detection
- Gaze-Based Predictive Models of Deep Reading Comprehension
- Aligning Tutor Discourse Supporting Rigorous Thinking with TUTEE Content Mastery for Predicting Math Achievement
You are viewing a record from the ECR Projects database. Click here to search the ECR Projects Database or the ECR Publications Database.


