What can eye movements reveal about reading, a complex skill ubiquitous in everyday life? Research suggests that gaze can reflect short-term comprehension for facts, but it is unknown whether it can measure long-term, deep comprehension. We tracked gaze while 147 participants read long, connected, informative texts and completed assessments of rote (factual) and inference comprehension (connecting ideas) while reading a text, after reading a text, after reading five texts, and after a seven-day delay. Gaze-based student-independent computational models predicted both immediate and long-term rote and inference comprehension with moderate accuracies. Surprisingly, the models were most accurate for comprehension assessed after reading all texts and predicted comprehension even after a week-long delay. This shows that eye movements can provide a lens into the cognitive processes underlying reading comprehension, including inference formation, and the consolidation of information into long-term memory, which has implications for intelligent student interfaces that can automatically detect and repair comprehension in real-time. © 2022 Copyright is held by the author(s).

The effectiveness of user interfaces are limited by the tendency for the human mind to wander. Intelligent user interfaces can combat this by detecting when mind wandering occurs and attempting to regain user attention through a variety of intervention strategies. However, collecting data to build mind wandering detection models can be expensive, especially considering the variety of media available and potential differences in mind wandering across them. We explored the possibility of using eye gaze to build cross-domain models of mind wandering where models trained on data from users in one domain are used for different users in another domain. We built supervised classification models using a dataset of 132 users whose mind wandering reports were collected in response to thought-probes while they completed tasks from seven different domains for six minutes each (five domains are investigated here: Illustrated Text, Narrative Film, Video Lecture, Naturalistic Scene, and Reading Text). We used global eye gaze features to build within- and cross- domain models using 5-fold user-independent cross validation. The best performing within-domain models yielded AUROCs ranging from .57 to .72, which were comparable for the cross-domain models (AUROCs of .56 to .68). Models built from coarse-grained locality features capturing the spatial distribution of gaze resulted in slightly better transfer on average (transfer ratios of .61 vs .54 for global models) due to improved performance in certain domains. Instance-based and feature-level domain adaptation did not result in any improvements in transfer. We found that seven gaze features likely contributed to transfer as they were among the top ten features for at least four domains. Our results indicate that gaze features are suitable for domain adaptation from similar domains, but more research is needed to improve domain adaptation between more dissimilar domains.

A large proportion of thoughts are internally generated. Of these, mind wandering-when attention shifts away from the current activity to an internal stream of thought-is frequent during reading and is negatively related to comprehension outcomes. Our goal is to review research on mind wandering during reading with an interdisciplinary and integrative lens that spans the cognitive, behavioural, computing and intervention sciences. We begin with theoretical developments on mind wandering, both in general and in the context of reading. Next, we discuss psychological research on how the text, context and reader interact to influence mind wandering and on associations between mind wandering and reading outcomes. We integrate the findings in a (working) theoretical account of mind wandering during reading. We then turn to computational models of mind wandering, including a short tutorial with examples on how to use machine learning to construct these models. Finally, we discuss emerging intervention research aimed at proactively reducing the occurrence of mind wandering or mitigating its effects. We conclude with open questions and directions for future research.

Student engagement is a key component of learning and teaching, resulting in a plethora of automated methods to measure it. Whereas most of the literature explores student engagement analysis using computer-based learning often in the lab, we focus on using classroom instruction in authentic learning environments. We collected audiovisual recordings of secondary school classes over a one and a half month period, acquired continuous engagement labeling per student (N=15) in repeated sessions, and explored computer vision methods to classify engagement from facial videos. We learned deep embeddings for attentional and affective features by training Attention-Net for head pose estimation and Affect-Net for facial expression recognition using previously-collected large-scale datasets. We used these representations to train engagement classifiers on our data, in individual and multiple channel settings, considering temporal dependencies. The best performing engagement classifiers achieved student-independent AUCs of .620 and .720 for grades 8 and 12, respectively, with attention-based features outperforming affective features. Score-level fusion either improved the engagement classifiers or was on par with the best performing modality. We also investigated the effect of personalization and found that only 60 seconds of person-specific data, selected by margin uncertainty of the base classifier, yielded an average AUC improvement of .084.

The prevalence of the acronym tl;dr (too long; didn't read) suggests that people intentionally disengage their attention from long sections of text. We studied this real-world phenomenon in an educational context by measuring rates of intentional and unintentional mind-wandering while undergraduate student participants (n = 80) read academic passages that were presented in either short sections of text (one sentence per screen)or relatively long sections (2-6 sentences per screen). We found that participants were significantly more likely tounintentionallydisengage their attention while reading the longer sections of text, whereas intentional mind-wandering rates were equivalent across short and long sections of text. The difference in unintentional mind-wandering rates suggests that section length may serve as a cue that people use to assess the cost-benefit tradeoffs involved in attending to (or disengaging from) text. We conclude that instructors should avoid presenting electronic reading material in long sections of text.

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level cross-validation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns = 104, 130, 147) answered multiple-choice comprehension questions similar to 30 min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372,ps < .001), in line with the strong association hypothesis. We found that making more, but shorter fixations, consistently predicted comprehension across all studies. Furthermore, models trained on one study's data could successfully predict comprehension on the others, suggesting generalizability across studies. Collectively, these findings suggest that there is a robust link between eye movements and subsequent comprehension of a long connected text, thereby connecting theories of low-level eye movements with those of higher order text processing during reading.

We propose that machine-learned computational models (MLCMs), in which the model parameters and perhaps even structure are learned from data, can complement extant approaches to the study of text and discourse. Such models are particularly useful when theoretical understanding is insufficient, when the data are rife with nonlinearities and interactivity, and when researchers aspire to take advantage of big data. Being fully instantiated computer programs, MLCMs can also be used for autonomous assessment and real-time intervention. We illustrate these ideas in the context of an eye movement-based MLCM of textbase comprehension during reading along connected text. Using a dataset where 104 participants read a 6,500-word text, we trained Random Forests models to predict comprehension scores from six eye movement features. The models were highly accurate (area under the receiver operating characteristic curve = .902; r = .661), robust, and generalized across participants, suggesting possible use in future studies. We conclude by arguing for an increased role of MLCMs in the future of discourse research.

During mind wandering, visual processing of external information is attenuated. Accordingly, mind wandering is associated with changes in gaze behaviors, albeit findings are inconsistent in the literature. This heterogeneity obfuscates a complete view of the moment-to-moment processing priorities of the visual system during mind wandering. We hypothesize that this observed heterogeneity is an effect of idiosyncrasy across tasks with varying spatial allocation demands, visual processing demands, and discourse processing demands and reflects a strategic, compensatory shift in how the visual system operates during mind wandering. We recorded eye movements and mind wandering (via thought-probes) as 132 college-aged adults completed a battery of 7 short (6 min) tasks with different visual demands. We found that for tasks requiring extensive sampling of the visual field, there were fewer fixations, and, depending on the specific task, fixations were longer and/or more dispersed. This suggests that visual sampling is sparser and potentially slower and more dispersed to compensate for the decreased sampling rate during mind wandering. For tasks that demand centrally focused gaze, mind wandering was accompanied by more exploratory eye movements, such as shorter and more dispersed fixations as well as larger saccades. Gaze behaviors were not reliably associated with mind wandering during a film comprehension task. These findings provide insight into how the visual system prioritizes external information when attention is focused inward and indicates the importance of task demands when assessing the relationship between eye movements, visual processing, and mind wandering.