Publications
Cognitive effort, defined as the relationship between cognitive load and task performance, offers insight into how individuals efficiently allocate mental resources during cognitively demanding activities. This metric is crucial in high-stakes public health and clinical training, where unmanaged cognitive overload has been linked to medical errors and workforce burnout. This study aims to examine whether cognitive effort varies systematically across task segments and whether it can be estimated at the individual level using brain signal data and machine learning. Method: Functional near-infrared spectroscopy (fNIRS) data were collected from 16 participants during a structured digital cognitive task comprising four sequential segments separated by short and long rest intervals. Cognitive effort was defined through relative neural efficiency and relative neural involvement, which combined measures of prefrontal hemodynamic activity with task performance. The analysis followed a two-stage approach. First, a segment-level group analysis assessed whether cognitive effort differed significantly across predefined task segments, thereby confirming that the task structure produced meaningful changes in cognitive demand. Second, participant-independent machine learning models predicted task performance from brain signal features. These predicted performance scores were then combined with neural measures to estimate cognitive effort at the individual level. Results: First, statistical analysis showed significant differences in cognitive effort across the four task segments. This confirms that even small changes in the assessment structure impact on the collective cognitive efficiency of the trainees. Then, we used machine learning on fNIRS data to predict individual performance scores. We found that the effort calculated from these predicted scores was nearly identical to that calculated from the actual scores. This result suggests that our effort metric is strongly based on brain signals. Conclusion: The findings demonstrate the feasibility of estimating cognitive effort from brain signals using artificial intelligence at both group and individual levels. Public Health Implications: Estimation of cognitive effort from digital task data may support scalable monitoring of cognitive workload and mental fatigue in technology-mediated environments, which complements subjective assessment methods used in public and digital health research.
An emergent challenge in geriatric care is improving the quality of care, which requires insight from stakeholders. Qualitative methods offer detailed insights, but they can be biased and have limited generalizability, while quantitative methods may miss nuances. To address these limitations, network-based approaches such as Epistemic Network Analysis (ENA) can bridge the methodological gap. By leveraging the strengths of both methods, ENA provides profound insights into healthcare expert interviews. In this paper, to better understand geriatric care attitudes, we interviewed ten nursing assistants, used ENA to analyze the data, and compared their real-life daily activities with training experiences. A two-sample t-test with a large effect size (Cohen’s d = 1.63) indicated a significant difference between real-life and training activities. The findings suggested incorporating more empathetic training scenarios into the future design of our geriatric care simulation. The results have implications for human-computer interaction and effective nursing training. This is illustrated by presenting an example of using quantitative ethnography to analyze expert interviews with nursing assistants as caregivers and inform subsequent simulation and design processes. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Virtual reality (VR) and interactive 3D visualization systems have enhanced educational experiences and environments, particularly in complicated subjects such as anatomy education. VR-based systems surpass the potential limitations of traditional training approaches in facilitating interactive engagement among students. However, research on embodied virtual assistants that leverage generative artificial intelligence (AI) and verbal communication in the anatomy education context is underrepresented. In this work, we introduce a VR environment with a generative AI-embodied virtual assistant to support participants in responding to varying cognitive complexity anatomy questions and enable verbal communication. We assessed the technical efficacy and usability of the proposed environment in a pilot user study with 16 participants. We conducted a within-subject design for virtual assistant configuration (avatar- and screen-based), with two levels of cognitive complexity (knowledge- and analysis-based). The results reveal a significant difference in the scores obtained from knowledge- and analysis-based questions in relation to avatar configuration. Moreover, results provide insights into usability, cognitive task load, and the sense of presence in the proposed virtual assistant configurations. Our environment and results of the pilot study offer potential benefits and future research directions beyond medical education, using generative AI and embodied virtual agents as customized virtual conversational assistants. © 2024 IEEE.
For serious games on education, understanding the effectiveness of different learning methods in influencing cognitive processes remains a significant challenge. In particular, limited research addresses the comparative effectiveness of serious games and videos in analyzing brain behavior for graph structure learning, which is an important part of the Science, Technology, Engineering, Math, and Computing (STEM+C) disciplinary education. This study investigates the impact of serious games on graph structure learning. For this, we compared our in-house game-based learning (GBL) and video-based learning (VBL) methodologies by evaluating their effectiveness on cognitive processes by oxygenated hemoglobin levels using functional near-infrared spectroscopy (fNIRS). We conducted a 2×1 between-subjects preliminary study with twelve participants, involving two conditions: game and video. Both groups received equivalent content related to the basic structure of a graph, with comparable session lengths. The game group interacted with a quiz-based game, while the video group watched a pre-recorded video. The fNIRS was employed to capture cerebral signals from the prefrontal cortex, and participants completed pre- and post-questionnaires capturing user experience and knowledge gain. In our study, we noted that the mean levels of oxygenated hemoglobin (ΔHbO) were higher in the GBL group, suggesting the potential enhanced cognitive involvement. Our results show that the lateral prefrontal cortex (LPFC) has greater hemodynamic activity during the learning period. Moreover, knowledge gain analysis showed an increase in mean score in the GBL group compared to the VBL group. Although we did not observe statistically significant changes due to participant variability and sample size, this preliminary work contributes to understanding how GBL and VBL impact cognitive processes, providing insights for enhanced instructional design and educational game development. Additionally, it emphasizes the necessity for further investigation into the impact of GBL on cognitive engagement and learning outcomes. © 2024 IEEE.
Anatomy education is an indispensable part of medical training, but traditional methods face challenges like limited resources for dissection in large classes and difficulties understanding 2D anatomy in textbooks. Advanced technologies, such as 3D visualization and augmented reality (AR), are transforming anatomy learning. This paper presents two in-house solutions that use handheld tablets or screen-based AR to visualize 3D anatomy models with informative labels and in-situ visualizations of the muscle anatomy. To assess these tools, a user study of muscle anatomy education involved 236 premedical students in dyadic teams, with results showing that the tablet-based 3D visualization and screen-based AR tools led to significantly higher learning experience scores than traditional textbook. While knowledge retention didn't differ significantly, ethnographic and gender analysis showed that male students generally reported more positive learning experiences than female students. This study discusses the implications for anatomy and medical education, highlighting the potential of these innovative learning tools considering gender and team dynamics in body painting anatomy learning interventions. © 2024 IEEE.
Educational games enhance learning experiences by integrating touchscreens, making interactions more engaging and intuitive for learners. However, the cognitive impacts of educational gameplay input modalities, such as the hand and stylus technique, are unclear. We compared the experience of using hands vs. a stylus for touchscreens while playing an educational game by analyzing oxygenated hemoglobin collected by functional Near-Infrared Spectroscopy and self-reported measures. In addition, we measured the hand vs. the stylus modalities of the task and calculated the relative neural efficiency and relative neural involvement using the mental demand and the quiz score. Our findings show that the hand condition had a significantly lower neural involvement, yet higher neural efficiency than the stylus condition. This result suggests the requirement of less cognitive effort while using the hand. Additionally, the self-reported measures show significant differences, and the results suggest that hand-based input is more intuitive, less cognitively demanding, and less frustrating. Conversely, the use of a stylus required higher cognitive effort due to the cognitive balance of controlling the pen and answering questions. These findings highlight the importance of designing educational games that allow learners to engage with the system while minimizing cognitive effort.


