STEM Learning and Learning Environments  

Investigating the Role of Interest in Middle Grade Science with a Multimodal Affect-Sensitive Learning Environment

Effective Years: 2020-2025

Student interest and affect play critical roles in shaping how individuals learn. Interest impacts student engagement with scientific ideas and practices, student learning outcomes, and the ability to set goals and self-regulate. Student interest is also a well-known precursor to STEM career development. Over the past several years, research on affect-sensitive learning environments has enabled data-rich investigations into the affective dynamics of student learning. However, designing adaptive learning environments that respond effectively to student affect is a key gap in the research. Recent developments in multimodal affect recognition and adaptive learning technologies have set the stage for the creation of affect-responsive interventions to support student learning, engagement, and critically, the development of science interest. This project centers on the design, development, and investigation of a multimodal affect-sensitive learning environment to enhance middle school students’ science learning, engagement, and interest in science. The project will investigate the relationship between student affect and interest in science, enabling the development of methods to support learning and interest development through multimodal, affect-sensitive interventions within an inquiry-based science learning environment. It is anticipated that the project will advance the national goal of providing effective, engaging science learning experiences for all students. With the overarching goal of developing methods and adaptive learning technologies that enable improved STEM education, the project has two major objectives: The first objective is to design, develop, and refine an affect-sensitive learning environment based on multimodal neural architectures to generate and sustain student interest in inquiry-based science learning. A suite of physical hardware sensors to capture rich multi-channel data (facial expression, eye gaze, posture, gesture, interaction trace logs) combined with quantitative observations of student affect and behavioral engagement (i.e., an established protocol for observations) will be utilized to train multimodal recurrent neural network-based models of student affect recognition. These models will drive adaptive interventions to guide students toward engaged problem solving by triggering and maintaining student interest in science inquiry. The affect-sensitive interventions will be integrated with Crystal Island, an inquiry-based learning environment for middle school science education. The second objective is to investigate the impact of the multimodal affect-sensitive learning environment on student learning, engagement, and interest in science. A culminating study with middle school students will examine the impact of these designs, comparing the multimodal affect-sensitive learning environment to a baseline environment without affect-sensitive interventions. This comparison will test the effectiveness of the learning environment in fostering enhanced learning and interest outcomes across a diverse range of learners, examining measures of knowledge, engagement, and interest, including interest in science and interest in STEM careers. The resulting findings will yield significant contributions to both theory and practice in student interest development and produce an empirical account of the effectiveness of multimodal neural architectures for modeling and responding to student affect. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

65 Publications Related to This Project: