STEM Learning and Learning Environments   Broadening Participation in STEM

Exploring Algorithmic Fairness and Potential Bias in K-12 Mathematics Adaptive Learning

Effective Years: 2020-2024

Students in middle school and high school often use adaptive learning software as part of their math education experience. Adaptive learning software works by automatically measuring how much students have learned about the topic, as well as their learning process and experiences, and then adjusting the instruction accordingly. This project will investigate potential ways in which adaptive learning software might be biased against students from certain groups, and how such biases can be reduced. Adaptive learning offers an opportunity to provide high quality instruction that is personalized to the needs of individual learners, but little is known about who benefits most from adaptive learning technologies. This project will address this issue by observing and interviewing students who use adaptive math learning software to discover what aspects of their identity are most salient in the adaptive learning context. This project will then investigate possible algorithmic biases related to the identities that students express. Findings from the project will contribute to understanding of the most relevant aspects of student identity in adaptive learning contexts, and how those identities affect their learning experience. Finally, this project will address the biases that are identified, thereby providing a more equitable mathematics education experience for students.

Modern adaptive learning platforms individualize learning support and improve learner outcomes by using algorithms that are typically derived through machine learning. Previous work has studied biases in educational model accuracy for large groups (e.g., ethnic and gendered categories, urban vs. rural, etc.); however, large groups may have a great deal of heterogeneity, and little is known about which specific groups of students suffer from biases in model accuracy and why. This project will approach the problem of potential bias in three steps. First, the project will begin by collecting data on educational software usage patterns (i.e., logs of actions and classroom observations of student experiences) for students using MATHia, a math education platform used by over half a million students across the United States. As part of this data collection, students will describe their identity in open-ended survey responses and interviews, which will be analyzed to discover identity characteristics that shape their learning experiences. Second, existing machine learning models will be applied to these data to predict knowledge, engagement, and self-regulated learning behaviors, and the predictions will be analyzed to reveal cases where models are systematically biased. Third, the project will compare various pre-processing, in-processing, and post- processing methods for bias reduction, and study the effects of the improved algorithms when applied in MATHia. Results from this project will contribute to scientific understanding of the role of student identity in adaptive learning software, biases in machine learning for educational software, and the effects of applying machine learning methods for bias reduction. This project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development, with co-funding by the Discovery Research PreK-12 (DRK-12) program.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.