Improving educators' trust in and effective uses of predictive learning analytics to support students

Effective Years: 2023-2028

Predictive learning analytics (PLA) that have a focus on identifying students at risk of failing are gaining traction with leaders at all levels of education. The effective adoption and use of PLA in the US education system can promote educational attainment, STEM workforce development, and national development by supporting lower-resourced educational institutions and members of underserved communities. Prior research has focused on enhancing the technical implementation of these AI-based systems, but studies of why deployments of existing technologies have failed identify issues that are not technical but social, psychological, or cultural in nature. For example, educators’ trust in and effective use of PLA are instrumental for realizing their potential benefits, motivating this work to study how changes in the framing, level of transparency, and training can improve educators’ use of PLA. The project will involve 3,600 US educators across several large-scale studies and advance an understanding of what influences educators’ trust and effective use of PLA, as well as AI-based education technology more broadly. These insights will guide the development of an evidence-based curriculum about PLA for educators to foster awareness, trust, and effective use. The PLA training materials will be made available online and via posts on social media channels. This project develops concrete and scalable interventions to help design technologies that affect the academic progress of learners of all ages.

Building on the Technology Acceptance Model and Academic Resistance Models, the project will investigate how educators respond to procedural justice framings and algorithmic transparency for AI-based education technology, especially during expectancy violation and high levels of uncertainty. Three studies will be conducted as online randomized controlled trials, replicated for robustness, that present educators with a simulated PLA to assess their perceptions, attitudes, and behavioral intentions. How different presentations of PLA influence educators’ usage intentions will be investigated by using an online noise audit, which quantifies how much the same information about students leads educators to the same conclusions about how to intervene. The cumulative insights of the project will be applied in a field experiment on a university campus to examine if evidence-based improvements to an existing at-risk identification process can encourage faculty to reach out to students and better support them. The project will contribute to significant advances in our causal understanding of ways to foster all educators’ trust and effective use of a learning analytics system. The award is funded in part by the EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. This project is also funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.