Developing a Scalable Measure of Inclusive STEM Teaching Practices for Diverse Institutions
Effective Years: 2022-2025
Recently, psychological research on broadening participation in STEM has shifted from student-focused interventions (e.g., belonging or growth mindset interventions given to students) to context-focused interventions (aimed at instructors to create cultures of belonging or growth). Tools for evaluating the changes in instructors’ practices induced by these programs have been lacking, however. Popular self-report methods for instructors can be biased due to faulty recall or social desirability bias, while more intensive observational methods are cumbersome and difficult to sustain. In addition, existing measures tend to be developed within one type of institution and may underemphasize inclusive practices from instructors at historically Black colleges and universities and Hispanic-serving institutions. These limitations have slowed progress toward the discovery of programs that could broaden participation in STEM. The present proposal seeks to develop a scalable, trustworthy system of instructor- and student-reports that could be the backbone for research evaluating the next generation of context-focused interventions.
This project will develop novel measures of inclusive postsecondary STEM teaching practices that create an experience of belonging and motivation for members of minoritized groups. Measures of inclusive practices will be developed using optimal survey design, including semi-structured interviews and cognitive pretesting co-developed with both instructors and students. The measures will be simultaneously developed and refined across three universities: at a predominantly white-serving institution, a predominantly Hispanic-serving institution, and a historically-Black university. This cross-site validation will ensure that measures are sensitive to the variability in students’ experiences and increase their potential application. The sensitivity of these measures in capturing the frequency and quality of practices will be validated using third-party observations, repeated student-report surveys, and course completion outcomes. A machine-learning algorithm, Bayesian Additive Regression Trees (BART), will inform the choice of the optimal number of practices items that minimize respondent burden while maximizing prediction. The resulting measures will be widely disseminated to unlock the creative potential of scientists and practitioners developing inclusive practices by allowing them to learn what is working, and why.
This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in education.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.