Publications
Educators increasingly recognize the importance of students' learning orientations, but relatively little is known about how these mindsets vary across and potentially shape educational settings. We use nationally representative data to document contextual variation in mathematics orientations in U.S. high schools. We find systematic variation in orientations between differentiated course levels within school, suggesting orientations are more a feature of proximate instructional contexts than general school climate. Between-course variation in orientations is comparable to analogous sorting on demographic characteristics and not primarily explained by prior achievement. Measures of individual learning orientations at scale hold promise for understanding collective educational contexts.
Group-based educational disparities are smaller in classrooms where teachers express a belief that students can improve their abilities. However, a scalable method for motivatthat overcame these obstacles and successfully motivated high-school teachers to adopt specific practices that support students' growth mindsets. The intervention used the values-alignment approach. This approach motivates behavioral change by framing a desired behavior as aligned with a core value-one that is an important criterion for status and admiration in the relevant social reference group. First, using qualitative interviews and a nationally representative survey of teachers, we identified a relevant core value: inspiring students' enthusiastic engagement with learning. Next, we designed a similar to 45-min, self-administered, online intervention that persuaded teachers to view growth mindset-supportive practices as a way to foster such student engagement and thus live up to that value. We randomly assigned 155 teachers (5,393 students) to receive the intervention and 164 teachers (6,167 students) to receive a control module. The growth mindset-supportive teaching intervention successfully promoted teachers' adoption of the suggested practices, overcoming major barriers to changing teachers' classroom practices that other scalable approaches have failed to surmount. The intervention also substantially improved student achievement in socioeconomically disadvantaged classes, reducing inequality in educational outcomes.
Large language models (LLMs), such as OpenAI's GPT-4, Google's Bard or Meta's LLaMa, have created unprecedented opportunities for analysing and generating language data on a massive scale. Because language data have a central role in all areas of psychology, this new technology has the potential to transform the field. In this Perspective, we review the foundations of LLMs. We then explain how the way that LLMs are constructed enables them to effectively generate human-like linguistic output without the ability to think or feel like a human. We argue that although LLMs have the potential to advance psychological measurement, experimentation and practice, they are not yet ready for many of the most transformative psychological applications - but further research and development may enable such use. Next, we examine four major concerns about the application of LLMs to psychology, and how each might be overcome. Finally, we conclude with recommendations for investments that could help to address these concerns: field-initiated 'keystone' datasets; increased standardization of performance benchmarks; and shared computing and analysis infrastructure to ensure that the future of LLM-powered research is equitable. Large language models (LLMs), which can generate and score text in human-like ways, have the potential to advance psychological measurement, experimentation and practice. In this Perspective, Demszky and colleagues describe how LLMs work, concerns about using them for psychological purposes, and how these concerns might be addressed.
Educational outcomes remain highly unequal within and across nations. Students' mindsets-their beliefs about whether intellectual abilities can be developed-have been identified as a potential lever for making adolescents' academic outcomes more equitable. Recent research, however, suggests that intervention programs aimed at changing students' mindsets should be supplemented by programs aimed at the changing the mindset culture, which is defined as the shared set of beliefs about learning in a school or classroom. This paper reviews the theoretical and empirical origin of the mindset culture and examines its potential to reduce group-based inequalities in education. In particular, experiments have identified two broad ways the mindset culture is communicated by teachers: via informal messages about growth (e.g., that all students will be helped to learn and succeed), and formal opportunities to improve (e.g., learning-focused grading policies and opportunities to revise and earn credit). New field experiments, applying techniques from behavioral science, have also revealed effective ways to influence teachers' culture-creating behaviors. This paper describes recent breakthroughs in the U.S. educational context and discusses how lessons from these studies might be applied in future, global collaborations with researchers and practitioners.
In this study, we identified multidimensional profiles in students' math anxiety, math self-concept, and math interest using data from a large generalizable sample of 16,547 9th grade students in the United States who participated in the National Study of Learning Mindsets. We also analyzed the extent that students' profile memberships are associated with related measures such as prior mathematics achievement, academic stress, and challenge-seeking behavior. Five multidimensional profiles were identified: two profiles which demonstrated relatively high levels of interest and self-concept, along with low math anxiety, in line with the tenets of the control-value theory of academic emotions (C-VTAE); two profiles which demonstrated relatively low levels of interest and self-concept, and high levels of math anxiety (again in accordance with C-VTAE); and one profile, comprising more than 37% of the total sample, which demonstrated medium levels of interest, high levels of self-concept, and medium levels of anxiety. All five profiles varied significantly from one another in their association with the distal variables of challenge seeking behavior, prior mathematics achievement, and academic stress. This study contributes to the literature on math anxiety, self-concept, and interest by identifying and validating student profiles that mainly align with the control-value theory of academic emotions in a large, generalizable sample.
Meta-analysts often ask a yes-or-no question: Is there an intervention effect or not? This traditional, all-or-nothing thinking stands in contrast with current best practice in meta-analysis, which calls for a heterogeneity-attuned approach (i.e., focused on the extent to which effects vary across procedures, participant groups, or contexts). This heterogeneity-attuned approach allows researchers to understand where effects are weaker or stronger and reveals mechanisms. The current article builds on a rare opportunity to compare two recent meta-analyses that examined the same literature (growth mindset interventions) but used different methods and reached different conclusions. One meta-analysis used a traditional approach (Macnamara & Burgoyne, 2023), which aggregated effect sizes for each study before combining them and examined moderators one-by-one by splitting the data into small subgroups. The second meta-analysis (Burnette et al., 2023) modeled the variation of effects within studies-across subgroups and outcomes-and applied modern, multilevel metaregression methods. The former concluded that growth mindset effects are biased, but the latter yielded nuanced conclusions consistent with theoretical predictions. We explain why the practices followed by the latter meta-analysis were more in line with best practices for analyzing large and heterogeneous literatures. Further, an exploratory re-analysis of the data showed that applying the modern, heterogeneity-attuned methods from Burnette et al. (2023) to the data set employed by Macnamara and Burgoyne (2023) confirmed Burnette et al.'s conclusions; namely, that there was a meaningful, significant effect of growth mindset in focal (at-risk) groups. This article concludes that heterogeneity-attuned meta-analysis is important both for advancing theory and for avoiding the boom-or-bust cycle that plagues too much of psychological science.
The use of Bayesian Knowledge Tracing (BKT) models in predicting student learning and mastery, especially in mathematics, is a well-established and proven approach in learning analytics. In this work, we report on our analysis examining the generalizability of BKT models across academic years attributed to "detector rot." We compare the generalizability of Knowledge Training (KT) models by comparing model performance in predicting student knowledge within the academic year and across academic years. Models were trained on data from two popular open-source curricula available through Open Educational Resources. We observed that the models generally were highly performant in predicting student learning within an academic year, whereas certain academic years were more generalizable than other academic years. We posit that the Knowledge Tracing models are relatively stable in terms of performance across academic years yet can still be susceptible to systemic changes and underlying learner behavior. As indicated by the evidence in this paper, we posit that learning platforms leveraging KT models need to be mindful of systemic changes or drastic changes in certain user demographics.
Teachers' growth mindset supportive language (GMSL)--rhetoric emphasizing that one's skills can be improved over time--has been shown to significantly reduce disparities in academic achievement and enhance students' learning outcomes. Although teachers espouse growth mindset principles, most find it difficult to adopt GMSL in their practice due the lack of effective coaching in this area. We explore whether large language models (LLMs) can provide automated, personalized coaching to support teachers' use of GMSL. We establish an effective coaching tool to reframe unsupportive utterances to GMSL by developing (i) a parallel dataset containing GMSL-trained teacher reframings of unsupportive statements with an accompanying annotation guide, (ii) a GMSL prompt framework to revise teachers' unsupportive language, and (iii) an evaluation framework grounded in psychological theory for evaluating GMSL with the help of students and teachers. We conduct a large-scale evaluation involving 174 teachers and 1,006 students, finding that both teachers and students perceive GMSL-trained teacher and model reframings as more effective in fostering a growth mindset and promoting challenge-seeking behavior, among other benefits. We also find that model-generated reframings outperform those from the GMSL-trained teachers. These results show promise for harnessing LLMs to provide automated GMSL feedback for teachers and, more broadly, LLMs' potentiality for supporting students' learning in the classroom. Our findings also demonstrate the benefit of large-scale human evaluations when applying LLMs in educational domains.
Social-evaluative stressors-experiences in which people feel they could be judged negatively-pose a major threat to adolescent mental health(1-3) and can cause young people to disengage from stressful pursuits, resulting in missed opportunities to acquire valuable skills. Here we show that replicable benefits for the stress responses of adolescents can be achieved with a short (around 30-min), scalable 'synergistic mindsets' intervention. This intervention, which is a self-administered online training module, synergistically targets both growth mindsets(4) (the idea that intelligence can be developed) and stress-can-be-enhancing mindsets(5) (the idea that one's physiological stress response can fuel optimal performance). In six double-blind, randomized, controlled experiments that were conducted with secondary and post-secondary students in the United States, the synergistic mindsets intervention improved stress-related cognitions (study 1, n = 2,717; study 2, n = 755), cardiovascular reactivity (study 3, n = 160; study 4, n = 200), daily cortisol levels (study 5, n = 118 students, n = 1,213 observations), psychological well-being (studies 4 and 5), academic success (study 5) and anxiety symptoms during the 2020 COVID-19 lockdowns (study 6, n = 341). Heterogeneity analyses (studies 3, 5 and 6) and a four-cell experiment (study 4) showed that the benefits of the intervention depended on addressing both mindsets-growth and stress-synergistically. Confidence in these conclusions comes from a conservative, Bayesian machine-learning statistical method for detecting heterogeneous effects(6). Thus, our research has identified a treatment for adolescent stress that could, in principle, be scaled nationally at low cost.


