Publications
Over the last decade, the "gender gap" in physics conceptual inventory scores has been extensively studied by the physics education research community. Researchers have identified many factors that influence the overall differences in post-test scores between men and women. More recently, it has been shown that the Force Concept Inventory (FCI) contains eight items that are substantially unfair; six are unfair to women, two are unfair to men. The Force and Motion Conceptual Evaluation (FMCE) and the Conceptual Survey of Electricity and Magnetism (CSEM), however, contain fewer unfair items. In this work, results from prior studies are used to further explore the gender gap in five large samples of conceptual inventory data: the FCI (N 1 3663), the FMCE (N 2 2551, N 3 3719), and the CSEM (N 4 1767, N 5 2439). The gender gap in these samples is partitioned into four components: the gender gap resulting from the student's academic performance, the gender gap resulting from prior preparation in physics, the gender gap resulting from instrumental fairness, and the gender gap of students with equal academic performance and physics preparation on the fair instrument. For all samples, very little of the gender gap was explained by differences in academic performance between men and women, measured by ACT or SAT math percentile scores or physics test average. The percentage of the gender gap resulting from instrumental fairness varied across samples from 30% in the FCI to 2% to 6% in the CSEM. A substantial part of the gender gap in four of the five samples (30%-40%) was explained by differences in prior physics preparation, measured by pretest scores on the conceptual inventories. Further correcting for conceptual physics prior preparation using the post-test score in the previous class reduced gender differences substantially.
The computer science community has struggled to assess student learning, especially at the early elementary level. Prior work has included one-on-one interviews, written assessments, and artifact analysis, each with their own benefits and drawbacks. Through our Personalized Assessment Worksheets for Scratch (PAWS) tool, we explore personalized assessments as an assessment technique that lies in between interviews, written assessments, and artifact analysis. PAWS creates personalized written assessments that integrates code from student Scratch projects. We hope that our PAWS tool, and more generally personalized assessments, will lead to an assessment technique that is both more accurate than written assessments and artifact analysis, and less time-consuming than interviews.
Proportional reasoning is an important skill that relates to fraction learning and math achievement. Because both proportional and analogical reasoning involve comparing relations, we hypothesized that supports for analogical reasoning (multiple exemplars and labels) would help children match discrete proportions. Fourth and 5th graders (N = 119) completed a 16-item proportional equivalence choice task in a 2 (exemplars: one, two) x 3 (script type: juice mixing narrative, novel adjectives, no labels) x 2 (trial type: part-foil, whole-foil) mixed-effects design. The juice mixing script included common labels and a story paradigm, whereas the novel adjectives script only utilized common labels. The least-informative no-labels script served as a baseline. Results showed a significant three-way interaction between exemplars, script type, and trial type. Viewing two exemplars led to no significant differences across the other factors. However, when viewing one exemplar, children exhibited a part-matching bias with novel adjectives and no labels, but not the juice narrative. In sum, multiple exemplars and the juice narrative reduced children's part-matching bias; we conclude that supports for analogical reasoning can aid proportional reasoning as well.
This study explores ways to support girls of color in forming their senses of selves in science, technology, engineering, and math (STEM) during the middle school years. Guided by social practice theory, we analyzed a large data set of survey responses ( n = 1,821) collected at five middle schools in low‐income communities across four states in the United States. Analyses focus on the extent to which key constructs that inform girls’ development of senses of self and relations among those indicators of STEM identities varied by their race/ethnicity. Though the means of indicators sometimes varied across racial/ethnic groups, multigroup structural equation modeling analyses indicate no significant racial/ethnic differences in the relations of STEM identities, suggesting that similar supports would be equally effective for all girls during the middle school years. Girls’ self‐perception in relation to science was the strongest predictor of their identification with STEM‐related careers, and this self‐perception was positively and distinctively associated with their experiences with science at home, outside of school, and in school science classes. This study argues for strategically expanding girls’ experiences with science across multiple settings during middle school in a way that increases their positive self‐perception in and with STEM.
Students need to accurately monitor and judge the difficulty of learning materials to effectively self-regulate their learning with advanced learning technologies such as intelligent tutoring systems (ITSs), including MetaTutorIVH. However, there is a paucity of research examining how metacognitive monitoring processes such as ease of learning (EOLs) judgments can be used to provide adaptive scaffolding and predict student performance during learning ITSs. In this paper, we report on a study investigating how students' EOL judgments can influence their performance and significantly predict their learning outcomes during learning with MetaTutorIVH, an ITS for human physiology. The results have important design implications for incorporating different types of metacognitive judgements in student models to support metacognition and foster learning of complex ITSs.
Self-regulated learning (SRL) and emotion regulation have been studied as separate constructs which impact students' learning with intelligent tutoring systems (ITSs). There is a general assumption that students who are proficient in enacting cognitive and metacognitive SRL processes during learning with ITSs are also proficient emotion regulators. In this paper, we investigated the relationship between metacognitive and cognitive SRL processes and emotion regulation by examining students' self-perceived emotion regulation strategies and comparing the differences between their (1) mean self-reported negative emotions, (2) proportional learning gains (PLGs), and the frequency of (3) metacognitive and (4) cognitive strategy use as they interacted with MetaTutor, an ITS designed to teach students about the circulatory system. Students were classified into groups based on self-perceived emotion regulation strategies and results showed students who perceived themselves as using adaptive emotion regulation strategies reported less negative emotions. Although no significant differences were found between the groups' learning outcomes, there were significant differences between the groups' frequency use of cognitive and metacognitive processes throughout the task. Our results emphasize the need to better understand how real-time emotion regulation strategies relate to SRL processes during learning with ITSs and can be used to enhance learning outcomes by encouraging adaptive emotion regulation strategies as well as increased frequencies of metacognitive and cognitive SRL processes.
We investigated how college students' (n = 40) different levels of action unit 4 (AU4: brow lowerer), metacognitive monitoring process use and pre-test score were associated with metacognitive monitoring accuracy during learning with a hypermedia-based ITS. Results revealed that participants with high pre-test scores had the highest accuracy scores with low levels of AU4 and use of more metacognitive monitoring processes, whereas participants with low pre-test scores had higher accuracy scores with high levels of AU4 and use of more metacognitive monitoring processes. Implications include designing adaptive ITSs that provide different types of scaffolding based on levels of prior knowledge, use of metacognitive monitoring processes, and emotional expressivity keeping in mind that levels of emotions change over time, and therefore must be monitored to provide effective scaffolding during learning.
The goal of this study was to examine 38 undergraduate and graduate students' note taking and summarizing, and the relationship between emotions, the accuracy of those notes and summaries, and proportional learning gain, during learning with MetaTutor, an ITS that fosters self-regulated learning while learning complex science topics. Results revealed that students expressed both positive (i.e., joy, surprise) and negative (i.e., confusion, frustration, anger, and contempt) emotions during note taking and summarizing, and that these emotions correlated with each other, as well as with proportional learning gain and accuracy of their notes and summaries. Specifically, contempt during note taking was positively correlated with proportional learning gain; note taking accuracy was negatively correlated with proportional learning gain; and confusion during summarizing was positively correlated with summary accuracy. These results reveal the importance of investigating specific self-regulated learning processes, such as taking notes or making summaries, with future research aimed at examining the differences and similarities between different cognitive and metacognitive processes and how they interact with different emotions similarly or differently during learning. Implications of these findings move us toward developing adaptive ITSs that foster self-regulated science learning, with specific scaffolding based on each individual student's learning needs.
Adapting ITSs that promote the use of metacognitive strategies can sometimes lead to intense prompting, at least initially, to the point that there is a risk of it feeling counterproductive. In this paper, we examine the impact of different prompting strategies on self-reported agent-directed emotions in an ITS that scaffolds students' use of self-regulated learning (SRL) strategies, taking into account students' prior knowledge. Results indicate that more intense initial prompting can indeed lead to increased frustration, and sometimes boredom even toward pedagogical agents that are perceived as competent. When considering prior knowledge, results also show that this strategy induces a significantly different higher level of confusion in low prior knowledge students when compared to high prior knowledge students. This result is consistent with the fact that higher prior knowledge students tend to be better at self-regulating their learning, and it could also indicate that some low prior knowledge students may be on their path to a better understanding of the value of SRL.
Cognitive, affective, metacognitive, and motivational (CAMM) processes are critical components of self-regulated learning (SRL) essential for learning and problem solving. Currently, ITSs are designed to foster cognitive, affective, and metacognitive (CAM) strategies and processes, presenting major gaps in the research since motivation is a key component of SRL and influences the remaining CAM processes. In our study, students interacted with MetaTutor, a hypermedia-based ITS, to investigate how 190 undergraduate students' proportional learning gain (PLG) related to sub-goals set, cognitive strategy use and metacognitive processes differed based on self-reported achievement goal orientation. Results indicated differences between approach, avoidance, and students who adopted both approach and avoidance goal orientations, but no differences between mastery, performance and students who adopted both mastery and performance goal orientations on PLG for content related to sub-goal 1. Conversely, no differences were found between goal orientation groups on PLG for sub-goal 2, revealing possible changes in goal orientation following sub-goal 1. Analyses indicated no differences between goal orientation groups on metacognitive processes and cognitive strategy use. Thus, we suggest turning away from self-report data, where future studies aim to incorporate multi-channel data over durations of tasks as students interact with ITSs to measure motivation and its tendency to fluctuate in real-time. Implications for using multiple data channels to measure motivation could contribute to adaptive ITS design based on all CAMM processes.


