Publications
A growing quantitative literature finds evidence that student teaching placements predict later outcomes of teacher candidates and their students, but there is little large-scale quantitative evidence about the mechanisms for these estimated relationships. We used data from a survey of STEM teachers in Washington State to better understand how their perceptions of preparation were related to student teaching placements and current classroom environment. We found evidence that the composition of students in student teaching classrooms was predictive of STEM teachers' perceptions of their preparation. For example, STEM teachers who student taught in classrooms with more English language learners and economically disadvantaged students reported feeling prepared to teach these specific student populations. Likewise, STEM teachers who student taught in high-poverty classrooms tended to report feeling better prepared to manage their current classroom, particularly if they were currently teaching in a high-poverty classroom.
Background and Context. With increasing efforts to bring computing education opportunities into elementary schools, there is a growing need for assessments, with arguments for validity, to support research evaluation at these grade levels. After successfully piloting a 10-question computational thinking assessment (Assessment of Computing for Elementary Students - ACES) for 4th graders in Spring 2020, we used our analyses of item difficulty and discrimination to iterate on the assessment. Objectives. To increase the number of potential items for ACES, we created isomorphic versions of existing questions. The nature of the changes varied from incidental changes that we did not believe would impact student performance to more radical changes that seemed likely to influence question difficulty. We sought to understand the impact of these changes on student performance. Method. Using these isomorphic questions, we created two versions of our assessment and piloted them in Spring 2021 with 235 upper-elementary (4th grade) students. We analyzed the reliability of the assessments using Cronbach's alpha. We used Chi-squared tests to analyze questions that were identical across the two assessments to form a baseline of comparison and then ran Chi-Squared and Kruskal-Wallis H tests to analyze the differences between the isomorphic copies of the questions. Findings. Both assessment versions demonstrated good reliability, with identical Cronbach's alphas of 0.868. We found statistically similar performance on the identical questions between our two groups of students, allowing us to compare their performance on the isomorphic questions. Students performed differently on the isomorphic questions, indicating the changes to the questions had a differential impact on student performance. Implications. This paper builds on existing work by presenting methods for creating isomorphic questions. We provide valuable lessons learned, both on those methods and on the impact of specific types of changes on student performance.
This study integrates an intersectional framework with data on 15,000 U.S. ninth graders from the High School Longitudinal Study of 2009 to investigate differences in ninth-grade math course placement at the intersection of adolescents' learning disability status, race, and socioeconomic status (SES). Descriptive results support an increased liability perspective, with the negative relationship between a learning disability and math course placement larger for adolescents more privileged in terms of their race and/or SES. Adjusted results suggest that the lower math course placements of youth with learning disabilities are due to cumulative disadvantage rather than disability-related inequities in the transition to high school for youth of diverse racial and socioeconomic backgrounds. In addition to demonstrating the importance of intersectional perspectives, this study provides a roadmap for future studies by introducing the new perspective of increased liability to be used in conjunction with the widely employed perspective of multiple marginalization.
There have been numerous efforts documenting the effects of open science in existing papers; however, these efforts typically only consider the author's analyses and supplemental materials from the papers. While understanding the current rate of open science adoption is important, it is also vital that we explore the factors that may encourage such adoption. One such factor may be publishing organizations setting open science requirements for submitted articles: encouraging researchers to adopt more rigorous reporting and research practices. For example, within the education technology discipline, the ACM Conference on Learning @ Scale (L@S) has been promoting open science practices since 2018 through a Call For Papers statement. The purpose of this study was to replicate previous papers within the proceedings of L@S and compare the degree of open science adoption and robust reproducibility practices to other conferences in education technology without a statement on open science. Specifically, we examined 93 papers and documented the open science practices used. We then attempted to reproduce the results with invitation from authors to bolster the chance of success. Finally, we compared the overall adoption rates to those from other conferences in education technology. Although the overall responses to the survey were low, our cursory review suggests that researchers at L@S might be more familiar with open science practices compared to the researchers who published in the International Conference on Artificial Intelligence in Education (AIED) and the International Conference on Educational Data Mining (EDM): 13 of 28 AIED and EDM responses were unfamiliar with preregistrations and 7 unfamiliar with preprints, while only 2 of 7 L@S responses were unfamiliar with preregistrations and 0 with preprints. The overall adoption of open science practices at L@S was much lower with only 1% of papers providing open data, 5% providing open materials, and no papers had a preregistration. All openly accessible work can be found in an Open Science Framework project(1).
Feedback-based iterative refinement is important in the development of any human-computer interface. The present work aims to evaluate and iteratively refine an immersive learning environment called Scale Worlds (SW), delivered via a head-mounted display (HMD). SW is a virtual learning environment encompassing scientific entities of a wide range of sizes that enables students an embodied experience while learning size and scale. Five usability experts performed think aloud while carrying out four interactive tasks in SW and compared three different design options during A/B testing. Improvement features based on the feedback from an earlier SW usability evaluation as well as HMD-specific features were examined. Usability experts completed the post-study system usability questionnaire, the NASA task load index, and a bipolar laddering survey that collected subjective perception of specific SW features. Results show that the progress panel (an improvement feature) was informative while the instructions (another improvement feature) caused clutter. The experts indicated clear usability preferences during A/B testing, which helped resolve three sets of theory-usability conflicts. The overall assessment of SW paved a path for theory-usability balance and provided valuable insights for designing and evaluating usability in immersive virtual learning environments.
Computer science learning in primary school classrooms has expanded, necessitating effective instructional strategies for this age group. GalleryWalks are a common activity to allow peers to share their work and give feedback on peers' work. Like other skills, providing effective feedback may require scaffolding and/or instruction for some students. Currently, there is little work exploring how CS students provide peer feedback without extensive instruction, especially at the primary school level (aged 5-11). We analyzed the feedback 4th grade students gave on a structured worksheet during a gallery walk. We found that students often provided both compliments and suggestions when prompted, but their feedback focused primarily on program aesthetics (e.g., characters, sounds, or storyline) rather than programmed elements, even when directed to focus on the program. In addition, bilingual and non-bilingual classes differed in how students followed the worksheet structure.
Motivation. Teachers can play a role in disrupting social inequities that are reflected in education, such as racial disparities in who succeeds in CS. Professional learning addressing inequities causes teachers to confront difficult topics, including how their own identities impact these problems. Understanding the differing ways teachers' identities surface can provide insights into designing better supports for their professional learning. Objectives. The goal of this paper is to examine the teaching and racial identities of two secondary CS teachers who participated in professional learning focused on combining CS content and equity pedagogy. The second goal of this paper is to demonstrate how discourse analytic methods can be used to examine interviews and other interactional data. Method. Teachers were interviewed individually about their teaching identity, racial identity, and professional learning. Drawing on Bucholtz and Hall's identity and interaction framework, interviews were examined for linguistic and discursive features reflecting positionality (i.e., how identity surfaces through the way individuals present themselves to and are perceived by others) and indexicality (i.e., various ways of referring to an identity). Results. Participants used personal deictics, quotative markers, code choice, and affective and epistemic stances when discussing and negotiating their identities with the interviewer. The data reflected ways teachers problematized questions about teaching identity, negotiated tensions in their disciplinary identities, found the topic of race difficult to address, and highlighted other aspects of their identities relevant to understanding and discussing race. Discussion.The study provides a demonstration of how discourse analytic methods can reveal nuances of teacher identity that may be overlooked with other qualitative approaches. Findings also revealed how teachers' ethnic identities might be used as a lever in helping teachers discuss the difficult topic of race in education. Discourse analytic methods are encouraged for future CS education research focused on interactional analyses.
This work proposes Dynamic Linear Epsilon-Greedy, a novel contextual multi-armed bandit algorithm that can adaptively assign personalized content to users while enabling unbiased statistical analysis. Traditional A/B testing and reinforcement learning approaches have trade-offs between empirical investigation and maximal impact on users. Our algorithm seeks to balance these objectives, allowing platforms to personalize content effectively while still gathering valuable data. Dynamic Linear Epsilon-Greedy was evaluated via simulation and an empirical study in the ASSISTments online learning platform. In simulation, Dynamic Linear Epsilon-Greedy performed comparably to existing algorithms and in ASSISTments, slightly increased students' learning compared to A/B testing. Data collected from its recommendations allowed for the identification of qualitative interactions, which showed high and low knowledge students benefited from different content. Dynamic Linear Epsilon-Greedy holds promise as a method to balance personalization with unbiased statistical analysis. All the data collected during the simulation and empirical study are publicly available at https://osf.io/zuwf7/.
This study examines how feminist academic administrators engender solidarity and practice feminist principles as leaders in United States higher education institutions. We draw from qualitative interview data with 27 self-identified feminist academic leaders about how they carry out this work, what obstacles they face, and the ways that their work disrupts—and is disrupted by—the intensifying neoliberal, managerial tendencies in higher education. Respondents shared experiences of promoting solidarity through their leadership and strove to create inclusive and equitable environments to benefit students, staff, and faculty, and especially minoritized individuals within these groups. Our analysis reveals how these feminist administrators applied a feminist ethic, engendered solidarity in their work, and were often keenly aware of—and willing to contest—the neoliberal context of their institutions and higher education more broadly. Our findings contribute to the sociological and cross-disciplinary literature on feminist leaders in academic institutions and the resistance against neoliberalism and managerialism practices from within academia.
Despite the rapid expansion of higher education, many young adults still enter the labor market without a college education. However, little research has focused on racial/ethnic earnings disadvantages faced by non-college-educated youth. We analyze the restricted-use data from the High School Longitudinal Study of 2009 to examine racial/ethnic earnings disparities among non-college-educated young men and women in their early 20s as of 2016, accounting for differences in premarket factors and occupation with an extensive set of controls. Results suggest striking earnings disadvantages for Black men relative to white, Latinx, and Asian men. Compared to white men, Latinx and Asian men do not earn significantly less, yet their earnings likely differ substantially by ethnic origin. While racial/ethnic earnings gaps are less prominent among women than men, women of all racial/ethnic groups have earnings disadvantages compared to white men. The results call for future studies into the heterogeneity within racial/ethnic groups and the intersectionality of race/ethnicity and gender among non-college-educated young adults.


