Publications
Handwriting practice is the most time-consuming activity for learners of Chinese as a foreign language (CFL). CFL instructors report allocating at least one third of their course time to handwriting practice although it prevents students from engaging in meaningful communication, especially in the earliest stages of learning. Given the amount of time students spend in a college course is relatively fixed, the preregistered study presented herein examines the best use of students' time when primary goals are word acquisition and communication. This work replicates a pilot study examining CFL word recognition in an online learning environment (ASSISTments) and the effects of supplemental handwriting practice. We examined word acquisition and recognition while manipulating condition (no-handwriting practice and with-handwriting practice), and posttest test point (1 [immediate], 2 [1 day delay], and 3 [1 week delay]). Two-way repeated measures analyses of variance revealed significant main effects for both condition and posttest test point in online and on-paper measures of word recognition and handwriting. Potential implications for CFL instruction and directions for future work are discussed.
Because of their complex structures, many natural-science categories are difficult to learn. Yet achieving accuracy in classification is crucial to scientific inference and reasoning. Thus, an emerging theme in cognitive-psychology and cognitive-science research has been to investigate better ways to instruct about categories. This article briefly reviews major findings that will help inform policies for teaching categories in the science classroom. Many of the examples come from our specific project that examines teaching rock classifications in the geologic sciences. This project uses formal models of human category learning—developed in cognitive psychology—to search for optimal teaching procedures. The model-suggested category-teaching procedures often lead to better learning outcomes than do alternative procedures motivated by teachers’ and students’ intuitive judgments. In addition to reviewing these enhanced procedures for teaching natural-science categories, the article points to recent broader efforts for fostering collaborations between cognitive-science researchers and education researchers. © The Author(s) 2018.
In this issue, Kindel et al. describe a new approach to managing survey data in service of the Fragile Families Challenge, which they call “treating metadata as data.” Although the approach they present is a good first step, a more ambitious proposal could improve survey data analysis even more substantially. The author recommends that data collection efforts distribute an open-source set of tools for working with a particular data set the author calls data-specific functions. The goal of these functions is to codify best practices for working with the data in a set of functions for commonly used statistical software. These functions would be jointly developed by the users and distributers of the data. Building such functions would both shorten the learning curve for new users and improve the quality of the data, by making tacit knowledge about problems with the data explicit and easy to act on. © SAGE Publications Inc.. All rights reserved.
Background Most science categories are hierarchically organized, with various high-level divisions comprising numerous subtypes. If we suppose that one's goal is to teach students to classify at the high level, past research has provided mixed evidence about whether an effective strategy is to require simultaneous classification learning of the subtypes. This past research was limited, however, either because authentic science categories were not tested, or because the procedures did not allow participants to form strong associations between subtype-level and high-level category names. Here we investigate a two-stage response-training procedure in which participants provide both a high-level and subtype-level response on most trials, with feedback provided at both levels. The procedure is tested in experiments in which participants learn to classify large sets of rocks that are representative of those taught in geoscience classes. Results The two-stage procedure yielded high-level classification performance that was as good as the performance of comparison groups who were trained solely at the high level. In addition, the two-stage group achieved far greater knowledge of the hierarchical structure of the categories than did the comparison controls. Conclusion In settings in which students are tasked with learning high-level names for rock types that are commonly taught in geoscience classes, it is best for students to learn simultaneously at the high and subtype levels (using training techniques similar to the presently investigated one). Beyond providing insights into the nature of category learning and representation, these findings have practical significance for improving science education.
Vision and Change challenged biology instructors to develop evidence-based instructional approaches that were grounded in the core concepts and competencies of biology. This call for reform provides an opportunity for new educational toots to be incorporated into biology education. In this essay, we advocate for learning progressions as one such educational tool. First, we address what learning progressions are and how they leverage research from the cognitive and learning sciences to inform instructional practices. Next, we use a published learning progression about carbon cycling to illustrate how learning progressions describe the maturation of student thinking about a key topic. Then, we discuss how learning progressions can inform undergraduate biology instruction, citing three particular learning progressions that could guide instruction about a number of key topics taught in introductory biology courses. Finally, we describe some challenges associated with learning progressions in undergraduate biology and some recommendations for how to address these challenges.
Faculty and peer interactions play a key role in shaping graduate student socialization. Yet, within the literature on graduate student socialization, researchers have primarily focused on understanding the nature and impact of faculty atone, and much less is known about how peer interactions also contribute to graduate student outcomes. Using a national sample of first-year biology doctoral students, this study reveals distinct categories that classify patterns of faculty and peer interaction. Further, we document inequities such that certain groups (e.g., underrepresented minority students) report constrained types of interactions with faculty and peers. Finally, we connect faculty and peer interaction patterns to student outcomes. Our findings reveal that, while the classification of faculty and peer interactions predicted affective and experiential outcomes (e.g., sense of belonging, satisfaction with academic development), it was not a consistent predictor of more central outcomes of the doctoral socialization process (e.g., research skills, commitment to degree). These and other findings are discussed, focusing on implications for future research, theory, and practice related to graduate training.
Economic downturns are known to spark periods of increased enrolment in traditional educational pursuits. The current study leverages 30-year longitudinal data from the Longitudinal Study of American Life (LSAL; N=1,556) to examine individual characteristics and experiences in adolescence, just prior to the Great Recession, and during it, to understand why some individuals chose to pursue new education or training in response to the recession whereas others did not. Indicators from adolescence include measures of self-esteem, locus of control, persistence, achievement in mathematics and achievement in science and were collected from 1987 to 1993. inclusive. Pre-recession indicators include level of education, occupational and marital status and were collected in 2007. Indicators of the impact of the recession were collected retrospectively in 2014 and include whether a job was lost, whether work hours were reduced, and whether there was difficulty making rent/mortgage payments. Binary logistic regression identified persistence in adolescence, pre-recession education level, reporting reduced hours and difficulty paying rent/mortgage during the recession as associated with the likelihood of pursuing new education during the recession. A follow-up analysis investigated whether the pursuit of additional education/training in response to the recession predicted the likelihood of being employed in 2017. Results indicate that obtaining new education during the recession was associated with later employment status, but the significance and direction of the effect depends on pre-recession education level. Implications of this longitudinal, life course analysis are discussed in addition to recommendation for future directions.
A growing number of methods aim to assess the challenging question of treatment effect variation in observational studies. This special section of Observational Studies reports the results of a workshop conducted at the 2018 Atlantic Causal Inference Conference designed to understand the similarities and differences across these methods. We invited eight groups of researchers to analyze a synthetic observational data set that was generated using a recent large-scale randomized trial in education. Overall, participants employed a diverse set of methods, ranging from matching and flexible outcome modeling to semiparametric estimation and ensemble approaches. While there was broad consensus on the topline estimate, there were also large differences in estimated treatment effect moderation. This highlights the fact that estimating varying treatment effects in observational studies is often more challenging than estimating the average treatment effect alone. We suggest several directions for future work arising from this workshop. © 2019 Carlos Carvalho, Avi Feller, Jared Murray, Spencer Woody, and David Yeager.
State accountability systems have been a primary school reform initiative in the US for the past 20 years, but often produce unintended negative consequences. In 2004, the Texas Education Agency (TEA) implemented the Performance Based Monitoring and Analysis System (PBMAS), which included an accountability indicator focused on the percentage of students found eligible for special education under the Individuals with Disabilities Education Act (IDEA), the nation’s special education law. From 2004 through 2016, the percentage of students found eligible for special education in Texas declined significantly, while the national rate held constant. Eventually, the U.S. Department of Education (ED) investigated TEA and the statewide implementation of IDEA. The purpose of this study is two-fold: (a) to evaluate the potential impact of the the PBMAS indicator on manipulation of special education identification practices; and (b) to describe how the indicator may have influenced school and district personnel. We highlight several concerning trends in state and district data and, through an analysis of publicly available reports from the ED, show how district and school personnel knowingly and unknowingly acted in ways that delayed and denied special education to potentially eligible students. We conclude with recommendations for TEA and implications for future research and policy. © 2019, Arizona State University. All rights reserved.
Randomized controlled trials (RCTs) admit unconfounded design-based inference--randomization largely justifies the assumptions underlying statistical effect estimates--but often have limited sample sizes. However, researchers may have access to big observational data on covariates and outcomes from RCT non-participants. For example, data from A/B tests conducted within an educational technology platform exist alongside historical observational data drawn from student logs. We outline a design-based approach to using such observational data for variance reduction in RCTs. First, we use the observational data to train a machine learning algorithm predicting potential outcomes using covariates, and use that algorithm to generate predictions for RCT participants. Then, we use those predictions, perhaps alongside other covariates, to adjust causal effect estimates with a flexible, design-based covariate-adjustment routine. In this way there is no danger of biases from the observational data leaking into the experimental estimates, which are guaranteed to be exactly unbiased regardless of whether the machine learning models are "correct" in any sense or whether the observational samples closely resemble RCT samples. We demonstrate the method in analyzing 33 randomized A/B tests, and show that it decreases standard errors relative to other estimators, sometimes substantially. [This is the online version of an article published in "Journal of Causal Inference." Additional funding was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN) program.]


