Publications
Countries, research institutions, and scholars are interested in identifying and promoting high-impact and transformative scientific research. This paper presents a novel set of text-and citation-based metrics that can be used to identify high-impact and transformative works. The 11 metrics can be grouped into seven types: Radical-Generative, Radical-Destructive, Risky, Multidisciplinary, Wide Impact, Growing Impact, and Impact (overall). The metrics are exemplified, validated, and compared using a set of 10,778,696 MEDLINE articles matched to the Science Citation Index Expanded (TM). Articles are grouped into six 5-year periods (spanning 1983-2012) using publication year and into 6,159 fields constructed using comparable MeSH terms, with which each article is tagged. The analysis is conducted at the level of a field-period pair, of which 15,051 have articles and are used in this study. A factor analysis shows that transformativeness and impact are positively related (rho = .402), but represent distinct phenomena. Looking at the subcomponents of transformativeness, there is no evidence that transformative work is adopted slowly or that the generation of important new concepts coincides with the obsolescence of existing concepts. We also find that the generation of important new concepts and highly cited work is more risky. Finally, supporting the validity of our metrics, we show that work that draws on a wider range of research fields is used more widely.
With the nationwide emphasis on improving outcomes for STEM undergraduates, it is important that we not only focus on modifying classroom instruction, but also provide students with the tools to maximize their independent learning time. There has been considerable work in laboratory settings examining two beneficial practices for enhancing learning: spacing and self-testing. In the current study, we examine biology students' study practices, particularly in the context of these two behaviors. We specifically investigate whether a light-touch study skills intervention focused on encouraging spacing and self-testing practices impacted their utilization. Based on pre- and post-course surveys, we found that students report utilizing both beneficial and ineffective study practices and confirm that usage of spacing and self-testing correlates with a higher course grade. We also found that students in the section of the course which received the study skills intervention were more likely to report continued use or adoption of spacing and self-testing compared to students in control sections without the intervention. Surprisingly, we found that underrepresented minorities (URMs) under-utilize self-testing, and that our intervention helped to partially ameliorate this gap. Additionally, we found that URMs who reported self-testing earned similar course grades compared to non-URMs who also self-tested, but that there was a much larger drop in performance for URMs who did not self-test relative to non-URMs who also did not selftest. Overall, we would encourage instructors to dedicate class time towards discussing the merits of beneficial study practices, especially for students that have historically underperformed in STEM disciplines.
Massive Open Online Courses (MOOCs) have the potential to democratize education by providing learners with access to high-quality free online courses. However, evidence supporting this democratization across countries is limited. We explored the question of MOOC democratization by conducting cross-national comparisons of gender differences in the enrollment in and completion of science, technology, engineering, and mathematics (STEM) MOOCs. We found that while females were less likely than males to enroll in STEM MOOCs, they were equally likely to complete them. Further, a higher probability to enroll in STEM MOOCs and smaller gender gaps in STEM MOOC enrollment and completion were found in less gender-equal and less economically developed countries.
In this research article, Ross J. Benbow and Matthew T. Hora explore the employability narrative, a view that focuses on whether colleges and universities provide students with the skills they need to be productively employed after graduation. Using sociocultural theory to problematize this narrative and qualitative methods to foreground the experiences of postsecondary educators and employers, the authors investigate conceptions of essential workplace skills in biotechnology and manufacturing fields. Their results show that though work ethic, technical knowledge, and technical ability represent core competencies valued across these communities, considerable variation exists in how members of different disciplinary and occupational subgroups value and conceptualize important skills. They found that respondents' conceptions of skills were also strongly tied to geography and organizational culture, among other contextual factors. With these results in mind, the authors conclude that skills are best viewed as multifaceted and situated assemblages of knowledge, skill, and disposition-or cultural models-and urge the adoption of more nuanced views among educators, employers, and policy makers that take into account the cultural and contextual forces that shape student success in the workplace.
Prior research reports mixed results regarding the economic impacts of crime. This study employs data from all regions of Mexico, including border regions in both the north and the south, to examine the effects of homicides on retail activity across Mexico during a period of escalating violence. The results indicate that one additional homicide within a municipality eliminates one retail establishment and one paid job in the retail sector. Furthermore, the negative consequences of violent crime for retailers are augmented by proximity to an international border. This is consistent with previous research findings that cross-border shopping is a key feature of commerce along the international boundaries of Mexico. It suggests that crime waves may disproportionately impact border city retail activity by partially diverting customer traffic to stores located in neighboring countries. This result is also consistent with the finding of recent research that violent conflict in northern Mexico resulted in increased retail activity in some United States border cities.
Distance learning is expanding rapidly in universities. While theoretical and qualitative literature stress the critical role of effective interpersonal interactions in motivating students and facilitating learning in online environments, quantitative evidence on the benefits of increased interpersonal interactions on student learning outcomes is limited. This study examines the effect of providing a voluntary in-person meeting time in a physical classroom and increasing instructor email activity in a fully online precalculus course at a public university. We examine student final exam score and course grade as outcome variables. Student selection into courses was minimal since students only had access to one treatment condition at a time. We further used a propensity score matching strategy to address demographic variations in student characteristics across cohorts. Our results indicate that the increased interpersonal interaction opportunities increased final exam scores by 0.22 standard deviations and improved passing rates by 19 percentage points. Rosenbaum's sensitivity analysis indicates that it is unlikely that these results are due to omitted variable bias.
The purpose of this article is to introduce and review the capability and performance of the Stata item response theory (irt) package that is available from Stata V.14, 2015. Using a simulated data set and a publicly available item response data set extracted from Programme of International Student Assessment, we review the irt package from applied and methodological researchers' perspectives. After discussing the supported item response models and estimation methods implemented in the package, we demonstrate the accuracy of estimation compared to results from other typically used software packages. Other application features for differential item function analysis, scoring, and the package generating graphs are also reviewed.
Despite the early development of causal reasoning (CR), and its potential for shaping scientific literacy, we have little understanding of its structural origins. Specifically, is CR a unique capability that develops relatively independently or is it largely dependent on broader, more fundamental, cognitive abilities? Executive Functioning (EF) is an especially promising contributor to CR based on its already established role in related skills like planning and problem solving (e.g., Diamond, 2013). To begin exploring this potential relationship, we assessed 123 three (Mage = 3.42 years) and 64 five year olds’ (Mage = 5.36 years) performance on two CR tasks (counterfactual reasoning and causal inference), each of which we expected might be influenced in different ways by distinct EF skills. The counterfactual reasoning task (Guajardo & Turley-Ames, 2004) required children to generate alternative courses of action that would lead to different outcomes in fictional vignettes. The causal inference task (Das Gupta & Bryant, 1989) required children to compare pictures taken before and after a transformation (e.g., broken flowerpot and intact flowerpot) and to select a tool (e.g., glue) that could have caused it. We measured EF with three tasks: flanker (inhibition), count and label (working memory), and dimensional change card sort (cognitive flexibility). Finally, we measured children’s vocabulary and processing speed. To explore the relationship between EF and CR, we conducted a series of four linear regressions predicting causal inference and counterfactual reasoning ability in 3 and 5 year olds. Of all our measures, only vocabulary and inhibitory control emerged as significant predictors of causal inference ability for both 3 (βvocab = .04, p = .002, and βinhib = .04, p = .04) and 5 year olds (βvocab = .03, p = .01, and βinhib = .02, p = .04). Similarly, inhibitory control emerged as the only significant predictor of counterfactual reasoning in 3 year olds, βinhib = .03, p = .03. In contrast, for 5 year olds, working memory was the only significantly predictor of counterfactual reasoning, βWM = .71, p = .02. These results suggest that causal inference skills are stably supported by inhibitory control throughout early childhood. The story for counterfactual reasoning, however, appears to be somewhat more complex. Consistent with previous work (Beck, Riggs & Gorniak, 2009), inhibitory control supported counterfactual reasoning ability in our 3-year-old sample. However, inhibitory control did not significantly predict counterfactual reasoning in 5 year olds, it was supported by working memory instead. One explanation for this difference might have to do with the sophistication of children’s counterfactual reasoning skills at these different ages. Taken together, these results suggest that CR does not develop as a unique capacity, but instead likely relies on EFs that influence different CR skills in distinct ways across development. This represents an initial step in understanding early CR skills, which are promising contributors to emerging scientific literacy.
Since the emergence of middle schools as distinct educational settings in the 1960s, proponents of the model have advocated for structures and approaches that best meet the particular developmental needs of young adolescents. Middle school researchers have developed frameworks of best practices for schools that have been widely, if not uniformly, adopted. However, there is a paucity of large-scale quantitative research on the efficacy of such best practices. In this study we used state-level administrative data from Texas to estimate the school-level contribution to standardized test scores in math and language arts, along with absenteeism. We then regressed these value-added quantities on indicators of middle school structures, along with research-supported predictors of school efficacy. Results showed that schools with fewer classes in the school day and higher quality teachers performed better, among other indicators. Findings from models using the campus contribution to absenteeism were similar. These results indicate that while elements of the middle school model may help transform individual schools, the equitable distribution of resources and the undoing of de facto segregation are vital to the success of all young adolescents.


