Publications
The past few years have seen a surge of interest in deep neural networks. The wide application of deep learning in other domains such as image classification has driven considerable recent interest and efforts in applying these methods in educational domains. However, there is still limited research comparing the predictive power of the deep learning approach with the traditional feature engineering approach for common student modeling problems such as sensor-free affect detection. This paper aims to address this gap by presenting a thorough comparison of several deep neural network approaches with a traditional feature engineering approach in the context of affect and behavior modeling. We built detectors of student affective states and behaviors as middle school students learned science in an open-ended learning environment called Betty's Brain, using both approaches. Overall, we observed a tradeoff where the feature engineering models were better when considering a single optimized threshold (for intervention), whereas the deep learning models were better when taking model confidence fully into account (for discovery with models analyses).
Online learning environments allow for the implementation of psychometric scales on diverse samples of students participating in authentic learning tasks. One such scale, the Intrinsic Motivation Inventory (IMI) can be used to inform stakeholders of students' subjective motivational and regulatory styles. The IMI is a multidimensional scale developed in support of Self-Determination Theory [1-3], a strongly validated theory stating that motivation and regulation are moderated by three innate needs: autonomy, belonging, and competence. As applied to education, the theory posits that students who perceive volition in a task, those who report stronger connections with peers and teachers, and those who perceive themselves as competent in a task are more likely to internalize the task and excel. ASSISTments, an online mathematics platform, is hosting a series of randomized controlled trials targeting these needs to promote integrated learning. The present work supports these studies by attempting to validate four subscales of the IMI within ASSISTments. Iterative factor analysis and item reduction techniques are used to optimize the reliability of these subscales and limit the obtrusive nature of future data collection efforts. Such scale validation efforts are valuable because student perceptions can serve as powerful covariates in differentiating effective learning interventions.
We model collaborative problem solving outcomes using data from 37 triads who completed a challenging computer programming task. Participants individually rated their group's performance, communication, cooperation, and agreeableness after the session, which were aggregated to produce group-level measures of subjective outcomes. We scored teams on objective task outcomes and measured individual students' learning outcomes with a posttest. Groups with similar personalities performed better on the task and had higher ratings of communication, cooperation, and agreeableness. Importantly, greater deviation in teammates' perception of group performance and higher ratings of communication, cooperation, and agreeableness negatively predicted individual learning. We discuss findings from the perspective of group work norms and consider applications to intelligent systems that support collaborative problem solving.
Affect and metacognition play a central role in learning. We examine the relationships between students' affective state dynamics, metacognitive judgments, and performance during learning with MetaTutorIVH, an advanced learning technology for human biology education. Student emotions were tracked using facial expression recognition embedded within MetaTutorIVH and transitions between emotions theorized to be important to learning (e.g., confusion, frustration, and joy) are analyzed with respect to likelihood of occurrence. Transitions from confusion to frustration were observed at a significantly high likelihood, although no differences in performance were observed in the presence of these affective states and transitions. Results suggest that the occurrence of emotions have a significant impact on students' retrospective confidence judgments, which they made after submitting their answers to multiple-choice questions. Specifically, the presence of confusion and joy during learning had a positive impact on student confidence in their performance while the presence offrustration and transition from confusion to frustration had a negative impact on confidence, even after accounting for individual differences in multiple-choice confidence.
Learning by drawing can be an effective strategy for supporting science text comprehension. However, drawing can also be cognitively demanding and time consuming, and students may not create quality drawings without sufficient guidance. Furthermore, evidence for drawing is often based on comparisons to weak control conditions, such as students who only read the text without provided illustrations. In this review, we synthesize past research to help draw boundary conditions for learning by drawing, focusing on the role of comparison conditions and drawing guidance. First, we analyze how drawing compares to each of four control conditions: reading only, text-focused strategies (e.g., summarizing), other model-focused strategies (e.g., imagining), or viewing instructor-provided illustrations. Next, we distinguish among four levels of drawing guidance: minimal guidance, drawing training, partially provided illustrations, and comparison to instructor-provided illustrations. Our findings indicate that when compared to only reading the text or using text-focused strategies, creating drawings is consistently more effective at fostering comprehension and transfer, regardless of the level of drawing guidance provided. However, when compared to other model-focused strategies or to viewing instructor-provided illustrations, effects of creating drawings are mixed and may depend on the level of drawing guidance provided, among other factors. We discuss the theoretical and practical considerations of our findings and suggest several directions for broadening research on drawing.
Social isolation is broadly associated with poor mental health and risky behaviors in adolescence, a time when peers are critical for healthy development. However, expectations for isolates' substance use remain unclear. Isolation in adolescence may signal deviant attitudes or spur self-medication, resulting in higher substance use. Conversely, isolates may lack access to substances, leading to lower use. Although treated as a homogeneous social condition for teens in much research, isolation represents a multifaceted experience with structurally distinct network components that present different risks for substance use. This study decomposes isolation into conceptually distinct dimensions that are then interacted to create a systematic typology of isolation subtypes representing different positions in the social space of the school. Each isolated position's association with cigarette, alcohol, and marijuana use is tested among 9(th) grade students (n = 10,310, 59% female, 83% white) using cross-sectional data from the PROSPER study. Different dimensions of isolation relate to substance use in distinct ways: unliked isolation is associated with lower alcohol use, whereas disengagement and outside orientation are linked to higher use of all three substances. Specifically, disengagement presents risks for cigarette and marijuana use among boys, and outside orientation is associated with cigarette use for girls. Overall, the adolescents disengaged from their school network who also identify close friends outside their grade are at greatest risk for substance use. This study indicates the importance of considering the distinct social positions of isolation to understand risks for both substance use and social isolation in adolescence.
Author name ambiguity in a digital library may affect the findings of research that mines authorship data of the library. This study evaluates author name disambiguation in DBLP, a widely used but insufficiently evaluated digital library for its disambiguation performance. In doing so, this study takes a triangulation approach that author name disambiguation for a digital library can be better evaluated when its performance is assessed on multiple labeled datasets with comparison to baselines. Tested on three types of labeled data containing 5000 to 6 M disambiguated names, DBLP is shown to assign author names quite accurately to distinct authors, resulting in pairwise precision, recall, and F1 measures around 0.90 or above overall. DBLP's author name disambiguation performs well even on large ambiguous name blocks but deficiently on distinguishing authors with the same names. Compared to other disambiguation algorithms, DBLP's disambiguation performance is quite competitive, possibly due to its hybrid disambiguation approach combining algorithmic disambiguation and manual error correction. A discussion follows on strengths and weaknesses of labeled datasets used in this study for future efforts to evaluate author name disambiguation on a digital library scale.
In supervised machine learning for author name disambiguation, negative training data are often dominantly larger than positive training data. This paper examines how the ratios of negative to positive training data can affect the performance of machine learning algorithms to disambiguate author names in bibliographic records. On multiple labeled datasets, three classifiers-Logistic Regression, Naive Bayes, and Random Forest-are trained through representative features such as coauthor names, and title words extracted from the same training data but with various positive-to-negative training data ratios. Results show that increasing negative training data can improve disambiguation performance but with a few percent of performance gains and sometimes degrade it. Logistic and Naive Bayes learn optimal disambiguation models even with a base ratio (1:1) of positive and negative training data. Also, the performance improvement by Random Forest tends to quickly saturate roughly after 1:10 similar to 1:15. These findings imply that contrary to the common practice using all training data, name disambiguation algorithms can be trained using part of negative training data without degrading much disambiguation performance while increasing computational efficiency. This study calls for more attention from author name disambiguation scholars to methods for machine learning from imbalanced data.
Initial research has shown that simulating data from models created with computer software may enhance students' understanding of concepts in introductory statistics; yet, there is little research investigating students' development of statistical models. The research presented here examines small groups of students as they develop a model for a situation where a music teacher plays ten notes for a student who tries to guess each of the notes correctly. As students constructed their models and described their thinking, their descriptions were narrative in nature, focusing on the story of notes played and guessed. In this context, their focus on narrative appeared to support the development of productive statistical models. In addition, when students investigated pre-built TinkerPlots models, they preferred models that they perceived as more communicative or narrative in nature. These results have important pedagogical implications in terms of designing modeling curriculum.
Real estate property value analysis is used for municipal taxation and budgeting. Commercial properties make up a large percentage of the property tax base in many, if not most, taxing jurisdictions. Data constraints limit the number of analyses conducted on commercial property value patterns. This study employs a fairly extensive data set to address that problem in the context of El Paso, Texas, a large metropolitan economy located on the United States border with Mexico. The sample contains data for 105,611 commercial real estate parcels. Empirical analysis is conducted using geographically weighted regression analysis. Results confirm that parameter estimation for the commercial property data in this sample should be conducted using methodologies that allow for spatial autocorrelation and heteroscedasticity.


