Publications
This study develops a framework to conceptualize the use and evolution of machine learning (ML) in science assessment. We systematically reviewed 47 studies that applied ML in science assessment and classified them into five categories: (a) constructed response, (b) essay, (c) simulation, (d) educational game, and (e) inter-discipline. We compared the ML-based and conventional science assessments and extracted 12 critical characteristics to map three variables in a three-dimensional framework:construct,functionality, andautomaticity. The 12 characteristics used to construct a profile for ML-based science assessments for each article were further analyzed by a two-step cluster analysis. The clusters identified for each variable were summarized into four levels to illustrate the evolution of each. We further conducted cluster analysis to identify four classes of assessment across the three variables. Based on the analysis, we conclude that ML has transformed-but notyetredefined-conventional science assessment practice in terms of fundamental purpose, the nature of the science assessment, and the relevant assessment challenges. Along with the three-dimensional framework, we propose five anticipated trends for incorporating ML in science assessment practice for future studies: addressing developmental cognition, changing the process of educational decision making, personalized science learning, borrowing 'good' to advance 'good', and integrating knowledge from other disciplines into science assessment.
Word learning is a social act. Because there is an arbitrary relation between words and their meaning, children must learn words from other people. Other people, however, are not always reliable sources of knowledge. People can be ignorant, hold false beliefs, or simply be deceptive. How do children evaluate the reliability of sources of knowledge for word learning? This chapter investigates the possibility that children possess multiple mechanisms for evaluating such reliability and possess such mechanisms very early in development. We suggest that infants not only track the accuracy of others using statistical learning mechanisms but also incorporate their existing knowledge of the world into judgments of reliability to make judicious inferences about the knowledge of a speaker and the pragmatics of a communicative act. Moreover, we suggest that as children get older, both low-level associative mechanisms and higher-level cognitive processes influence the way in which children track and use others’ reliability as sources of knowledge. (PsycInfo Database Record (c) 2022 APA, all rights reserved)
Women are traditionally underrepresented in science, technology, engineering, and mathematics (STEM). While the representation of women in STEM classrooms has grown rapidly in recent years, it remains pedagogically meaningful to understand whether their learning outcomes are achieved in different ways than male students. In this study, we explored this issue through the lens of language in the context of an asynchronous online discussion forum. We applied Linguistic Inquiry and Word Count (LIWC) to examine linguistic features of students' reflective posting in an online chemistry class at a four-year university. Our results suggest that cognitive linguistic features significantly predict the likelihood of passing the course and increases perceived sense of belonging. However, these results only hold true for female students. Pronouns and words relevant to social presence correlate with passing the course in different directions, and this mixed relationship is more polarized among male students. Interestingly, the linguistic features per se do not differ significantly between genders. Overall, our findings provide a more nuanced account of the relationship between linguistic signals of social/cognitive presence and learning outcomes. We conclude with implications for pedagogical interventions and system design to inclusively support learner success in online STEM courses.
Commercial facial affect detection software is typically trained on large databases and achieves high accuracy in detecting basic emotions, but their use in educational settings is unclear. The goal of this research is to determine how basic emotions relate to the achievement emotion states that are more relevant in academic settings. Such relations, if accurate and consistent, may be leveraged tomake more effective use of the commercial affect-detection software. For this study, we collected affect data over four days from a classroom study with 65 students using Betty's Brain. Basic emotions obtained from commercial software were aligned to achievement emotions obtained using sensor-free models. Interpretable classifiers enabled the study of relationships between the two types of emotions. Our findings show that certain basic emotions can help infer complex achievement emotions such as confusion, frustration and engaged concentration. This suggests the possibility of using commercial software as a less context-sensitive and more development-friendly alternative to the affect detector models currently used in learning environments.
Assessment in the context of foreign language learning can be difficult and time-consuming for instructors. Distinctive from other domains, language learning often requires teachers to assess each student's ability to speak the language, making this process even more time-consuming in large classrooms which are particularly common in post-secondary settings; considering that language instructors often assess students through assignments requiring recorded audio, a lack of tools to support such teachers makes providing individual feedback even more challenging. In this work, we seek to explore the development of tools to automatically assess audio responses within a college-level Chinese language-learning course. We build a model designed to grade student audio assignments with the purpose of incorporating such a model into tools focused on helping both teachers and students in real classrooms. Building upon our prior work which explored features extracted from audio, the goal of this work is to explore additional features derived from tone and speech recognition models to help assess students on two outcomes commonly observed in language learning classes: fluency and accuracy of speech. In addition to the exploration of features, this work explores the application of Siamese deep learning models for this assessment task. We find that models utilizing tonal features exhibit higher predictive performance of student fluency while text-based features derived from speech recognition models exhibit higher predictive performance of student accuracy of speech.
Persistence is a crucial trait for learners. However, a common issue in mastery learning is that persistence is not always productive, a construct termed wheel-spinning. In this paper, we extend on prior work to develop wheel-spinning detectors in the ASSISTments learning system that distinguish between non-persistence, productive persistence and wheel-spinning. To understand how quickly we can detect each state, we use data from different numbers of practice opportunities and compare model performance across student-problem set pairs. We identify that a model constructed using data from the first nine practice opportunities outperforms models using less practice data. However, it is possible to differentiate students who will eventually wheel-spin from learners who will persist productively using data from only the first three opportunities. Wheel-spinning can be differentiated from non-persistence from the first five opportunities, and non-persistence can be differentiated from productive persistence from the first seven opportunities. These results show that early differentiation between wheel-spinning and productive persistence is feasible. These detectors relied upon hint requests, the correctness of prior opportunities, and the amount of practice and time on the skill. Identifying predictive features offer insights into the impact of in-system behaviors on wheel-spinning and guide the system design.
Collaborative game-based learning environments integrate gamebased learning and collaborative learning. These environments present students with a shared objective and provide them with a means to communicate, which allows them to share information, ask questions, construct explanations, and work together toward their shared goal. A key challenge in collaborative learning is that students may engage in unproductive discourse, which may affect learning activities and outcomes. Collaborative game-based learning environments that can detect this off-task behavior in real-time have the potential to enhance collaboration between students by redirecting the conversation back to more productive topics. This paper investigates the use of dialogue analysis to classify student conversational utterances as either off-task or on-task. Using classroom data collected from 13 groups of four students, we trained off-task dialogue models for text messages from a group chat feature integrated into Crystal Island: EcoJourneys, a collaborative game-based learning environment formiddle school ecosystem science. We evaluate the effectiveness of the off-task dialogue models, which use different word embeddings (i.e., word2vec, ELMo, and BERT), as well as predictive off-task dialogue models that capture varying amounts of contextual information from the chat log. Results indicate that predictive off-task dialogue models that incorporate a window of recent context and represent the sequential nature of the chat messages achieve higher predictive performance compared to models that do not leverage this information. These findings suggest that off-task dialogue models for collaborative game-based learning environments can reliably recognize and predict students' off-task behavior, which introduces the opportunity to adaptively scaffold collaborative dialogue.
Game-based learning environments enable students to engage in authentic, inquiry-based learning. Reflective thinking serves a critical role in inquiry-based learning by encouraging students to think critically about their knowledge and experiences in order to foster deeper learning processes. Freeresponse reflection prompts can be embedded in game-based learning environments to encourage students to engage in reflection and externalize their reflection processes, but automatically assessing student reflection presents significant challenges. In this paper, we present a framework for automatically assessing students' written reflection responses during inquiry-based learning in Crystal Island, a game-based learning environment for middle school microbiology. Using data from a classroom study involving 153 middle school students, we compare the effectiveness of several computational representations of students' natural language responses to reflection prompts-GloVe, ELMo, tf-idf, unigrams-across severalmachine learning-based regression techniques (i.e., random forest, support vector machine, multi-layer perceptron) to assess the depth of student reflection responses. Results demonstrate that assessment models based on ELMo deep contextualized word representations yield more accurate predictions of students' written reflection depth than competing techniques. These findings point toward the potential of leveraging automated assessment of student reflection to inform real-time adaptive support for inquiry-based learning in game-based learning environments.
We examine the use of computer-based learning in the classroom and the effect of immediate feedback on student performance. Since it is well known in educational research that it is possible to observe a Matthew Effect in which the rich get richer, we wanted to see if feedback was useful for low prior knowledge students, as defined by students whose pretest score was at or below the median. In this counterbalanced randomized controlled trial, 243 tenth and eleventh grade mathematics students were exposed to one of two conditions, as we measured their learning from: 1) immediate feedback (where the computer told them correctness and they could also ask for hints) or 2) practice only (where they received feedback only after taking a posttest). Results suggest that immediate feedback from computer-based learning tasks benefit both high and low prior knowledge students, with low prior knowledge students exhibiting greater gains. The implications of these findings support further investigation into the use of computer-based learning tasks that provide immediate feedback.
Cardiomyocyte L-type Ca2+ channels (Ca(v)s) are targets of signaling pathways that modulate channel activity in response to physiologic stimuli. Ca(v)regulation is typically transient and beneficial but chronic stimulation can become pathologic; therefore, gaining a more complete understanding of Ca(v)regulation is of critical importance. Intracellular O-linked glycosylation (O-GlcNAcylation), which is the result of two enzymes that dynamically add and remove single N-acetylglucosamines to and from intracellular serine/threonine residues (OGT and OGA respectively), has proven to be an increasingly important post-translational modification that contributes to the regulation of many physiologic processes. However, there is currently no known role for O-GlcNAcylation in the direct regulation of Ca-v activity nor is its contribution to cardiac electrical signaling and EC coupling well understood. Here we aimed to delineate the role of O-GlcNAcylation in regulating cardiomyocyte L-type Ca-v activity and its subsequent effect on EC coupling by utilizing a mouse strain possessing an inducible cardiomyocyte-specific OGT-null-transgene. Ablation of the OGT-gene in adult cardiomyocytes (OGTKO) reduced OGT expression and O-GlcNAcylation by > 90%. Voltage clamp recordings indicated an similar to 40% reduction in OGTKO Ca(v)current (I-Ca), but with increased efficacy of adrenergic stimulation, and Ca-v steady-state gating and window current were significantly depolarized. Consistently, OGTKO cardiomyocyte intracellular Ca2+ release and contractility were diminished and demonstrated greater beat-to-beat variability. Additionally, we show that the Ca-v alpha 1 and beta 2 subunits are O-GlcNAcylated while alpha 2 delta 1 is not. Echocardiographic analyses indicated that the reductions in OGTKO cardiomyocyte Ca2+ handling and contractility were conserved at the whole-heart level as evidenced by significantly reduced left-ventricular contractility in the absence of hypertrophy. The data indicate, for the first time, that O-GlcNAc signaling is a critical and direct regulator of cardiomyocyteI(Ca)achieved through altered Ca(v)expression, gating, and response to adrenergic stimulation; these mechanisms have significant implications for understanding how EC coupling is regulated in health and disease.


