Publications
The new capabilities of generative artificial intelligence tools Generative AI, such as ChatGPT, allow users to interact with the system in intuitive ways, such as simple conversations, and receive (mostly) good-quality answers. These systems can support students' learning objectives by providing accessible explanations and examples even with vague queries. At the same time, they can encourage undesired help-seeking behaviors by providing solutions to the students' homework. Therefore, it is important to better understand how students approach such tools and the potential issues such approaches might present for the learners. In this paper, we present a case study for understanding student-AI collaboration to solve programming tasks in the CS1 introductory programming course. To this end, we recruited a gender-balanced majority non-white set of 15 CS1 students at a large public university in the US. We observed them solving programming tasks. We used a mixed-method approach to study their interactions as they tackled Python programming tasks, focusing on when and why they used ChatGPT for problem-solving. We analyze and classify the questions submitted by the 15 participants to ChatGPT. Additionally, we analyzed user interaction patterns, their reactions to ChatGPT's responses, and the potential impacts of Generative AI on their perception of self-efficacy. Our results suggest that in about a third of the cases, the student attempted to complete the task by submitting the full description of the tasks to ChatGPT without making any effort on their own. We also observed that few students verified their solutions. We discuss the results and their potential implications.
The notion of being smart is a concept that underpins the culture of engineering classrooms. That said, it is a topic that is not discussed or addressed by educators directly. Through this special session, we aim to give light to the concept of smartness and the problematic and oppressive practices that result from it given the extent literature on the topic including our own research into the domain. With participants, we aim to generate practical approaches to addressing smartness that work in a variety of contexts to broaden participation in engineering via more inclusive classrooms.
This study explores the experiences of three Networked Improvement Communities (NICs) within mathematics departments as they work to critically transform their introductory mathematics programs. Drawing on the framework of dominant and critical axes of equity, we identify three key tensions experienced by the NICs: identity neutrality versus identity centrality, power over versus power with, and students as novices versus students as experts. These tensions are framed as productive enactments of working to change systems from within, highlighting the challenges and opportunities inherent in navigating the liminal space of nepantla. We argue that engaging with these tensions is crucial for fostering critical transformations and offer recommendations for leveraging the roles of identity, power, and student partnership to promote more equitable and inclusive mathematics education environments.
High-quality science assessments are multi-dimensional. They promote disciplinary practices, core ideas, cross-cutting concepts, and science sense-making. In this paper, we investigate the feasibility of using generative artificial intelligence (GenAI), specifically multimodal large language models (MLLMs), to annotate and provide improvement ideas for K-12 science assessments. The AI-generated annotations critique how the assessments align with the three dimensions of the Next Generation Science Standards (NGSS) and suggest ideas to elicit students' science sense-making. We outline our process with various prompting strategies: few-shot and zero-shot learning (prompting with and without examples), chain of thought (eliciting the MLLM's reasoning), and sampling strategies (outputting high or low level of randomness). Overall, the AI annotations can reason about the alignment between the assessments and NGSS dimensions and overlap with annotations from K-12 educators. Annotations generated with few-shot learning generally score higher overall and provide more details than zero-shot prompts. Further, interviews with science teachers reveal that the MLLM annotations can support teachers' reflection on instructional practices and assessment revision. We discuss the application of MLLMs to develop three-dimensional science assessments.
Background: Learning analytics (LA) research often aggregates learning process data to extract measurements indicating constructs of interest. However, the warranty that such aggregation will produce reliable measurements has not been explicitly examined. The reliability evidence of aggregate measurements has rarely been reported, leaving an implicit assumption that such measurements are free of errors. Objectives: This study addresses these gaps by investigating the psychometric pros and cons of aggregate measurements. Methods: This study proposes a framework for aggregating process data, which includes the conditions where aggregation is appropriate, and a guideline for selecting the proper reliability evidence and the computing procedure. We support and demonstrate the framework by analysing undergraduates' academic procrastination and programming proficiency in an introductory computer science course. Results and Conclusion: Aggregation over a period is acceptable and may improve measurement reliability only if the construct of interest is stable during the period. Otherwise, aggregation may mask meaningful changes in behaviours and should be avoided. While selecting the type of reliability evidence, a critical question is whether process data can be regarded as repeated measurements. Another question is whether the lengths of processes are unequal and individual events are unreliable. If the answer to the second question is no, segmenting each process into a fixed number of bins assists in computing the reliability coefficient. Major Takeaways: The proposed framework can be a general guideline for aggregating process data in LA research. Researchers should check and report the reliability evidence for aggregate measurements before the ensuing interpretation. © 2024 John Wiley & Sons Ltd.
A large body of existing research indicates that practice after an initial reading improves learning outcomes and is more beneficial than rereading. But can practice with feedback be effective even without upfront reading? Across three laboratory studies (n = 1,554), participants learned the same amount from practice with feedback regardless of whether they did the introductory reading or not, and they saved 60% of time when skipping the reading. These results suggest that if students have the opportunity to practice and receive feedback on their performance, additional passive instruction like textbook reading may be redundant and inefficient. Students might be better off focusing on practice and feedback for robust and efficient learning.
Gender disparities persist in postsecondary computing fields, despite improvements in postsecondary equity overall and STEM fields as an aggregate. The entrenchment of this issue requires a comprehensive, longitudinal lens. Building on expectancy-value theory, the present study examines the relationships among students’ gender-ability stereotypes, attainment values, course-taking, and major choices. Using data from the High School Longitudinal Study of 2009 (HSLS: 2009), we applied weighted t-tests and multiple-group structural equation modeling to investigate how motivational beliefs (i.e., gender-ability stereotypes, attainment values) and course-taking patterns in math and science may predict major choice in computing. Overall, we find gender differences in identity-based mathematics and science motivational beliefs have long-term effects. Gender-ability stereotypes in math and science shape attainment values in each domain, whereby stereotypes suppress girls’ attainment values and enhance boys? attainment values (p < 0.001), in turn shaping course-taking and major decisions. Math- and sciencerelated motivational and curricular factors affect “other” STEM more than computing major outcomes. Specifically, computer science course-taking is completed more by boys (d = 0.21), but girls’ chances of declaring computing majors are especially enhanced by completing these courses in high school. Advanced science course-taking and science attainment value positively predict boys’ but not girls’ likelihood of declaring computing majors. We discuss the implications of these findings for research, policy, and practice. © The Author(s), under exclusive licence to Springer Nature B.V. 2023.
For decades, politicians, policy makers, and industry leaders have told a disturbing narrative: An inferior educational system, this account goes, prevents many young Americans from entering STEM (science, technology, engineering, and mathematics) fields, weakening the nation's innovation and scientific enterprise. The resulting talent shortage, it reasons, can only be overcome by admitting larger numbers of foreign nationals who have the required skills. But this narrative misrepresents what actually drives large numbers of capable, ambitious, well-educated Americans away from STEM fields and toward careers in business, law, medicine, and other nonacademic endeavors: a university labor system that exploits would-be scientists.
Objects and places are foundational spatial domains represented in human symbolic expressions, like drawings, which show a prioritization of depicting small-scale object-shape information over the large-scale navigable place information in which objects are situated. Is there a similar object-over-place bias in language? Across six experiments, adults and 3- to 4-year-old children were asked either to extend a novel noun in a labeling phrase, to extend a novel noun in a prepositional phrase, or to simply match pictures. To dissociate specific object and place information from more general figure and ground information, participants either saw scenes with both place information (a room) and object information (a block in the room), or scenes with two kinds of object information that matched the figure-ground relations of the room and block by presenting an open container with a smaller block inside. While adults showed a specific object-over-place bias in both extending novel noun labels and matching, they did not show this bias in extending novel nouns following prepositions. Young children showed this bias in extending novel noun labels only. Spatial domains may thus confer specific and foundational biases for word learning that may change through development in a way that is similar to that of other word-learning biases about objects, like the shape bias. These results expand the symbolic scope of prior studies on object biases in drawing to object biases in language, and they expand the spatial domains of prior studies characterizing the language of objects and places.
This study aims to better understand norms that are potentially relevant for student engagement in peer critique within the modeling context.By analyzing multiple data from a 5 th /6 th grade classroom, we identified social and socioscientific norms within small groups.Students demonstrated productive within-group engagement in peer critique governed by norms, while encountering some difficulty with adopting certain socioscientific norms.


