Publications
With many school districts nationwide integrating Computer Science (CS) and Computational Thinking (CT) instruction at the K-8 level, it is crucial researchers closely inspect the relationship between program expression and student understandings. In this study, we propose and report on our use of Scratch Charades, a game in which students act out Scratch scripts while others build them. The purpose of Scratch Charades is to familiarize students with scripts and blocks without the cognitive overhead of the complex user interface. However, in this study, we also used it to elicit student understandings about Scratch blocks and scripts to design mnemonics to help students debug their code. We propose two building and/or debugging strategies based on our observations.
The computer science community has struggled to assess student learning via Scratch programming at the primary school level (ages 7-12). Prior work has relied most heavily on artifact (student code/projects) analysis, with some attempts at one-on-one interviews and written assessments. In this paper, we explore the relationship between artifact analysis and written assessments. Through this study of a large-scale introductory computing implementation, we found that for students who had code in their projects, student performance on specific questions on the written assessments is only very weakly correlated to specific attributes of final projects typically used in artifact analysis as well as attributes we use to define candidate code (r < 0.2, p < 0.05). In particular, the correlation is not nearly strong enough to serve as a proxy for understanding. © 2020 ACM.
With the rise of Computational Thinking (CT) instruction at the elementary level, it is imperative for elementary computing instruction to support a variety of learners. TIPP&SEE is a meta-cognitive learning strategy that scaffolds student learning when learning from example code. Results from a previous study show statistically-significant performance differences favoring students using the TIPP&SEE strategy on a written assessment [43]. In this work, our goal is gain insight as to why such dramatic learning differences may have occurred. We analyze the students' computational artifacts and TIPP&SEE worksheets. Artifact analysis reveals that students in the TIPP&SEE group are more thorough in their work, completing more elements of the required tasks. In addition, they build open-ended projects with longer scripts that utilize more learned blocks. Worksheet analysis shows that students were highly accurate on some types of questions but largely skipped others. Despite these positive behaviors, there was little statistical correlation between student worksheet correctness, project completion, and written assessment performance. Therefore, while students in the TIPP&SEE group performed actions we believe lead to more success, no individual actions directly explain the results. Like other meta-cognitive strategies, the value of TIPP&SEE may lie in cognitive processes not directly observable, and may vary based upon individual student differences.
Major metropolitan school districts around the United States are implementing computer science in elementary school classrooms as part of the CS for All (CS4All) initiative. Little is known, however, about the success of such a large-scale rollout, especially in terms of equity. In this study we analyze the performance of 4th grade classrooms completing three modules of an introductory computational thinking curriculum, looking at not only overall results but also the variance in performance between high-, mid-, and low-performing schools (as identified by their school report cards). We find that all classrooms are benefiting from the computational thinking (CT) curriculum, making great strides in providing equitable access to CT education. However, statistically-significant differences in performance are present, especially between the high- and low-performing schools, showing that there is still room for improvement in developing strategies and curricula for struggling learners.
The computer science community has struggled to assess student learning, especially at the early elementary level. Prior work has included one-on-one interviews, written assessments, and artifact analysis, each with their own benefits and drawbacks. Through our Personalized Assessment Worksheets for Scratch (PAWS) tool, we explore personalized assessments as an assessment technique that lies in between interviews, written assessments, and artifact analysis. PAWS creates personalized written assessments that integrates code from student Scratch projects. We hope that our PAWS tool, and more generally personalized assessments, will lead to an assessment technique that is both more accurate than written assessments and artifact analysis, and less time-consuming than interviews.


