Publications
Computer science learning in primary school classrooms has expanded, necessitating effective instructional strategies for this age group. GalleryWalks are a common activity to allow peers to share their work and give feedback on peers' work. Like other skills, providing effective feedback may require scaffolding and/or instruction for some students. Currently, there is little work exploring how CS students provide peer feedback without extensive instruction, especially at the primary school level (aged 5-11). We analyzed the feedback 4th grade students gave on a structured worksheet during a gallery walk. We found that students often provided both compliments and suggestions when prompted, but their feedback focused primarily on program aesthetics (e.g., characters, sounds, or storyline) rather than programmed elements, even when directed to focus on the program. In addition, bilingual and non-bilingual classes differed in how students followed the worksheet structure.
Background and Context. With increasing efforts to bring computing education opportunities into elementary schools, there is a growing need for assessments, with arguments for validity, to support research evaluation at these grade levels. After successfully piloting a 10-question computational thinking assessment (Assessment of Computing for Elementary Students - ACES) for 4th graders in Spring 2020, we used our analyses of item difficulty and discrimination to iterate on the assessment. Objectives. To increase the number of potential items for ACES, we created isomorphic versions of existing questions. The nature of the changes varied from incidental changes that we did not believe would impact student performance to more radical changes that seemed likely to influence question difficulty. We sought to understand the impact of these changes on student performance. Method. Using these isomorphic questions, we created two versions of our assessment and piloted them in Spring 2021 with 235 upper-elementary (4th grade) students. We analyzed the reliability of the assessments using Cronbach's alpha. We used Chi-squared tests to analyze questions that were identical across the two assessments to form a baseline of comparison and then ran Chi-Squared and Kruskal-Wallis H tests to analyze the differences between the isomorphic copies of the questions. Findings. Both assessment versions demonstrated good reliability, with identical Cronbach's alphas of 0.868. We found statistically similar performance on the identical questions between our two groups of students, allowing us to compare their performance on the isomorphic questions. Students performed differently on the isomorphic questions, indicating the changes to the questions had a differential impact on student performance. Implications. This paper builds on existing work by presenting methods for creating isomorphic questions. We provide valuable lessons learned, both on those methods and on the impact of specific types of changes on student performance.
With the growth of Computer Science (CS) and Computational Thinking (CT) instruction in the primary/elementary domain, it is important that such instruction supports diverse learners. Four categories of students-students in poverty, multi-lingual students, students with disabilities, and students who have below-grade-level proficiency in reading and math, may face academic challenges that can hinder their learning in CS/CT curricula. However, little is known about how to support these students in CS/CT instruction, especially at this young age. TIPP&SEE, a meta-cognitive strategy that scaffolds learning by proceduralizing engagement through example code, may offer some support. A quasi-experimental study revealed that the gaps between students with and without academic challenges narrowed when using the TIPP&SEE strategy, indicating its promise in providing equitable learning opportunities in CS/CT. © 2021 ACM.
Development and Preliminary Validation of the Assessment of Computing for Elementary Students (Aces)
As reliance on technology increases in practically every aspect of life, all students deserve the opportunity to learn to think computationally from early in their educational experience. To support the kinds of computer science curriculum and instruction that makes this possible, there is an urgent need to develop and validate computational thinking (CT) assessments for elementary-aged students. We developed the Assessment of Computing for Elementary Students (ACES) to measure the CT concepts of loops and sequences for students in grades 3-5. The ACES includes block-based coding questions as well as non-programming, Bebras-style questions. We conducted cognitive interviews to understand student perspectives while taking the ACES. We piloted the assessment with 57 4th grade students who had completed a CT curriculum. Preliminary analyses indicate acceptable reliability and appropriate difficulty and discrimination among assessment items. The significance of this paper is to present a new CT measure for upper elementary students and to share its intentional development process. © 2021 Owner/Author.
Worldwide, national initiatives have led to many school districts implementing computing curricula at the primary level. At that age, students are learning the foundational skills of reading and math. It is important to understand how computing can influence the development of these skills. While some argue that learning computing sharpens problem-solving skills that are applicable to other subjects, evidence supporting this belief is thin. In a quasi-experimental study of fourth-grade (ages 9-10) students, we compared state reading and math test scores of students receiving computing instruction with students who did not. Our findings demonstrated that a more open-ended, less scaffolded form of computing instruction was linked to performance gains in math, but not in reading (F(2, 232)=11.08, p< .01, p2=.0625). When looking at students who face academic challenges that can impact reading and math, the same trend applied to students with economic disadvantages and students with limited English proficiency, but not for students with disabilities. These results suggest that moderately scaffolded computing instruction supports the development of skills applicable to math, a step towards better understanding the relationship between learning opportunities in computing and outcomes in other subjects. © 2021 ACM.
The demand for computational thinking (CT) problem solving abilities surge as every aspect of life becomes more dependent on complex digital technologies. Just as in math and language, a strong CT foundation needs to be established in early education in order for students to develop an instinctive CT perspective of the world. The urgent demand for CT instruction in elementary school quickly draws attention to the shortage of elementary school-level teachers qualified and interested in CT. Additionally, with a commitment to equity in the United States education system and knowledge of the high percentage of English language learning (ELL) students in schools, the obligation to create curricula that will provide access to CT knowledge, skills, and practices for elementary-level ELL students is loudly apparent. In response to these two needs, our team has adapted existing Scratch-based CT curriculum to support classroom teachers with minimal CT experience and to be more accessible to English language learners. The purpose of this paper is to share the framework that guided the curriculum adaptations, to describe the specific changes that were made, and to discuss discoveries made during the process. This journey may be helpful to anyone who is tasked with modifying a curriculum to meet the needs of novice content teachers and ELL students. © 2021 Owner/Author.
With the global movement to incorporate computer science instruction into elementary education, learners are being introduced to computer science and computational thinking (CS/CT) ideas at increasingly younger ages. At these early ages, young learners are developing cognitive abilities foundational to their education. While other discipline-based education fields, such as math, science, and reading, have long studied the role of cognitive abilities, such as short-term working memory and long-term retrieval, in their respective fields, similar research in computer science education is relatively sparse. In this exploratory study, we examined the relationship between cognitive abilities and CS/CT performance of fourth-grade students (ages 9-10) who underwent either an introductory CT curriculum based on Use->Modify->Create or the same curriculum with additional scaffolding from the TIPP&SEE metacognitive learning strategy. Our analysis revealed performance on CT assessments to be weakly correlated with working memory and long-term retrieval, with correlations increasing as the CT concepts grew more complex. This suggests that scaffolding beyond TIPP&SEE may be needed with more complex CT concepts. We also found that when using TIPP&SEE, students scoring below average on cognitive ability tests performed as well as students in the control condition with average cognitive ability scores. These results indicate TIPP&SEE's potential in creating more equitable computing instruction. We hope that results from this initial exploration can help encourage further study into the role of cognitive abilities in CS/CT education for young learners.
With the rise of Computational Thinking (CT) instruction at the elementary level, it is imperative that elementary computing instruction support a variety of learners. A popular pedagogical approach for this age group is Use->Modify->Create, which introduces a concept through a more scaffolded, guided instruction before culminating in a more open-ended project for student engagement. Yet, there is little research on student learning during the Use->Modify step, nor strategies to promote learning in this step. This paper introduces TIPP&SEE, a metacognitive learning strategy that further scaffolds student learning during this step. Results from an experimental study show statistically-significant performance gains from students using the TIPP&SEE strategy on nearly all assessment questions of moderate and hard difficulty, suggesting its potential as an effective CS/CT learning strategy.
As many school districts nationwide continue to incorporate Computer Science (CS) and Computational Thinking (CT) instruction at the K-8 level, it is crucial that we understand the factors and skills, such as reading and math proficiency, that contribute to the success of younger learners in a computing curriculum and are typically developed at this age. Yet, little is known about the relationship between reading and math proficiency, and the learning of key CS concepts at the elementary level. This study focused on 4th-grade students (ages 9-10) who were taught events, sequence, and repetition through an adaptation of the Creative Computing Curriculum. While all students benefited from access to such a curriculum, there were statistically-significant differences in learning outcomes, especially between students whose reading and math proficiency are below grade-level, and students whose proficiency are at or above grade-level. This performance gap suggests the need for curricular improvement and learning strategies that are CS specific for students who struggle with reading and math.
The CS community has struggled to assess student learning at the K-8 level, with techniques ranging from one-on-one interviews to written assessments. While scalable, automated techniques exist for analyzing student code, a scalable method for assessing student comprehension of their own code has remained elusive. This study is a first step in bridging the gap between the knowledge gained from interviews and the time efficiency and scalability of written assessments and automated analysis. The goal of this study is to understand how student answers on various types of questions differ depending on whether they are being asked about their own code or generic code. We find that while there were no statistically-significant differences in overall scores, questions about generic and personalized code of comparable complexity are far from equivalent. Our qualitative analyses revealed interesting patterns in student responses, inviting further research into this assessment technique. In particular, students answered differently from students with generic code when presented with individual blocks from their code taken out of context and placed into different code snippets, and students answered in a way that demonstrates a functional, instead of structural, understanding on Explain in Plain English (EiPE) questions.


