STEM Learning and Learning Environments  

Using Neural Networks for Automated Classification of Elementary Mathematics Instructional Activities

Effective Years: 2020-2025

This research project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development. In the last decade, there has been a tremendous increase in the use of video for preparing teachers and studying teaching quality. Prominent approaches to summative evaluation of teaching candidates and beginning teachers feature video recordings of instructional practices. In addition, large-scale research studies have included video of instruction. Teacher preparation programs increasingly use video in methods courses and for formative assessment purposes. Moreover, the growing use of interactive simulation in pre-service preparation and in-service professional development relies on having video recorded examples of exemplary instructional practices. Despite the significant growth in the use of video to measure and promote instructional quality in recent years, there remain some key challenges to employing it at scale. First, it is very time-consuming for trained human raters to view hundreds of hours of video recorded lessons. Second, financial costs and time demands increase with the use of multiple human raters per video. Third, manually cataloging, labeling, and indexing large volumes of classroom video for later viewing is time consuming. Recent advances in computer vision, machine learning, and deep learning may provide solutions to these challenges and could make the process of analyzing and scoring videos more efficient. In particular, deep learning has become the state-of-the-art choice in problems related to analyzing the content of video. This proposed NSF Core Research study will draw on videos of elementary mathematics instruction that were collected for two NSF-funded studies that featured the Mathematics-Scan (M-Scan) classroom observation tool. The research will use these videos to explore several ways that deep neural networks can be used to classify instructional activities in videos of math instruction. This study will advance knowledge and understanding by examining the degree to which three types of artificial neural networks can accurately classify (a) objects and (b) instructional activities in videos of elementary mathematics instruction. For example, it may be straightforward for neural networks to determine whether an elementary teacher is engaged in lecture vs. facilitating discussion with students. On the other hand, it may be harder for such networks to assess the ways teachers represent math content, the nature of their questions, and whether student gestures signify understanding. The research design concerns three aspects of using computer vision, machine learning, and deep learning: (a) the type of neural network, (b) the type of video label (i.e., object labels and instructional activity labels), and (c) the subject of math instruction (e.g., number and operations; patterns, functions, and algebra; geometry). A few types of neural networks have recently proven effective for video classification: convolutional neural networks (CNNs), long short-term memory (LSTM) neural networks, and hybrid CNN-LSTMs. Ultimately, this project is aimed at beginning to build systematic infrastructure for classifying videos of classroom instruction at scale in efficient and affordable ways. The findings will potentially have key implications for (a) large-scale research studies that feature videos of instruction and (b) pre-service teacher preparation programs, in-service professional development activities, and efforts to evaluate teaching candidates and practicing teachers. In particular, the results from this study will inform decisions about the types of neural networks that can be used to correctly classify videos of instruction and the practical limitations of using networks for this purpose. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

7 Publications Related to This Project: