STEM Learning and Learning Environments  

Using Educational Data Mining Techniques to Uncover How and Why Students Learn from Erroneous Examples

Effective Years: 2017-2022

In this research project, a team from Carnegie Mellow University and a team from the University of Pennsylvania will use data mining techniques to explore how and why interactive erroneous examples can benefit mathematics learners. The project team will iteratively refine existing, computer-based pedagogical materials based on an analysis of log data of hundreds of students who have used those materials to learn. The work will be done with middle school mathematics students, specifically sixth grade students, and the target topic will be decimals. Recent research and studies have indicated that learning from the errors of others -- a prominent pedagogical approach used in medical education, but less commonly used in other areas of education -- has considerable potential to help students learn. Yet, erroneous examples are not used widely, in part because it is not yet understood how and why they provide this benefit. The broader impact of the project is that it will provide improved mathematics lessons, infused with erroneous examples, vetted with empirical evidence of their effectiveness, which can be deployed on the world-wide web to the significant number of U.S. students who are at risk of poor mathematics achievement due to poor understanding of core STEM concepts. The project will also provide a blueprint that can be used widely in other projects and systems, supported by both theory and empirical evidence, for how to help students learn from erroneous examples. The project is funded by the EHR Core Research (ECR) program, which supports work that advances the fundamental research literature on STEM learning.

The project will employ an iterative research plan in which log data from online learning of erroneous examples will be datamined and analyzed. The project team will start with log data from over 600 middle school mathematics students who have used erroneous examples to learn in online instructional technology and games. The team will then collect data from approximately two thousand more students over the course of the three-year project to explore whether enhancements to the erroneous examples materials based on revised theory improves student learning. Moment-by-moment learning models will be used to understand whether students learn immediately from erroneous examples, or whether erroneous examples lead to future learning. The project team will also study whether confusion and other forms of affect mediate successful learning from erroneous examples, using previously validated automated detectors of confusion and other affective states. The project team will also investigate how each of the steps of the student interaction with the erroneous examples -- for example, identifying errors, fixing errors, and explaining errors -- promote learning and positive affect. By the end of the third year of the project, the project team anticipates having a comprehensive theoretical model of how learning with erroneous examples works and how erroneous examples should be integrated into instruction more generally.