Publications
Many studies have examined the structure and properties of the Force Concept Inventory (FCI); however, far less research has investigated the Force and Motion Conceptual Evaluation (FMCE). This study applied Multidimensional Item Response Theory (MIRT) to a sample of N=4528 FMCE post-test responses. Exploratory factor analysis showed that 5, 9, and 10-factor models optimized some fit statistics. The FMCE uses extensive blocking of items into groups with a common stem; these blocks factored together in most models. A confirmatory analysis, which constrained the MIRT models to a theoretical model constructed from expert solutions, produced a model requiring only 8 principles, fundamental reasoning steps. This was substantially fewer than the 19 principles identified in the FCI by a previous study. Correlation analysis also demonstrated that the two instruments were very dissimilar. The reduced number of principles and the repetition of items using a single principle allowed the extraction of eight single-principle subscales, seven with Cronbach's alpha greater than the 0.7 required for acceptable internal consistency. The differences between the FCI and the FMCE suggest that the two instruments could provide complementary, but different, information about student understanding of Newton's laws with the FCI measuring an integrated Newtonian force concept and the FMCE measuring details of that force concept. © 2019 authors. Published by the American Physical Society. Published by the American Physical Society under the terms of the "https://creativecommons.org/licenses/by/4.0/" Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.
The use of machine learning and data mining techniques across many disciplines has exploded in recent years with the field of educational data mining growing significantly in the past 15 years. In this study, random forest and logistic regression models were used to construct early warning models of student success in introductory calculus-based mechanics (Physics 1) and electricity and magnetism (Physics 2) courses at a large eastern land-grant university. By combining in-class variables such as homework grades with institutional variables such as cumulative GPA, we can predict if a student will receive less than a B in the course with 73% accuracy in Physics 1 and 81% accuracy in Physics 2 with only data available in the first week of class using logistic regression models. The institutional variables were critical for high accuracy in the first four weeks of the semester. In-class variables became more important only after the first in-semester examination was administered. The student's cumulative college GPA was consistently the most important institutional variable. Homework grade became the most important in-class variable after the first week and consistently increased in importance as the semester progressed; homework grade became more important than cumulative GPA after the first in-semester examination. Demographic variables including gender, race or ethnicity, and first generation status were not important variables for predicting course grade.
While many studies have examined the structure, validity, and reliability of the Force Concept Inventory, far less research has been performed on other conceptual instruments in widespread use in physics education research. This study performs a confirmatory analysis of the Conceptual Survey of Electricity and Magnetism (CSEM) guided by a theoretical model of expert understanding of electricity and magnetism. Multidimensional Item Response Theory (MIRT) with the discrimination matrix constrained to the theoretical model was used to investigate two large datasets (N-1 = 2014 and N-2 = 2657) from two research universities in the United States. The optimal model identified by MIRT was similar, but not identical, for the two datasets and had very good model fit with comparative fit indices of 0.975 and 0.984, respectively. The most parsimonious optimal model required 23 independent principles of electricity and magnetism and was significantly better fitting than a more general model dividing the CSEM into 6 general topics. The optimal models for the two samples were quite similar, sharing 22 of a possible 26 conceptual principles. Most of the overall item difficulties and discriminations were significantly different between the two samples; however, the rank order of the overall difficulty and discrimination were generally similar. There was much more similarity between the discrimination by item of the individual principles. Five items had a difficulty ranking that was substantially different between the two samples, indicating that while generally similar, relative difficulty does depend on the student population and instructional environment.


