Harvesting Actionable Results for Learning and Instruction: A Novel Mixed Methods Approach to Extracting and Validating Information from Diagnostic Assessment
Effective Years: 2023-2026
Cognitive diagnostic models (CDMs) are an area of psychometric research that has seen substantial growth in the past decade. These assessment tools have received more attention because a simple overall test score does not serve teaching goals as a richer evaluation of the student's skills is needed to support tailored instruction. The investigator proposes to extend existing CDM psychometric approaches to develop the diagnostic facet status model (DFSM) for formative assessment design based on the high school physics curriculum. There are a number of working concerns with existing CDMs. They do not scale well to high-dimensional facet spaces, they require complex coding schemes based on expert input, they cannot explore intricate facet relationships, or option-based nominal responses. This project will build on existing CDM approaches to address these limitations. The project develops a new psychometric model, the DFSM, and a longitudinal extension, which can simultaneously model high-dimensional goal and intermediate understandings at item option level. The PIs develop a machine learning method to identify relations among facets, yielding a comprehensive "facet map" that reveals both attribute hierarchies and conjoined facets for the learning of physics. Finally, the researchers conduct qualitative studies to validate DFSM and longitudinal DFSM's output for learning and instruction. The project is supported by NSF's EDU Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, which is designed to build investigators' capacity to carry out high-quality STEM education research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
8 Publications Related to This Project:
- Using Multilabel Classification Neural Network to Detect Intersectional Dif With Small Sample Sizes
- Review of Cognitive Diagnostic Models ( CDMs ): Recent Methodological Advancements for Addressing Practical Challenges
- Flexible Bayesian Slice-Sampling Algorithms: An Illustration Using the Hierarchical Piecewise Constant Proportional Hazards Latent Trait Model
- Challenges and Potential Solutions of Measurement in the Era of Educational Big Data: Introduction to the Jem Special Issue
- A Novel Method for Detecting Intersectional DIF: Multilevel Random Item Effects Model With Regularized Gaussian Variational Estimation
- A Note on Standard Errors for Multidimensional Two-Parameter Logistic Models Using Gaussian Variational Estimation
- A Diagnostic Facet Status Model (Dfsm) for Extracting Instructionally Useful Information from Diagnostic Assessment
- A Diagnostic Facet Status Model (Dfsm) for Extractinginstructionally Useful Information from Diagnostic Assessment (Apr, 10.1007/S11336-024-09971-8, 2024)
You are viewing a record from the ECR Projects database. Click here to search the ECR Projects Database or the ECR Publications Database.


