To Use or Not to Use: Method Selections in Cognitive Diagnosis

Identifying the best model using fit indices is a crucial step in cognitive diagnosis (CD) data analysis. Previous research has examined the performance of popular relative and absolute model fit indices in the context of cognitive diagnosis models (CDMs). However, the effectiveness of these indices under uncommon conditions, such as small sample sizes, remains unclear. Therefore, the primary goal of this study is to investigate the impact of sample size on commonly used model fit indices in CD. Four relative model fit indices, including AIC, BIC, CAIC, and SABIC, and three absolute model fit indices, SRMSR, RMSEA 2, and M 2, were considered. The study also investigates the performance of two general approaches in CD, the generalized DINA (G-DINA) model and the general nonparametric classification (GNPC) method, in contrast to the studied model fit indices. The goal of this part of the study is to explore whether these two general approaches can serve as effective alternatives when the model fit indices do not perform well. Practical guidelines that would be useful for researchers and practitioners were provided based on the simulation results under various conditions. Additionally, a real data analysis was conducted to demonstrate the practical utility of the study.