Two-Step Approach to Topic Modeling to Incorporate Covariate and Outcome

This study investigates the applicability of topic modeling to analyze educational data. Topic modeling is useful because it reveals the latent topic structure underlying a collection of texts. Because metadata provides useful information about the topics, this study explores a way of including metadata as a covariate predicting topics and outcomes by topic in a topic model using a two-step approach. In Study 1, we use structural topic model (STM) and regression model because STM estimates the topic structure and how covariate is related to the topics. In Study 2, supervised Dirichlet allocation (sLDA) model is used to investigate the relationship between topics and the outcome variable: we incorporate sLDA with ANOVA. We demonstrate that the inclusion of multiple metadata improved the interpretability of the topic modeling techniques' results by examining the relationship among the examinees' written answers, problem-solving strategies, and scores using the empirical data of 246 middle school mathematics teachers' written responses to an item.