Proof Blocks is a software tool that allows students to practice writing mathematical proofs by dragging and dropping lines instead of writing proofs from scratch. Proof Blocks offers the capability of assigning partial credit and providing solution quality feedback to students. This is done by computing the edit distance from a student’s submission to some predefined set of solutions. In this work, we propose an algorithm for the edit distance problem that significantly outperforms the baseline procedure of exhaustively enumerating over the entire search space. Our algorithm relies on a reduction to the minimum vertex cover problem. We benchmark our algorithm on thousands of student submissions from multiple courses, showing that the baseline algorithm is intractable, and that our proposed algorithm is critical to enable classroom deployment. Our new algorithm has also been used for problems in many other domains where the solution space can be modeled as a DAG, including but not limited to Parsons Problems for writing code, helping students understand packet ordering in networking protocols, and helping students sketch solution steps for physics problems. Integrated into multiple learning management systems, the algorithm serves thousands of students each year. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Students use learning analytics systems to make day-to-day learning decisions, but may not understand their potential flaws. This work delves into student understanding of an example learning analytics algorithm, Bayesian Knowledge Tracing (BKT), using Cognitive Task Analysis (CTA) to identify knowledge components (KCs) comprising expert student understanding. We built an interactive explanation to target these KCs and performed a controlled experiment examining how varying the transparency of limitations of BKT impacts understanding and trust. Our results show that, counterintuitively, providing some information on the algorithm’s limitations is not always better than providing no information. The success of the methods from our BKT study suggests avenues for the use of CTA in systematically building evidence-based explanations to increase end user understanding of other complex AI algorithms in learning analytics as well as other domains. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Prompted by findings of gender differences in learning game preferences and outcomes, education researchers have proposed adapting games by gender to foster learning and engagement. However, such recommendations typically rely on intuition, rather than empirical data, and are rooted in a binary representation of gender. On the other hand, recent evidence from several disciplines indicates that gender is best understood through multiple dimensions, including gender-typed occupational interests, activities, and traits. Our research seeks to provide learning game designers with empirical guidance incorporating this framework in developing digital learning games that are inclusive, equitable, and effective for all students. To this end, we conducted a survey study among 333 5th and 6th grade students in five urban and suburban schools in a mid-sized U.S. city, with the goal of investigating how game preferences differ by gender identity or gender-typed measures. Our findings uncovered consistent differences in game preferences from both a binary and multi-dimensional gender perspective, with gender-typed measures being more predictive of game preferences than binary gender identity. We also report on preference trends for different game genres and discuss their implications on learning game design. Ultimately, this work supports using multiple dimensions of gender to inform the customization of learning games that meet individual students' interests and preferences, instead of relying on vague gender stereotypes.

The analysis of fine-grained data on students’ learning behavior in intelligent tutoring systems using machine learning is a starting point for personalized learning. However, findings from such analyses are commonly viewed as uninterpretable and, hence, not helpful for understanding learning processes. The explainable AI method SHAP, which generates detailed explanations, is a promising approach here. It can help to better understand how different learning behaviors influence students’ learning outcomes and potentially produce new insights for adaptable intelligent tutoring systems in the future. Based on K-12 data (N = 472 students), we trained several ML models on student characteristics and behavioral trace data to predict whether a student will answer an exercise field correctly or not—a low-level approximation of academic performance. The optimized machine learning models (lasso regression, random forest, XGBoost) performed well (AUC= 0.68 ; F1= [ 0.63 ; 0.66 ; 0.69 ] ), outperforming logistic regression (AUC= 0.57 ; F1= 0.52 ). SHAP importance values for the best model (XGBoost) indicated that, besides prior language achievement, several behavioral variables (e.g., performance on the previous attempts) were informative predictors. Thus, specific learning behaviors can help explain exercise field performance, independent of the student’s basic abilities and demographics—providing insights into areas of potential intervention. Moreover, the SHAP values highlight heterogeneity in the effects, supporting the notion of personalized learning. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Collaboration analytics are a necessary step toward implementing intelligent systems that can provide feedback for teaching and supporting collaborative skills. However, the wide variety of theoretical perspectives on collaboration emphasize assessment of different behaviors toward different goals. Our work demonstrates rigorous measurement of collaboration in small group discourse that combines coding schemes from three different theoretical backgrounds: Collaborative Problem Solving, Academically Productive Talk, and Team Cognition. Each scheme measured occurrence of unique collaborative behaviors. Correlations between schemes were low to moderate, indicating both some convergence and unique information surfaced by each approach. Factor analysis drives discussion of the dimensions of collaboration informed by all three. The two factors that explain the most variance point to how participants stay on task and ask for relevant information to find common ground. These results demonstrate that combining analytical tools from different perspectives offers researchers and intelligent systems a more complete understanding of the collaborative skills assessable in verbal communication. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.