The present study examines the assistance dilemma of providing combinations of perceptual supports in math education by fitting a Bayesian knowledge tracing model (BKT) to an open-source, online order-of-operations intervention dataset. This online intervention dataset contained 688 U.S. middle schoolers' accuracy data who solved order-of-operations problems in pretest, intervention, post-test, and delayed post-test sessions. Combinations of spacing and color cues were embedded in intervention materials, alongside immediate correctness feedback and worked examples, creating nine between-subject cue conditions (congruent, incongruent, or neutral spacing and/or color). To examine the effect of perceptual cues on immediate performance and long term learning, we trained a BKT variant, conditioning guess, slip, learn, and forget parameters on perceptual cue conditions. The estimated cue-specific BKT parameters suggest two main findings. First, the presence of congruent perceptual cues (color and/or spacing, e.g., 2 - 8 + 3×11) increased guessing rate and reduced slipping rate, suggesting that visually highlighting high-order suboperations facilitated students' immediate problem solving. Second, students who underwent intervention with incongruent color cues (e.g., 2 - 8 + 3 × 11) showed reduced guessing rate but exhibited the highest learning rate and the lowest forgetting rate. These results imply that providing misleading visual highlights may hinder immediate performance but can be beneficial for long term learning. Furthermore, the impact of perceptual cues on immediate performance and long term learning indicated by BKT parameters approximately aligned with the patterns revealed by behavioral measures (i.e., intervention, post-test, and delayed post-test accuracy). Such consistencies demonstrate the feasibility of using the BKT framework to understand the assistance dilemma of perceptual supports. The code and data are available at: osf.io/q8kn6.

Perceptual cues in instructional materials may help students notice structures of math notation and improve problem-solving performance. The present study examined whether and how congruent perceptual cues (spacing and color) affected students' performance on order-of-operations problems and considered potential moderators. The analysis included 482 middle-school students, randomly assigned to one of three conditions: spacing (3 − 9 + 4×7), color (“4 × 7” highlighted in 3 − 9 + 4 × 7), or no cues. Students participated via an online platform during a 45-min block in their classrooms. Beyond examining performance during the experiment and posttest, we considered moderators of prior order-of-operations knowledge, perception of equivalence, and math anxiety. Neither congruent spacing nor congruent color affected performance on the experimental items or posttest. However, students with higher perception of equivalence benefited more from congruent spacing cues than students with lower perception of equivalence. Students with higher math anxiety struggled more during the experiment, but there was no moderation. Future work should include specific intervention scaffolding and feedback alongside the presentation of cues, as well as delve into variations in perceptual structures at the individual and problem levels to fully understand effects of perceptual cues on math problem-solving performance.