The present study examines the assistance dilemma of providing combinations of perceptual supports in math education by fitting a Bayesian knowledge tracing model (BKT) to an open-source, online order-of-operations intervention dataset. This online intervention dataset contained 688 U.S. middle schoolers' accuracy data who solved order-of-operations problems in pretest, intervention, post-test, and delayed post-test sessions. Combinations of spacing and color cues were embedded in intervention materials, alongside immediate correctness feedback and worked examples, creating nine between-subject cue conditions (congruent, incongruent, or neutral spacing and/or color). To examine the effect of perceptual cues on immediate performance and long term learning, we trained a BKT variant, conditioning guess, slip, learn, and forget parameters on perceptual cue conditions. The estimated cue-specific BKT parameters suggest two main findings. First, the presence of congruent perceptual cues (color and/or spacing, e.g., 2 - 8 + 3×11) increased guessing rate and reduced slipping rate, suggesting that visually highlighting high-order suboperations facilitated students' immediate problem solving. Second, students who underwent intervention with incongruent color cues (e.g., 2 - 8 + 3 × 11) showed reduced guessing rate but exhibited the highest learning rate and the lowest forgetting rate. These results imply that providing misleading visual highlights may hinder immediate performance but can be beneficial for long term learning. Furthermore, the impact of perceptual cues on immediate performance and long term learning indicated by BKT parameters approximately aligned with the patterns revealed by behavioral measures (i.e., intervention, post-test, and delayed post-test accuracy). Such consistencies demonstrate the feasibility of using the BKT framework to understand the assistance dilemma of perceptual supports. The code and data are available at: osf.io/q8kn6.

ABSTRACT Undergraduate research experiences (UREs) and course-based UREs (CUREs) promote students' scientific self-efficacy growth. Yet, how self-efficacy develops during research is not understood. Furthermore, what students do during research varies in ways that likely affect self-efficacy development. We sought to address these knowledge gaps by collecting scientific self-efficacy data from CURE and URE students at nine universities at the beginning, middle, and end of a single term of research. We leveraged a theoretical advancement, latent state-trait theory-revised, to disaggregate the components of students' self-efficacy into stable or trait-like self-efficacy and dynamic or state-like self-efficacy. We determined that students' scientific self-efficacy was moderately stable during their research, with the most malleable component being beliefs in their abilities to figure out data collection and explain results. We also surveyed students ∼45 times throughout their research experience to test the extent to which research hours and types of research tasks contributed to self-efficacy development. We found that students who completed more analytic tasks experienced significantly more self-efficacy growth than students who completed other types of tasks, while time spent on research was not influential. Our results illustrate the importance of engaging students in analytic tasks during CUREs and UREs for fostering their self-efficacy development. Highlight for table of contents Using latent state-trait theory-revised, we found that students' scientific self-efficacy was more stable than malleable over one research term. Beliefs about data collection and explaining results were most dynamic. Conducting more analytic tasks fostered self-efficacy, while the time spent and completion of other tasks had no effect.