The presence of "big data"in higher education has led to the increasing popularity of predictive analytics for guiding various stakeholders on appropriate actions to support student success. In developing such applications, model selection is a central issue. As such, this study presents a comprehensive examination of five commonly used machine learning models in student success prediction. Using administrative and learning management system (LMS) data for nearly 2,000 college students at a public university, we employ the models to predict short-term and long-term academic success. Beyond the tradeoff between model interpretability and accuracy, we also focus on the fairness of these models with regard to different student populations. Our findings suggest that more interpretable models such as logistic regression do not necessarily compromise predictive accuracy. Also, they lead to no more, if not less, prediction bias against disadvantaged student groups than complicated models. Moreover, prediction biases against certain groups persist even in the fairest model. These results thus recommend using simpler algorithms in conjunction with human evaluation in instructional and institutional applications of student success prediction when valid student features are in place. © 2020 Owner/Author.

The emergence of big data in educational contexts has led to new data-driven approaches to support informed decision making and efforts to improve educational effectiveness. Digital traces of student behavior promise more scalable and finer-grained understanding and support of learning processes, which were previously too costly to obtain with traditional data sources and methodologies. This synthetic review describes the affordances and applications of microlevel (e.g., clickstream data), mesolevel (e.g., text data), and macrolevel (e.g., institutional data) big data. For instance, clickstream data are often used to operationalize and understand knowledge, cognitive strategies, and behavioral processes in order to personalize and enhance instruction and learning. Corpora of student writing are often analyzed with natural language processing techniques to relate linguistic features to cognitive, social, behavioral, and affective processes. Institutional data are often used to improve student and administrational decision making through course guidance systems and early-warning systems. Furthermore, this chapter outlines current challenges of accessing, analyzing, and using big data. Such challenges include balancing data privacy and protection with data sharing and research, training researchers in educational data science methodologies, and navigating the tensions between explanation and prediction. We argue that addressing these challenges is worthwhile given the potential benefits of mining big data in education.

This data article includes information on institutional data at a large public research university in Southern California. In particular, data on undergraduate student enrollments in online and face-to-face courses during summer terms from 2014 to 2017 cumulating in 72,441 course enrollments from 23,610 undergraduate students in 433 courses is provided. This data includes additional information on the statistical models examining factors influencing student enrollment by course modality and the associations of course modality with course grades. This includes descriptive data and data derived from multi-level logistic regression analyses and multi-way fixed effects linear regression analyses. This data article is associated with the article Effects of course modality in summer session: Enrollment patterns and student performance in face-to-face and online classes [1]. (c) 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons. org/licenses/by/4.0/).

Online summer courses offer opportunities to catch-up or stay on-track with course credits for students who cannot otherwise attend face-to-face summer courses. While online courses may have certain advantages, participation patterns and student success in summer terms are not yet well understood. This quantitative study analyzed four years of institutional data cumulating in 72,441 course enrollments of 23,610 students in 433 courses during summer terms at a large public research university. Multi-level logistic regression models indicated that characteristics including gender, in-state residency, admission test scores, previous online course enrollment, and course size, among others, can influence student enrollment by course modality. Multi-way fixed effects linear regression models indicated that student grades were slightly lower in online courses compared to face-to-face courses. However, at-risk college student populations (low-income students, first-generation students, low-performing students) were not found to suffer additional course performance penalties of online course participation.

Background Project-based learning has shown promise in improving learning outcomes for diverse students. However, studies on its impacts have largely focused on the perceptions of students and instructors or students' immediate performance. This study reports the impact of taking a project-based introductory engineering course on students' subsequent academic success. Purpose/Hypothesis This quantitative study examines characteristics related to enrollment in the project-based introductory engineering course and subsequent academic performance. We hypothesized that participation in the course would be associated with higher academic performance in subsequent engineering courses. In addition, we examined heterogeneity effects for students traditionally underrepresented in engineering education. Design/Method This study utilized data on students' demographics, academic preparation, course enrollment, and course performance from 1,318 engineering students from a large public university in Southern California. Logistic regression analysis with robust standard errors examined enrollment patterns. We applied propensity scores as inverse-probability weights in multiple linear models to calculate the average treatment effect on the treated for participants from the project-based introductory engineering course in five subsequent engineering courses. This analysis was conducted for all students and for selected student subgroups. Results Enrollment in the project-based introductory engineering course was positively associated with students' performance in some subsequent engineering courses and did not adversely affect students traditionally underrepresented in engineering. Conclusions This study provides an example of a project-based introductory engineering course that can support students' academic success in engineering. The benefits detected for some student populations (e.g., female) are encouraging for broadening engineering pathways.

The ability to regulate one's own learning is essential for success in online courses. Recent efforts have used clickstream data to create timely, fine-grained, and comprehensive measures of self-regulated learning (SRL) in online courses in an attempt to shed light on the process of SRL and to improve the identification of students who lack SRL skills and are at risk of low achievement. However, key questions remain: to what extent do these clickstream measures correspond to traditional self-reported measures about specific SRL constructs? Do these clickstream measures provide more information than existing self-reported measures in predicting course performance? This study used the clickstream data collected from a learning management system to measure two aspects of SRL: time management and effort regulation. We found that the clickstream measures were significantly associated with students' self-reported time management and effort regulation after the course. In addition, these clickstream measures significantly improved predictions of students' performance in the current and subsequent courses over predictions based on self-reported measures alone. These results provide evidence for the validity of the clickstream measures and guide the use of clickstream data to understand the process of SRL and identify students who might not be well served by taking classes online.

Informed by cognitive theories of learning, this work examined how students' self-reported study patterns (spacing vs. cramming) corresponded to their engagement with the Learning Management System (LMS) across two years in a large biology course. We specifically focused on how students accessed non-mandatory resources (lecture videos, lecture slides) and considered whether this pattern differed by underrepresented minority (URM) status. Overall, students who self-reported utilizing spacing strategies throughout the course had higher grades than students who reported cramming throughout the course. When examining LMS engagement, only a small percentage of students accessed the lecture videos and lecture slides. Applying a negative binomial regression model to daily counts of click activities, we also found that students who utilized spacing strategies accessed LMS resources more often but not earlier before major deadlines. Moreover, this finding was not different for underrepresented students. Our results provide some initial evidence showing how spacing behaviors correspond to accessing learning resources. However, given the lack of general engagement with LMS resources, our results underscore the value of encouraging students to utilize these resources when studying course material.

Addressing high demand for developmental math instruction and low rates of successful completing of the developmental coursework, with cost and space constraints, has been an ongoing challenge for post-secondary institutions. With advances in online instructional technology, particularly those based on artificial intelligence, web-based instruction is increasingly considered as a way to alleviate these burdens. This is among one of the first efforts that uses a quasi-experimental design to compare the academic outcomes of students who take a developmental mathematics course in a blended setting that combines face-to-face instruction with an online intelligent tutorial system, ALEKS, to the academic outcomes of students who take the same course in a fully online setting. Results suggest that students receiving online-only instruction perform worse on the final exam and receive lower course grades. However, a cost-effectiveness analysis suggests that fully online instruction has both a lower cost per student enrolled and a lower cost per student passing the course.

Time management skills are an essential component of college student success, especially in online classes. Through a randomized control trial of students in a for-credit online course at a public 4-year university, we test the efficacy of a scheduling intervention aimed at improving students' time management. Results indicate the intervention had positive effects on initial achievement scores; students who were given the opportunity to schedule their lecture watching in advance scored about a third of a standard deviation better on the first quiz than students who were not given that opportunity. These effects are concentrated in students with the lowest self-reported time management skills. However, these effects diminish over time such that we see a marginally significant negative effect of treatment on the last week's quiz grade and no difference in overall course scores. We examine the effect of the intervention on plausible mechanisms to explain the observed achievement effects. We find no evidence that the intervention affected cramming, procrastination, or the time at which students did work.

This quantitative study examines the impact of a three-week online organic preparatory course for chemistry undergraduates that is designed to improve student performance in the subsequent organic chemistry course series (N = 1,289). Organic chemistry often serves as a gatekeeper for students pursuing careers in science, technology, engineering, or mathematics (STEM). Because many students are underprepared for the rigorous organic chemistry series, and consequently are at greater risk of failing it, an online preparatory course was offered that emphasized topics that students frequently struggle with when they enter organic chemistry. The average treatment effects of participation in the online preparatory course on students' subsequent organic chemistry course grades were analyzed utilizing inverse-probability weights with regression adjustment. The analyses indicate that participation in the online preparatory course led to an improvement in subsequent organic chemistry course performance of approximately one-third of a letter grade (e.g., C+ to B-). Notably, students typically at-risk in college environments (i.e., low-income students, first-generation college students, underrepresented minorities) showed commensurate gains when compared to their non-at-risk counterparts. Consequently, this study provides an example of a low-cost intervention that can increase student learning and achievement in organic chemistry. In addition, this study contributes to the nascent research base that examines more distal effects of online course participation.