Predicting Student Success -- More is Not Always Better

[MarisaBluestone]

As classes start to fill up in the virtual and in-person classrooms this fall, many questions remain about how the semester will be impacted by the pandemic. When the pandemic began, plans for instruction and assignments needed to be changed quickly; many instructors weighed the costs and benefits of the right workload for students during the time of change and crisis. Unfortunately, there wasn’t much data available to influence their decisions about what the right workload looked like.

At Macmillan Learning, we’ve been conducting research over the past year to better learn about how students are interacting with their course materials, and what benefits doing so may bring. We sought to discover more details about how class assignments within our learning platform, Achieve, have been used, both synchronously and asynchronously, and the impact that completing them had on student success. 

We questioned everything. We wondered whether or not the quantity of assignments, the type of assignments, and engagement with assignments impacted grades. And after analyzing results from more than 11,000 students using Achieve in 225 beta courses from fall of 2019 to spring of 2021 to better understand the impact of assignments, we have some answers about how assignments in Achieve impact exam scores. 

Our big picture takeaway -- student success, as it relates to their exam scores, is less about the number of assignments that instructors give out, and more about engagement with the assignments. The number of assignments given to students had no impact on their exam scores. Rather, the amount students engaged with the assignments that were given was the factor that made an impact. This data is consistent across all disciplines we studied, including chemistry, math, psychology, economics, biology, and English. 

students are projected to score 9% higher on their exams when they have better grades --almost a full letter gradeStudents with higher scores on Achieve assignments tend to perform better on their course exams. Further, students who completed fewer assignments on Achieve than was typical for their course, also performed lower on their exams. Students engaging with 50%-75% of the Achieve assignments could expect to score 4% points lower on their exams. Students engaging with less than 50% of the typical course load in Achieve could expect to score 8% lower on their exams, again almost a whole letter grade. 

These two effects work in conjunction with each other. This means students performed better when they did the work they were assigned, and better still when they did well on that work. 

In sum, we have very good evidence of an overall relationship between Achieve usage, better assignment grades and higher exam scores. We also learned that more isn’t necessarily better and also that all assignments are not created equal. We plan to share more information about which assignments work best once we’re done with our analysis. Stay tuned for more about that as well as our next blog post -- recommendations on what instructors can do to support student success, as it relates to the assignments they give.