In the fall of 2020, we conducted research with 29 unique instructors at 26 institutions, teaching 67 sections to 2,670 students about how Achieve worked for them. ... View more

    This checklist demonstrates the design, development, and testing process of Achieve and illustrates the evidence currently available to help administrators and educators evaluate whether it will support student success in their educational context. ... View more

SDI Productions/Getty Images       One strategy that has gained popularity for effectively implementing an active learning course design is a “flipped classroom’’ where students engage with the course material before coming to class. The literature on the effectiveness of using pre-class activities to improve learner outcomes is mixed. Consequently, we evaluated the use of pre-class activities in a new digital learning tool, Achieve, to investigate whether engaging in pre-class activities influenced assessment scores. We partnered with 40 instructors on the evaluation of pre-class activities in Achieve. Instructors chose whether or not to implement pre-class activities in their course, naturally categorizing students into “pre-class users” or “non pre-class users”. In total, 2,251 students consented to participate in the study (74% of the population), 1,372 engaged in pre-class activities and 879 did not. Groups were compared on three dependent variables: student likelihood to recommend a course using Achieve to a friend, summative assessment scores in Achieve, and in-course final exam scores. We also evaluated the effect of utilization of pre-class activities in Achieve on the dependent variables. Engagement in pre-class activities in Achieve had a significant effect on each of the dependent variables, even when prior academic performance, baseline level of motivation, and the instructor were controlled. We also found a small (based on Cohen’s classification), but significant relationship between the extent to which a student engaged in pre-class activities in Achieve and assessment scores. Finally, results uncovered evidence that instructors and students perceive pre-class activities in Achieve to contribute to positive student behaviors. And, there is evidence that use of pre-class activities significantly contributes to a student’s likelihood to recommend a course where Achieve will be used to a friend, but that it accounts for a small proportion of the variability in the outcome. Achieve for Readers and Writers (1-Term Access)  Achieve Read & Practice for The American Promise, Value Edition (1-Term Access)  Achieve for Interactive General Chemistry (1-Term Access)  ... View more

    Advances in the learning sciences combined with the fast evolution of powerful digital technologies and advances in the design of user experience have the potential to transform the future of higher education. There is now a large body of research demonstrating that learning science principles such as self-regulation, formative assessment, and active learning support the development of lifelong learners (Bell & Kozlowski, 2008). Digital tools can increasingly offer content, resources, and assessments that are personalized, adaptive, and relevant. (Cook et. al, 2013). And, seamless, efficient, engaging digital experiences that make the lives of students and instructors easier can meet the needs of busy educators and a diverse student population (Herrington & Oliver, 2000). However, despite a host of digital learning tools available to instructors there is a lack of rigorous and relevant evidence researching effectiveness and this has often led to false starts and frustrations of what to use and how to use it to best effect to improve student success. When building Achieve, a new digital learning platform, Macmillan Learning began by founding the solution on research-based learning science principles, and co-designing the platform with instructors and students. Researchers at Macmillan Learning then took the unusual approach of beginning to investigate the effectiveness and efficacy of Achieve in its infancy by conducting rapid-cycle evaluations of tool features as they were developed and implementing increasingly rigorous validity and efficacy studies as the platform matured through beta testing. ... View more

This study investigated the differential efficacy of a new digital learning tool, Achieve, among students less and more academically prepared to succeed.           ... View more

Educational technology has the potential to vastly improve teaching and learning in higher education. Rapid advancements in technology and the pace at which new tools are being developed are leading to alpha products being tested with students in live classrooms, which can have a negative impact on teaching and learning. There are many constraints of educational technology that is early in development, and student success in higher education is critically important. Therefore, it is necessary to develop alternative methods to requiring students to use an alpha version of a learning tool as their primary course material for a full semester. This paper discusses a new method for conducting a formative evaluation of digital learning tools. The method has proven to enable relevant, timely, and actionable insights on product optimization, implementation patterns, and professional development—without requiring the use of an alpha product in a high-stakes environment. A case study of a formative evaluation of new digital platform, Achieve, is used to illustrate the approach. The formative evaluation of the alpha version of Achieve was a longitudinal study conducted with a set of instructors teaching a course in which the solution might be used, but the study was conducted outside of and independent from their live classrooms. The evaluation was comprised of eight unique rapid-cycle evaluations, each lasting one week, that simulated the arc of a traditional semester. Qualitative and quantitative data were collected to develop insights into platform use, implementation patterns, perception, and expectations. Results from the evaluation were used for real-time remediation and optimization of the tool, to understand instructors’ chosen implementation patterns, and to inform professional development for future users. Real-time results implemented by the development teams provided confidence among researchers and instructors that a beta version of the platforms used in live classrooms in subsequent studies would not adversely impact the student and instructor experience, but rather contribute positively to important learner outcomes.     ... View more