In the fall of 2020, we conducted research with 29 unique instructors at 26 institutions, teaching 67 sections to 2,670 students about how Achieve worked for them. ... View more

SDI Productions/Getty Images       One strategy that has gained popularity for effectively implementing an active learning course design is a “flipped classroom’’ where students engage with the course material before coming to class. The literature on the effectiveness of using pre-class activities to improve learner outcomes is mixed. Consequently, we evaluated the use of pre-class activities in a new digital learning tool, Achieve, to investigate whether engaging in pre-class activities influenced assessment scores. We partnered with 40 instructors on the evaluation of pre-class activities in Achieve. Instructors chose whether or not to implement pre-class activities in their course, naturally categorizing students into “pre-class users” or “non pre-class users”. In total, 2,251 students consented to participate in the study (74% of the population), 1,372 engaged in pre-class activities and 879 did not. Groups were compared on three dependent variables: student likelihood to recommend a course using Achieve to a friend, summative assessment scores in Achieve, and in-course final exam scores. We also evaluated the effect of utilization of pre-class activities in Achieve on the dependent variables. Engagement in pre-class activities in Achieve had a significant effect on each of the dependent variables, even when prior academic performance, baseline level of motivation, and the instructor were controlled. We also found a small (based on Cohen’s classification), but significant relationship between the extent to which a student engaged in pre-class activities in Achieve and assessment scores. Finally, results uncovered evidence that instructors and students perceive pre-class activities in Achieve to contribute to positive student behaviors. And, there is evidence that use of pre-class activities significantly contributes to a student’s likelihood to recommend a course where Achieve will be used to a friend, but that it accounts for a small proportion of the variability in the outcome. Achieve for Readers and Writers (1-Term Access)  Achieve Read & Practice for The American Promise, Value Edition (1-Term Access)  Achieve for Interactive General Chemistry (1-Term Access)  ... View more

    Advances in the learning sciences combined with the fast evolution of powerful digital technologies and advances in the design of user experience have the potential to transform the future of higher education. There is now a large body of research demonstrating that learning science principles such as self-regulation, formative assessment, and active learning support the development of lifelong learners (Bell & Kozlowski, 2008). Digital tools can increasingly offer content, resources, and assessments that are personalized, adaptive, and relevant. (Cook et. al, 2013). And, seamless, efficient, engaging digital experiences that make the lives of students and instructors easier can meet the needs of busy educators and a diverse student population (Herrington & Oliver, 2000). However, despite a host of digital learning tools available to instructors there is a lack of rigorous and relevant evidence researching effectiveness and this has often led to false starts and frustrations of what to use and how to use it to best effect to improve student success. When building Achieve, a new digital learning platform, Macmillan Learning began by founding the solution on research-based learning science principles, and co-designing the platform with instructors and students. Researchers at Macmillan Learning then took the unusual approach of beginning to investigate the effectiveness and efficacy of Achieve in its infancy by conducting rapid-cycle evaluations of tool features as they were developed and implementing increasingly rigorous validity and efficacy studies as the platform matured through beta testing. ... View more

In order to provide instructors and other faculty the most valuable and actionable evidence of how a digital tool will work for their students in their educational environments, evaluation of product effectiveness needs to begin in development and continue throughout the lifetime of the product. By beginning testing early in a product’s life cycle, instructors have valuable insights they can use when making adoption and implementation decisions. This research brief presents the results from an implementation study of the iClicker student app and provides a discussion about how this study will provide early evidence of effectiveness to support instructors when they make adoption and usage decisions.   ... View more

In order to provide instructors and other faculty the most useful, practical, and actionable evidence of whether a digital learning tool will work for students in their educational environments, evaluation of product effectiveness should begin in development and continue once a product is on the market. By defining implementation use cases and examining how implementation decisions relate to student outcomes, instructors have valuable insights to refer to when using the iClicker student app in the classroom. This paper presents the results from the iClicker student app implementation study and includes a discussion about how identifying and defining use cases can help instructors make research-driven decisions about how to use a digital learning tool.     ... View more