This checklist demonstrates the design, development, and testing process of Achieve and illustrates the evidence currently available to help administrators and educators evaluate whether it will support student success in their educational context. ... View more

SDI Productions/Getty Images       One strategy that has gained popularity for effectively implementing an active learning course design is a “flipped classroom’’ where students engage with the course material before coming to class. The literature on the effectiveness of using pre-class activities to improve learner outcomes is mixed. Consequently, we evaluated the use of pre-class activities in a new digital learning tool, Achieve, to investigate whether engaging in pre-class activities influenced assessment scores. We partnered with 40 instructors on the evaluation of pre-class activities in Achieve. Instructors chose whether or not to implement pre-class activities in their course, naturally categorizing students into “pre-class users” or “non pre-class users”. In total, 2,251 students consented to participate in the study (74% of the population), 1,372 engaged in pre-class activities and 879 did not. Groups were compared on three dependent variables: student likelihood to recommend a course using Achieve to a friend, summative assessment scores in Achieve, and in-course final exam scores. We also evaluated the effect of utilization of pre-class activities in Achieve on the dependent variables. Engagement in pre-class activities in Achieve had a significant effect on each of the dependent variables, even when prior academic performance, baseline level of motivation, and the instructor were controlled. We also found a small (based on Cohen’s classification), but significant relationship between the extent to which a student engaged in pre-class activities in Achieve and assessment scores. Finally, results uncovered evidence that instructors and students perceive pre-class activities in Achieve to contribute to positive student behaviors. And, there is evidence that use of pre-class activities significantly contributes to a student’s likelihood to recommend a course where Achieve will be used to a friend, but that it accounts for a small proportion of the variability in the outcome. Achieve for Readers and Writers (1-Term Access)  Achieve Read & Practice for The American Promise, Value Edition (Six Months Access)  Achieve for Interactive General Chemistry (1-Term Access)  ... View more

    Advances in the learning sciences combined with the fast evolution of powerful digital technologies and advances in the design of user experience have the potential to transform the future of higher education. There is now a large body of research demonstrating that learning science principles such as self-regulation, formative assessment, and active learning support the development of lifelong learners (Bell & Kozlowski, 2008). Digital tools can increasingly offer content, resources, and assessments that are personalized, adaptive, and relevant. (Cook et. al, 2013). And, seamless, efficient, engaging digital experiences that make the lives of students and instructors easier can meet the needs of busy educators and a diverse student population (Herrington & Oliver, 2000). However, despite a host of digital learning tools available to instructors there is a lack of rigorous and relevant evidence researching effectiveness and this has often led to false starts and frustrations of what to use and how to use it to best effect to improve student success. When building Achieve, a new digital learning platform, Macmillan Learning began by founding the solution on research-based learning science principles, and co-designing the platform with instructors and students. Researchers at Macmillan Learning then took the unusual approach of beginning to investigate the effectiveness and efficacy of Achieve in its infancy by conducting rapid-cycle evaluations of tool features as they were developed and implementing increasingly rigorous validity and efficacy studies as the platform matured through beta testing. ... View more

This study investigated the differential efficacy of a new digital learning tool, Achieve, among students less and more academically prepared to succeed. ... View more

In 2018, we worked with a variety of institutions to evaluate the impact of Achieve Read & Practice. These are the resulting educator studies. Prince George's Community College (ES5) Community College of Rhode Island (ES7)     ... View more

In 2018, we worked with a variety of institutions to evaluate the impact of Achieve Read & Practice. These are the resulting educator studies.  Pearl River Community College (ES3) Seminole State College of Florida (ES4)       ... View more

In 2018, we worked with a variety of institutions to evaluate the impact of Achieve Read & Practice. These are the resulting educator studies.  Bellevue  College (ES1) Eastern Florida State College (ES2 +ES6)         ... View more

  This paper is an evaluation of Achieve Read& Practice and the investigation of whether retrieval practice helps close the achievement gap in Psychology courses. ... View more

Educational technology has the potential to vastly improve teaching and learning in higher education. Rapid advancements in technology and the pace at which new tools are being developed are leading to alpha products being tested with students in live classrooms, which can have a negative impact on teaching and learning. There are many constraints of educational technology that is early in development, and student success in higher education is critically important. Therefore, it is necessary to develop alternative methods to requiring students to use an alpha version of a learning tool as their primary course material for a full semester. This paper discusses a new method for conducting a formative evaluation of digital learning tools. The method has proven to enable relevant, timely, and actionable insights on product optimization, implementation patterns, and professional development—without requiring the use of an alpha product in a high-stakes environment. A case study of a formative evaluation of new digital platform, Achieve, is used to illustrate the approach. The formative evaluation of the alpha version of Achieve was a longitudinal study conducted with a set of instructors teaching a course in which the solution might be used, but the study was conducted outside of and independent from their live classrooms. The evaluation was comprised of eight unique rapid-cycle evaluations, each lasting one week, that simulated the arc of a traditional semester. Qualitative and quantitative data were collected to develop insights into platform use, implementation patterns, perception, and expectations. Results from the evaluation were used for real-time remediation and optimization of the tool, to understand instructors’ chosen implementation patterns, and to inform professional development for future users. Real-time results implemented by the development teams provided confidence among researchers and instructors that a beta version of the platforms used in live classrooms in subsequent studies would not adversely impact the student and instructor experience, but rather contribute positively to important learner outcomes.     ... View more

We have been researching the impact of Achieve Read & Practice and here is an infographic that show some of our early data.    ... View more

In order to provide instructors and other faculty the most valuable and actionable evidence of how a digital tool will work for their students in their educational environments, evaluation of product effectiveness needs to begin in development and continue throughout the lifetime of the product. By beginning testing early in a product’s life cycle, instructors have valuable insights they can use when making adoption and implementation decisions. This research brief presents the results from beta tests of ACHIEVE Read & Practice for psychology and provides a discussion about how we have used early evaluation to further refine the product, to inform subsequent impact testing, and to provide early evidence of effectiveness to support instructors when they make adoption and usage decisions.     Achieve Read & Practice for OpenStax Introductory Psychology (Six Months Access)  ... View more

We have been researching the impact of iClicker for years and here are two infographics that show some of our results.      ... View more

This research note offers an overview of the quasi-experimental, mixed-methods study of the impact of the iClicker engagement model on student academic performance. Results from this study indicate that using iClicker in the higher education classroom increases average end of course grades. Additional analyses suggest that this impact can be explained by an increase in student confidence to participate, engagement in class, and preparedness for in-class exams.     ... View more

In order to provide instructors and other faculty the most valuable and actionable evidence of how a digital tool will work for their students in their educational environments, evaluation of product effectiveness needs to begin in development and continue throughout the lifetime of the product. By beginning testing early in a product’s life cycle, instructors have valuable insights they can use when making adoption and implementation decisions. This research brief presents the results from an implementation study of iClicker Reef and provides a discussion about how this study will provide early evidence of effectiveness to support instructors when they make adoption and usage decisions. ... View more

In order to provide instructors and other faculty the most useful, practical, and actionable evidence of whether a digital learning tool will work for students in their educational environments, evaluation of product effectiveness should begin in development and continue once a product is on the market. By defining implementation use cases and examining how implementation decisions relate to student outcomes, instructors have valuable insights to refer to when using iClicker Reef in the classroom. This paper presents the results from the iClicker Reef implementation study and includes a discussion about how identifying and defining use cases can help instructors make research-driven decisions about how to use a digital learning tool. ... View more

  Educational technology has the potential to substantially improve student success, particularly when it comes to active learning. Yet, many instructors are concerned about the impact of internet-enabled devices like laptops and phones in their classrooms, and research demonstrates that using devices in the classroom can divide student’s attention in class (Glass & Kang, 2017; Sana, Weston & Cepeda, 2013; Hembrooke & Gay, 2003; Grace-Martin & Gay, 2001). The division of  student attention between classroom instruction and content on mobile devices is sometimes referred to as multitasking behavior. Previous research has indicated that multitasking behavior (Payne-Carter, Greenburg & Walker, 2017; Jacobsen & Forste, 2011; Fried, 2007) is negatively related to student performance on classroom exams and overall course grades, as well as student perception of content mastery. Macmillan Learning’s new Focus feature in iClicker Reef was created to help students manage the distractions that divide their attention while maintaining active learning in class. Focus is a setting instructors can enable in iClicker to help students stay on task when using mobile devices for participation in iClicker classroom activities. Students log into their iClicker sessions during class and are sent a reminder to remain focused in class if they choose to leave the iClicker session to browse the web, answer texts, or watch videos. Students and their instructor are provided a summary of information about students’ time spent in and out of Focus while using iClicker. The goal of providing this information is to increase students’ awareness of any multitasking behavior and to help them self-regulate their ability to stay on task during class. In the Fall 2019 semester, eight instructors partnered with Macmillan Learning to better understand whether iClicker Focus supports student learning and self-regulation of multitasking behavior. This research brief outlines those partnerships and highlights the results of the study. The report begins with an overview of iClicker Reef and the Focus feature, the research procedures including samples and methods are presented next, followed by results and implications for instructors.   ... View more

Infographics on the data we've gotten regarding students' use of LaunchPad in their psychology courses.      ... View more

In this paper, we propose a framework for evaluating the effectiveness of digital learning tools that begins during development and continues once a product is in use. The framework starts by building a clear understanding of the variety of ways users choose to use a learning product, and progresses through increasingly rigorous studies, repeated across different educational environments and use cases. This approach provides a continuously growing body of evidence that, we believe, provides more relevant and reliable insights into how a product will be effective and under what circumstances.   ... View more

In this paper, we outline the approach to educational data mining and learning analytics we have developed at Macmillan Learning, illustrate how this approach is aiding the development of a next generation of digital educational tools, informing textbook authors and content developers, and enabling better insights and intervention strategies for institutions, faculty and staff, and students themselves. But first, we will begin with a brief survey of the current state of analytics.         ... View more

  Here, we have assembled Six Key Principles for Learning Experience Design that educational research and cognitive psychology indicate are instrumental to driving better learner outcomes. These are divided into six “Foundations” and a corresponding set of six “Principles” derived from these foundations. These underly Macmillan’s other Learning Science Foundations. ... View more

A learning model is a visualization of the instructional and assessment elements that underlie a learning experience and help instructors and institutions understand how a well-designed experience may drive impact. This Learning Model is based on research and practices in Active Learning, a pedagogy that has a substantial body of research demonstrating that it drives student engagement, satisfaction, and performance.  ... View more

A Learning Model is a visualization of the instructional and assessment elements that underlie a learning experience and help instructors and institutions understand how a well-designed experience may drive impact. This Learning Model is based on research and practices in Active Learning, a pedagogy that has a substantial body of research demonstrating that it drives student engagement, satisfaction, and performance. ... View more

A Learning Model is a visualization of the instructional and assessment elements that underlie a learning experience and help instructors and institutions understand how a well-designed experience may drive impact. This Learning Model is based on research and practices in Active Learning, a pedagogy that has a substantial body of research demonstrating that it drives student engagement, satisfaction, and performance. ... View more

Here we provide an evidence-based Learning Objective Strategy derived from a synthesis of the learning science literature. We begin by discussing the benefits of using learning objectives to drive learning experience design, then outline a research-based approach to the design of learning objectives. ... View more

Here we provide an evidence-based strategy for assessment practice derived from a synthesis of the research literature. We begin by outlining the importance and benefits of addressing cognitive and noncognitive aspects of learning, then discuss the guiding principles of an effective assessment strategy, best practices in measurement, and features, capabilities of learning experience design that can enhance the assessment strategy and improve learner outcomes. ... View more

Here we provide overarching principles for the design of effective analytics for instructors derived from a synthesis of the learning science literature. In addition, we provide examples of insights and dashboard reporting elements for instructors that leverage these principles. ... View more

Here we provide overarching principles for the design of effective analytics for students derived from a synthesis of the learning science literature. In addition, we provide examples of insights and dashboard reporting elements for students that leverage these principles. ... View more