In this paper, we propose a framework for evaluating the effectiveness of digital learning tools that begins during development and continues once a product is in use. The framework starts by building a clear understanding of the variety of ways users choose to use a learning product, and progresses through increasingly rigorous studies, repeated across different educational environments and use cases. This approach provides a continuously growing body of evidence that, we believe, provides more relevant and reliable insights into how a product will be effective and under what circumstances.   ... View more

In this paper, we outline the approach to educational data mining and learning analytics we have developed at Macmillan Learning, illustrate how this approach is aiding the development of a next generation of digital educational tools, informing textbook authors and content developers, and enabling better insights and intervention strategies for institutions, faculty and staff, and students themselves. But first, we will begin with a brief survey of the current state of analytics.         ... View more