How do we measure the success of our students? Given the increased role of assessment in justifying resources both within and outside a university, the question of how we examine, describe, and improve college student success is more important than ever. As part of this process, we sometimes get stuck in discussions about how to measure outcomes. One of the common questions is whether we should use direct or indirect measures. Some advocate for only direct measures, saying that indirect measures aren’t needed. Some believe a survey or indirect measure is sufficient because that is often easiest. Others believe that a more comprehensive approach, including both direct and indirect measures, is needed. Let’s explore these different approaches.
Direct measures, particularly of learning, are important. The most obvious direct measures are tests of knowledge. However, direct measures also include portfolios of sample work, performances, simulations, and other actions that can be observed and rated. We trust these measures because we can see the outcome. Thus, direct measures are powerful and needed.
Indirect measures are different than direct measures. Instead of having the learner demonstrate their knowledge or skill, indirect measures ask individuals to reflect upon and describe the learning or outcome. These individuals could be students, alumni, supervisors, faculty, or staff. At first glance, indirect measures may pale in comparison to direct measures. Why would we ask for descriptions when we can observe? However, indirect measures are valuable for multiple reasons:
- Some outcomes are difficult to observe. How would you observe a change in attitude? Asking someone to describe their attitudes or even how they changed can be simpler and clearer. Or, how would you observe something like lifelong learning? Determining the range of behaviors that may demonstrate lifelong learning is challenging.
- Imagine we want to measure citizenship and are interested in community service, voting, and other forms of civic engagement. Instead of observing someone over a long period of time, we can simply ask about those behaviors. Sometimes the cost of observations isn’t worth it.
- Individuals who can provide solid information about outcomes, such as alumni and employers, are more likely to engage in an indirect measure. Alumni and employers will fill out a survey, but often do not want to take a test.
- Indirect measures such as surveys are efficient. They can be used to gather information about many outcomes, at various points in time, and for many students. Tests, performances, and portfolios require significant effort from the students who complete them and can place a heavy load on the faculty and staff who evaluate them. It often isn’t feasible to use direct measures for all learning outcomes for all students and graduates at various points in time.
- Finally, surveys measure other aspects of the collegiate experience that contribute to learning and other student outcomes (classroom and out of class experiences, course content, relationships with faculty, staff and peers, etc.). To understand how to improve learning and student success, this information is critical.
Indirect measures are often easier and more effective when considering time and resource constraints. However, indirect measures cannot take the place of direct measures. Ultimately, both have their own power and are indispensable parts of an accreditation or program review. But, given some of the challenges in using direct measures, why would we ignore something as valuable as indirect measures when measuring student success?
Are you interested in learning more about direct and indirect measures and when to use them? Watch a recent webinar recording, “Measures Matter: Exploring Direct versus Indirect Measures.”
