Levels of Evaluation
(LEV-uhls ov EEval-YOU-ayshun)
GAS-LIKE
What Are Levels of Evaluation?
Levels of evaluation are a variety of distinct ways to measure a training program. These include number of attendees, number of course completions, participant reaction, knowledge gain, on-the-job skill transfer, impact on a department or organization, and return on investment.
Other Common Names
Other common names for dialogue education include:
- Kirkpatrick Evaluation Model
- The Four Levels of Learning Evaluation
Key Properties
Element 30, levels of evaluation, is characterized by three properties:
- Determines the effectiveness or success of a training program
- Uses qualitative and quantitative measures
- Gathered at various points between the actual training event and a specified time following the event
Resources From The Train Like a Champion Blog
Insights from Brian Washburn in What's Your Formula (not at all gaseous)
- "Have you ever collected end-of-training evaluation forms and, as soon as the last participant is out the door, hurried to scan through the comments? Lots of “that was the best training I’ve ever received” can absolutely make your day. Of course, reading “that was a total waste of time”—even if it’s the only negative comment in 50 evaluation forms—can be soul crushing. The thing is, those post-training evaluation forms are only one way to evaluate the program, and we shouldn’t put too much weight on any particular evaluation component. Each one provides a data point that tells a story—the more data points, the more complete the story we can tell."
- "Each level provides a glimpse of program effectiveness from a different point in time, and each level offers imperfect data. Keep in mind that not every training program needs to be measured at every single level. To determine which levels are most appropriate for your next training initiative, you’ll have to determine what you want success to look like. And be sure to ask that question before you begin designing the program. It’s very difficult to collect baseline data, for example, after you’ve already launched the learning program.
- "There are six commonly used levels of evaluation. Levels 1–4 are commonly attributed to Donald Kirkpatrick and collectively referred to as Kirkpatrick’s Four Levels of Evaluation. Various attributions have been given to Levels 0 and 5, which may offer a more complete view, depending on how you use the data."
CLOSELY RELATED ELEMENTS
Supervisor Support
When you find ways to enlist the support of your learners’ supervisors, they can help the learners find the relevance and importance of your content, which can boost Level 1 scores. Supervisor observations are also key to a more objective Level 3 evaluation.
Audience Response
While pre- and post-testing are perhaps the most common way to formally conduct Level 2 evaluations, there are many opportunities to gather in-the-moment data around how much and whether learning is taking place. Using audience response software can be an informal way to collect Level 1 (reaction) and Level 2 (knowledge) evaluation data.
Quiz Software
Similar to audience response tools, quiz software can help you gather Level 2 information during the flow of a training program. Most quiz software allows you to review participant results and generate reports after participants have completed the quiz (which is often conducted in the form of a game).
Dialogue Education
Dialogue EducationSimilar to several other examples discussed, the element of dialogue education can be an intentional yet informal way to measure Levels 1 and 2 during the course of a training program. When participants are engaged in conversation and dialogue, it’s much easier to observe how much they’re picking up and retaining.
Learning Objectives Taxonomy
Identifying the appropriate objectives on which to focus and connecting them with the appropriate activities is the only way to ensure you’re able to accurately measure the right things.
Instructional Design
As with everything else in this chapter, once you’ve determined which levels of evaluation are important, they should be part of the overall design of the learning program. Pre- and post-tests will take up precious session time. Efforts at Level 3 and Level 4 evaluation should be announced and even designed as part of the overall program, meaning learners must respond to post-training surveys so you can collect that data before they receive their certificate of completion for the course.
Data
While all levels of evaluation are data points, it may be necessary to collect other information before you can make sense of the evaluation results. Baseline data and data on outside factors can all offer greater context, which will make any data collected as part of the levels of evaluation more meaningful.
Other Sources of Information on Levels of Evaluation
Books on Levels of evaluation
