Measuring for Effectiveness
(MEH-zure-een fur EEfek-tiv-nuss)
LIQUID
What does Measuring for Effectiveness Mean?
Measuring for effectiveness (or training evaluation) is a learning program strategy in which the efficacy of a program is measured using qualitative and/or quantitative means.
Other Common Names
Other common names for measuring for effectiveness include:
- Training evaluation
Key Properties
With measuring for effectiveness, you are gauging the impact of a learning against the intended outcomes. Some of the primary properties of this element include:
- A baseline has been established
- One or more measurement instruments are used
- Outside factors are controlled for (to the fullest extent possible)
- Answers the question: Would people have been better off if this training program didn’t exist?
Resources From The Train Like a Champion Blog
- Training Measurement Demystified: Why and how to measure training.
- What it means to be an “L&D Detective”: Discussion with Kevin Yates, a self-described L&D detective doing some of the most important work that can be done in our industry, uncovering the impact and value of learning initiatives.
Insights that Flow Freely through What's Your Formula
- "The tricky thing about trying to measure any training program for effectiveness is that there are so many other variables that could influence the results. Is a boost in sales following a training program the result of a sales team who is now better equipped and more knowledgeable, or is it the result of a strong overall economy? Is increased retention and job satisfaction the result of a new coaching program, or are employees simply happier because the weather has turned nicer? There are a lot of factors at play and very few (if any) organizations are willing to invest the time, money, and resources needed for a scientific study of the impact or effectiveness or a training program. That said, learning professionals still shouldn’t disregard the need to measure whether or not a learning program was “effective.” After all, why should your organization bother to allocate money for learning programs? Why should employees bother logging into the LMS to access your e-learning library or attending your next session on leadership development?"
- "The data you collect and how you collect it will go a long way in determining if it’s actually good data that paints an objective picture of your program’s results. Are you relying primarily on self-reported information? People tend to think more highly of their own performance. Does anyone have incentive to paint a rosier (or bleaker) picture of the impact of the training? Sometimes in an effort to prove something we want to be true (that our training program was super effective, for example), we look for data that supports our case while ignoring or not collecting data that suggests otherwise."
- "There will always be factors outside the control of your instructional design and facilitation skills. Things such as the economy, company culture, workload of your participants, language or culture barriers, technological savvy, and even things like a participant’s (or their boss’s) mood after the Super Bowl, can all have an impact on the effectiveness of your training program."
Elements That Bond with Measuring for Effectiveness
Smile Sheets
While good post-training evaluations don’t guarantee learning happened, poor post-training evaluations often indicate that learning probably didn’t happen. There’s only so much useful information (when it comes to measuring training effectiveness) that can be gleaned from smile sheets, but they do offer one data point by providing insight into your learners’ reactions to the learning Experience.
Assessment
You can examine participants’ increase in knowledge or skills using the activities that take place during a learning experience and/or following a learning experience through post-testing or on-the-job observation. Again, no single measurement instrument will give you all the information you need to know, but having multiple data points allows you to put together a story about the effectiveness (or lack thereof) of your learning experience.
Levels of Evaluation
Working your way up the four levels of evaluation, you can collect data points that paint a clearer picture of your program’s effectiveness by gauging learner reaction, knowledge, transfer to the job, and on-the-job results.
Instructional Design
You can’t wait to plan measurement until after your boss says: “You know that training program you launched three months ago? Do we know if it was effective at all?” If you’re going to try to measure the effectiveness then you’ll want to bake that measurement into the original program design, beginning with assessing for the needs you’re going to fill and determining the program’s learning objectives.
Role Play
While role play and simulation activities are done in the relative safety and security of the training environment, observing changes in participant behavior and knowledge during these activities can offer preliminary information as to whether you can reasonably expect to see real world results.
Data
Every single suggested element to bond with measuring for effectiveness has provided a way to collect data. Data—both qualitative and quantitative—offers insight into the impact and effectiveness of a training program. Some programs such as teambuilding may not need many data points while other programs such as executive coaching or sales training may require a variety of data points and a longer period to gauge effectiveness.
Other Sources of Information on TRaining Evaluation
Books on Training Evaluation
