Apr 08, 2015
So many organisations use the above ‘line’ as their sole way of evaluating the learning effectiveness of their employee training programs.
Why is this? Upon returning from training why employees are not asked questions like "what skills or behaviours will you now apply or demonstrate from the training" or "what will you do differently now that you have been trained?" or "what will you stop doing or start doing as a result of the training?" is beyond me.
So if your organisation is serious, and why wouldn’t it be, about evaluating the learning effectiveness of employee training today could be your lucky day.
Therefore, for my post today I want to introduce you to the king of training evaluation and learning effectiveness…Donald L. Kirkpatrick.
Donald Kirkpatrick first published his ideas in 1959, in a series of articles in the US Training and Development Journal. The articles were subsequently included in Kirkpatrick’s book Evaluating Training Programs (1975), published by the American Society for Training and Development (ASTD), with whom Kirkpatrick still maintains (as at 2005) close connections. Donald Kirkpatrick has written several other significant books about training and evaluation, and has consulted with some of the world’s largest corporations.
Kirkpatrick’s book Evaluating Training Programs defined his originally published ideas of 1959, thereby further increasing awareness of them. Therefore, his theory has now become arguably the most widely used and popular model for evaluation of training and learning.
Kirkpatrick’s four-level model is now considered an industry standard across HR and training communities. The four levels of training evaluations model was later redefined and updated in Kirkpatrick’s 1998 book called Evaluating Training Programs: The Four Levels.
The four levels of Kirkpatrick’s evaluation model essentially measure:
- Reaction of student – what they thought and felt about the training.
- Learning – the resulting increase in knowledge or capability.
- Behaviour – extent of behaviour and capability improvement and implementation application.
- Results – the effects on the business or environment resulting from the trainee’s performance.
All these measures are recommended for full and meaningful evaluation of learning in organisations, although their application broadly increases in complexity, and usually cost, through the four levels.
Level 1 Evaluation – Reactions
Just as the word implies, evaluation at this level measures how participants in a training program react to it. It attempts to answer questions regarding the participants’ perceptions – did they like it and was the material relevant to their work? This type of evaluation is often referred to as the ‘happy-sheet.’ According to Kirkpatrick, every program should be evaluated to at least this level to provide for the improvement of a training program. Further, the participants’ reactions have important consequences for learning (level 2). Although a positive reaction does not guarantee learning, a negative reaction almost certainly reduces its possibility.
Level 2 Evaluation – Learning
Assessing at this level moves the evaluation beyond learner satisfaction and attempts to assess the extent students have advanced in skills, knowledge, behaviour or attitude. Measurement at this level is more difficult and time-consuming. Methods range from formal to informal testing to team assessment and self-assessment. If possible, participants take the test or assessment before the training (pre-test) and after training (post-test) to determine the amount of learning that has occurred.
Level 3 Evaluation – Transfer
This level measures the transfer that has occurred in learners’ behaviour due to the training program. Evaluating at this level attempts to answer the question – "Are the newly acquired skills, knowledge, behaviour or attitude being used in the everyday environment of the learner?" For many trainers this level represents the truest assessment of a program’s effectiveness. However, measuring at this level is difficult as it is often impossible to predict when the change in behaviour will occur. Therefore, important decisions in terms of when to evaluate, how to evaluate and how often to evaluate need to be made.
Level 4 Evaluation – Results
Frequently thought of as the ‘bottom line,’ this level measures the success of the program in terms that the managers and executives can understand – increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment. From an organisation standpoint, this is the overall reason for a training program, yet level four results are not typically addressed. Determining results in financial terms is difficult to measure, and is hard to link directly to the training.
Here are some methods for long-term (Level 3 and Level 4) evaluation techniques:
- Send post-training surveys (3 to 6 months) after a training program
- Offer ongoing, sequenced training and coaching over a period of time
- Conduct follow-up needs assessment
- Check key metrics to measure if participants achieved training objectives
- Interview trainees and their managers, or their customer groups
In summary, learning evaluation is a widely researched area. This is understandable since the subject is fundamental to the existence and performance of education around the world.
While Kirkpatrick’s model is not the only one of its type, for most industrial and commercial applications it suffices. Indeed, most organisations would be absolutely thrilled if their training and evaluation, and therefore their ongoing people development, were planned and managed according to Kirkpatrick’s model.
To learn more about our range of training and evaluation programs and our other professional development programs, take a look at our website.
Until next time, keep learning, keep listening, and keep moving.