Learning and Development: A Strategic Function, Stage 4: Evaluation

Over the past few months, we have been looking at the stages of the training cycle with the aim of creating a better understanding of L&D as a strategic function.  In this newsletter, we talk about the final stage of the cycle i.e. Evaluation, which is undoubtedly an area of interest for all businesses. As a reminder, the four stages of the training cycle are illustrated below:

Although Evaluation is the final stage of the training cycle, it is very closely linked to stage 1 which is the Needs Assessment and Analysis stage. This link is important as all L&D activities should begin with a clear understanding of the business priorities and the expected outcomes, and end with a robust mechanism to measure the impact of the investment. It is important to measure the effectiveness of any L&D intervention at both an organisational and individual level as this can actually help to steer the investment in terms of time, money and effort so that it complements and supports the actual needs of the business. This calls for the L&D function to evolve into a more consultative role and consider business priorities as the basis of employees’ development.

The outcome of the Needs Assessment (stage 1 as discussed in our March newsletter), determines what the business expects to achieve at the end of the training. It is the responsibility of the L&D department to ensure that there is a process, metric and tool in place to measure the expected outcomes. The two most commonly used models for the purposes of evaluation are:

• The Kirkpatrick Model of Evaluation by Donald L. Kirkpatrick
• The Phillips ROI Methodology by Jack J. Phillips PhD

Both models are based on the notion that providing the individual with an opportunity to learn triggers a chain reaction that can confirm effectiveness of the intervention. A comparison of the two models is given in the table below:

Level	Kirkpatrick Model	Phillip ROI Methodology
1	Reaction	Reaction, satisfaction, and planned application
2	Learning	Learning
3	Behaviour	Behaviour, application, implementation
4	Result	Business impact
5	Return on expectations (ROE)	Return on Investment (ROI)

Adapted from: http://www.elearninglearning.com

Which one of the two models is more effective and relevant to the business is debatable. Both models address Evaluation at similar levels and ultimately assess the effectiveness by analysing the impact on the business. Both models are being challenged for appropriateness and effectiveness by leading L&D researchers now, but in actual fact most businesses only apply Level 1 of the models which is not sufficient to evaluate the impact on the business in financial or non-financial terms.

Phillips sees Return on Investment (ROI) as a separate level because it requires a new set of data that is not found in the other levels of Evaluation. The ROI level in the Phillips approach indicates that the impact of training should be evaluated in isolation to ensure results are tangible and credible. Unlike Phillips, the Kirkpatrick approach does not see Return on Investment (ROI) as a separate level and focuses on the business partnership approach. The Kirkpatrick model emphasizes the need to focus on collective efforts to accomplish return on stakeholder expectations without attempting to isolate the effects of training. That is why Return on Expectations (ROE) is being viewed as the fifth level of the Kirkpatrick model.

ROE is a collaborative agreement that unites an organisation in working towards a common goal. It is becoming quite popular in progressive organisations but it is important to understand that even though the measurements for ROE may not be as scientific and formulaic as ROI, it is not very different from ROI. ROE can in fact be a strong measurement that may include a formal metric based ROI calculation if that is what the business stakeholders expect. The key is to understand the objectives of L&D intervention and define a metric that can be used to monitor and track progress. Some examples of typical ‘expectations’ converted into successful outcomes are:

• Revenue from new customers
• Increased operational efficiencies with decreased training costs
• Reduced turn-around times
• Increased retention of top talent

Many people struggle with implementing ROE / ROI because they do not always understand the business expectations and can overlook the flaws of the process. It is important to avoid getting caught up in the use of jargon as this can impact the credibility and substance of the training and the business objectives. Asking the right questions such as “What will success look like to you?”, “What is the expected outcome?” etc. can help to convert broad and un-quantified expectations into observable and measurable indicators. The focus of ROE is on self reflection to address identified gaps in business skills and desired level of competencies.

Evaluation involves ‘the formal or informal assessment of the quality and effectiveness of an employer’s L&D provision, usually either by some measure of the merit of the provision itself (…) and/or by monitoring its impact (…)’ (CIPD 2015). Whether you want to focus on ROE or potentially calculate ROI, just like so many things in life, it all depends. In this case, it all depends on the goals of your Evaluation, the metrics you intend to use to measure success, and the data your stakeholders need. Regardless of the model used by your organisation, the important consideration is what is the business aiming to achieve from training?

It could be argued that many L&D functions have been caught up in administrative activities for far too long. Therefore it is now crucial for L&D functions, to break away from the monotony of training administration and think of development as a function that is embedded in the strategic intention of the business so that it can serve as the key driver of the organisation’s success.