Focus on getting results - measure performance not learning

Introduction of the 70:20:10 model means many changes for learning and development teams, one of which is in evaluation. A common question asked in our webinars is, “How do you measure learning in a 70:20:10 program?” In this blog post I'm going to explore that question.

Why do we measure learning?

There are two broad reasons why learning and development teams measure learning programs:

• as a means to continuously improve the program

• to gauge the business outcomes that the program has generated.

The problem with focusing on continuous improvement

An end-of-program survey is normally the main method used to gain data for continuous improvement of a training program. The reality is, however, that these surveys tend to seek only information about components like the quality of the catering and the performance of the trainer. If it’s an elearning module the usability of the interface is often the subject.

Some also include some type of self assessment of learning and behaviour change, for example was the course relevant, and what are you going to change in the workplace after doing the course? The data is collected and used to drive decisions about the next time the course is run. Getting learner feedback is important, but this is only one dimension of what learning evaluation can be.

Collecting data based on reactions works okay in an event-driven approach to learning, but it doesn't work in a 70:20:10 learning model where learning is continuous and embedded into the flow of work. It's not a mature approach to learning measurement. What is amiss here is that this approach focuses on the measurement of the process of learning, not on the outcome. Learning should result in a change of behaviour and an on-the-job performance improvement. One way to think about learning is as an input for performance improvement.

Measuring business outcomes

The other way to think about learning measurement and evaluation is to focus on measuring the performance outcome from a program at an individual and organisational level. This is not always easy to do. I often talk about learning not being a straight line, by which I mean simply designing and implementing a program doesn't always mean you will get good results. Organisations are complex, and there are numerous factors that affect the success of a program. One learning and development manager I worked with used to run a coaching development company and was opposed to outcome-focused measures because he felt that as an external provider he couldn't influence all the factors.

What I've found is that focusing on business outcomes means that in the early stage of designing a program the conversations quickly transform. Often there is a realisation that a training solution is only part of the whole, which quickly leads to an approach that is more focused on performance improvement. Focusing on outcomes is a great approach to gain support for a more integrated, holistic 70:20:10 approach. It's more likely to lead to conversations about the integration of learning into the flow of work, performance support, and how managers are involved in the implement and rollout of programs.

The first challenge in a performance-focused evaluation approach is deciding what metrics are to be measured and how that data will be collected. Learning and development teams are used to collecting data from surveys that they control, or data from the learning management system, but are normally not involved in the collection of wider business performance data. A few approaches I've seen work well in the past are:

• using business metrics that are already being collected and reported on

• benchmarking approaches – where data is collected before, during and after the program has run.

Focusing on performance outcomes means that the outcomes are clear to everyone including the participants, managers, and anyone involved in the implementation of the program.

Measuring how often employees access a learning portal is measuring an input, not an outcome. While this type of data can still be useful for continuous improvement, a performance-focused approach would instead correlate the data about what employees are doing (the inputs) with what the performance outcomes are. This means you might look at a sample of employees who have improved their performance as part of the program, and then, if the program is focused on performance supports, you would look at what resources they have access to. A further interview about the factors that contributed to their performance improvement would be useful too.

The 70:20:10 model means changes in the evaluation methods for learning and development professionals. It means moving beyond just collecting data about learning input, and it provides the opportunity to focus evaluation and measurement on the business outcomes. This can also lead to the wider business understanding of learning’s effect on business metrics.