We've Measured Training for 60 Years. So Why Are We Still Measuring the Wrong Thing?

In 1959, Donald Kirkpatrick introduced a four-level framework for evaluating training effectiveness.

It became one of the most widely referenced models in Learning & Development. Most HR and L&D leaders know it. Many strategy documents mention it.

Yet in practice, many organizations still stop at Level 1.

They measure whether participants enjoyed the program. Whether the facilitator was engaging. Whether the workshop scored 4.7 out of 5.

Those numbers are not useless. They tell us something about the learning experience. But they do not tell us whether anything changed.

They do not tell us whether a manager gave better feedback. They do not tell us whether a salesperson handled objections more effectively. They do not tell us whether a team became more accountable. They do not tell us whether the business improved.

That is the measurement gap StrataVant was designed to close.

The real question is not “Did they like the training?”

The real question is:

Did the learning change behaviour in real work?

Kirkpatrick’s model gives us a clear way to think about this.

Level 1 — Reaction: Did participants find the program useful and engaging?

Level 2 — Learning: Did they gain the knowledge, skill, confidence, and commitment to apply what they learned?

Level 3 — Behaviour: Are they using the new behaviour back at work?

Level 4 — Results: Has that behaviour change contributed to business outcomes such as retention, productivity, revenue, customer satisfaction, or team performance?

Most organizations are not wrong to measure Level 1. The problem is when Level 1 becomes the finish line.

A positive workshop rating does not prove behaviour change. A completed course does not prove application. A confident participant does not prove business impact.

This is where many development investments lose credibility with business leaders. The CEO and CFO are rarely asking, “Did people enjoy the program?” They are asking, “What changed because of it?”

What the New World Kirkpatrick Model clarified

In 2016, James and Wendy Kirkpatrick updated the model for a new world of learning. Their key message was simple but important:

Do not design training first and measure impact last. Start with the business outcome and work backwards.

That means we should first ask: What business result are we trying to support? Then: What behaviour must change to support that result? Then: What capability, confidence, and commitment must learners build?

Only after that should we design the learning experience.

This shifts L&D from a content-delivery function to a behaviour-change function. That is a very different mandate.

It also explains why many programs struggle to reach Level 3 and Level 4. They were never designed backwards from business outcomes in the first place.

Why organizations stopped at Level 1

It is easy to criticize organizations for measuring the easy things. But the honest reason is more practical.

For years, measuring behaviour change was difficult. It required observation after the workshop. It required managers to stay involved. It required reinforcement, follow-up, coaching, tracking, and data. Most L&D teams did not have the time, tools, or infrastructure to do that consistently.

So they measured what was available. Attendance. Completion. Satisfaction. Workshop ratings.

These became the easiest indicators to report, even though they were not the strongest indicators of impact.

The issue was not always lack of intent. It was lack of system.

What AI has changed

AI does not change learning because it can generate content. That is the least interesting use of AI.

The bigger shift is that AI makes behaviour practice, feedback, and reinforcement easier to scale.

A leader can now rehearse a difficult feedback conversation with an AI character before having the real conversation with a team member. The AI can push back, deflect, become defensive, or ask difficult questions.

The learner is not just recalling the SBI feedback model. They are trying to use it under pressure.

That matters. Because real leadership does not happen in a worksheet. It happens in moments of discomfort, conflict, ambiguity, and emotion.

AI role play gives learners a safe place to practise those moments before they matter.

It also creates useful data: Did the learner describe the situation clearly? Did they use evidence instead of judgement? Did they explain the impact? Did they invite dialogue? Did they stay calm when challenged?

This gives organizations a stronger view of whether learning is moving toward behaviour. Not perfect proof. But much better evidence than a smile sheet.

The missing bridge: reinforcement

The New World Kirkpatrick Model also emphasizes “required drivers” — the reinforcement structures that help learning become behaviour.

This is where many programs fail. Participants attend a workshop, feel motivated, return to work, and get pulled back into old habits.

Their managers are not involved. There is no practice rhythm. There are no structured check-ins. There is no application task. There is no accountability loop. So the learning fades.

At StrataVant, this is why every development journey is designed around four phases: Learn. Rehearse. Execute. Reflect.

Learning alone is not enough. Learners must rehearse the behaviour before applying it. They must execute it in real work. They must reflect on what happened. Managers must reinforce the behaviour after the formal session ends.

This is how development moves from event to system.

What StrataVant measures

StrataVant’s Leadership Excellence program is not built around the question “Did participants enjoy the workshop?” It is built around a more important question:

Did leadership behaviour change in real work?

That is why the program starts with the business outcome the client wants to support — for example, stronger accountability, better feedback conversations, improved coaching quality, clearer performance expectations, stronger manager ownership, lower avoidable turnover, and better team execution.

From there, the program works backwards into the behaviours leaders need to practise.

The measurement approach then looks beyond satisfaction scores. It can include AI practice engagement, role-play performance against behavioural rubrics, workplace application tasks, manager check-ins, reflection submissions, 360 pulse shifts, and 90-day leadership commitments.

Together, these provide a stronger Level 3 view and a clearer bridge toward Level 4 business outcomes.

This is more useful for sponsors because it answers a more serious question: What behaviour are we seeing after the development investment?

The future of L&D is not more content

Organizations do not need more workshops that end with good feedback scores but little follow-through. They need development systems that help people practise, apply, and sustain new behaviours in real work.

Kirkpatrick gave us the framework more than 60 years ago. The New World Kirkpatrick Model reminded us to start with outcomes and work backwards. AI now gives us a practical way to make behaviour practice, reinforcement, and evidence more scalable.

The opportunity is clear. L&D can move from measuring learning activity to measuring behaviour change. And that is where development finally starts to earn its place in business strategy.

Final question

When your organization evaluates leadership or sales development, which level does it actually measure — and which level does your board expect an answer at?