“Efficiency is doing things right. Effectiveness is doing the right things.” – Peter Drucker
Dan Pontefract is quite clear in Dear Kirkpatrick’s: You still don’t get it:
Let me be clear – training is not an event; learning is a connected, collaborative and continuous process. It can and does occur in formal, informal and social ways every day in and out of your job. In your email, with the statement “what happens after the training event”, you have cemented (again) the root cause of the Kirkpatrick model. The ‘event’ is not solely how learning occurs. Whether in the original model, or the weakly updated model, the single largest flaw with the Kirkpatrick Four Levels model is the fact its basic premise is that learning starts with an event. Once you ultimately get past this stumbling block, the Kirkpatrick Four Levels model will potentially become relevant again, should it be suitably updated again.
Dan is not the first person to show the limitations of the Kirkpatrick model. Eric Davidove and Craig Mindrun wrote in Verifying Virtual Value:
The key to determining the business value of networked learning, however, is a more expansive view of the kinds of outcomes delivered. Traditional training analyses, such as Kirkpatrick’s four levels of evaluation, were designed to assess solutions that are delivered in a linear manner. Since networked or collaborative learning solutions are informal, integrated with the workflow and driven by the learners, these traditional assessments will not work.
Event-based instructional interventions, or the course as learning vehicle, is an outdated and useless way to look at workplace learning. Courses are an artifact of a time when information was scarce and connections were few. The internet is an environment optimized for ABC learning [Anything But Courses].
In “Not Your Father’s ROI”, Jay Cross suggests:
Make a hypothesis of cause and effect. Interview a statistically significant sample of the workforce to see if the hypothesis holds up. Often, results obtained from social science research methods will produce more meaningful feedback than solid counts of the wrong thing.
Changing our training evaluation models shouldn’t be a management focus anyway. That’s looking at the wrong thing. Even if we get 100% efficiency, and some level of effectiveness, we’re missing 90% of the picture, as shown in this graphic by Charles Jennings.
Training more efficiently is a mug’s game. Managers and workplace performance professionals should focus on Working Smarter, by helping people learn and develop socially.