Learning Assessment

For March the LCBQ is:

How do you assess whether your informal learning, social learning, continuous learning, performance support initiatives have the desired impact or achieve the desired results?

Jay Cross responds:

You need to wait a while before taking the assessment. Smile-sheets and test scores prove nothing because they are administered before the forgetting curve sets in. The reason only 10%-15% of what is learned shows up on the job is that most of what you learn disappears rapidly unless it’s reinforced by reflection and practice. That’s why it’s a good idea to wait three to six months — to see what sticks.

In the Canadian military, it is called Validation, as opposed to Evaluation. The latter is to make sure your training program is using appropriate methods and resources. The former is to ensure that you’re meeting operational needs.

We used rolling validations in the Air Force. First, they were based on training that had clear Performance Objectives (PO’s). We would follow up with a quick survey to course graduates, their immediate supervisors and perhaps other senior operational staff. Six months after each course we sent out a survey. It listed all the PO’s on the course. Graduates were asked if 1) they had learned the skill on the course; 2) if they had to use that skill in the past six months and 3) if they felt competent with that skill [yes, it’s subjective]. Supervisors were asked if that graduate had performed the skill in the past six months and if so, whether the graduate was competent.

This was a very quick pulse-taking that could then be followed up with more in-depth interviews.

Could you do this with informal or social learning? Of course you could. You just need to describe the capability you wish to measure.

For example, after 6 months on the job, with access to micro-sharing, company blogs and the development of a personal knowledge management framework, you could ask similar questions.

Do you have to regularly access information and knowledge to do your job?

Have you learned to access information and knowledge faster than when you started 6 months ago?

Supervisors can be asked if people are effective at getting access to information in a timely fashion.

The Big Question is, what do you expect to achieve? In a complex environment, there is no linear relationship between cause and effect, so we know these measurements are only indicators. Used correctly, and administered as lightly as possible, they can help make informed decisions on priorities, support or resource allocations.



5 Responses to “Learning Assessment”

  1. Kelly Meeker

    Thank you for this post. There’s such an important difference between ensuring that the student has actually attended & participated in a learning experience and actually ensuring that the learning met your overall objectives and improved the student’s performance. Not easy to measure unless you have the objectives clearly in mind in the first place.

  2. Gilbert Babin

    KMs response is illustrative of how we tend to thinki in term of students but often forget that it is the organisation as a whole that must learn and perform.

    Evaluation, assessment and validation are different functions with different purposes. They all have their place in a healthy ecosystem. This said they are not goals.


    • Harold Jarche

      I agree, Gilbert, these are only indicators. The organization has to be cognizant of its mission and principles and use these data to check whether its on course.

  3. Sibrenne Wagenaar

    I like the question of how to measure the impact of informal and social learning. Would be interesting to see if you can stretch the questions even more… What in your work goes easier due to…? What are moments of informal learning and how has this effected your work? What do you think Harold?


Leave a Reply

  • (will not be published)