How do we measure training, and why? Are L&D departments tracking, measuring and reporting meaningful learning data, or just data that makes a solid case for their own s
urvival?
It’s not easy to determine learning effectiveness and learning transfer. But sometimes I wonder if the metrics we usually gather and when we gather them – learner satisfaction right after the end of the course – are good predictors of learning transfer or simply a good Kirkpatrick’s Level 1 study. Yes, satisfied learners will come back for more, but isn’t that a bit of a brute force approach?
Rummler, Brache, Hale and other Human Performance Technologists remind us to take a systemic look, and when I see the data that is usually collected as part of a learning project, I wonder if we should be teaching a bit less and measuring a bit more. More about the system, to be precise.
In a study of corporate learning efficiency published in the American Journal of Distance Education, Gunawardena et al. (2010) suggest that collegial support is the strongest predictor of learning transfer. This is the result of a study conducted in a multinational delivering a highly technical online learning program. Collegial support is part of the system rather than the learning solution alone, although there is much that can be done at the learning solution level to support it.
What do they mean by “collegial support” exactly? These are the questions that encapsulate collegial support:
- My colleagues encourage me to implement what I have learned in this course
- I share what I have learned with my colleagues so that more employees benefit from my learning opportunities
- I have worked together with my colleagues in troubleshooting complications when implementing the newly acquired skills
In this same study, learner satisfaction was driven by factors that were unrelated to learning transfer. Specifically, the main predictor of learner satisfaction was self-efficacy, understood as the ability and confidence to use the online learning solution.
The findings of this study seem to point to a different set of metrics that are not that difficult to capture, although perhaps a bit unpopular in environments where individual performance has taken such a prominent role that collaborative attitudes have lost all incentives.
In another study of corporate online training, Joo et al. (2011) conclude that organizational support has a direct effect on learning transfer. Again, another system-level, not learner-level variable. Here, “organization support” was defined as “supervisor support, peer support and organizational culture”. It is tempting to assume that such variables are constant across the company, but having supported many teams in multiple countries across Europe, I can confidently say this is far from constant, and depends not only on country culture but also division, department, and team culture.
Learner satisfaction is great to have. But are you measuring the right things?
By the way, if you believe that collegial support is the best predictor of learning transfer, what do you think of “social learning” in the corporate space?