Below is a piece written by our CEO James Judge on Learning and Development Spending for the People and Capability section of the online publication – themandarin.com.au “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” — John Wanamaker Learning and Development
In my last article introducing our People and Capability section I ventured that while fads and fashions permeate every human endeavour from science to art — the field of human resources seems particularly susceptible to their influence. In this post I focus on learning and development (L&D) – what used to be simply called ‘training’ when I first worked in the public service. Normally treated as sub-set of human resources, but often with input from other functional areas and line managers, it faces its own fashions and challenges.
It’s not too contentious to suggest that L&D programs that are not evidence informed in their design and delivery will more likely be driven by fads. An associated challenge in this field, one that is rarely publicly ventilated, is the issue of meaningfully measuring return on investment (ROI).
There is an old aphorism in the advertising world, attributed to both the famous English industrialist Lord Lever and US department store magnate John Wanamaker, speaking to a similar problem. In an era before the advent of google analytics the conundrum they both faced was that advertising spending seemed to yield results, but it was impossible to know what part of the spend was working and what wasn’t.
More on fads
Marcus Buckingham and Donald O. Clifton wrote a book in 2001 that was to become an influential bestseller. Now, Discover Your Strengths introduced the reader to a free online assessment tool, or ‘StrengthsFinder,’ that would uncover their personal signature strengths. Reportedly derived from a lengthy study of nearly 2 million people (said to have excelled in their careers) the premise of the book was that the existing learning and development focus in organisations was all wrong. Rather than devoting energy trying to remedy employee weaknesses, organisations and individuals should instead be focused on building upon existing strengths.
Despite criticism from psychologists and others, Buckingham went onto write a series of very popular books, mostly building on this strengths focus principle. He formed the eponymous The Marcus Buckingham Company which still offers leader development and employee engagement solutions using a ‘strengths based’ approach. Gallup, a company which Buckingham had a long association, also offers a (paid) strengths assessment and a range of services to help create ‘Strengths based workplaces.’ His legacy can be seen in the CVs of many coaches and leadership experts who reference their ‘strengths based’ approach’.
Fast-forward 12 years to another extremely influential writer of bestsellers, Malcolm Gladwell, and his 2013 David and Goliath. In this book Gladwell conjectures that what is learned out of necessity (what he terms ‘Compensation Learning’) is inevitably more powerful than learning that builds on the strengths we have been naturally given (what he calls ‘Capitalization Learning’). Again there has been criticism — I leave to history the impact on Learning and Development programs.
The challenge
Every year in every jurisdiction, every government agency will spend something on learning and development. Decisions will be made as to what needs to be included in annual programs for which cohort of workers. As an example, it would be a safe bet that public hospitals in Australia will run short courses on safe-lifting techniques while public service commissions will be offering longer duration, leadership programs.
On another level, decisions will be made as to how desired learning and development outcomes or objectives are best met. Should this be achieved through more traditional off-site training over a block of days or perhaps an intensive online course? Conceivably the newly emerging trend of using micro-learning modules is worth exploring or possibly an external coach or an internal mentor will be considered to assist in behavioural change? Maybe a mix of methodologies, like the APSC endorsed and adopted 70:20:10 model, is the best way to embed learning? This refers to a model that recommends 70% of learning be on the job, 20% through peer based supports and 10% in a formal setting.
What the evidence suggests
Returning to the example of the challenges faced by the manager of hospital training unit or an executive implementing a leadership program at a central agency, consider the following.
Multiple research reports show that providing lifting technique training is not effective in minimising the risk of injury from manual tasks.
In relation to the global leadership development industry (estimated to be worth $50 billion), research published last month indicates only 10% of 500 CEOs polled globally believed their leadership-development initiatives had a clear business impact.
A survey by the Australian Human Resources Institute (AHRI) on L&D revealed that around 33% of respondents said they didn’t know or they didn’t measure the ROI on L&D spend in their organisation. Around 17% said it was measured through participant self-assessment (note this study finding that participant evaluations have no validity as a general indicator of learning outcomes). Adding up the numbers gets us to half of all respondents either not measuring, not knowing if they measure, or ineffectively measuring, the ROI on their L&D spend.
A white paper last year published by AIM (now AIML) on the 70:20:10 model in the Public Sector, written by Dr Johnson and Prof Blackman of UNSW’s Public Service Research Group, suggested that although the model is sound, middle-managers across the public sector indicated it had not yet brought the strong, positive results they expected.
Measuring Success
There is a standard model used for evaluating the effectiveness of training. Called the Kirkpatrick Model, it considers the value of any mode of training across four levels. Level 1 evaluates how participants respond to the training. Level 2 measures if they actually learned the material. Level 3 considers if they are using what they learned, and Level 4 evaluates if the training positively impacted upon the organisation. More recently a 5th level has been added to compare the monetary value of the business outcomes with the costs of the training. The 17% referenced in the AHRI survey were using Level 1 evaluation. Often derived from participant evaluation forms, sometimes referred to as ‘happy sheets’ by trainers, they are highly ineffective as a measure of ROI as they may simply indicate how entertaining a presenter was — or the quality of the single origin coffee and bagel varieties on offer. It’s difficult to measure ROI on Levels 3, 4 and 5 (which is probably why it isn’t done more often). Thankfully in the realm of public sector leadership training, there is a specific multi-year study being undertaken at the moment on the effectiveness of the Public Sector Management Program by Dr Vicki Browning from QUT. Such research can only help answer the question of what is wasted and what works.
As to why the 70:20:10 might not be yielding more positive results, that’s a topic for next week!