I had a really disheartening encounter last week at the CIPD Festival of Work.

I was mooching about and chatting with people there about what the event was like (interesting), people they’d met (diverse), and the topics they’d seen which interested them (loads and little about AI). I happened on one vendor and was talking about the work we’re doing on evaluation and I asked ‘How do you know your training makes a difference?’.

The response was along the lines of ‘we ask the usual in the form of happy sheets, but nothing else’.

I was disheartened for a few reasons.

Firstly, this organisation are having to push – and push hard – to get new business. The energy they obviously have to throw into marketing their 150+courses could be so much better used in understanding the specific issues with their clients and supporting them differently.

There’s a whole question about effectiveness. As I say in the book which is out soon:

Qualitative data has been generally in the feared ‘Comments’ section where users are asked to describe the experience they have been through. In some cases, these are in the form of questions which are unerringly answered in generic and limited ways.
For example, look at the list below of typical questions and responses:
Q. What did you enjoy?
A. All of it.
Q. What would you change?
A. Nothing.
Q. What else do you want to say?
A. Thanks.
These forms do little to shift the perception of the L&D function as people pleasers. When our colleagues challenge these clichés, we smile, shrug, and suggest we can’t do anything else.

Proving Impact – Andrew Jacobs

The most disappointing point to the whole interaction was the comment that this vendor has no plans to do anything differently in terms of evaluation because ‘my customers don’t ask for it’. My biggest fear for this vendor is that when these customers do ask for it, they will be high and dry and wondering why the business has failed.

As I have said SO many times before, learning needs to move from being a shopkeeper to an engineer or we will disappear.

One of the typical questions above is what would YOU change? Let me know in the comments.