01926 633333February 12th, 2018
With the performance of the NHS one of the most contentious subjects in the news recently, it’s no surprise there is a lot of attention paid to its key performance indicators (KPIs). There must be a huge amount of pressure on NHS Trusts to report positive results, and it is in that climate that the UK Statistics Authority has told NHS England that it must explain changes it has made to its Accident & Emergency (A&E) reporting.
Their concern is that the target to have 95% of A&E patients seen within four hours has been achieved partly through adding in data from minor injuries clinics and care centres, sites which were not previously included, but which have a much better performance than traditional A&E departments.
It’s far from the first-time public bodies have come under criticism for manipulation of KPIs – a previous government was memorably accused of reacting to worsening child poverty figures by changing the definition of poverty. It’s not only the public sector where the pressure to produce good numbers can be so great it leads to creative data collection though. One of the most common high-level business KPIs in industry is On Time In Full delivery (OTIF), but it is also one of the most flexible depending upon the story you want to tell.
On Time In Full delivery
OTIF has become a critical measure of performance in many sectors, whether delivering the end product direct to consumers or business to business. The theory is that it gets as close as possible to measuring customer experience without actually asking them what they think, because surveys can be costly and complicated with results which are difficult to interpret. The truth is though that while OTIF sounds simple – how much was delivered on time and in full divided by how much was ordered – it can hide more than it reveals, and it can also be measured in a way that suits the results the business wants to project.
On Time
The first thing to consider is how you define what “on time” means. Are we measuring against when the customer wanted the product, when
the business has a contractual obligation to supply it by, or when the customer was told during their initial contact they could have it by? The three things could be entirely different to each other, and OTIF being a pass or fail measure, the decision of where you set the boundaries for your measurement could completely alter your results.
I was once on a 50 minute flight, the touch down of which was greeted by a fanfare and an announcement that this was yet another on time arrival for Europe’s most punctual airline. High fives all round at the airline’s head office, meanwhile we then taxied to the terminal and waited 20 minutes for someone to bring a set of steps so we could disembark.
In Full
Then there’s the question of “in full”, are we talking about order lines or whole orders? The case against whole orders is that you’re not properly measuring business performance if you consider delivering a large volume delivery to a big customer to be the same as an individual product being delivered to a small one. The argument against order line level OTIF is the questions of whether a business should really be patting itself on the back for delivering part of an order on time and in full. There is a running joke in the packaged gas industry that suppliers give themselves top marks for delivering acetylene to welding companies on time. Their oxygen, without which the acetylene is useless, will be with them in a week!
Conclusion
Of course, the solution for any organisation trying to measure itself is to answer all of these questions from the customer’s point of view. If the point of OTIF, or for that matter A&E waiting time measures, is to truly represent the customer’s experience, then questions about measurement boundaries should answer themselves. Even so, there will always be a temptation to think first of what we want the results to look like, instead of how we can most honestly represent the customer’s experience.
In the public sphere we have the UK Statistics Authority looking out for us, in business a degree of scepticism and an understanding of the importance of not always taking results at face value can go a long way.
Like what you’re reading? Sign up here to be part of our Lean and Leaders community or click here for more articles!
Becoming part of our community will enable you to keep up to date with best practice about Lean and Leadership.
Mike joined Bourton Group following on from a successful career in business performance improvement.
He is a certified Lean Six Sigma Black Belt with nine years’ experience of leading hard benefit generating projects, across multiple functions of a large organisation. He has delivered numerous Continuous Improvement training courses, and coached colleagues at all stages of the Lean Six Sigma training process.