Show Me !!!- The Art of Dealing with “Data Denial”

During the course of managing projects, consultants are often faced with the age old problem of “data denial”. You know the feeling- when you have to look in the mirror and face the fact that its your data, your conclusions, and your recommendations that could affect the lives of many. While some would say it’s a feeling of power or influence, most would argue it’s nothing more than a big headache and major source of stress.

That notwithstanding, let’s look at the source of “data denial”. Data denial stems from those people within organizations that have a lot to lose from accepting the conclusions of your “data story”. Think about it like this- if you knew the conclusion of a book before you started reading, and didn’t like it, would you even begin reading? Such is the case with data. A conclusion that is disliked will breed data denial every time, regardless of how good or bad the data story hangs together. Its a fact of life , that there will always be a large percentage of your clients who will not like what the data has to say. Accept that. Their reaction is something that you have no control over.

What you can do, however, is make damn sure the data, and the data story hang together. If it does, the naysayers will soon meet their ultimate destiny all by themselves. The will assert your conclusions are wrong based on a single piece of data, but soon find out that it’s not one piece of data, but rather a a body of evidence pointing in one direction, that’s driving your conclusion- a direction that likely runs counter to them. And then, with far less fanfare than they arrived with- poof- they’re gone!

So how do you create that defendable “data story”. First, make sure that you’ve scrubbed the information before building it into your conclusions. That should be obvious, but remember, we’re not talking only about spreadsheet errors and omissions here. We’re also talking about the reliability and validity of the data. How was it captured? Was it captured the same way from each respondent? Are the data capture systems reliable? Are clear standards in place to ensure an apple is always an apple?

Secondly, make sure your conclusions aren’t based on a single dimension of performance. For example, a conclusion about high cost will almost always be met with the “oh but we ‘re high cost because we are the high quality producer!”. Maybe true, maybe not. Your conclusion will be a lot more defendable if the service quality dimension of performance is built into the equation, rather than being absent or tangential to the argument. A data story like “your cost is x..and your service level is y. And while it appears you pay for having that high service level, companies a and b generate the same service level at 70% of your cost”. That’ll take quite a bit of wind out of your adversary’s sails, and with any luck, get his energy refocused on problem solving rather than data denial.

Finally, avoid being absolute with your data components. Nothing is perfect. No room for black or white answers. There are times where statistical accuracy and hairline confidence intervals are important (like sending the space shuttle into orbit!), but most of the time, directional accuracy is more than enough. Spend your time finding 10 metrics that point directionally to your conclusion, rather than finding one lone measure that is squeaky clean statistically. I’m not diminishing the importance of statistical accuracy, but what I am saying is that there is a time and place for it. Very often, you can prove your conclusion much faster and feel very confident in your recommendation without the comfort of statistical precision. I’ll take directional accuracy over “analysis paralysis” every time. Oh yeah, and did I mention that for every statistically precise datapoint, there is a statistically perfect rebuttal. Here, the old adage, “you can make statistics say anything” rings oh so true. Pick your battles wisely, and spend your time on obtaining a larger volume of directionally accurate supporting metrics rather than shooting for data perfection.

There are many more ways to “tighten up” your conclusions and avoid falling victim to the data naysayers. The above are just some of the more important ones- the ones that can’t be ignored.

Remember though that “data denial” comes with the territory if you’re in the business of performance management. It can’t be avoided. And even if it could be, most of us would find that to be a very boring place to work. Instead, embrace the challenge knowing that you are armed and well prepared for whatever they throw at you. If you’ve done your best at this, trust the right answer will emerge based on the data you’ve prepared. The naysayers will usually take care of themselves.