One of my all-time favorite quotes from Andrew Lang (1844-1912), a poet, novelist, literary critic and anthropologist:
“He uses statistics as a drunken man uses lamp-posts…for support rather than illumination.”
I think the reason I like this quote so much (besides the wonderful visual analogy this provides) is that it reinforces what I have always believed – that any metric or measure you can come up with is simply a tool for exploration and discussion.
Back in my early days in compensation, I remember spending a lot of time on a market analysis for a senior leadership position. I did the analysis the way I’d been taught, and was rather proud of the detailed statistics I had collected and presented.
The only problem was that the data was significantly lower than the senior leader was making at the time, and they were looking for a reason to increase his compensation. Oops. This was my first lesson in “creative statistics”….I learned how to “massage” the numbers to arrive at the place they wanted.
Ever since, I have been skeptical about presenting metrics and measures as gospel.
As I migrated from compensation to Organization Development, the light bulb turned on for me….metrics, statistics, data….its all about people, not numbers. Don’t get me wrong, I am a data geek and hugely supportive of data driven decisions.
But using statistics without questioning them, I believe, is foolhardy. If the statistics are bad, I have watched the smartest people rationalize why they really aren’t correct. When they statistics are good, I have watched those same very smart people pat themselves on the back and say “good job!”
For me, the necessary question about statistics is “why?” Why do the numbers say what they say? If our numbers show that we were successful, do we know why and could we repeat the process?
If our numbers were not good, is the data accurate? If the data is accurate, what are the top 3 things we will do to change the numbers? How will we motivate our employees and leaders to achieve those goals? If the data is not accurate, what are we doing to fix it, and when will it be accurate?
Bottom line, I think metrics, measures and statistics are a catalyst for communication, dialogue and understanding. They are not an end in themselves.