Making the numbers dance

Written by Leda Glyptis
11th November 2021

“Nice bar chart, boss,” said fresh-faced me, many moons ago… “What does it represent?”

Numbers are always true. But they don’t always mean what you say they do.

“How we fare against our biggest global competitors,” said the boss.

“We have to make some important decisions about the next couple of quarters around where we will invest budget, effort and mind-share, and such decisions should be informed. Facts and figures, young padawan. Facts and figures.”

Duh, I thought.

Of course we need to make informed decisions to manage risk.

And I know what the bar charts are trying to show. I can read. It says ‘competitor positioning’ at the top of the page and the graph is helpfully colour-coded in ours and our competitors’ brand colours.

What I don’t know is what went into the formula that spat out the bar chart. What do we measure our performance in units of? I don’t see dollar signs, which was my first and only guess, and I want to learn.

I was still fairly new to the industry.

Coming from academia, I didn’t find the need to lead with facts and numbers daunting. That, I knew how to do. That, I was used to.

But there was always something elusive in how the numbers were deployed. What I saw when I looked at them and what I seemed to be expected to see didn’t match up and that is not a good way to feel at work.

Anyway, back to my story.

I was young and it was a long time ago, when junior employees in banks didn’t tell their bosses “Duh”.

So instead, I said, “No, yeah, I got that, just wondering what metrics we are using: percentage of deals won? Deals we know we lost to the competition? Net share of wallet? Quarterly results?”

My boss wasn’t going to be tricked.

“What do you mean?” he said. Not unkindly but also not making me feel like the brightest spark in the room.

I squirmed, but it was too late to back down.

Nervous throat clearing.

Cheeks flushing.

“What do the bar charts represent?” I insisted. “What’s the unit of measurement by which we are proving that we are ahead of the curve by what looks like half an inch. Half an inch of what?”

“Gut feel,” he said.

And with a triumphant look around the room, he clicked onto the next slide. Entirely unchallenged by everyone in the room. His boss. The sales team. Even finance.

Without giving the game away, the position he was advocating for won the day. What with the bar charts and all.

It was also, as we eventually found out, entirely wrong. Fancy that.

I don’t know who needs to hear this, but gut feel is not a measurable thing. It is not a fact. And it does not become fact because you expressed it in a graph. Doing so says more about you and the wilful delusion of the organisations we often find ourselves in, frankly, than anything else. And that’s the point: making up a bar chart doesn’t make your argument true. But if everyone acts like it does, the emperor has, for all intents and purposes, new clothes.

And that is often the story of the data we use to inform decisions inside organisations because we want the comfort of having done our homework even when that comfort may be false.

Because all decisions entail risk, not all things are knowable and often looking hard at irrelevant data passes for analysis.

Although completely fictitious visual representations of data do happen, they are, admittedly, rare. What is common is actual representation of data that is true but doesn’t prove the thing that it is being used to prove, leaving us in a place that isn’t exactly fiction but isn’t exactly a firm footing either because the hypothesis (the thing you wish was true) is not proven by the data provided (the thing that is true). So although you have facts, your position remains fiction. Or at least wishful thinking.

And this matters.

Not just because it helped explain the disconnect I was experiencing during my first few years in the industry. But because, ultimately, we need to make decisions.

Often. Regularly. And occasionally with uncertainty.

So of course we want as much data as possible to inform said decision.

But what we often do, intentionally or unintentionally, is try to bend the data we have into a narrative of inevitability for the answer and certainty we crave. And fill in the gaps with smoke.

Because most of the time the question we answer is not ‘What do I need to know to make this decision?’ but rather ‘What do I need to show to convince my superiors as to the quality of my work or the validity of my view?’ Although the former could lead to the latter, the journey doesn’t work the other way round and the two are most definitely not interchangeable.

Because you can always make the numbers dance.

And that’s where the problems begin.

And it matters.

Because dressing up preference, instinct or risk acceptance in the language of objective fact-finding isn’t the same as furnishing facts. And trying to find the answer is not the same as trying to elicit a preferred outcome. And claiming objectivity doesn’t level the two.

Let me give a different example to illustrate the same point.

Have you seen value pool analyses used to persuade banks to go down path A vs path B?

They look swish, don’t they?

13 pages of Excel tables with a covering memo that fundamentally says ‘people are paying money to do stuff, we have no way of knowing whether they will pay you to do this stuff or if you actually understand what the stuff is for, but look… numbers… this is all true somewhere for someone so it must be true for you’, somehow leaping from single proof point to universal truth in one flick of my magic wand.

And don’t get me wrong.

That flick of the wand is the leap of faith, the choice, the decision point.

That comes with no guarantees. That’s where risk and leadership come in.

But the more we make the numbers dance in our ‘thinking time’, the more we project preference into the facts, the safer the decision may feel for decision makers. And that is the game being played, ultimately. And it is a dangerous one.

Now. Some players know this for what it is.

This is very similar to the 10-year projection figures pre-revenue start-ups put in their investor pitches.

Here’s how much money people made before me.

Here’s how much money people pay my competition.

And here’s how much of it I expect will flow into my pockets.

Q: What about this is not true?

A: The bit that matters. The bit that says ‘ergo, this will apply to me’.

Numbers are always true.

But they don’t always mean what you say they do.

Giving your gut feel a seven doesn’t make your guesswork any more scientific.

And the fact that other people made money doing a thing, doesn’t mean that you will.

VCs know that, by the way.

Or at least the good ones do.

They know which bits of your story are fact and which are the narrative you overlay, your hopes, dreams and business strategy.

VCs are checking your homework, not swallowing it whole. Also, they don’t hide their gut feelings behind nonsense bar charts. They look at the science but often go for religion: most VCs and investors will tell you they will go through the numbers and do DD and kick the tyres and test the market and check the tech, but they will also tell you that ‘there’s something about the team’ they chose to back, and there you have it. Faith and hope and gut in their appropriate place. Not masquerading as facts.

And this is the lesson, boys and girls.

Because in life and business a lot of the time there is no right answer. At least not one that is clear from the outset. A lot of decisions require a leap of faith.

You need the facts, or at least as many facts as you can gather. You need to know the facts you wish you had but are missing. And you need to know the difference.

Inside big organisations, the difference is often intentionally blurred. Because the juniors want the pass mark, and the seniors don’t want the pressure of taking a risk on behalf of the org, so you end up with a whole lot of bad science.

A few years after the bar chart incident, I said to a team member that I didn’t really understand what drove some decisions on the ground in that team. How do you choose A over B?

“Ah,” she said with pure joy (my obsession with seeing people’s homework having become quite notorious by then). “I have a model.”

“A’right. Show me.”

Out comes this highly convoluted and admittedly very pretty Excel spreadsheet that modelled a whole host of scenarios, if this then that. Great. I like this. So if something changes we don’t go the way we’ve always gone like lemmings. I like it a lot.

“What are the original inputs?” I said. “Where did you get the sizing parameters to determine tipping points?”

“Oh, the numbers are theoretical,” beamed the team member.

Yeah.

No.

Numbers, man.

The one thing I had counted on in life to never be theoretical. And don’t you talk to me about dummy data. I know all about it. But the point of dummy data is that it proves the model and then goes back to where it came from. And real data comes in, to generate proof points, to feed into facts that can be challenged and evolved.

It’s science, innit?

Truth needs to withstand scrutiny. And facts don’t stretch without breaking. And yours just broke.

So what is my point?

When the question in business is ‘What should I do next?’, no amount of science will take the fear and uncertainty out of the need to lead and make decisions. Facts, figures and proof points are there to inform your thinking, not replace your agency. The temptation to try and not make it so is a cottage industry inside banking. It is understandable because people want to look after their career. But it has veered us way, way off course.

And that should be way scarier than making decisions on incomplete information, which is what life is all about anyway.

We all wish there was a right and wrong answer and that numbers held it.

And for the right question, they do.

So ask them the right questions.

And then ask the rest of yourself.

#LedaWrites

Leda Glpytis