Mark Twain, the fiction writer, first popularised the phrase “there are three kinds of lies: lies, damned lies and statistics”. Twain erroneously attributed the line to a British Prime Minister, who in fact, was never recorded saying any such thing. It’s fitting, given what I’m about to describe, that one of the best-known claims about statistics is grounded in the absence of evidence.
Statistics – the interpretation of data according to scientific method – is a robust discipline of its own, but more importantly, contributes heavily to almost all quantitative enquiry in any field. In theory, this includes consulting. However, this post argues that statistics are less important in consulting than an outsider may realise.
The purpose of statistics
The most simple type of statistics are descriptors about a set of data: means, medians, ranges. Numbers that tell you about other numbers. Yet data sets are often only samples of a larger population. Many statistical tests seek to prove whether the sample data says anything of statistical significance about the underlying population. Patterns in the data, such as the differences between groups, may emerge because of a real difference or through pure chance (what statisticians call random variation). A simple example: if I want to compare the heights of children in grade four versus grade five, my inferences will be inaccurate if my samples only captured grade four girls and grade five boys, because sex differences between samples is more likely to drive height variation than grade.
Another way of saying this is that statistics offers objective methods to separate signal from noise – or at least to understand the relative balance of signal and noise within the data.
Statistics in consulting (or the lack thereof)
Data analysis is, of course, a crucial part of consulting. But this is not to be confused with statistical analysis, which to my surprise, was used less often than I’d expected in top-tier management consulting.
(There are, of course, heavily analytical cases which use advanced statistics, or proprietary out-of-the-box methodologies which use statistical analysis in the background, such as A/B testing or optimisation problems. Such cases usually employ specialist expert consultants, rather than generalists.)
But in ‘regular’ work, statistics are barely used. I never saw a p-value used in internal or client work; not once. Internal presentations often suggested small changes (such as quarterly averages for working hours) in firm performance of 1 to 2 hours per quarter were due to changes the firm had made, rather than random quarterly variation. Analyses for clients often had sensitivities attached, to give a sense of the limits of major assumptions, but never presented using statistical methods.
Below, I hypothesise several reasons for why statistics aren’t used more widely:
Poor client data: A classic problem for any consultant – messy, noisy data, not fit for basic analysis. The maxim of garbage in, garbage out dictates that robust statistics will not be worth the effort.
Forwards-looking, not backwards-looking: Consulting is often about predicting and modelling the future, not describing the past. However, statistical analysis is only useful on historical data. There are no statistical analyses to prove whether a ten-year cash flow projection will be correct.
The correlation / causation problem: In the absence of experimental or quasi-experimental trials, statistics seldom show causality. Yet consulting can be the art of suggesting causality in order to make recommendations.
Random variation doesn’t matter: If revenue has declined, a company has a problem. Suggesting it’s due to random variation doesn’t solve anything.
Keep it simple, stupid: Who really understands what a p-value means? Or what a chi-square test does? Even if consultants understand their methods, they may not see the value in having to connive clients to ‘trust the maths’. It’s better to find a method of communicating the answer that resonates.
Narrative is more important than numbers: Statistical metrics are not material to the decision – they are details that executives don’t care about. Consulting is about crafting narratives around problems.
Statistics are important when you are aiming for certainty or trying to quantify uncertainty. But certainty is hard to find in the real world; it also takes longer. Financial services aside, the operational measures of many companies don’t come with perfect, quantified accompanying data.
Consulting is often about getting a good idea of what the right answer is, as quickly as possible, and providing forward-looking recommendations based on a strong narrative. Statistics aren’t always useful to the core product.
Sam Smith worked in a top-tier management consulting firm for two years before taking time out for study. They write under a pseudonym to bring you honest reflections and insider information.
Image: Pexels
🔴 Interested in consulting?