Lies, damned lies and content marketing

There are three types of lies: lies, damned lies and statistics. 

Statistics and research are rarely as objective or factual as you think. There’s perhaps a certain irony in the fact that the above quote is generally attributed to Mark Twain, whereas it in fact originates from 19th century British Prime Minister Benjamin Disraeli.  

As we all know, the internet has never been renowned for its strict love of the truth. And this extends to the world of content marketing. Many of the facts you read are far less objective than they may seem. Whether that’s because they’ve been poorly researched, taken out of context, or outright fabricated – you’ll often find a more complicated situation lying beneath the black and white statistic.  

As content marketers, thoroughly fact-checking statistics can be difficult and, at times, time consuming. But that doesn’t mean it’s not important. To quickly identify poor research, we need to understand where it comes from, how it develops, and ultimately what it looks like.  

This blog explains how to banish poor research from your content marketing strategy for good.  

Where does poor research come from? 

In recent years, the term ‘zombie statistic’ has arisen to describe a particular type of poorly researched statistic that gets so widely circulated around the digital sphere it begins to be viewed as fact. The Independent recently dug into some common examples of these, including some that you’ll almost certainly have encountered: 

• There are more people alive today than have ever lived  
• People only use 10% if their brains 
• You’re never more than six feet from a rat in a city 
• You should drink eight glasses of water a day

At one point or another, each one of these common statistics has been thoroughly debunked. And if you try to chase back the source of any of them, you’ll likely find yourself falling down a rabbit hole of missing citations, defunct links and hearsay.  

What do all these statistics have in common? They all tend to depict complicated and nuanced situations in objective terms, often using quite eye-catching language. In reality, if you dig into any statistic or fact, you’ll inevitably find the truth to be more complicated than the black and white percentage sign makes it seem. But when internet users are looking for easy facts and digestible content, the authority of a statistic can often go wrongly unquestioned.   

How does poor research spread online? 

Many zombie statistics actually predate the internet, but there’s no doubting that the internet has breathed new life into the phenomenon. It’s ushered in an unprecedented democratization of the written word, meaning everyone with an internet connection can post content online. The benefits of this are obvious. But naturally, it also lends itself to unsubstantiated facts and statistics.   

As well as this, the fast–paced nature of the digital sphere lends itself to brief, fast content; quickly written and quickly consumed. Digital content writers are under pressure to produce content quickly that’s fresh and relevant. This means facts and research, quickly researched, often go unchecked. When neither writer not reader has the time to properly check the validity of statistics, poor research can travel through many stages of the citation chain without being debunked.  

Why content marketers must remain vigilant 

As time-pressed content marketers, falling into the trap of relying on shaky statistics to back up your claims can be all too easy. In the short term, the chances of it coming back to haunt you are slim. But, in reality, it’s a bad precedent to set for a content marketing strategy, whether anybody finds out or not.  

Inbound marketing is based around the principle of being honest with your customer. The logic follows that if your product or service has tangible value, all you need to do is identify and educate your customer for that product to essentially sell itself. If the marketing messages you use to prove that value require a certain ‘creative application’ of the truth, then that contract is fundamentally broken – and you customers will find out about it sooner or later.  

It’s also important to remember that marketers don’t just rely on research to prove why their products are valuable. We rely on research to inform the decisions we make, the products we sell and the messages we use to sell them. It can be easy for us to assume truth in research that confirms the way we think of the world – however short-sighted, one-dimensional, or lacking in nuance that conclusion might be. In contrast, research that forces us to challenge our assumptions can often pass by unnoticed.  

If people assume certain facts and beliefs to be true, they’ll likely believe research that backs it up. Researchers who set out to prove their existing assumptions will inadvertently use methodologies that skew results in one direction or another, perhaps by using leading questions or employing a small, unrepresentative sample size. Over time, the research informs the bias, and the bias creates the research, resulting in something of a self-confirming feedback loop. It becomes easy for an echo chamber of ideas to form. Often, the assumption isn’t even necessarily wrong, it just becomes an oversimplification of a more nuanced situation.  

How to avoid bad research 

Time pressed digital marketers will be happy to discover that identifying poor research doesn’t always have to be as long-winded as they think. In fact, following just a few simple tips can help you easily root out a lot of bad statistics. Here’s a look through some simple steps you can take.  

• Avoid speculative research

How many times have you read that “recognisable person or institution has ‘predicted’ that something will happen, or something will increase by X% in 2020” Even worse is when a report that ‘predicts’ something will happen in the future gets quoted as saying it will happen in the future.  

It’s too easy for readers to skim over such speculative statistics and assume that “there will be 6 billion internet users by 2024!” is an objective fact. But take a moment to think through the implications, and you’ll quickly realise how wildly impossible it is to account for all the factors that could possibly influence such a statistic.  

Such research can’t be completely discarded, and there are instances where an educated guess about the future is better than nothing. If you must include speculative research in your work, make it clear that it is a prediction that is being made, and who is doing the predicting.  

• Avoid correlational data

This refers to when simultaneous, self-contained trends are assumed to influence each other, when no evidence for this link exists. What would you think, for example, if you read that a group of researchers had gone into a primary school to discover if tall children were cleverer, before discovering overwhelming evidence to suggest they were? You’d probably be quite shocked, before eventually realising that in a primary school, taller children also tend to be older. Nobody said the test subjects were all from the same age group. In this case, a third and entirely unrelated factor determined the results. 

Statistics that are based on correlational data pervade the digital sphere. With the right sample size, leading questions, and tactical omission of important information, it’s possible to make it seem like almost anything causes something else.  

If you head over to spurious correlations, you can browse through a whole range of ridiculous, eye-catching, but ultimately coincidental correlations, inspired by the same methodologies behind otherwise well–respected research. If you were to believe every bizarre correlation you see, you’d end up thinking that, among other things, eating margarine causes divorce, eating cheese causes people to suffocate in their own bedsheets, and eating mozzarella gives you a doctorate.   

These things are, obviously, not true. Neither is much of the correlation–based research or facts that get circulated around the internet. If in doubt, check the methodology of a research report, and see if you can work out how they came upon their statistics. If it’s correlational, take it with a hefty pinch of salt.  

• Consider the sample

Unrepresentative samples are among the most common features of bad research. The amount and type of people a survey asks will greatly affect the quality of results it finds. Consider for example if two researchers wanted to run simultaneous surveys to discover the nation’s favourite supermarket. One of them decides to stand outside Tesco and one outside Waitrose, asking people what their supermarket preferences are. Both, unsurprisingly, publish wildly contradictory statistics that are presented as objective fact. 

In B2B research, companies are guilty of such tactics all the time. Consider for example, statistics like “90% of our clients are satisfied with our services”. It seems fairly objective on a first read, until you stop to consider that, perhaps, if someone wasn’t satisfied, they wouldn’t be a client anymore – and therefore wouldn’t have been asked.  

Size also affects how effective a sample group is. If two people ask the same question of different sample sizes – the first of 20 and the second of 100 – you’d logically expect the second to be more representative. If neither publish the sample size or type in their methodologies, you’d do well to ignore anything either of them say. 

Seek disconfirming evidence 

If there’s one takeaway you should remember from this blog it’s this: seek disconfirming evidence. That means never settle for a piece of research being correct just because it seems right. Search for facts and statistics that challenge, expand or diversify your worldview. In many cases, simply looking for a wider range of research is enough to negate the worst effects of zombie statistics and unsubstantiated ‘facts’.

There’s a whole range of reasons why reality isn’t quite as objective as much of the world’s research makes it seem. In reality, it’s extremely difficult to be entirely vigilant and analyse every piece of unsubstantiated research in the world. But if you follow the simple guidance in this blog, you’ll be well on the way to identifying some of the most common types of false statistical claims.

Alternatively, get in touch with the experts at Fifty Five and Five, where we specialise in creating effective, thought-provoking and well-researched content for our B2B technology clients.