Four tips to avoid being bamboozled by political statistics and data

With a plethora of elections around the world^[1] in 2024, voters considering their options can expect to be presented with all kinds of numbers and statistics aimed at giving credibility to various claims from various parties. But data, graphs, percentages and surveys^[2] can be misleading or manipulated^[3].

So here are four things to remember if you’re trying to navigate what’s behind the numerical spin – or what Mark Twain (and others) referred to^[4] as “lies, damned lies and statistics”.

1. Get things in proportion

Political debate often comes down to various sides throwing big numbers^[5] around in support of an opinion – whether it’s to do with immigration, the NHS, the economy, or any other issue people care about.

For example, in arguments about public debt^[6], there are always ways to make the other side look bad.

So if I wanted to convince you that the UK’s debt is out of control, I might explain that it is officially expected^[7] to increase by £400 billion in the next five years. That works out at almost £10,000 per adult – a huge number.

But it is also forecast to decrease by around 5%^[8] as a share of GDP, which is a much more positive situation.

To make sense of big numbers, we need to compare them to the right thing. Most economists would argue that what truly matters in this case is whether the debt is sustainable^[9] – an imperfect assessment which uses a combination of its size relative to GDP, interest rate and economic growth.

The same principle applies to other policies. It’s true for instance that, between 2018 and 2022, 2,496 pedestrians were killed^[10] in the UK. But it’s also true that in 2022, pedestrians killed by cars accounted for 0.06% of the country’s deaths.

A more meaningful approach might be to compare practical steps to protect pedestrians. A recent study^[11] for example, found that reducing speed limits to 20mph reduces casualties by 10%. We could then use that number to estimate the number of years of life saved^[12], and compare it to other choices such as banning smoking, taxing alcohol or investing in new NHS treatments.

2. Check the context

In 2023 a UK survey^[13] tested public understanding of a headline which said the inflation rate had fallen from 10.1% to 6.1%. Half of the respondents correctly understood that this meant prices were still increasing, but not as quickly as before. But more than one-third thought a drop in inflation meant lower prices.

A key pledge of the British prime minister, Rishi Sunak, was that the inflation rate would halve^[14]. This means he promised prices would continue to increase, but half as fast as before. Yet how many people will have thought that he promised a decrease in prices?

Many of the big economic numbers used in political debates are just like inflation in that they measure a gradient, and gradients need context. If economic growth is 5% in China^[15] versus 0.4% in the UK^[16], it does not mean that the Chinese are wealthier than the British, as the UK’s GDP per person is still more than four times^[17] that of China’s.

3. Change is relative

Gradients can also be difficult to interpret accurately because they change all the time. It was correct, for instance, to claim in May 2024^[18] that the UK economy was growing faster than any other wealthy country. But only if you look at the previous three months.

This is partly because the UK’s growth rate was negative in the nine months before^[19], so any improvement from such a low point can be seen as relatively quick. And if you take the graph back to 2022^[20], the UK has performed worse than most major economies.

Meanwhile 2021 saw the UK’s highest growth in recent history^[21] (at 8.7%) – because it followed the severe economic damage^[22] done by COVID in 2020.

Another example is Libya, which has statistics showing huge growth rates^[24] of 86.8% in 2012, 32.5% in 2017 and 31.4% in 2021. But every one of those impressive-looking percentages came after periods of devastating war and destruction.

You can almost always find a graph that fits a certain time line for the story you want to tell.

4. Graphs don’t show conclusions

Correlation is not causation. That is, just because two things happen at the same time, doesn’t mean they are related. But when a big change in a particular trend occurs, it is always tempting to look at what else was happening.

Take the issue of healthcare^[25]. Some may note that since the Conservative party took power in 2010 the NHS has been deteriorating^[26] with regard to waiting lists and stagnating life expectancy.

But the change of government is not the only thing to have happened around 2010. The global financial crisis of 2008 had a particularly large effect on the UK and its huge financial sector.

The immediate effect of the crisis was to reduce the size of the economy by 10%^[27], forcing all countries to choose between much higher debt and cutting spending. To put blame on the party who took power at that time, one needs to dig into the actual investment choices^[28], and possible alternatives^[29].

Or consider concerns about the impact of social media^[30] on the mental health of teenagers. In the UK and the US^[31], some commentators have attributed a rise in teenage suicide since 2010 to the availability of smartphones.

But in other countries^[32] the trend is different. In France and Canada teenage suicides have actually decreased since 2010. Yet they have phones too.

So while it can be tempting to link two coinciding events – either to blame or give credit – that does not mean they are necessarily related. Coincidences happen and are often so hard to believe^[33] that they have led to wrongful murder convictions. Some people have won the lottery twice^[34].

And some people this year will win power, helped in their campaigns by the use of numbers and statistics. But only if voters properly understand how they are used can that data really improve the public debate.

References

1. ^{^} around the world (theconversation.com)
2. ^{^} surveys (academic.oup.com)
3. ^{^} manipulated (www.amstat.org)
4. ^{^} referred to (www.york.ac.uk)
5. ^{^} big numbers (www.edmundconway.com)
6. ^{^} arguments about public debt (fullfact.org)
7. ^{^} it is officially expected (www.ons.gov.uk)
8. ^{^} decrease by around 5% (obr.uk)
9. ^{^} is sustainable (academic.oup.com)
10. ^{^} 2,496 pedestrians were killed (www.gov.uk)
11. ^{^} recent study (jech.bmj.com)
12. ^{^} years of life saved (www.gov.uk)
13. ^{^} UK survey (www.escoe.ac.uk)
14. ^{^} inflation rate would halve (www.bbc.co.uk)
15. ^{^} 5% in China (www.reuters.com)
16. ^{^} 0.4% in the UK (commonslibrary.parliament.uk)
17. ^{^} more than four times (data.worldbank.org)
18. ^{^} claim in May 2024 (fullfact.org)
19. ^{^} nine months before (www.theguardian.com)
20. ^{^} back to 2022 (fullfact.org)
21. ^{^} recent history (data.worldbank.org)
22. ^{^} severe economic damage (www.ons.gov.uk)
23. ^{^} Ink Drop/Shutterstock (www.shutterstock.com)
24. ^{^} huge growth rates (data.worldbank.org)
25. ^{^} issue of healthcare (yougov.co.uk)
26. ^{^} the NHS has been deteriorating (www.ft.com)
27. ^{^} by 10% (ifs.org.uk)
28. ^{^} the actual investment choices (lordslibrary.parliament.uk)
29. ^{^} possible alternatives (www.instituteforgovernment.org.uk)
30. ^{^} the impact of social media (www.bbc.co.uk)
31. ^{^} UK and the US (www.ft.com)
32. ^{^} other countries (ourworldindata.org)
33. ^{^} often so hard to believe (www.science.org)
34. ^{^} won the lottery twice (metro.co.uk)