How figures can lie and how to spot the lies.

Data is constantly used to support and validate claims. This is ideally what readers want. You want to know that what you’re reading is based on unbiased, truthful evidence. As you’ve probably heard, four out of five dentists recommend Colgate . Four out of five dentists also recommend sugarless Trident gum . I wonder if they’re the same dentists? Although, this might seem humorous, ask yourself, is the data being manipulated to sway your opinion? Is it the truth, or is it just truthy?

What is truthiness? When Stephen Colbert coined the term in the Colbert Report, he defined it as a “quality characterizing a ‘truth’ that a person making an argument or assertion claims to know intuitively ‘from the gut’ or because it ‘feels right’ without regard to evidence or facts”. An individual who is being truthy might make a few little tweaks to their final figures to convey their message. Unfortunately, when truthiness seeps into data analysis and particularly figures, the results can range from persuasive to flat out manipulative.  

In this post I’ll present three figures, each one analyzing visits to the emergency department at Kelowna General Hospital (KGH), located in British Columbia (BC). Our goal is to show the true trend in emergency department visits from 2015 to 2017.  Only one of these figures will be truthful. The remaining two will be truthy. In fact, the figures will showcase the good, the bad, and the ugly of data manipulation.  After seeing these examples, you will be able to examine a figure for yourself and decide if it is truthful.

The Truth

To begin, let’s look at our raw data. The table below shows the numbers that will be used to create the figures. The annual percent increase is calculated as the difference in visits between the two years divided by the earlier year’s total visits. These numbers were collected and reported by the Canadian Institute for Health Information (CIHI). As you read through this post, come back to this table and remind yourself of the truth. Remember we are interested in the trend in visits to the KGH emergency department.

Table 1: Emergency Department Visits at Kelowna General Hospital (KGH) compared to the province of British Columbia.

The Good

The first figure below shows a bar graph of raw emergency department visits to Kelowna General Hospital (KGH) over the last three years.  It is arguably the most truthful. The baseline is included on the y-axis and you can clearly see the increase in visits between 2015 and 2016. The visits appear to plateau between 2016 and 2017. When we examine Table 1, we can see that this is the truth. There is only a small increase of 7.3% in visits between 2015 and 2016 and an even smaller increase of 1.7% between 2016 and 2017.

The Bad

Let’s look at our second visualization. Do you see what happened? This graph is almost identical to the one above, but with one key difference. Look closely at the y-axis. You’ll notice that it doesn’t start at zero. Now our baseline is 70 000 visits to the emergency department. That increase in visits between 2015 and 2016 looks deceptively larger than 7.3% on this scale.

The Ugly

Finally, we look at the ugly. What story is the figure below telling? Looking closely, we really can’t see a difference in the Kelowna General Hospital (KGH) or the British Columbia (BC) trends across the three years. Both the visits to KGH and the overall visits in BC don’t seem to be changing. The bold black colouring of the BC visits doesn’t help either. The black is overpowering compared to the grey. You might not see the KGH data if you were only skimming this post.    

Looking back at Table 1, we know that the number of visits to emergency rooms in BC is on average 20 times larger than the number of visits to KGH. These two trends belong on two different scales, or better yet, two different graphs.

Judging for Yourself

We’ve looked at three figures in this post, each with their own story. But as you know, only one is true. There are many ways a figure can be manipulated to change its narrative and influence perception. Scales can be modified to hide trends in the figure. Scales can also be used create and emphasize false trends. Colour will draw your attention. But be careful, it might be drawing your attention away from something important. So, the next time you’re reading a post or watching the news, and the shiny graphic appears on the screen, ask yourself, is it truthful, or simply truthy? 

Additional Truthful Resources

• Truthiness (Ben Zimmer, The New York Times Magazine)
• Truthiness (en.wikipedia.org)
• The Canadian Institute for Health Information (www.cihi.ca)