Proper statistics are powerful tools, but shitty statistics are magic spells: crude incantations of more belief than science desperately attempting to convince people of changes in the world. Like most evil spells, they're often seen in books with faces on them. And like the Necronomicon, Facebook reveals the darkness and evil within people you thought you knew. Statistics mislead people because the only time most of them care about standard deviations is reading about how Fifty Shades of Grey got BDSM wrong. You'd get more useful life advice from Mario Kart 8. Which is why we need to arm ourselves against these crappiest of illusions. Behold, eight math-free ways to work out which numbers to ignore.
#8. Who's Sharing It?
George Doyle/Stockbyte/Getty Images
The first way to check data is to check who's sharing it. This sounds unscientific, as we're meant to respect the data instead of being biased by who's presenting it, but it's already been biased by their presentation of it. Science only works when you use the whole thing. You have to consider all the results, not just the ones that support things you like. But pundits Google titles that match what they already believe, then wave quotes out of context to claim the power of SCIENCE. It's like ripping the shiniest part out of a Lamborghini engine and claiming you have the ability to sprint at 300 kilometers per hour.
"And according to this, when I take my top down I'm sexy as hell!"
Properly prepared scientific data is free of bias, but most of that preparation is boiling out those biases, intentional or otherwise. The whole point of numerical analysis is not screwing up the second we get emotional. Which is why you have to ignore people who are already frothing at the mouth when they start screaming about the statistics backing them up. Even if it sounds interesting, you should take the title of the paper and go read it at the source. If a study has truly proven that 100 percent of all pies are infected with diarrhetic toxodeathmosis, you're going to hear about that shit on the news, not just on Crazy CakeFan Karl's "Rather Die Than Pie" Cake Hour on 106.4 CAKE FM.
#7. Who's the Source?
There isn't a special "A Study" committee convening to make sure that only eminent scientists are allowed to release bar charts and percentages. "A study" just means "some people wrote some shit down." A child with a calculator can calculate up some numbers just as easily, and you might be 5,318,008 times as compelled to look at them when they're twisted around, but using numbers doesn't mean they're smart. It's your responsibility to check the source before getting upset about what they say.
The average Internet infographic is a full page of graphic design, one sentence of numbers, and a tiny bottom-right corner of source written in sky-blue letters on an aqua blue background. You don't see who's actually making the claims until the very end. Which is weird. In real life, we check who's speaking before listening to their entire spiel. We don't evaluate a half-hour presentation on the problems with current capitalism and how it's the fault of the foreign carp weevil infesting the ears of our politicians through imported furniture in government buildings, and only then check whether we're listening to the British Broadcasting Corporation on the radio or Big Bearded Carl after a bottle of homemade bleach-wine.
"The weevils hate this stuff; that's why their doctor-servants try to stop me!"
First, check if the figure provides a source. If it doesn't, it's garbage. Then, check the source. It might be the World Health Organization, or it might be the American Medical Liberty League for Living Whole. One of those is a specialized expert agency of the United Nations. The other is just a collection of words hoping you think they sound nice. But even the nicest name could be claimed by Bond villains. Talking about wholeness and liberty doesn't mean they're not fundamentalist sects dedicated to the incubation and release of biological plagues on the civilian population.
#6. What's the Sample?
The latest study of modern work environments revealed that 33 percent of staff displayed increased productivity when they drank Manhattans, and 100 percent improved productivity when they weren't wearing pants. I know it's the latest study, because I just did it. The sample was me and two cats.
The Cats Have Won
Like me, they produce "stuff for the Internet."
If the sample size isn't given in the same breathlessly excited paragraph as the results, those results are potentially garbage. If the sample size isn't given at all, those results are definitely garbage, rubbish intended to bury your brain under spoofed input and steer you toward the writer's opinions instead of informing you to make your own.
There have been some amazing cohort studies over decades of medical professionals, gigantic efforts to gather priceless medical and sociological data. There have also been wastes of potential toilet paper where bored students walked around campus one sunny morning and decided, "We won't just ask the people who can afford to come here, we'll ask the ones who don't even understand that this has to be paid for and are screwing around instead of being in class." Lo and behold, the resulting survey shows that "people" don't really need food stamps. When all it really reveals is that arseholes are involved in every step of that paper's publication. From the source that thought like that, to the writer who wrote about it, to the publication which accepted the paper for a submission fee instead of for peer review.
#5. What Are the Error Margins?
How large are the error bars on the figure? If they're small, the data could be useful; if they're large, the data is less accurate but still useful; and if they're zero, the data is garbage. Exact numbers sound more scientific, but at a sociological level they only reveal innumeracy. We can quantify the precise percentage points on particle physics experiments, but anyone claiming that anything on the human scale is exactly 37.8 percent is trying to garner your respect by reciting sacred digits. You couldn't measure exactly 37.8 percent of anything human even if you used only one of them, a good ruler, and an extremely sharp knife. The act of cutting would cause extra percentage points to squirt and slop all over the place.
"If you truly believe you only use 10 percent of your brain, this won't make any difference."
I used to compile data on the depth of holes blasted in material by lasers. A clear length under literally microscopic examination, and laser-blasted craters are the most definite mark you can make on anything with modern technology. We still included error bars in the results. Because it's not possible to measure things perfectly. It's still possible to get useful results by accounting for error, but it's also possible -- and much, much easier -- to mislead the world by ignoring it. For more detail, here's a great post on the importance of error bars and a detailed follow-up, explaining how they help you avoid being fooled out of $20,000 by bananas.
Or here's an example: I offer you a million dollars to wear a bulletproof vest with a 100 percent bullet-stopping record while I fire one bullet at it.
Why, you'd be a fool NOT to take this deal!
Would you take it? Oh, didn't I mention that my measurements had an error of 90 percent? Because I'm sure as hell not going to tell you that if you don't ask.
Fools don't check the error bars.
So, the effectiveness might be as low as 10 percent, and even then, that's only if I'm telling the truth. How good does the job look now? People assume that not mentioning any errors means it's accurate. Not mentioning any errors means they don't want to show you the errors. And you should really want to know why.