Whenever I see a headline that begins with “New Study Proves…”, I skip over it and move on to the next one. I love science, and I even like studies, but I have a big problem with the way studies are framed today, especially in the media. There are two major things wrong with these so-called scientific studies, which, combined, give us misleading and often outright incorrect headlines which many of us use to inform our decisions.
The first principle that is crucial to understand is that correlation is not the same thing as causation. For example, people who send their children to private schools are more likely to be convicted of stock market fraud than those who don’t.
New Study Shows That Sending Children to Private School Could Lead to Criminal Behavior in Parents!
Well, no. People who have the money to send their kids to private school are more likely to be in a position to conduct stock frauds. There is a link between the two things, but it is not a causal link. In other words, sending your children to private school is not going to turn you into a criminal.
You see this in diet and health studies all the time. Scientists realized that people who ate a lot of saturated fat had high levels of saturated fat. That caused people to eat less saturated fat and replace those calories with refined sugars.
But what happened then? People’s saturated fat went even higher. It turns out that dietary saturated fat does not increase bodily saturated fat, but refined carbohydrate does. People who ate a lot of red meat also happened to eat a lot of refined carbohydrate. In this case, following the advice of the “research” actually made the problem worse.
This happens all the time, because the motivation of many writers and “journalists” is to get clicks and views, not to inform people and improve their health. This is a major conflict of interest.
Let’s say, though, that you read the study and determine that it actually did hone in on causality by altering only one variable, and having a control. They determine, with 99% confidence that eating red crayons cures cancer. Do you eat red crayons to cure your cancer?
Not so fast…
What if a mad scientist, sure that the cure to cancer lay in crayons, hired 100 teams to test 100 different colors? If they required a 99% confidence interval, it’s very likely that one of the teams would have a false positive. Small percentages, even 1% ones, pop up sometimes.
The other teams have no reason to publish their work. They weren’t able to prove anything, so they start work on their next study. Meanwhile, to the red crayon team, it appears they’ve discovered a miracle. It’s time to publish and get their name in the lights!
Later, of course, another team would try to replicate the results and realize that red crayons don’t actually cure cancer. But no one who read the buzzfeed article saying that they did work would ever hear about that study. It’s not a good headline.
Crayon-eating may be farfetched, but think of how many teams there are across the world, all desperately trying to figure things out. Many improbable results will appear to be true, the scientists themselves will believe them, and the public gets misled.
I love science. Even the process by which these studies are done deserves little blame. Finding correlation and preliminary results are just the first step. The problem is when we take this first step to mean more than it does and make changes based on it. This problem is exacerbated by the fact that the most unlikely “cures” and “hidden dangers” are the ones that we want most to hear about. Solid incremental research takes time, but fluke findings and correlation as causation is quick, easy and punchy.
Trust science, be skeptical of studies.
###
Picture is some agave plants in the desert outside of Las Vegas.
CruiseSheet is currently down for upgrading… should be up later today with more cruises than ever and more accurate price updates.
Leave a Reply