4.10: Stupid Journal Tricks

There’s no more dreaded phrase to the fact-checker than “a recent study says”. Recent studies say that chocolate cures cancer, prevents cancer, and may have no impact on cancer whatsoever. Recent studies say that holding a pencil in your teeth makes you happier. Recent studies say that the scientific process is failing, and others say it is just fine.

Most studies are data points — emerging evidence that lends weight to one conclusion or another but do not resolve questions definitively. What we want as a fact checker is not data points, but the broad consensus of experts. And the broad consensus of experts is rare.

The following chapters are not meant to show you how to meticulously evaluate research claims. Instead, they are meant to give you, the reader, some quick and frugal ways to decide what sorts of research can be safely passed over when you are looking for a reliable source. We take as our premise that information is abundant, and time is scarce. As such, it’s better to err on the side of moving onto the next article than to invest time an article that displays warning signs regarding either expertise or accuracy.