perfidy

There was no way I wasn't going to read an Economist article that started like this:

THEODORE STURGEON, an American science-fiction writer, once observed that “95% of everything is crap”. John Ioannidis, a Greek epidemiologist, would not go that far. His benchmark is 50%. But that figure, he thinks, is a fair estimate of the proportion of scientific papers that eventually turn out to be wrong.

Dr. Ionannidis' article, entitled "Why Most Published Research Findings Are False" turns out to be a really interesting read. It uses numerical and statistical methods in an attempt to prove that, in fact, 50% of everything scientific is crap.

I'm sure it's just me, pretending to think too deeply on a stormy day in Houston, but there's a circular irony to his assertion that got me chuckling.

Once past that, the reasons that his statistics seem reasonable are many and credible. As a ferinstance:

Dr Ioannidis began by looking at specific studies, in a paper published in the Journal of the American Medical Association in July. He examined 49 research articles printed in widely read medical journals between 1990 and 2003. Each of these articles had been cited by other scientists in their own papers 1,000 times or more. However, 14 of them—almost a third—were later refuted by other work. Some of the refuted studies looked into whether hormone-replacement therapy was safe for women (it was, then it wasn't), whether vitamin E increased coronary health (it did, then it didn't), and whether stents are more effective than balloon angioplasty for coronary-artery disease (they are, but not nearly as much as was thought).

Not to worry, though - that's all part of the process, as we're reminded, not by Dr. Ioannidis, but by the Economist's science editor:

Science is a Darwinian process that proceeds as much by refutation as by publication. But until recently no one has tried to quantify the matter.

A fun fact to remember next time I hear of breathless new scientific hypotheses. Also important to remember? Even highly authoritative sources, as measured by number of citations elsewhere, are not guaranteed to be the truth might be crap.