Promising = over-hyped + under-tested

I first wrote about the tendency of "promising" treatments to metamorphose into "disappointing" treatments in a BMJ piece about evidence based mistakes. Early results, after all, can't promise anything at all.

The graph depicts a cumulative meta-analysis: each new study is being absorbed into a summation of the evidence so far. With 4 studies, it's shifted from the "this helps" side of the ledger over to the "this harms" side. See more about cumulative analyses in this classic article.

"Promising" is a media and marketing staple, too. Several wonderful initiatives keep the media to account on this, story by story: Behind the Headlines, Germany's Median-Doktor.de, Japan's Media Doctor, and the US Health News Review.