You will meet too much false precision

Precise numbers and claims - as though there is no margin for error - are all around us. When someone tells you that 54.3% of people with some disease will have a particular outcome, they're basically predicting the future of all groups of people based on what happened to another group of people in the past. Well, what are the chances of exactly that always happening, eh?

If our fortune teller was quoting the mean of a study here, it could be written like this: 67.5% (95% CI: 62%-73%). The CI stands for "confidence interval" and it gives you an idea of how much imprecision or uncertainty there is around the estimate. The confidence level - 95% here, which is common - is chosen when a confidence is calculated. The 95% level means the significance level is at 0.05 (or 5%) - more about that here. It has set the level of uncertainty being measured - how probable it is, that roughly that result would occur.

The chances of the result always being precisely 67.5% can be pretty slim or very high, depending on lots of things. If there is a lot of data, the confidence interval will be narrow: the best case scenario and the worst case scenario will be close together (say, 66% to 69%).

We give ranges for estimates all the time. If someone asks, "How long does it take to get to your house?", we don't say "39.35 minutes". We say, "Usually about half an hour to 45 minutes, depending on the traffic."

In a systematic review, you will often see an outcome of an individual study shown as a line. The length of that line is showing you the width of the confidence interval around the result. It looks something like this:

This is called a forest plot. Find more from Statistically Funny on this in The Forest Plot Trilogy.

What a confidence interval isn't: it doesn't mean that 95% of people's outcomes will be between those upper and lower boundaries. It's where the mean is expected to be likely to fall (or median, or whatever other statistic is being measured).

If the statistical estimates are made with Bayesian methods, the range you will see around an estimate isn't a confidence interval: it's a credible interval. I explain a bit about Bayesian statistics in this post. Unlike a confidence interval, a credible interval has incorporated extra data about the probability of the result falling inside the interval.

Update [4 June 2016]: The American Statistical Association (ASA) issued a statement encouraging people to consider estimates like confidence intervals instead of only looking at p-values and statistical significance. I've written an explanation of that in this post: 5 Tips for Avoiding P-Value Potholes.

Comments

Austerity-A Fancy Word for Destitute.

The reason for this post is not for the folks who have been caught in the first wave of personal economic hard reality, but the next wave. Regardless of the optimism espoused by grinning leaders and sycophant press, we are entering the final stage of global economic collapse. It began in 2008 and was forestalled for five years with fudge putty, but the weight of global indebtedness cannot be propped any longer and the final crunch is imminent. Austerity measures herald the final throes. Indications of coming austerity. Austerity measures are the final last ditch effort, futile or not! Back in the day many of us old-timers went through periods of "hard-times". In retrospect I realize there is no comparison to yesteryear hard times and today's version. Back then, expectations were never very high for the working class, there were no sophisticated systems or conveniences anyway. In fact the difference between being "set" or not was about having treats or not. Si...

Terrifying Arctic methane levels

A peak methane level of 3026 ppb was recorded by the MetOp-B satellite at 469 mb on December 11, 2021 am. This follows a peak methane level of 3644 ppb recorded by the MetOp-B satellite at 367 mb on November 21, 2021, pm. A peak methane level of 2716 ppb was recorded by the MetOp-B satellite at 586 mb on December 11, 2021, pm, as above image shows. This image is possibly even more terrifying than the image at the top, as above image shows that at 586 mb, i.e. much closer to sea level, almost all methane shows up over sea, rather than over land, supporting the possibility of large methane eruptions from the seafloor, especially in the Arctic. Also, the image was recorded later than the image at the top with the 3026 ppb peak, indicating that even more methane may be on the way. This appears to be confirmed by the Copernicus forecast for December 12, 2021, 03 UTC, as illustrated by the image below, which shows methane at 500 hPa (equivalent to 500 mb). Furthermore, ...

Women and children overboard

It's the Catch-22 of clinical trials: to protect pregnant women and children from the risks of untested drugs....we don't test drugs adequately for them. In the last few decades , we've been more concerned about the harms of research than of inadequately tested treatments for everyone, in fact. But for "vulnerable populations," like pregnant women and children, the default was to exclude them. And just in case any women might be, or might become, pregnant, it was often easier just to exclude us all from trials. It got so bad, that by the late 1990s, the FDA realized regulations and more for pregnant women - and women generally - had to change. The NIH (National Institutes of Health) took action too. And so few drugs had enough safety and efficacy information for children that, even in official circles, children were being called "therapeutic orphans." Action began on that, too. There is still a long way to go. But this month there was a sign that ...

Humor4u

Search This Blog