Goldilocks and the three reviews

Goldilocks is right: that review is FAR too complicated. The methods section alone is 652 pages long! Which wouldn't be too bad, if it weren't that it is a few years out of date. It took so long to do this review and go through rigorous enough quality review, it was already out of date the day it was released. Something that happens often enough to be rather disheartening.

When methodology for systematic reviewing gets overly rococo, the point of diminishing returns will be passed. That's a worry, for a few reasons. For one, it's inefficient and more reviews could be done with the resources. Secondly, more complex methodology can both be daunting, and it can be hard for researchers to accomplish with consistency. Thirdly, when a review gets very elaborate, reproducing or updating it isn't going to be easy either.

It's unavoidable for some reviews to be massive and complex undertakings, though, if they're going to get to the bottom of massive and complex questions. Goldilocks is right about review number 2, as well: that one is WAY too simple. And that's a serious problem, too.

Reviewing evidence needs to be a well-conducted research exercise. A great way to find out more about what goes wrong when it's not, is reading Testing Treatments. And see more on this here at Statistically Funny, too.

You need to check the methods section of every review before you take its conclusions seriously - even when it claims to be "evidence-based" or systematic. People can take far too many shortcuts. Fortunately, it's not often that a review gets as bad as the second one Goldilocks encountered here. The authors of that review decided to include only one trial for each drug "in order to keep the tables and figures to a manageable size." Gulp!

Getting to a good answer also quite simply takes some time and thought. Making real sense of evidence and the complexities of health, illness and disability is often just not suited to a "fast food" approach. As the scientists behind the Slow Science Manifesto point out, science needs time for thinking and digesting.

To cover more ground, people are looking for reasonable ways to cut corners, though. There are many kinds of rapid review, including reliance on previous systematic reviews for new reviews. These can be, but aren't always, rigorous enough for us to be confident about their conclusions.

You can see this process at work in the set of reviews discussed at Statistically Funny a few cartoons ago. Review number 3 there is in part based on review number 2 - without re-analysis. And then review number 4 is based on review number 3.

So if one review gets it wrong, other work may be built on weak foundations. Li and Dickersin suggest this might be a clue to the perpetuation of incorrect techniques in meta-analyses: reviewers who got it wrong in their review, were citing other reviews that had gotten it wrong, too. (That statistical technique, by the way, has its own cartoon.)

Luckily for Goldilocks, the bears had found a third review. It had sound methodology you can trust. It had been totally transparent from the start - included in PROSPERO, the international prospective register for systematic reviews. Goldilocks can get at the fully open review quickly via PubMed Health, and its data are in the Systematic Review Data Repository, open to others to check and re-use. Ahhh - just right!

I'm grateful to the Wikipedians who put together the article on Goldilocks and the three bears. That article pointed me to the fascinating discussion of "the rule of three" and the hold this number has on our imaginations.

Benefits Of Healthy eating Turmeric every day for the body

One teaspoon of turmeric a day to prevent inflammation, accumulation of toxins, pain, and the outbreak of cancer. Yes, turmeric has been known since 2.5 centuries ago in India, as a plant anti-inflammatory / inflammatory, anti-bacterial, and also have a good detox properties, now proven to prevent Alzheimer's disease and cancer. Turmeric prevents inflammation: For people who

Women and children overboard

It's the Catch-22 of clinical trials: to protect pregnant women and children from the risks of untested drugs....we don't test drugs adequately for them. In the last few decades , we've been more concerned about the harms of research than of inadequately tested treatments for everyone, in fact. But for "vulnerable populations," like pregnant women and children, the default was to exclude them. And just in case any women might be, or might become, pregnant, it was often easier just to exclude us all from trials. It got so bad, that by the late 1990s, the FDA realized regulations and more for pregnant women - and women generally - had to change. The NIH (National Institutes of Health) took action too. And so few drugs had enough safety and efficacy information for children that, even in official circles, children were being called "therapeutic orphans." Action began on that, too. There is still a long way to go. But this month there was a sign that

Not a word was spoken (but many were learned)

Video is often used in the EFL classroom for listening comprehension activities, facilitating discussions and, of course, language work. But how can you exploit silent films without any language in them? Since developing learners' linguistic resources should be our primary goal (well, at least the blogger behind the blog thinks so), here are four suggestions on how language (grammar and vocabulary) can be generated from silent clips. Split-viewing Split-viewing is an information gap activity where the class is split into groups with one group facing the screen and the other with their back to the screen. The ones facing the screen than report on what they have seen - this can be done WHILE as well as AFTER they watch. Alternatively, students who are not watching (the ones sitting with their backs to the screen) can be send out of the classroom and come up with a list of the questions to ask the 'watching group'. This works particularly well with action or crime scenes with

Humor4u

Search This Blog