Systematic reviews incorporating evidence from non-randomized study designs: Reasons for caution when estimating health effects

BC Reeves, J van Binsbergen, C van Weel

Research output: Contribution to journalArticle (Academic Journal)peer-review

17 Citations (Scopus)

Abstract

Systematic reviews that include nonrandomized studies (NRS) face a number of logistical challenges. However, the greatest threat to the validity of such reviews arises from the differing susceptibility of randomized controlled trials (RCTs) and NRS to selection bias. Groups compared in NRS are unlikely to be balanced because of the reasons leading study participants to adopt different health behaviours or to be treated differentially. Researchers can try to minimize the susceptibility of NRS to selection bias both at the design stage, for example, by matching participants on key prognostic factors, and during data analysis, for
example, by regression modelling. However, because of logistical difficulties in matching, imperfect knowledge about the relationships between prognostic factors and between prognostic factors and outcome, and measurement limitations, it is inevitable that estimates of effect size derived from NRS will be confounded to some extent. Researchers, reviewers and users of evidence alike need to be aware of the consequences of residual confounding. In poor quality RCTs, selection bias tends to favour the new treatment being evaluated. Selection bias need not necessarily lead to systematic bias in favour of one treatment
but, even if it acts in an unpredictable way, it will still give rise to additional, nonstatistical uncertainty bias around the estimate of effect size. Systematic reviews of NRS studies run the risk of compounding these biases. Nutritional choices and uptake of health education about nutrition are very likely to be associated with potential confounding factors. Therefore, pooled estimates
of the effects of nutritional exposures and their confidence intervals are likely to be misleading; reviewers need to take into account both systematic and uncertainty bias.
Translated title of the contributionSystematic reviews incorporating evidence from non-randomized study designs: Reasons for caution when estimating health effects
Original languageEnglish
Article numberWOS:000231545400041
Pages (from-to)S155 - S161
JournalEuropean Journal of Clinical Nutrition
Volume59 Suppl 1
DOIs
Publication statusPublished - 2005

Structured keywords

  • BTC (Bristol Trials Centre)

Fingerprint Dive into the research topics of 'Systematic reviews incorporating evidence from non-randomized study designs: Reasons for caution when estimating health effects'. Together they form a unique fingerprint.

Cite this