Quasi-experimental study designs series—paper 6: risk of bias assessment

Hugh Waddington, Ariel Aloe, Betsy Becker, Eric W. Djimeu, Jorge Garcia Hombrados, Peter Tugwell, George Wells, Barnaby Reeves

Research output: Contribution to journalArticle (Academic Journal)peer-review

64 Citations (Scopus)
604 Downloads (Pure)


Rigorous and transparent critical appraisal is a core component of high quality systematic reviews. Well-conducted quasi-experiments have been empirically shown to estimate credible, unbiased treatment quantities. Conversely, when inappropriately designed or executed, these estimates are likely to be biased. This paper draws on recent advances in risk of bias assessment. It presents an approach to evaluating the internal validity of credible quasi-experiments. These are non-randomised studies using design-based approaches to control for unobservable sources of confounding such as difference studies, instrumental variables, interrupted time series, natural experiments and regression discontinuity designs. Our review suggests that existing risk of bias tools provide, to different degrees, incomplete transparent criteria to assess the validity of credible quasi-experiments. We argue that a tool is needed to assess risk of bias consistently across credible quasi-experiments. Drawing on existing tools, in particular Cochrane’s new tool for non-randomized studies of interventions (Sterne et al., 2014), we discuss domains of bias and suggest directions for evaluation questions.
Original languageEnglish
Pages (from-to)43-52
Number of pages10
JournalJournal of Clinical Epidemiology
Early online date27 Mar 2017
Publication statusPublished - Sept 2017

Structured keywords

  • BTC (Bristol Trials Centre)


  • risk of bias
  • systematic review
  • meta-analysis
  • quasi-experiment
  • natural experiment
  • instrumental variables
  • regression discontinuity
  • interrupted time series
  • difference in differences
  • propensity score matching


Dive into the research topics of 'Quasi-experimental study designs series—paper 6: risk of bias assessment'. Together they form a unique fingerprint.

Cite this