Skip to content

GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence

Research output: Contribution to journalArticle

  • GRADE Working Group
Original languageEnglish
Pages (from-to)105-114
Number of pages10
JournalJournal of Clinical Epidemiology
Early online date9 Feb 2018
DateAccepted/In press - 11 Jan 2018
DateE-pub ahead of print - 9 Feb 2018
DatePublished (current) - 1 Jul 2019


Objective: To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty rating process. Study Design and Setting: The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group. Results: We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. Conclusions: The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.

Additional information

Copyright © 2018 Elsevier Inc. All rights reserved.

    Research areas

  • Certainty of the evidence, GRADE, Nonrandomized studies, Quality of evidence, Risk of bias, ROBINS

Download statistics

No data available



  • Full-text PDF (accepted author manuscript)

    Rights statement: This is the accepted author manuscript (AAM). The final published version (version of record) is available online via Elsevier at . Please refer to any applicable terms of use of the publisher.

    Accepted author manuscript, 736 KB, PDF document

    Licence: CC BY-NC-ND


View research connections

Related faculties, schools or groups