Confidence in recommendations based on Network Meta-Analysis: threshold analysis as an alternative to GRADE

David Phillippo, Sofia Dias, Nicky Welton, Deborah Caldwell, Nichole Taske, Tony Ades

Research output: Contribution to conferenceConference Abstractpeer-review

88 Downloads (Pure)


Background: Guideline development requires synthesising evidence on multiple treatments of interest, typically using Network Meta-Analysis (NMA). Often the studies included are assessed as having flaws and the reliability of results from the NMA can be in doubt. Therefore, guideline developers need to assess the robustness of recommendations made based on the NMA to potential biases in the evidence. Recent approaches proposed to do this include GRADE NMA and threshold analysis.
Objectives: We apply threshold analysis retrospectively to published NICE guidelines for headaches and social anxiety, and compare with GRADE NMA.
Methods: Threshold analysis derives thresholds to quantify how much the evidence could be adjusted for bias before the recommendation changes, and what the revised recommendation would be. GRADE NMA combines quality assessments for each piece of evidence into an overall judgement of confidence in the recommendation.
Results and Discussion: The quality of each piece of evidence is typically unrelated to its influence on the NMA results. In our examples, recommendations are only sensitive to plausible biases in a small proportion of the evidence. In larger networks with greater numbers of trials, recommendations are robust against almost any plausible biases.
Implications for guideline developers/users: Threshold analysis can give guideline developers more confidence in recommendations where thresholds are large and can highlight decision-sensitive studies and comparisons.
Conclusion: GRADE NMA assesses evidence quality, but does not account for the influence of evidence on the recommendation. Threshold analysis directly indicates the sensitivity to and impact of potential bias in each piece of evidence. This knowledge can be used to make better-informed recommendations.
Original languageEnglish
Number of pages17
Publication statusUnpublished - 12 Sept 2018
EventGuidelines International Network Conference 2018 - The Principal, Manchester, United Kingdom
Duration: 11 Sept 201814 Sept 2018


ConferenceGuidelines International Network Conference 2018
Country/TerritoryUnited Kingdom
Internet address


Dive into the research topics of 'Confidence in recommendations based on Network Meta-Analysis: threshold analysis as an alternative to GRADE'. Together they form a unique fingerprint.

Cite this