A threshold analysis assessed the credibility of conclusions from network meta-analysis

Deborah M Caldwell, A E Ades, Sofia Dias, Sarah H Watkins, Tianjing Li, Nichole Taske, Nicky Welton, Bhash Naidoo

Research output: Contribution to journalArticle (Academic Journal)peer-review

18 Citations (Scopus)
562 Downloads (Pure)


Objective: To assess the reliability of treatment recommendations based on network meta-analysis (NMA)

Study design: We consider evidence in an NMA to be potentially biased. Taking each pair-wise contrast in turn we use a structured series of threshold analyses to ask: (a) “How large would the bias in this evidence-base have to be before it changed our decision?” and (b) “If the decision changed, what is the new recommendation?” We illustrate the method via two NMAs in which a GRADE assessment for NMAs has been implemented: weight-loss and osteoporosis.

Results. Four of the weight-loss NMA estimates were assessed as “low” and 6 as “moderate” quality by GRADE; for osteoporosis 6 were “low”, 9 “moderate” and 1 “high”. The threshold analysis suggests plausible bias in 3 of 10 estimates in the weight-loss network could have changed the treatment recommendation. For osteoporosis plausible bias in 6 of 16 estimates could change the recommendation. There was no relation between plausible bias changing a treatment recommendation and the original GRADE assessments.

Conclusions. Reliability judgements on individual NMA contrasts do not help decision makers understand whether a treatment recommendation is reliable. Threshold analysis reveals whether the final recommendation is robust against plausible degrees of bias in the data.
Original languageEnglish
Pages (from-to)68-76
Number of pages9
JournalJournal of Clinical Epidemiology
Early online date16 Jul 2016
Publication statusPublished - Dec 2016

Structured keywords

  • ConDuCT-II


  • mixed treatment comparison
  • comparative effectiveness
  • health technology assessment
  • reliability
  • quality assessment
  • bias


Dive into the research topics of 'A threshold analysis assessed the credibility of conclusions from network meta-analysis'. Together they form a unique fingerprint.
  • NICE TSU: NICE Guidelines Technical Support Unit

    Dias, S., Ades, A. E., Welton, N. J. & Keeney, E.

    1/04/16 → …

    Project: Other

  • ConDuCT-II

    Blazeby, J.


    Project: Research

  • Research Fellowship

    Caldwell, D. M.


    Project: Research

Cite this