Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics

Kirsty M Rhodes; Rebecca M Turner; Jelena Savović; Hayley Jones; David Mawdsley; Julian Higgins

doi:10.1016/j.jclinepi.2017.11.025

Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics

Kirsty M Rhodes, Rebecca M Turner, Jelena Savović, Hayley Jones, David Mawdsley, Julian Higgins

Research output: Contribution to journal › Article (Academic Journal) › peer-review

24 Citations (Scopus)

312 Downloads (Pure)

Abstract

Objective
We investigated the associations between risk of bias judgments from Cochrane reviews for sequence generation, allocation concealment and blinding and between-trial heterogeneity.

Study Design and Setting
Bayesian hierarchical models were fitted to binary data from 117 meta-analyses, to estimate the ratio λ by which heterogeneity changes for trials at high/unclear risk of bias, compared to trials at low risk of bias. We estimated the proportion of between-trial heterogeneity in each meta-analysis that could be explained by the bias associated with specific design characteristics.

Results
Univariable analyses showed that heterogeneity variances were, on average, increased among trials at high/unclear risk of bias for sequence generation (Math Eq 1.14, 95% interval: 0.57 to 2.30) and blinding (Math Eq 1.74, 95% interval: 0.85 to 3.47). Trials at high/unclear risk of bias for allocation concealment were on average less heterogeneous (Math Eq 0.75, 95% interval: 0.35 to 1.61). Multivariable analyses showed that a median of 37% (95% interval: 0% to 71%) heterogeneity variance could be explained by trials at high/unclear risk of bias for sequence generation, allocation concealment and/or blinding. All 95% intervals for changes in heterogeneity were wide and included the null of no difference.

Conclusion
Our interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.

Original language	English
Pages (from-to)	45-54
Number of pages	10
Journal	Journal of Clinical Epidemiology
Volume	95
Early online date	5 Dec 2017
DOIs	https://doi.org/10.1016/j.jclinepi.2017.11.025
Publication status	Published - Mar 2018

Keywords

Meta-analysis
Heterogeneity
Sequence generation
Allocation concealment
Blinding
Randomized trials

Access to Document

10.1016/j.jclinepi.2017.11.025Licence: Unspecified

Full-text PDF (final published version)
This is the final published version of the article (version of record). It first appeared online via Elsevier at http://www.sciencedirect.com/science/article/pii/S0895435617304353 . Please refer to any applicable terms of use of the publisher.
Final published version, 497 KBLicence: CC BY

Handle.net

Persistent link

Embargoed Document

Full-text PDF (accepted author manuscript)
Accepted author manuscript, 536 KB
Request copy

NIHR BRC BEI
Sterne, J. A. C.
1/04/17 → 31/03/22
Project: Research, Parent
Evidence synthesis of diagnostic test performance from a decision-making perspective
Jones, H. E.
12/08/15 → 11/08/19
Project: Research
ACROBAT-RT: Refinment of and extensions to the Cochrane Risk of Bias tool for Randomised trials
Savovic, J.
1/05/15 → 31/10/16
Project: Research

Dr Jelena Savovic
- jelena.savovicbristol.acuk
- Bristol Medical School (PHS) - Senior Lecturer in Evidence Synthesis
- Bristol Population Health Science Institute
Person: Academic , Member

Cite this

@article{f893bf7e4c5f47318d0278804428c023,

title = "Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics",

abstract = "ObjectiveWe investigated the associations between risk of bias judgments from Cochrane reviews for sequence generation, allocation concealment and blinding and between-trial heterogeneity.Study Design and SettingBayesian hierarchical models were fitted to binary data from 117 meta-analyses, to estimate the ratio λ by which heterogeneity changes for trials at high/unclear risk of bias, compared to trials at low risk of bias. We estimated the proportion of between-trial heterogeneity in each meta-analysis that could be explained by the bias associated with specific design characteristics.ResultsUnivariable analyses showed that heterogeneity variances were, on average, increased among trials at high/unclear risk of bias for sequence generation (Math Eq 1.14, 95% interval: 0.57 to 2.30) and blinding (Math Eq 1.74, 95% interval: 0.85 to 3.47). Trials at high/unclear risk of bias for allocation concealment were on average less heterogeneous (Math Eq 0.75, 95% interval: 0.35 to 1.61). Multivariable analyses showed that a median of 37% (95% interval: 0% to 71%) heterogeneity variance could be explained by trials at high/unclear risk of bias for sequence generation, allocation concealment and/or blinding. All 95% intervals for changes in heterogeneity were wide and included the null of no difference.ConclusionOur interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.",

keywords = "Meta-analysis, Heterogeneity, Sequence generation, Allocation concealment, Blinding, Randomized trials",

author = "Rhodes, {Kirsty M} and Turner, {Rebecca M} and Jelena Savovi{\'c} and Hayley Jones and David Mawdsley and Julian Higgins",

year = "2018",

month = mar,

doi = "10.1016/j.jclinepi.2017.11.025",

language = "English",

volume = "95",

pages = "45--54",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier Inc.",

}

TY - JOUR

T1 - Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics

AU - Rhodes, Kirsty M

AU - Turner, Rebecca M

AU - Savović, Jelena

AU - Jones, Hayley

AU - Mawdsley, David

AU - Higgins, Julian

PY - 2018/3

Y1 - 2018/3

N2 - ObjectiveWe investigated the associations between risk of bias judgments from Cochrane reviews for sequence generation, allocation concealment and blinding and between-trial heterogeneity.Study Design and SettingBayesian hierarchical models were fitted to binary data from 117 meta-analyses, to estimate the ratio λ by which heterogeneity changes for trials at high/unclear risk of bias, compared to trials at low risk of bias. We estimated the proportion of between-trial heterogeneity in each meta-analysis that could be explained by the bias associated with specific design characteristics.ResultsUnivariable analyses showed that heterogeneity variances were, on average, increased among trials at high/unclear risk of bias for sequence generation (Math Eq 1.14, 95% interval: 0.57 to 2.30) and blinding (Math Eq 1.74, 95% interval: 0.85 to 3.47). Trials at high/unclear risk of bias for allocation concealment were on average less heterogeneous (Math Eq 0.75, 95% interval: 0.35 to 1.61). Multivariable analyses showed that a median of 37% (95% interval: 0% to 71%) heterogeneity variance could be explained by trials at high/unclear risk of bias for sequence generation, allocation concealment and/or blinding. All 95% intervals for changes in heterogeneity were wide and included the null of no difference.ConclusionOur interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.

AB - ObjectiveWe investigated the associations between risk of bias judgments from Cochrane reviews for sequence generation, allocation concealment and blinding and between-trial heterogeneity.Study Design and SettingBayesian hierarchical models were fitted to binary data from 117 meta-analyses, to estimate the ratio λ by which heterogeneity changes for trials at high/unclear risk of bias, compared to trials at low risk of bias. We estimated the proportion of between-trial heterogeneity in each meta-analysis that could be explained by the bias associated with specific design characteristics.ResultsUnivariable analyses showed that heterogeneity variances were, on average, increased among trials at high/unclear risk of bias for sequence generation (Math Eq 1.14, 95% interval: 0.57 to 2.30) and blinding (Math Eq 1.74, 95% interval: 0.85 to 3.47). Trials at high/unclear risk of bias for allocation concealment were on average less heterogeneous (Math Eq 0.75, 95% interval: 0.35 to 1.61). Multivariable analyses showed that a median of 37% (95% interval: 0% to 71%) heterogeneity variance could be explained by trials at high/unclear risk of bias for sequence generation, allocation concealment and/or blinding. All 95% intervals for changes in heterogeneity were wide and included the null of no difference.ConclusionOur interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.

KW - Meta-analysis

KW - Heterogeneity

KW - Sequence generation

KW - Allocation concealment

KW - Blinding

KW - Randomized trials

U2 - 10.1016/j.jclinepi.2017.11.025

DO - 10.1016/j.jclinepi.2017.11.025

M3 - Article (Academic Journal)

C2 - 29217451

SN - 0895-4356

VL - 95

SP - 45

EP - 54

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics

Abstract

Keywords

Access to Document

Handle.net

Embargoed Document

Fingerprint

Projects

NIHR BRC BEI

Evidence synthesis of diagnostic test performance from a decision-making perspective

ACROBAT-RT: Refinment of and extensions to the Cochrane Risk of Bias tool for Randomised trials

Profiles

Dr Jelena Savovic

Cite this