A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses

Dean Langan, Julian P T Higgins, Dan Jackson, Jack Bowden, Areti Angeliki Veroniki, Evangelos Kontopantelis, Wolfgang Viechtbauer, Mark Simmonds

Research output: Contribution to journalArticle (Academic Journal)

43 Citations (Scopus)
138 Downloads (Pure)

Abstract

Studies combined in a meta-analysis often have differences in their design and conduct that can lead to heterogeneous results. A random-effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian-Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta-analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian-Laird estimator is negatively biased in scenarios with small studies, and in scenarios with a rare binary outcome. Results also show the Paule-Mandel method has considerable positive bias in meta-analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta-analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta-analysis. The estimated summary effect of the meta-analysis and its confidence interval derived from the Hartung-Knapp-Sidik-Jonkman method is more robust to changes in the heterogeneity variance estimate and shows minimal deviation from the nominal coverage of 95% under most of our simulated scenarios.

Original languageEnglish
Number of pages16
JournalResearch Synthesis Methods
Early online date6 Sep 2018
DOIs
Publication statusE-pub ahead of print - 6 Sep 2018

Keywords

  • Heterogeneity
  • simulation
  • random-effects
  • DerSimonian-Laird
  • REML

Fingerprint Dive into the research topics of 'A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses'. Together they form a unique fingerprint.

  • Cite this

    Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2018). A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Research Synthesis Methods. https://doi.org/10.1002/jrsm.1316