TY - JOUR
T1 - Analysis of Categorical Moderators in Mixed-effects Meta-analysis
T2 - Consequences of Using Pooled vs. Separate Estimates of the Residual Between-studies Variances
AU - Rubio-Aparicio, María
AU - Sánchez-Meca, Julio
AU - Lopez-Lopez, Jose A
AU - Botella, Juan
AU - Marín-Martínez, Fulgencio
PY - 2017/11
Y1 - 2017/11
N2 - Subgroup analyses allow to examining the influence of a categorical moderator on the effect magnitude in meta-analysis. We conducted a simulation study using a dichotomous moderator, and compared the impact of pooled versus separate estimates of the residual between-studies variance on the statistical performance of the QB(P) and QB(S) tests for subgroup analyses assuming a mixed-effects model. Our results suggested that a similar performance can be expected as long as there are at least 20 studies and these are approximately balanced across categories. Conversely, when subgroups were unbalanced, the practical consequences of having heterogeneous residual between-studies variances were more evident, with both tests leading to the wrong statistical conclusion more often than in the conditions with balanced subgroups. A pooled estimate should be preferred for most scenarios, unless the residual between-studies variances are clearly different and there are enough studies in each category to get precise separate estimates.
AB - Subgroup analyses allow to examining the influence of a categorical moderator on the effect magnitude in meta-analysis. We conducted a simulation study using a dichotomous moderator, and compared the impact of pooled versus separate estimates of the residual between-studies variance on the statistical performance of the QB(P) and QB(S) tests for subgroup analyses assuming a mixed-effects model. Our results suggested that a similar performance can be expected as long as there are at least 20 studies and these are approximately balanced across categories. Conversely, when subgroups were unbalanced, the practical consequences of having heterogeneous residual between-studies variances were more evident, with both tests leading to the wrong statistical conclusion more often than in the conditions with balanced subgroups. A pooled estimate should be preferred for most scenarios, unless the residual between-studies variances are clearly different and there are enough studies in each category to get precise separate estimates.
KW - meta-analysis
KW - mixed-effects model
KW - subgroup analysis
KW - between-studies variance
U2 - 10.1111/bmsp.12092
DO - 10.1111/bmsp.12092
M3 - Article (Academic Journal)
C2 - 28164265
SN - 0007-1102
VL - 70
SP - 439
EP - 456
JO - British Journal of Mathematical and Statistical Psychology
JF - British Journal of Mathematical and Statistical Psychology
IS - 3
ER -