Avoiding Bias When Estimating the Consistency and Stability of Value-Added School Effects

George Leckie*

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

7 Citations (Scopus)
181 Downloads (Pure)


The traditional approach to estimating the consistency of school effects across subject areas and the stability of school effects across time is to fit separate value-added multilevel models to each subject or cohort and to correlate the resulting empirical Bayes predictions. We show that this gives biased correlations and these biases cannot be avoided by simply correlating “unshruken” or “reflated” versions of these predicted random effects. In contrast, we show that fitting a joint value-added multilevel multivariate response model simultaneously to all subjects or cohorts directly gives unbiased estimates of the correlations of interest. There is no need to correlate the resulting empirical Bayes predictions and indeed we show that this should again be avoided as the resulting correlations are also biased. We illustrate our arguments with separate applications to measuring the consistency and stability of school effects in primary and secondary school settings. However, our arguments apply more generally to other areas of application where researchers routinely interpret correlations between predicted random effects rather than estimating and interpreting these correlation directly.

Original languageEnglish
Pages (from-to)440-468
Number of pages29
JournalJournal of Educational and Behavioral Statistics
Issue number4
Early online date9 Feb 2018
Publication statusPublished - 1 Aug 2018

Structured keywords

  • SoE Centre for Multilevel Modelling


  • consistency
  • multilevel model
  • multivariate response
  • school effects
  • stability
  • value-added

Fingerprint Dive into the research topics of 'Avoiding Bias When Estimating the Consistency and Stability of Value-Added School Effects'. Together they form a unique fingerprint.

Cite this