Avoiding Bias When Estimating the Consistency and Stability of Value-Added School Effects

George Leckie*

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

17 Citations (Scopus)
292 Downloads (Pure)

Abstract

The traditional approach to estimating the consistency of school effects across subject areas and the stability of school effects across time is to fit separate value-added multilevel models to each subject or cohort and to correlate the resulting empirical Bayes predictions. We show that this gives biased correlations and these biases cannot be avoided by simply correlating “unshruken” or “reflated” versions of these predicted random effects. In contrast, we show that fitting a joint value-added multilevel multivariate response model simultaneously to all subjects or cohorts directly gives unbiased estimates of the correlations of interest. There is no need to correlate the resulting empirical Bayes predictions and indeed we show that this should again be avoided as the resulting correlations are also biased. We illustrate our arguments with separate applications to measuring the consistency and stability of school effects in primary and secondary school settings. However, our arguments apply more generally to other areas of application where researchers routinely interpret correlations between predicted random effects rather than estimating and interpreting these correlation directly.

Original languageEnglish
Pages (from-to)440-468
Number of pages29
JournalJournal of Educational and Behavioral Statistics
Volume43
Issue number4
Early online date9 Feb 2018
DOIs
Publication statusPublished - 1 Aug 2018

Research Groups and Themes

  • SoE Centre for Multilevel Modelling

Keywords

  • consistency
  • multilevel model
  • multivariate response
  • school effects
  • stability
  • value-added

Fingerprint

Dive into the research topics of 'Avoiding Bias When Estimating the Consistency and Stability of Value-Added School Effects'. Together they form a unique fingerprint.

Cite this