Estimating reliability statistics and measurement error variances using instrumental variables with longitudinal data

Harvey Goldstein*, George Leckie, Michele Haynes, Phuong Tran

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)

Abstract

The presence of randomly distributed measurement errors in scale scores such as those used in educational and behavioural assessments implies that careful adjustments are required to statistical model estimation procedures if inferences are required for ‘true’ as opposed to ‘observed’ relationships. In many cases this requires the use of external values for ‘reliability’ statistics or ‘measurement error variances’ which may be provided by a test constructor or else inferred or estimated by the data analyst. Popular measures are those described as ‘internal consistency’ estimates and sometimes other measures based on data grouping. All such measures, however, make particular assumptions that may be questionable but are often not examined. In this paper we focus on scaled scores derived from aggregating a set of indicators, and set out a general methodological framework for exploring different ways of estimating reliability statistics and measurement error variances, critiquing certain approaches and suggesting more satisfactory methods in the presence of longitudinal data. In particular, we explore the assumption of local (conditional) item response independence and show how a failure of this assumption can lead to biased estimates in statistical models using scaled scores as explanatory variables. We illustrate our methods using a large longitudinal dataset of mathematics test scores from Queensland, Australia.
Original languageEnglish
Number of pages26
JournalLongitudinal and Life Course Studies
Publication statusAccepted/In press - 10 Mar 2020

Structured keywords

  • SoE Centre for Multilevel Modelling

Keywords

  • RELIABILITY
  • LONGITUDINAL
  • instrumental variable

Fingerprint Dive into the research topics of 'Estimating reliability statistics and measurement error variances using instrumental variables with longitudinal data'. Together they form a unique fingerprint.

  • Cite this