Measurement and evaluation issues with PISA

This chapter comments on the restricted nature of the data modelling and analysis that underpins PISA, and the resulting interpretations. It points to certain features that raise questions about the provenance of the data collected, the ways in which it is disseminated and the limitations these impose on the interpretations that can be made in terms of international and over-time comparisons. Specifically, it discusses the issue of item translation and selection which PISA addresses through simplistic latent variable, or item response, modelling. It also looks at the ways in which data are provided to users and the technical limitations of this, and finally issues concerned with the legitimacy of causal inferences that have driven educational policy changes.