The impact of teaching experience upon marking reliability in Key Stage 3 English

L Royal-Dawson, J-A Baird

Research output: Contribution to journalArticle (Academic Journal)peer-review

27 Citations (Scopus)

Abstract

Hundreds of thousands of raters are recruited internationally to score examinations, but little research has been conducted on the selection criteria for these raters. Many countries insist upon teaching experience as a selection criterion and this has frequently become embedded in the cultural expectations surrounding the tests. Shortages in raters for some of England’s national examinations has led to non-teachers being hired to score a small minority of items and changes in technology have fostered this approach. For a National Curriculum test in English taken at age 14, this study investigated whether teaching experience was a necessary selection criterion for all aspects of the examination. Fifty-seven raters with different backgrounds were trained in the normal manner and scored the same 97 students’ work. Accuracy was investigated using a cross-classified multilevel model of absolute score differences with accuracy measures at level 1 and raters crossed with candidates at level 2. By comparing the scoring accuracy of graduates with a degree in English, teacher trainees, experienced teachers and experienced raters, this study found that teaching experience was not a necessary selection criterion. A rudimentary model for allocation of raters to different question types is proposed and further research to investigate the limits of necessary qualifications for scoring is suggested.
Translated title of the contributionThe impact of teaching experience upon marking reliability in Key Stage 3 English
Original languageEnglish
Pages (from-to)2 - 8
Number of pages6
JournalEducational Measurement: Issues and Practice
Volume28 ( 2)
DOIs
Publication statusPublished - Jun 2009

Fingerprint

Dive into the research topics of 'The impact of teaching experience upon marking reliability in Key Stage 3 English'. Together they form a unique fingerprint.

Cite this