Skip to content

Students' voices in the evaluation of their written summaries: empowerment and democracy for test takers?

Research output: Contribution to journalArticle

Original languageEnglish
Pages (from-to)539 - 572
Number of pages33
JournalLanguage Testing
Volume24 (4)
DOIs
DatePublished - Oct 2007

Abstract

Native speaker experts have been widely used to create scoring templates in language testing, despite arguments concerning “the futility of the definition of native speaker� (Savignon, 2003) and the conceptual and practical ambiguity of native speakerness. This paper reports the use of two kinds of scoring templates to evaluate students’ summaries produced under various experimental conditions. In this research, the two scoring templates were generated from the summaries of five English-speaking experts and 100 EFL students respectively, hence the terms – “expert template� and “popular template�. The use of both templates to evaluate students’ summaries was investigated from two perspectives (i) how they affected the scores the summaries would be assigned, and (ii) whether and to what extent the students would (de)value this “democracy� in test development. It was found that a summary consistently received higher scores (right statement credits, and overall quality scores) when it was judged by the popular scoring template, regardless of the language and the language order (English then Chinese or Chinese then English) in which a summary was produced. However, the scores of the expert template were better than those of the popular template in predicting the summarizers’ reading comprehension abilities as measured by FCE and TOEFL. The majority of the students had a strong preference for the expert rather than the popular template. The arguments for, or against, the use of these two templates centered mainly around i) the degree of experience and language abilities of students and experts in understanding and summarizing a source text, ii) the stereotypical status and common practice of using students and experts in educational assessment, and iii) the dialectical interpretations of “quantity� and “quality�. Implications of these findings are discussed with specific reference to the involvement of test takers in development of assessment criteria.

Additional information

Publisher: Sage

View research connections

Related faculties, schools or groups