Native speaker experts have been widely used to create scoring templates in language testing, despite arguments concerning â€œthe futility of the definition of native speakerâ€� (Savignon, 2003) and the conceptual and practical ambiguity of native speakerness. This paper reports the use of two kinds of scoring templates to evaluate studentsâ€™ summaries produced under various experimental conditions. In this research, the two scoring templates were generated from the summaries of five English-speaking experts and 100 EFL students respectively, hence the terms â€“ â€œexpert templateâ€� and â€œpopular templateâ€�. The use of both templates to evaluate studentsâ€™ summaries was investigated from two perspectives (i) how they affected the scores the summaries would be assigned, and (ii) whether and to what extent the students would (de)value this â€œdemocracyâ€� in test development. It was found that a summary consistently received higher scores (right statement credits, and overall quality scores) when it was judged by the popular scoring template, regardless of the language and the language order (English then Chinese or Chinese then English) in which a summary was produced. However, the scores of the expert template were better than those of the popular template in predicting the summarizersâ€™ reading comprehension abilities as measured by FCE and TOEFL. The majority of the students had a strong preference for the expert rather than the popular template. The arguments for, or against, the use of these two templates centered mainly around i) the degree of experience and language abilities of students and experts in understanding and summarizing a source text, ii) the stereotypical status and common practice of using students and experts in educational assessment, and iii) the dialectical interpretations of â€œquantityâ€� and â€œqualityâ€�. Implications of these findings are discussed with specific reference to the involvement of test takers in development of assessment criteria.
|Translated title of the contribution||Students' voices in the evaluation of their written summaries: empowerment and democracy for test takers?|
|Pages (from-to)||539 - 572|
|Number of pages||33|
|Publication status||Published - Oct 2007|