The C-test as a Second Language Proficiency Estimate and Screening Test in Turkish
: An Argument-based Validation Study

  • Merve Demiralp

Student thesis: Doctoral ThesisDoctor of Philosophy (PhD)

Abstract

Assessing the language proficiency of second language (L2) learners and bilinguals is essential in second language acquisition (SLA) research; to control for language proficiency or to select participants into an SLA study (i.e., Gaillard, 2015; Drackert, 2016; Norris & Ortega, 2012; Tremblay, 2011). L2 proficiency testing is also important in educational programs to make decisions such as placing students into appropriate levels of language programs and screening students to identify those with limited language skills (i.e., Elder & von Randow, 2008; Norris, 2006, 2008). At present, there is an insufficient body of standardized and validated measurement instruments in SLA research (Norris & Ortega, 2012). Similarly, validating low-stakes screening and diagnostic tests has been underestimated in educational context despite their impact when they are published online open to hundreds of thousands of learners (i.e., Alderson et al, 2015; Schmidgall et al, 2017). To address these issues, this present research developed a Turkish C-test for adult L2 learners of Turkish and validated it by using Kane’s argument-based approach (2006) for two different uses: (1) SLA research purposes where researchers need to control the language proficiency of their research participants (study 1); (2) educational purposes as a screening test for the Turkish Proficiency Exam (TYS) used to admit Turkish L2 learners into Turkish-medium universities (study 2). Both validation studies adopted a mixed-methods approach in order to gain better insight into stakeholders’ perception of the uses of the Turkish C-test. For study 1, the Turkish C-test was administered to 85 Turkish L2 learners in the UK and USA along with a background questionnaire and feedback survey. This was augmented with a second survey administered to 10 SLA researchers, and interviews were conducted with five of these researchers. The Turkish C-test was found to discriminate between four different levels of Turkish L2 learners with an IRT separation reliability of .94. Furthermore, the internal consistency of the texts was high with a reliability value of .92, and texts functioned consistently across both UK and USA settings. Regarding stakeholders’ perceptions of the test, although they found the test practical, they were sceptical towards what it can measure and be used for. For study 2, the Turkish C-test was administered to 79 TYS candidates alongside a background questionnaire and feedback survey. Interviews were also conducted with 13 of these participants. This was augmented with a second survey administered to 34 instructors of Turkish, and interviews were conducted with two of these instructors. The Turkish C-test was found to moderately to strongly correlate with each TYS section (reading, writing, listening, oral) as well as TYS total score. It was also successful in placing 68% of the TYS candidates in the right TYS levels although it couldn’t discriminate between C1 and C2 levels. Qualitative data suggested that test takers and instructors were sceptical about the relevance of the Turkish C-test to the spoken sections of TYS despite the strong quantitative findings. Nevertheless, test takers reported that the C-test helped them understand their need to learn more and thus would be useful in exam preparation for TYS. Overall, the findings suggest that the Turkish C-test can predict success or failure in TYS and could therefore be used as a screening test. This can help TYS candidates save time, money, and energy. This dissertation is unique in showing the development stages of a Turkish C-test step by step with language specific factors. Through an argument-based approach to validation, it provided researchers and learners with a freely available Turkish C-test that can be effectively used for research and screening purposes on the condition that findings are replicated with a follow-up study. If future researchers or practitioners wanted to use a Turkish C-test for different populations or uses, they can follow the steps and guidance stated in this dissertation while developing their own C-tests.
Date of Award23 Jun 2020
Original languageEnglish
Awarding Institution
  • The University of Bristol
SupervisorGeorge B Leckie (Supervisor) & Shelley McKeown Jones (Supervisor)

Keywords

  • language assessment
  • test development and validation
  • argument-based approach to validation
  • mixed-methods
  • Turkish language

Cite this

'