The impact of visual cues on second language listening comprehension process and performance
: An eye-tracking study

  • Suh Keong Kwon

Student thesis: Doctoral ThesisDoctor of Philosophy (PhD)


Video is now a prevalent resource for language learning but its use in language assessment remains tentative. This study explores how inclusion of visual cues in a listening comprehension test has an impact on second language learners’ test-taking process and test performance. Previous studies have explored the effect of visual cues in a listening test, mainly by comparing the scores between audio-only and video-based tests but few studies have yet revealed how and what specific visual cues are viewed and used by candidates when completing the listening tasks.
This mixed-method study addressed two overarching research questions that investigated the effect of visual cues on candidates’ 1) listening test performance and 2) cognitive and metacognitive processes. Under these questions, five specific sub-questions were further addressed that investigated the extent to which the effect of visual cues varies by the candidates’ proficiency or type of prompts. To answer these questions, this study employed an eye-tracking technology as a main data collection method and examined the extent to which second language (L2) learners’ individual variability in processing multimodal input varies by the given test condition (audio-only vs. video), prompt types (dialogue vs. lecture), and their baseline proficiency levels (low-mid-high). In total 117 EFL learners in Korea were involved in the eye-tracking listening test and their test scores, eye-movement data, and responses to the cognitive and metacognitive strategy use questionnaire and stimulated-recall interviews were carefully analysed. For this, this present study followed Creswell and Plano Clark (2007)’s classification of triangulation model, ‘validating quantitative data,’ to compensate for any limitation that a single method may have.
Findings demonstrated that candidates in the video condition performed better than the candidates in the audio-only condition, and this was more evident in the lecture items than in the dialogue items. In terms of the test-takers’ eye-movements, it was found that the candidates in the video group viewed the visual cues (speaker and PPT slide) for a greater proportion of time, longer, and more frequently than the item components (stem, answer key, distractors). Mixed results were found in terms of the difference in eye-movements between the two types of prompts (dialogue and lecture), and there was no significant difference in eye-movements among the three proficiency levels of candidate. In addition, a few patterns of showing either positive or negative relationships between the eye-movements on certain areas-of-interests (AOIs) in the test and test scores were found. Specifically, we found looking at the PPT slides in the lecture items and the answer key was predicted to increase the test scores while looking at the distractors had a debilitating effect. Findings of the stimulated-recall interview confirmed that most test-takers found the visual cues provided in the video helpful and did not find the tasks in such condition cognitively more demanding. Evidence drawn from the interviews also contributed to explain the pattern of eye-movements that candidates showed while completing the video-mediated listening comprehension test. In summary, this present study has effectively corroborated meaningful original findings to support the inconclusive debates over the validity of inclusion of visual cues in a listening comprehension test.
Date of Award1 Oct 2019
Original languageEnglish
Awarding Institution
  • The University of Bristol
SupervisorGuoxing Yu (Supervisor) & William J Browne (Supervisor)

Cite this