The impact of Japanese EFL test-takers’ listening comprehension on their notetaking and summarization performance in two distinct listen-to-summarize tasks: an eye-tracking mixed-methods study

Recently, there has been a growing trend towards assessing test-takers’ ability to summarize different sources (e.g., lectures, texts, and charts). While integrated tasks often focus on read-to-write tasks, listen-to-write tasks are increasingly used to evaluate summarization skills. This study aimed to deepen our understanding of how listening comprehension influences note-taking strategies and summarization performance in two distinct listen-to-summarize tasks. In this mixed-methods study, we used a wearable eye-tracking device (Tobii Pro Glasses 3, ver. 2020) to monitor eye movements of 20 participants during listen-to-summarize tasks with different levels of task difficulty. Participants completed a listening comprehension quiz following each listen-to-summarize task. Quantitative and qualitative data were collected through the listening strategy and test-taking questionnaire, semistructured interviews, notes, and summaries to compare the different strategies they applied to the two listen-to-summarize tasks. The findings revealed that the more challenging listen-to-summarize task required greater cognitive effort to recall information for summary production, resulting in longer pauses on the screen. Ordinal Logistic Regression analysis showed that only the quiz result of the more challenging listen-to-summarize task could predict the performance in the listen-to-summarize tasks. The interview results revealed the challenges test-takers faced in completing the tasks, confirming that listening comprehension influenced notetaking strategies, which in turn affected summary writing. The eye-tracking results suggested that test-takers adapted different strategies depending on the task difficulty. These findings offer insights into how listening comprehension mediates notetaking and summary writing. For example, participants who take more structured and well-organized notes tend to demonstrate stronger listening comprehension. These results also provide implications for improving teaching methods, such as guiding instructors to explicitly teach notetaking strategies for identifying main ideas and to scaffold listening tasks with previewing activities to improve comprehension.