Abstract
Identifying indices of effort in post-editing of machine translation can have a number of applications, including estimating machine translation quality and calculating post-editors’ pay rates. Both source-text and machine-output features as well as subjects’ traits are investigated here in view of their impact on cognitive effort, which is measured with eye tracking and a subjective scale borrowed from the field of Educational Psychology. Data is analysed with mixed-effects models, and results indicate that the semantics-based automatic evaluation metric Meteor is significantly correlated with all measures of cognitive effort considered. Smaller effects are also observed for source-text linguistic features. Further insight is provided into the role of the source text in post-editing, with results suggesting that consulting the source text is only associated with how cognitively demanding the task is perceived in the case of those with a low level of proficiency in the source language. Subjects’ working memory capacity was also taken into account and a relationship with post-editing productivity could be noticed. Scaled-up studies into the construct of working memory capacity and the use of eye tracking in models for quality estimation are suggested as future work.
Original language | English |
---|---|
Pages (from-to) | 187-216 |
Number of pages | 30 |
Journal | Machine Translation |
Volume | 28 |
Issue number | 3 |
Early online date | 21 Nov 2014 |
DOIs | |
Publication status | Published - Dec 2014 |
Keywords
- Post-editing
- Cognitive effort
- Eye tracking
- Meteor
- Working memory capacity