Indices of cognitive effort in machine translation post-editing

Research output: Contribution to journalArticle (Academic Journal)peer-review

29 Citations (Scopus)
719 Downloads (Pure)


Identifying indices of effort in post-editing of machine translation can have a number of applications, including estimating machine translation quality and calculating post-editors’ pay rates. Both source-text and machine-output features as well as subjects’ traits are investigated here in view of their impact on cognitive effort, which is measured with eye tracking and a subjective scale borrowed from the field of Educational Psychology. Data is analysed with mixed-effects models, and results indicate that the semantics-based automatic evaluation metric Meteor is significantly correlated with all measures of cognitive effort considered. Smaller effects are also observed for source-text linguistic features. Further insight is provided into the role of the source text in post-editing, with results suggesting that consulting the source text is only associated with how cognitively demanding the task is perceived in the case of those with a low level of proficiency in the source language. Subjects’ working memory capacity was also taken into account and a relationship with post-editing productivity could be noticed. Scaled-up studies into the construct of working memory capacity and the use of eye tracking in models for quality estimation are suggested as future work.
Original languageEnglish
Pages (from-to)187-216
Number of pages30
JournalMachine Translation
Issue number3
Early online date21 Nov 2014
Publication statusPublished - Dec 2014


  • Post-editing
  • Cognitive effort
  • Eye tracking
  • Meteor
  • Working memory capacity


Dive into the research topics of 'Indices of cognitive effort in machine translation post-editing'. Together they form a unique fingerprint.

Cite this