Why rate when you could compare? Using the “EloChoice” package to assess pairwise comparisons of perceived physical strength

Andrew P. Clark*, Kate L. Howard, Andy T. Woods, Ian S. Penton-Voak, Christof Neumann

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)

8 Citations (Scopus)
227 Downloads (Pure)

Abstract

We introduce “EloChoice”, a package for R which uses Elo rating to assess pairwise comparisons between stimuli in order to measure perceived stimulus characteristics. To demonstrate the package and compare results from forced choice pairwise comparisons to those from more standard single stimulus rating tasks using Likert (or Likert-type) items, we investigated perceptions of physical strength from images of male bodies. The stimulus set comprised images of 82 men standing on a raised platform with minimal clothing. Strength-related anthropometrics and grip strength measurements were available for each man in the set. UK laboratory participants (Study 1) and US online participants (Study 2) viewed all images in both a Likert rating task, to collect mean Likert scores, and a pairwise comparison task, to calculate Elo, mean Elo (mElo), and Bradley-Terry scores. Within both studies, Likert, Elo and Bradley-Terry scores were closely correlated to mElo scores (all rs > 0.95), and all measures were correlated with stimulus grip strength (all rs > 0.38) and body size (all rs > 0.59). However, mElo scores were less variable than Elo scores and were hundreds of times quicker to compute than Bradley-Terry scores. Responses in pairwise comparison trials were 2/3 quicker than in Likert tasks, indicating that participants found pairwise comparisons to be easier. In addition, mElo scores generated from a data set with half the participants randomly excluded produced very comparable results to those produced with Likert scores from the full participant set, indicating that researchers require fewer participants when using pairwise comparisons.

Original languageEnglish
Article numbere0190393
JournalPLoS ONE
Volume13
Issue number1
DOIs
Publication statusPublished - 2 Jan 2018

Structured keywords

  • Cognitive Science
  • Social Cognition

Fingerprint Dive into the research topics of 'Why rate when you could compare? Using the “EloChoice” package to assess pairwise comparisons of perceived physical strength'. Together they form a unique fingerprint.

  • Cite this