Understanding effects of subjectivity in measuring chord estimation accuracy

Yizhao Ni, Matt McVicar, Raul Santos-Rodriguez, Tijl De Bie

Research output: Contribution to journalArticle (Academic Journal)peer-review

21 Citations (Scopus)


To assess the performance of an automatic chord estimation system, reference annotations are indispensable. However, owing to the complexity of music and the sometimes ambiguous harmonic structure of polyphonic music, chord annotations are inherently subjective, and as a result any derived accuracy estimates will be subjective as well. In this paper, we investigate the extent of the confounding effect of subjectivity in reference annotations. Our results show that this effect is important, and they affect different types of automatic chord estimation systems in different ways. Our results have implications for research on automatic chord estimation, but also on other fields that evaluate performance by comparing against human provided annotations that are confounded by subjectivity.

Original languageEnglish
Article number6587770
Pages (from-to)2607-2615
Number of pages9
JournalIEEE Transactions on Audio, Speech, and Language Processing
Issue number12
Publication statusPublished - 15 Nov 2013


  • automatic chord estimation
  • music information retrieval
  • sequence crowd learning
  • Subjectivity evaluation


Dive into the research topics of 'Understanding effects of subjectivity in measuring chord estimation accuracy'. Together they form a unique fingerprint.

Cite this