Bayesian Methods for Intelligent Task Assignment in Crowdsourcing Systems

Edwin Simpson, Stephen Roberts

Research output: Chapter in Book/Report/Conference proceedingChapter in a book

46 Downloads (Pure)

Abstract

In many decision-making scenarios, it is necessary to aggregate information from a number of different agents, be they people, sensors or computer systems. Each agent may have complementary analysis skills or access to different information, and their reliability may vary greatly. An example is using crowdsourcing to employ multiple human workers to perform analytical tasks. This chapter presents an information-theoretic approach to selecting informative decision-making agents, assigning them to specific tasks and combining their responses using a Bayesian method. For settings in which the agents are paid to undertake tasks, we introduce an automated algorithm for selecting a cohort of agents (workers) to complete informative tasks, hiring new members of the cohort and identifying those members whose services are no longer needed. We demonstrate empirically how our intelligent task assignment approach improves the accuracy of combined decisions while requiring fewer responses from the crowd.
Original languageEnglish
Title of host publicationScalable Decision Making
Subtitle of host publication Uncertainty, Imperfection,Deliberation
EditorsTatiana Guy, Miroslav Kárný, David Wolpert
PublisherSpringer
Chapter1
Pages1-32
Number of pages33
ISBN (Electronic)978-3-319-15144-1
ISBN (Print)978-3-319-15143-4
DOIs
Publication statusE-pub ahead of print - 10 Feb 2015

Publication series

Name Studies in Computational Intelligence
PublisherSpringer, Cham
Volume538
ISSN (Print)1860-949X
ISSN (Electronic)1860-9503

Keywords

  • Combine Decision
  • Confusion Matrice
  • Search Query
  • Latent Dirichlet Allocation
  • Confusion Matrix

Fingerprint

Dive into the research topics of 'Bayesian Methods for Intelligent Task Assignment in Crowdsourcing Systems'. Together they form a unique fingerprint.

Cite this