β3-IRT: A New Item Response Model and its Applications

Yu Chen, Telmo M Silva Filho, Ricardo B. C. Prudêncio, Tom Diethe, Peter Flach

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

70 Downloads (Pure)

Abstract

Item Response Theory (IRT) aims to assess latent abilities of respondents based on the correctness of their answers in aptitude test items with different difficulty levels. In this paper, we propose the β3-IRT model, which models continuous responses and can generate a much enriched family of Item Characteristic Curves. In experiments we applied the proposed model to data from an online exam platform, and show our model outperforms a more standard 2PL-ND model on all datasets. Furthermore, we show how to apply β3-IRT to assess the ability of machine learning classifiers.This novel application results in a new metric for evaluating the quality of the classifier’s probability estimates, based on the inferred difficulty and discrimination of data instances.
Original languageEnglish
Title of host publicationProceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS 2019)
Subtitle of host publicationApril 16-18, 2019, Naha, Okinawa, Japan
EditorsKamalika Chaudhuri, Masashi Sugiyama
PublisherProceedings of Machine Learning Research
Pages1013-1021
Number of pages9
Publication statusPublished - 10 Mar 2019

Publication series

NameProceedings of Machine Learning Research
Volume89
ISSN (Print)2640-3498

Fingerprint

Dive into the research topics of '<i>β</i><sup>3</sup>-IRT: A New Item Response Model and its Applications'. Together they form a unique fingerprint.

Cite this