UWEE Tech Report Series

How to Analyze Paired Comparison Data


UWEETR-2011-0004

Author(s):
Kristi Tsukida and Maya R. Gupta

Keywords:
Thurstone's Law, paired comparisons, Bradley-Terry model

Abstract

Thurstone's Law of Comparative Judgment provides a method to convert subjective paired comparisons into one-dimensional quality scores. Applications include judging quality of different image reconstructions, or different products, or different web search results, etc. This tutorial covers the popular Thurstone-Mosteller Case V model and the Bradley-Terry logistic variant. We describe three approaches to model- fitting: standard least-squares, maximum likelihood, and Bayesian approaches. Proofs of key results are given in the appendix. This tutorial assumes basic knowledge of random variables and probability distributions.

Download the PDF version