UWEE Tech Report Series

How to Analyze Paired Comparison Data


Kristi Tsukida and Maya R. Gupta

Thurstone's Law, paired comparisons, Bradley-Terry model


Thurstone's Law of Comparative Judgment provides a method to convert subjective paired comparisons into one-dimensional quality scores. Applications include judging quality of different image reconstructions, or different products, or different web search results, etc. This tutorial covers the popular Thurstone-Mosteller Case V model and the Bradley-Terry logistic variant. We describe three approaches to model- fitting: standard least-squares, maximum likelihood, and Bayesian approaches. Proofs of key results are given in the appendix. This tutorial assumes basic knowledge of random variables and probability distributions.

Download the PDF version