Blog

21 November 2026

Pairwise comparison beats individual scoring

An AI judge ranked 100 essays from the ELLIPSE dataset by comparing them in pairs. The ranking matched trained human raters on most pairs, and on nearly all where the raters saw a clear difference. Scoring each essay individually did much worse.