acceptodds
Under review as a conference paper at ICLR 2027

The Sample Cost of Accurate Model Comparison in Total Variation

Abstract

Choosing between two generative models asks for one decision rather than an estimate of either model's error. How much data can this distinction save? We study two completely known distributions on an alphabet of size , with independent evaluation data from an unknown target distribution. A classical selector achieves a factor-3 approximation to the smaller total variation distance using a number of samples independent of . We show that every fixed improvement below factor 3 requires samples in the worst case, at sufficiently small fixed additive error. The known upper bound for estimating total variation gives a matching dependence on . Thus accurate comparison and numerical distance estimation have the same domain-size complexity, even with only two simple candidate distributions. The proof preserves a reversed model ranking while matching logarithmically many moments of the target probabilities. We give a general moment construction, a direct Poisson-thinning argument, and an explicit reduction between exact-factor comparison and distance estimation.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.