Precision Thresholds for Shared Evaluation Caches

Model evaluation is often staged: a library is scored once on a shared set of examples, and later reports ask for more precise comparisons without rebuilding the entire evaluation table. We ask how far such a finite cache can be reused before fresh evaluation must scale with the size of the library. We consider $h$ complete i.i.d. score vectors in $[0,1{]}^K$, arbitrary unknown cross-model dependence, and a fixed collection of mean differences that must satisfy a uniform marginal MSE target under a hard cap on fresh work. The key obstruction is residual candidate uncertainty: additional measurements of a shared reference improve every contrast, but cannot uniformly remove the uncertainty left in each candidate's finite cache. For a unit-price reference star with $M$ candidates, this yields an exact threshold $d_0(h)=1/[4(\sqrt{h}+1{)}^2]$ separating bounded, $\sqrt{M}$, and linear fresh work in $M$; matching lower bounds hold for fully adaptive acquisition, with critical work $(2+o(1))h\sqrt{M}$ and a $1/(h\sqrt{M})$ transition window. A stronger layout-wise accuracy contract changes the critical law to $\Theta (M+h\sqrt{M})$, with an integer crossover governed by $M/h^2$. For partial observations on general comparison graphs, we give an exact bounded-score reduction to edgewise union counts and asymptotically exact high-precision work limits.