DEPLOYRANK: RANKING AMBIGUITY AND CERTIFIED CHOICE UNDER MISSING CONTEXT LINKAGE
Abstract
Benchmarks may publish outcomes and all candidates' predictions for each evaluation record, yet omit the records' links to context available at deployment. We study what these releases identify and what evidence justifies model choice. For any finite number of candidates, we construct one complete-joint prediction release that preserves a strict standalone Blackwell order yet admits every contextual Blackwell order arbitrarily close to a fixed redundancy completion. Each requested order holds in one compatible world simultaneously for all regular strictly proper losses. Ranking ambiguity, however, need not prevent an -optimal choice. For frozen rules and binary context, DeployRank exactly tests zero-link decision adequacy and certifies choices through selectively disclosed row–context links. We derive exact pairwise minimum-disclosure frontiers and multi-candidate bounds, distinguishing holder-known proof size from online discovery cost. In a post-hoc minimax analysis of one protocol-frozen UCI Heart task, the release-only choice incurs nats of aligned regret at . An alignment-informed holder certifies the corrected choice with 86 of 417 links; the precommitted policy uses 278 paid links, counting quota-implied labels as free. Other audits demonstrate zero-link sufficiency and guarantee-only repair.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.