acceptodds
Under review as a conference paper at ICLR 2027

Auditable Ranking-Slice Redeployment under Conflicting Refresh Metrics

Abstract

Maintaining a model ranking requires deciding whether a refreshed slice can replace the evidence that produced the current order. We show that conventional refresh scores resolve this decision differently even when candidate pools, budgets, filters, and five evaluation cycles are held fixed: Paraphrase leads acceptance and Kendall's , while Adversarial-LLM produces the largest anchor–fresh gap. This metric-winner reversal motivates a redeployment contract with four distinct evidence units: candidate throughput, sampled admission validity, semantic preservation of deployed items, and ranking usefulness. SliceLedger operationalizes the contract with declared intent invariants and cue-family transformations that make each item's latent task traceable through generation, symbolic validation, expert arbitration, and external review; a blinded -item audit validates its admission decisions at precision and recall. At near-matched ( versus ), SliceLedger exceeds Adversarial-LLM by acceptance points, , and 8.2 deployed-preservation points (stratified-bootstrap CI [3.9,12.6]), reaching 91.1% preservation under a symmetric three-rater audit. Matched-time expert-guided maintenance reaches 95.1% preservation while accepting 31.2% of candidates, compared with for SliceLedger, exposing an automatic–human Pareto frontier. Removing the intent schema loses acceptance points and ; the selection remains stable on a model-disjoint panel and under a frozen five-cycle replay. Across GSM8K, SQuAD 2.0, HumanEval, and held-out MMLU, the contract turns recurring refresh into an auditable redeployment methodology and identifies a high-throughput automatic operating point with strong semantic and ranking preservation.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.