acceptodds
Under review as a conference paper at ICLR 2027

Historical Evaluation Without a Unique Past: Identification, Complexity, and Evidence from a Real-Time Archive

Abstract

Historical evaluation often uses the archive available today to approximate the data available at a past decision. Even when the decision protocol is known, incomplete provenance can leave several historical states consistent with it. We represent this ambiguity by a set of admissible worlds. A loss, loss contrast, or winner is identified when it is invariant across that set, even if the historical state itself remains unknown. We prove that strict winner reversal is NP-complete even for one cutoff, binary availability, fixed targets, frozen linear predictors, and squared loss. Ordered revisions admit an exact cutoff quotient, while fixed-dimensional affine timing permits exact evaluation over hyperplane-arrangement faces. An audit of 38 quarterly decisions using 1,180 official ALFRED snapshots shows why the admissible set matters. When one feed age is shared across feeds and fixed across decisions, same-world comparisons certify all ten pairwise rankings on a declared four-age support; separate marginal ranges leave four unresolved. Adding intermediate daily ages exposes one reversal, and allowing feed-specific fixed ages exposes two. If ages may instead vary by decision, only four rankings remain certified. Reusing identical resolved states reduces 256 worlds with fixed feed-specific ages to 32 states and cuts model calls eightfold. These results connect historical claim identification to explicit temporal assumptions, exact computational structure, and verifiable archival evidence.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.