Evaluating Evidence Under Change: A Categorical Semantics for Decision-Relative Reuse
Abstract
Evaluation results often outlive the procedures that produced them, and when tasks, verifiers, environments or reporting rules change, what an old archive still warrants depends on the question asked of it. We give a finite categorical semantics in which evaluation settings and changes form a category, typed compatibility relations carry retained evidence to possible target records and each downstream use observes them through a decision profile. Tracking record multiplicity and use dependence separates exact reuse from decision-sufficient multiplicity, unresolved disagreement, empty compatibility and invalid comparison. Since changes com- pose, alternative routes can be compared through the profile sets they induce, and any discrepancy yields a finite witness of decision-relevant non-transport. For a fixed family of future observations we characterize the coarsest quotient preserving them and when compressed interfaces preserve propagation exactly. In exploratory case studies, a coarse CI archive still settles release but loses triage detail, eICU report contracts agree at current thresholds yet separate at every prespecified split- decile, and a corrected DeepSWE v1-to-v1.1 re-evaluation finds no qualifying same-state use-specific sufficiency witness. The semantics shows what historical evidence still supports for a declared use, where it falls short and which distinctions an archive must keep.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.