acceptodds
Under review as a conference paper at ICLR 2027

When Routing Accuracy Misleads: Auditing Framework Selection with Certified Twins

Abstract

Many reasoning tasks require committing to a single epistemic framework before solving: blending a -value with a posterior yields incoherence, not a partial truth. We study framework routing — selecting the applicable belief-space "universe" before reasoning — and report that high routing accuracy is a poor measure of it. Auditing our own benchmark, we find that a classifier that cannot read the words, only their shape and punctuation, reaches % against a % majority baseline, and that most Ill-posed training items announced their own deficiency in a parenthetical present in % of train but % of test items. We release EpistemicBench v5, an explicitly versioned repair — v4 is untouched — that removes that marker together with the train-only augmentation tags, and discloses the leakage it does not fix. The repair works and does not buy the capability: retraining lifts ill-posed recall by pp, yet on a certified subset whose labels are machine-checkable properties of the generating parameters, our router detects ill-posed items, trained with the marker or without it. A pre-registered, unaudited replication over eight disjoint generator families repeats this at . Items within a family are near-duplicates, so these are finite-suite counts, not binomial ceilings. Benchmark accuracy on this class can therefore be a severe overestimate of epistemic competence, and we make no claim that ill-posedness detection was learned. On the repaired data we detect no effect of gate hardness: every paired 95% CI among matched M hard, soft and Bayesian gates includes zero — a failure to detect a difference, not a demonstrated equivalence. A M fine-tuned specialist beats all three and ties RoBERTa-large at its trainable budget, but it differs from them in architecture and objective at once, so that comparison is observational. EpistemicBench v5, both certified sets, the audit tooling and code will be released upon publication. Audit the benchmark before believing the number.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.