acceptodds
Withdrawn from ICLR 2027

ROUTEAUDIT: Interaction-Aware Identification for Budgeted Multi-Verifier Routing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Adaptive multi-verifier systems are commonly compared through endpoint quality–cost gaps, even when the verifier catalog, availability, accounting, information filtration, or scorer changes with the policy. We formulate verifier routing as a contract-conditioned identification problem. The contract records request support, verifier catalog, realized availability, resource accounting, online filtration, and post-trace scoring; a matched route contrast changes only the policy coordinate. ROUTEAUDIT adds three measurable objects to this contract. A contract lattice averages coordinate increments over every admissible bridge order and reports the resulting attribution together with its path sensitivity. A policy-independent response tape identifies paired sequential contrasts when adaptive policies reveal different observations. For incomplete matching, request-level bounds use whichever potential outcome remains observed and give a sharp finite-population interval. The protocol commits paid observations and ledger events before the oracle join and returns an attribution certificate for each comparison. On two held-out raw-tail caches, matched static SF+SA equals the cascade, assigning the apparent gains of 0.1797 and 0.1250 over full static to the verifier-set edge. On 1,319 held-out task requests, the supplied learned and RLVR studies report quality 0.9522 and 0.9553 versus 0.9484 for matched static; the RLVR–static paired difference is with a request-paired interval and a training-seed-by-request hierarchical interval . Controlled contract factorials, bridge-order tests, and stochastic-provider replays are specified under the same certificate interface, with missing measurements kept explicit. RLVR is used as a stress test for the same identification contract rather than as its definition.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Multi-Verifier Systems, Verifier Routing, Causal Attribution, Evaluation Protocols, Budgeted Inference, Auditable AI, Sequential Decision Making

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_xn6pOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes ROUTEAUDIT, a framework for separating routing-policy effects from changes in verifier access, availability, resource accounting, and scoring. It combines contract-lattice attribution, shared response tapes, and bounds for incomplete matching to produce audit certificates.

Critical strengths
  1. The paper addresses a practically important problem: apparent routing gains may arise from changes in verifier access, budgets, or scoring rather than the routing policy itself.
  2. The evaluation reports both quality and resource metrics and distinguishes full-set quality from accepted-set quality and coverage.
Critical weaknesses
  1. The audit certificate conflates checkable conditions with external assumptions. Hash and trace checks establish record consistency, not environment stability, non-interference, or leakage-free training.
  2. C6 assumes complete, valid policy outcomes under a stable, interference-free environment, while R1–R5 assume a policy-independent, call-consistent response tape. Identification then follows almost definitionally from paired averaging, assuming away the central difficulty and making the argument nearly circular, with limited theoretical contribution.
  3. Exact contract-lattice attribution requires exponentially many contract evaluations, yet the paper provides no complexity analysis or scalable approximation with error guarantees. Restricting bridge orders changes the attribution target rather than resolving the computational burden.
  4. The main experiments emphasize routing quality and cost rather than identification accuracy. Attribution recovery, incomplete matching, and stochastic-response claims lack substantive main-text validation and uncertainty analysis.
  5. The motivating cascade always calls SF and SA, making its zero contrast with matched static largely predetermined. Full static also introduces an LF-priority decision rule, confounding verifier access with fusion logic and weakening the attribution to verifier-set changes alone.
  6. The quality of the manuscript’s writing and presentation is poor. The narrative in the 'Abstract' and 'Introduction' lacks coherence. The equation numbering is also inconsistent, as only some equations are numbered. Although the authors state that LLMs were used solely for polishing, the final text still contains clear problems in expression, indicating insufficient human review. Further, when enlarged, Figure 1 exhibits distinct signs of being AI-generated, which is inconsistent with the scope of AI usage disclosed in the authors' statement.
Questions for authors

See Weakness

Reviewer_cLRtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RouteAudit, an interaction-aware identification framework for evaluating budgeted multi-verifier routing systems. It formalizes routing comparisons through an explicit contract covering request support, verifier access, availability, resource accounting, online information, and scoring. The reported results show that apparent routing gains can arise from verifier-set differences, while RLVR achieves improved task quality and reduced resource consumption under matched evaluation conditions.

Critical strengths
  1. Well-motivated identification framework for routing evaluation. The paper explicitly separates routing-policy effects from verifier-set effects under matched evaluation contracts.
  2. Systematic and auditable evaluation protocol. The framework incorporates online information restrictions, pre-scoring trace closure, resource accounting, and identification certificates.
  3. Comprehensive evaluation across multiple dimensions. The paper considers task quality, inference cost, latency, verifier availability, and policy robustness, with additional learned-routing baselines and multi-task experiments.
Critical weaknesses
  1. Potential inconsistencies in contract-lattice attribution results. Table 43 implies an endpoint route-contrast difference of 0.0114, with positive availability-coordinate increments, whereas Table 47 reports negative availability contributions and path-wise mean contributions summing to 0.0068. Unless these tables concern distinct estimands, the results appear inconsistent with the telescoping attribution formulation in Section 3.3.
  2. Limited methodological novelty beyond established identification techniques. The permutation-averaged attribution corresponds to Shapley allocation, the response-tape construction resembles common-random-number coupling, and the incomplete-matching bounds follow standard worst-case partial-identification arguments. Although their integration is useful, the paper does not sufficiently establish what fundamentally new identification guarantees arise beyond combining these existing techniques.
  3. Insufficient empirical validation of the proposed partial-identification mechanism. Although Section 3.3 derives sharp request-level bounds, Table 48 evaluates a more conservative envelope with width 4ε rather than directly reporting the tightness and coverage of the proposed sharp intervals.
  4. Limited evidence of superiority over competitive learned-routing baselines. Although RLVR significantly improves over matched static routing, its advantages over stronger learned baselines are less convincing. In Table 39, the paired improvements over Bayesian Thompson routing (+0.0023) and cost-constrained online routing (+0.0015) both have confidence intervals containing zero. Consequently, the evidence for additional gains from RLVR over competitive routing methods remains limited.
Questions for authors
  1. Clarification of attribution consistency: Do Tables 43–44 and Table 47 use identical contract endpoints and attribution estimands? If not, please clarify their differences. Otherwise, could the authors provide a detailed reconstruction of the path-wise contributions, particularly for the V→P→A ordering?
  2. Clarification of methodological novelty: What specific identification guarantees or theoretical insights does RouteAudit provide beyond Shapley attribution, common-random-number coupling, and conventional partial-identification bounds? Clarifying this distinction would materially affect the assessment of novelty.
  3. Validation of sharp partial-identification bounds: Could the authors report the actual request-level sharp intervals, their widths, and empirical coverage under the missingness settings in Table 48? How do these results compare with the reported conservative envelope, particularly under difficulty-biased and verifier-dependent missingness?
  4. Clarification of verifier accounting and fusion: Since SA uses both small- and large-model cached responses while LF separately parses the large-model response, how are reused model outputs and generation costs distinguished from charged verifier calls? Could alternative static fusion rules materially change the reported verifier-set attribution?
Reviewer_RECtOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is extremely hard to read, to the extend that I can not even summarize the high levels of the paper.

Critical strengths

Look at the first paragraph in the abstract: "Adaptive multi-verifier systems are commonly compared through endpoint quality–cost gaps, even when the verifier catalog, availability, accounting, information filtration, or scorer changes with the policy. We formulate verifier routing as a contract-conditioned identification problem. The contract records request support, verifier catalog, realized availability, resource accounting, online filtration, and post-trace scoring; a matched route contrast changes only the policy coordinate."

I can not understand a single word , beyond the first few words. what are the following buzz words: {verifier catalog, accounting, contract, contract-conditioning, online filtration, post-trace scoring, ...}? What are all of those multi digit numbers in the abstract? Who writes abstract like that (but AI)?

This issue drags through all of the paper. This is obviously AI generated, and not properly revised.

This paper lacks the very basics of academic writing, hence, this is a strong reject.

Critical weaknesses

explained above.

Questions for authors

NO.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.