- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes ROUTEAUDIT, a framework for separating routing-policy effects from changes in verifier access, availability, resource accounting, and scoring. It combines contract-lattice attribution, shared response tapes, and bounds for incomplete matching to produce audit certificates.
Critical strengths
- The paper addresses a practically important problem: apparent routing gains may arise from changes in verifier access, budgets, or scoring rather than the routing policy itself.
- The evaluation reports both quality and resource metrics and distinguishes full-set quality from accepted-set quality and coverage.
Critical weaknesses
- The audit certificate conflates checkable conditions with external assumptions. Hash and trace checks establish record consistency, not environment stability, non-interference, or leakage-free training.
- C6 assumes complete, valid policy outcomes under a stable, interference-free environment, while R1–R5 assume a policy-independent, call-consistent response tape. Identification then follows almost definitionally from paired averaging, assuming away the central difficulty and making the argument nearly circular, with limited theoretical contribution.
- Exact contract-lattice attribution requires exponentially many contract evaluations, yet the paper provides no complexity analysis or scalable approximation with error guarantees. Restricting bridge orders changes the attribution target rather than resolving the computational burden.
- The main experiments emphasize routing quality and cost rather than identification accuracy. Attribution recovery, incomplete matching, and stochastic-response claims lack substantive main-text validation and uncertainty analysis.
- The motivating cascade always calls SF and SA, making its zero contrast with matched static largely predetermined. Full static also introduces an LF-priority decision rule, confounding verifier access with fusion logic and weakening the attribution to verifier-set changes alone.
- The quality of the manuscript’s writing and presentation is poor. The narrative in the 'Abstract' and 'Introduction' lacks coherence. The equation numbering is also inconsistent, as only some equations are numbered. Although the authors state that LLMs were used solely for polishing, the final text still contains clear problems in expression, indicating insufficient human review. Further, when enlarged, Figure 1 exhibits distinct signs of being AI-generated, which is inconsistent with the scope of AI usage disclosed in the authors' statement.
Questions for authors
See Weakness