COVER: Identifiable Evaluation of Structured Routing Interventions

A routing system may choose a model, a team of agents, a tool bundle, or a complete model configuration. Exhaustively executing every legal action can be costly when the goal is only to compare two frozen routing policies. We introduce COVER, an identification-first framework that asks which actions must actually be executed to determine that policy difference while keeping the downstream execution process fixed. Under a declared linear action-value representation, a policy contrast is exactly identifiable when the difference between the policies’ expected features lies in the span of the executed action features. This yields a minimum identifying support and an explicit conditioning measure for noise amplification. Controlled experiments reduce a 220-action coalition problem to 66 actions for exact second-order table recovery and to 8 actions for exact frozen-policy contrasts. On 43 untouched MuSiQue tasks, COVER uses 9 actions per task versus 11 matched and 15 exhaustive, with all 43 contrasts within ϵ = .01; however, the content-only ablation has only three distinct table patterns, so this cohort demonstrates prospective identification under favorable natural structure rather than broad natural-table compressibility. On 42 fresh MMLU-Pro tasks over 12 hosted configurations, COVER uses 5 actions per task versus 7.24 matched and 12 exhaustive, with 38/42 contrasts within ϵ = .05 and fresh token/latency reductions of 29.9%/40.5%; charging shared calibration reduces savings to 11.6% actions, 12.8% tokens, and 14.7% latency. ToolSandbox and MMLU-Pro expose the failure boundary: a singlecheck safeguard is conservative, while a stronger residual-correction safeguard can cost more than the assumption-free baseline. COVER therefore turns routing evaluation into an auditable question about what information is actually required for the comparison.