Attempt-Matched Evaluation of Model-Routing Cascades in LLM Agents

Routing between language models inside an agent is motivated by an oracle: send each task to the right model and quality rises ten points or more. Operationalised as a cheap-then-strong reactive fallback cascade, the form cost-aware cascade work uses, that oracle rests on an unequal comparison: the cascade spends two attempts, the always-strong baseline one. The control that isolates routing is retrying that same strong model, and on two independent benchmark families it gives opposite answers. On ${\tau }^2$-bench the routing-specific effect is indistinguishable from zero on most datasets and significantly negative on one; on BFCL's multi-turn split it is significantly positive. An exact identity reconciles them, regrouping the same sum into a first-slot cost plus a rescue term. What differs across families is the strong model's self-rescue rate, how often it succeeds when retried on its own failure (rank correlation $-0.84$ over $11$ cells): where retrying already recovers failures, switching adds nothing. Where the capability gap is large the first-slot cost dominates instead, and that is why our one negative dataset is negative. A one-sided screen follows: every significantly positive effect we measure sits at a self-rescue rate at or below $0.182$, which one extra rollout per task estimates. A cascade oracle needs the same-model-retry control, since retry's share of the naive gain reaches $6.0$ times it. And on ${\tau }^2$ what survives is price, not quality, at matched budget and without needing a positive effect; where the effect is positive, as on BFCL, the cascade wins on both axes. Two scope limits. The confound comes from spending two attempts, so nothing here bounds a one-call pre-execution dispatch router; its selection-corrected oracle excludes zero on $1$ of $4$ datasets. And our retry is blind, so a failure-conditioned one could only push these effects down.