Agent-as-a-Router: Closing the Feedback Loop in Model Routing for Coding Tasks

Users of coding agents increasingly have access to several frontier Large Language Models (LLMs) that excel on different tasks, so routing each task to a suitable model matters for both quality and cost. Most prior open-source routers treat this as a one-shot classification problem whose information state is frozen after training. We argue that routing for agents is instead a streaming decision problem with verifiable feedback since gathering information matters for routing. A controlled diagnosis supports this view: supplying per-dimension performance statistics to an LLM router raises its score by 3–9 points over the same statistics with shuffled model labels, whereas scaling the router from 9B to 27B parameters or eliciting longer reasoning does not. We therefore formalize routing as a Context$\to$Action$\to$Feedback (C-A-F) loop evaluated by cumulative regret against a per-task oracle, instantiate it as ACRouter (an Orchestrator, a Verifier, and an execution-grounded Memory), and release CodeRouterBench, $\sim$10K coding tasks with per-task outcomes for 8 frontier LLMs plus a 176-task out-of-distribution (OOD) agentic-programming split. ACRouter attains the lowest cumulative regret on in-distribution streams, and component ablations show that closing the loop accounts for most of the gain over retrieval from a frozen memory. On OOD tasks, static routers trained offline fall to 24–39% resolve rate, at or below random routing. With an exact verifier, verification-driven escalation resolves 73.3% of OOD tasks, 9.1 points above the strongest single model; with deployable verifiers that do not read the grading tests, the gain falls within noise.