acceptodds
Under review as a conference paper at ICLR 2027

BitRouter: Learning Interpretable Model Routing Policies for Agentic Workflows from Deployment Feedback

Abstract

Agentic workflows involve state-dependent model choices whose combined effects are often evaluated through delayed, session-level outcomes. Building on bandit-based routing and cost-subsidy formulations, we introduce BitRouter, a framework for learning interpretable workflow routing policies under cost and quality requirements. Each complete policy is a bandit arm retained throughout a session, while explicit rules adapt model choices to workflow state. We reuse session-level evidence across policies with shared rules, accounting for uncertainty in the rules’ combined effects relative to quality-feasibility margins when deciding whether additional policy-specific evaluation is needed. Across repeated runs on 80 common-valid Terminal-Bench 2.1 tasks, an initial routing configuration achieved 81.25% task success, versus 81.56% for a fixed strong-model baseline and 76.50% for same-pool random routing. Mean nominal API cost per trial, counting accepted valid-path requests, was 40.93% lower than that of the strong-model baseline. End-to-end learning experiments show lower combined execution and verification costs than independent policy evaluation, while keeping the observed rate of selecting policies that fail to meet the quality requirement within the prescribed limit.

Primary area:
reinforcement learning
Keywords:
Large Language Model Routing, Multi-Armed Bandits, Agentic Workflows, Interpretable Policies, Evidence Sharing, Quality-Constrained Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KjeiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper under review presents a new framework (called BitRouter) for learning when a cheaper routing policy can replace a reference policy. The authors consider complete routing policies as “bandit arms”, using the session’s terminal quality and cost as evidence about the policy. Policies are organised as edits to a reference configuration, allowing evidence to be reused and shared among related policies. A new policy is only adopted when the evidence supports the required quality and cost requirements and the expected future savings can recover the extra cost of exploration. The proposed approach has been evaluated by experiments against baselines.

Critical strengths
  • The paper addresses a practical problem in the field of agentic workflows.

  • The edit representation for data exchange between related policies is promising.

  • Different types of costs are used for learning.

  • The discussion of limitations is quite solid.

Critical weaknesses
  • The empirical evaluation only uses one benchmark and a single strong-model baseline (plus the random-routing method).

  • The cost metric in the experiments only covers accepted-path nominal API cost, while there might be other relevant costs in practice.

  • There is no ablation study to analyse the impact of the different components of the overall framework.

  • A theoretical analysis of the approximations used within the proposed framework would be useful.

Questions for authors
  • Did you analyse the impact of your structural evidence sharing on the other components of your framework?

  • How well does the proposed approach generalise to different workloads or different domains?

Reviewer_vSwyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a bandit-based routing policy that defines arms as complete workflow policies and uses session outcomes as feedback.

Critical strengths

The problem statement seems sound, both in terms of the use of a bandit-based appraoch and in terms of how arms and rewards are defined.

Critical weaknesses

I found the writing to be impenetrable, and it felt like it was largely AI generated. IMO the presentation of this work clearly warrants rejection. The question I asked below is good evidence in support of this claim: I had difficulty understanding even the most basic facts about this work, namely how the headline G1 policy was created in the first place.

Questions for authors

Where does the G1 policy came from? This is crucially important b/c the performance of the G1 policy on Terminal Bench 2.1 is the main result. It looks like this policy was just applied to Terminal Bench 2.1 though as a static policy. How was it learned?

Reviewer_1WxNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BitRouter, a framework for learning interpretable model-routing policies in agentic workflows using session-level feedback. It combines bandit-based policy evaluation, shared evidence across related policies, and quality–cost constraints for policy replacement. Experiments on Terminal-Bench 2.1 demonstrate substantial cost savings while maintaining a similar observed task success rate to a strong-model baseline.

Critical strengths

The paper addresses an important practical problem in agentic workflows. Modeling complete routing policies as learning arms and distinguishing execution cost from evaluation expense provide a reasonable and interpretable framework for quality–cost optimization. The experimental methodology is relatively rigorous. The authors report repeated evaluations, bootstrap confidence intervals, detailed cost accounting, and extensive implementation and reproducibility information. The approximately 41% reduction in measured nominal API cost is a promising result.

Critical weaknesses

The experimental results provide limited evidence that the cost reductions can be achieved without compromising task performance. Although BitRouter reduces execution costs by approximately 41%, its success rate is slightly lower than that of the strong-model baseline (81.25% vs. 81.56%). The reported confidence interval is relatively wide, making it difficult to establish that the two approaches achieve comparable task success. In addition, the random-routing baseline does not fully control for differences in model usage, limiting the ability to isolate the advantages of the proposed routing strategy. The evaluation of online learning is relatively limited, involving only four candidate policies derived from two types of edits. It remains unclear whether the approach can consistently improve performance with larger policy spaces or more diverse workflows. Additional experiments demonstrating successful policy updates would strengthen the paper's main claims.

Questions for authors

Q1. Effectiveness of policy learning Could the authors provide experiments in which the learning procedure successfully identifies and adopts a better policy? How does the proposed shared-evidence acquisition strategy compare with independent policy estimation or simpler exploration strategies under the same budget? Q2. Quality–cost comparison Can the authors provide stronger evidence that task quality is preserved within a specified tolerance? How would the results change under a baseline with matched model-selection proportions and a more comprehensive cost-accounting scheme? Q3. Scalability and robustness How does BitRouter perform with larger policy libraries, more interacting configuration edits, and different task orders? Could the authors provide additional evidence showing how the learning and adoption decisions evolve as more session feedback becomes available?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.