acceptodds
Under review as a conference paper at ICLR 2027

RIDGE: Role-Adaptive Routing for LLM-based Multi-Agent Systems

Abstract

LLM-based multi-agent systems (MASs) can support complex iterative workflows. However, they commonly face a challenging trade-off between performance and cost. Existing systems typically assign a fixed model to each agent role, ignoring variations in subtask difficulty. Furthermore, current LLM routers are mainly designed for single-turn conversations and rely on paired model outputs that are difficult to obtain in multi-agent workflows. To address these issues, we propose RIDGE, a lightweight router that dynamically identifies model capability boundaries during task execution. It combines online Critic evaluations with forensic attribution of failed tasks to distill a strong model’s assessment of execution quality, failure risk, and failure-critical steps into a role-conditioned hypernetwork. The hypernetwork generates role-specific prediction heads that separately estimate the potential execution states of fast and strong models from the current task semantics. An online MemoryBank further supports routing decisions. On SWE-bench Lite and WebArena-Verified, RIDGE matches the success rate of a static strong-model baseline while reducing strong-model token usage by up to 41.6%. It also routes 76.19% of failure-critical steps to the strong model, compared with 21.99% for OmniRouter and 15.06% for RouteLLM.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Multi-Agent Systems, Model Routing, Dynamic Model Selection, Cost-Efficient Inference, Role-Adaptive Routing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CgwfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RIDGE, a role-adaptive model routing framework for LLM-based multi-agent systems. Existing work normally assigns a fixed LLM to each agent role, the proposed methods trying to dynamically selects between fast and strong models at each execution step. The framework contains five modules to guide the routing decisions. Experiments on SWE-bench and WebArena show task success rates comparable to a static strong-model baseline but can significantly reduce strong-model token usage.

Critical strengths
  1. The paper addresses a trade-off problem between efficiency and performance in LLM-based multi-agent systems. Moving from fixed role-level model allocation to step-level routing is well motivated, as subtask difficulty can vary substantially within the same agent role.
  2. The experiments demonstrate that RIDGE can achieve success rates comparable to the strong-model baseline while substantially reducing strong-model token usage.
Critical weaknesses

I have three major concerns.

First, RIDGE combines several components, making the overall framework somewhat difficult to follow. It is unclear why such a complex architecture is necessary. The ablation study in Table 3 only reports failure-critical escalation rates, without showing the corresponding task success rates or computational costs. This makes it difficult to assess whether each component actually improves routing performance. Several important components, such as forensic supervision and MemoryBank, also lack direct ablations.

Second, the experiments do not adequately support the claimed capability-boundary estimation.The paper uses forensic attribution to identify failure-critical steps and treats these signals as supervision for model capability-boundary estimation. However, a step being causally responsible for task failure does not necessarily imply that executing it with a stronger model would prevent the failure. Therefore, it remains unclear whether RIDGE actually learns the capability differences between fast and strong models.

Third, the efficiency improvements are not convincing. The evaluation primarily emphasizes strong-model token savings rather than overall computational cost. In Figure 3, the total token consumption of RIDGE is actually higher than that of Force-Strong, despite the reported reduction in strong-model tokens. Without end-to-end latency or compute-cost measurements, it is difficult to determine whether RIDGE provides meaningful efficiency improvements in practice.

Questions for authors
  1. Can the authors provide more comprehensive ablation studies, including task success rates and computational costs? In particular, what are the individual contributions of the 5 modules? How does RIDGE compare with simpler role-conditioned routing methods?

  2. Can the authors report end-to-end latency or total computational costs, including routing overhead? Given the increased total token consumption on WebArena, how much practical improvement does RIDGE achieve?

  3. How do you evaluate the impact of frequent switching between fast and strong models on inference latency, context reuse, and multi-agent execution?

  4. The experiments only consider Qwen-based models. Does RIDGE generalize to cross-family model pairs?

Reviewer_8GdHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RIDGE, a step-level router for LLM-based multi-agent systems that chooses between a fast model (Qwen3-14B) and a strong model (Qwen3.5-35B-A3B-AWQ) for each routable call. A role-conditioned hypernetwork generates role-specific heads that predict a six-dimensional execution-risk state and uncertainty for both models. The heads are supervised by a mix of online Critic decisions and offline forensic attribution from Qwen3.7-Max. Routing then uses per-role thresholds, same-role/same-model semantic support, and a MemoryBank. On role-structured versions of SWE-bench Lite and WebArena-Verified, the paper reports success comparable to Force-Strong with 35–42% fewer strong-model tokens, and higher escalation of failure-critical steps than OmniRouter and RouteLLM.

Critical strengths
  1. The role-structured SWE-bench Lite and WebArena-Verified workflows keep the official tasks and evaluators, and the solvability check with DeepSeek-V4-Flash (53.4% / 69.7%) shows the decomposition does not break the tasks. If released, these workflows could be useful to others studying routing in MAS.

  2. The forensic attribution protocol penalizes only causally responsible steps using a but-for test, rather than spreading the task failure over the whole trajectory. This is a more reasonable step-level supervision signal than binary task outcome.

Critical weaknesses
  1. The cost premise is not supported by the experimental setup. The “strong” model is Qwen3.5-35B-A3B, an MoE with about 3B active parameters, while the “fast” model is a dense 14B model. Per token, the strong model is likely no more expensive, and possibly cheaper, than the fast one. So “strong-model tokens saved” does not translate into any real cost or latency saving. The paper reports only token counts. On WebArena (Figure 3b), RIDGE actually uses more total tokens than Force-Strong (807k vs. 765k). Without wall-clock time, FLOPs, or a setup where the strong model is clearly more expensive, the main efficiency claim does not hold.

  2. The routing results do not show step-level discrimination, and the key baseline is missing. In Table 1, RIDGE escalates 76.19% of failure-critical steps but also 73.81% of non-critical rejected steps and 27.6% of successful steps. It barely separates the two kinds of rejected steps. The comparison with OmniRouter and RouteLLM is not made at a matched escalation budget. Those methods escalate much less overall (88–97% strong-token savings on WebArena), so a higher recall on failure-critical steps is expected from simply escalating more. Figures 6 and 7 suggest the router mostly learns a role prior. Navigator goes to strong 64–80% of the time at every difficulty level, PreGround about 5%, and changing only the role label moves the rate from 4.3% to 85.7%. This looks close to the fixed role-level assignment in Figure 2 that the paper argues against. Yet a static “Navigator/Critic on strong, others on fast” baseline is never compared. MasRouter, the most directly related MAS routing method, is discussed but not evaluated.

Questions for authors
  1. The routing mechanism may rely on workflow history rather than semantics alone. Section 3.5.2 says routing uses only the current role and semantics. However, four of the six state dimensions in sis_i (sierrs^{err}_i, sirejects^{reject}_i, sireviewers^{reviewer}_i, sidepths^{depth}_i) describe the workflow history rather than the current request. If the semantic core excludes this history, how can these dimensions be predicted from the request alone? If retries and reviewer feedback remain visible in the request, the router may simply be learning to escalate after a rejection, which is close to the FrugalGPT-style cascade the paper criticizes. Relatedly, for the "other rejected steps" in Table 1, is the escalation decision made on the original attempt or on the retry?

  2. The reward design lacks positive supervision and introduces conflicting objectives. From Figure 1 and Appendix B, Critic approval gives riC=0r^C_i = 0, rejection gives riC=−1r^C_i = -1, and forensic severity satisfies riF∈[−1,−0.4]r^F_i \in [-1, -0.4]. This means ρi≤0\rho_i \le 0 always, and the boundary target qi=(1−ρi)/2q_i = (1-\rho_i)/2 is always ≥0.5\ge 0.5, so steps that went well and steps with no evidence both map to qi=0.5q_i = 0.5. Is there any positive supervision indicating that the fast model is sufficient? In addition, the state loss (Eq. 7) pulls each predicted dimension toward the observed state, while the boundary loss (Eq. 9) pulls the mean of the predicted dimensions toward qiq_i, and these two targets generally disagree. Which one dominates in practice?

  3. The evaluation does not establish whether escalation improves outcomes. Tables 1 and 2 measure how often failure-critical steps are escalated, but not whether escalating them changes the outcome. A step attributed as a culprit under MfM_f does not imply that MsM_s would handle it correctly. If the failure-critical steps are re-executed with MsM_s, how often do the step or the task recover? This is the quantity the capability-boundary claim actually rests on.

Reviewer_Zm6wOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates role-adaptive routing for LLM-based multi-agent systems.

Critical strengths

None

Critical weaknesses

The overall quality of the submission falls substantially below the standards expected for ICLR, making it difficult to conduct a meaningful technical review in its current form. The paper requires fundamental improvements before its technical contributions can be properly assessed.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.