- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes RIDGE, a role-adaptive model routing framework for LLM-based multi-agent systems. Existing work normally assigns a fixed LLM to each agent role, the proposed methods trying to dynamically selects between fast and strong models at each execution step. The framework contains five modules to guide the routing decisions. Experiments on SWE-bench and WebArena show task success rates comparable to a static strong-model baseline but can significantly reduce strong-model token usage.
Critical strengths
- The paper addresses a trade-off problem between efficiency and performance in LLM-based multi-agent systems. Moving from fixed role-level model allocation to step-level routing is well motivated, as subtask difficulty can vary substantially within the same agent role.
- The experiments demonstrate that RIDGE can achieve success rates comparable to the strong-model baseline while substantially reducing strong-model token usage.
Critical weaknesses
I have three major concerns.
First, RIDGE combines several components, making the overall framework somewhat difficult to follow. It is unclear why such a complex architecture is necessary. The ablation study in Table 3 only reports failure-critical escalation rates, without showing the corresponding task success rates or computational costs. This makes it difficult to assess whether each component actually improves routing performance. Several important components, such as forensic supervision and MemoryBank, also lack direct ablations.
Second, the experiments do not adequately support the claimed capability-boundary estimation.The paper uses forensic attribution to identify failure-critical steps and treats these signals as supervision for model capability-boundary estimation. However, a step being causally responsible for task failure does not necessarily imply that executing it with a stronger model would prevent the failure. Therefore, it remains unclear whether RIDGE actually learns the capability differences between fast and strong models.
Third, the efficiency improvements are not convincing. The evaluation primarily emphasizes strong-model token savings rather than overall computational cost. In Figure 3, the total token consumption of RIDGE is actually higher than that of Force-Strong, despite the reported reduction in strong-model tokens. Without end-to-end latency or compute-cost measurements, it is difficult to determine whether RIDGE provides meaningful efficiency improvements in practice.
Questions for authors
-
Can the authors provide more comprehensive ablation studies, including task success rates and computational costs? In particular, what are the individual contributions of the 5 modules? How does RIDGE compare with simpler role-conditioned routing methods?
-
Can the authors report end-to-end latency or total computational costs, including routing overhead? Given the increased total token consumption on WebArena, how much practical improvement does RIDGE achieve?
-
How do you evaluate the impact of frequent switching between fast and strong models on inference latency, context reuse, and multi-agent execution?
-
The experiments only consider Qwen-based models. Does RIDGE generalize to cross-family model pairs?