- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RouteSage, a self-evolving routing framework that reduces LLM agent costs by augmenting cheaper models with reusable Experiences extracted from execution histories. It introduces Capability Regions, quality-constrained routing, and validated Capability Patches. Experiments across three agent benchmarks demonstrate promising cost reductions and quality improvements.
Critical strengths
-
The paper addresses the important problem of reducing inference costs in LLM-based agent services while maintaining task quality. This problem is highly relevant to real-world deployments, where repeated model calls can lead to substantial operational costs.
-
RouteSage integrates experience reuse, quality-constrained routing, and capability expansion into a unified framework. The reported improvements across three different agent benchmarks suggest that this approach has practical potential for improving cost efficiency without retraining models.
Critical weaknesses
-
The paper does not sufficiently establish its novelty over existing routing and experience-based agent learning methods, such as FrugalGPT, Agent Workflow Memory, and ReMe. Since improving weaker models through reusable execution experiences has already been explored, the unique contribution of RouteSage remains unclear, which weakens its claimed novelty.
-
The baselines primarily focus on conventional model routing, without comparisons to experience-enhanced alternatives. Moreover, the ablation studies do not adequately isolate the contributions of key components, such as Capability Regions and patch validation. This makes it difficult to determine whether the improvements arise from the proposed framework or simply from incorporating reusable Experiences.
Questions for authors
-
How does RouteSage compare with existing routing methods equipped with the same reusable Experiences?
-
Can the authors provide additional ablations to demonstrate the necessity of Capability Regions and the proposed patch validation mechanism?
-
How does RouteSage fundamentally differ from existing cost-efficient routing and experience-based agent optimization methods? Could the authors further clarify its unique advantages and explain why existing approaches are insufficient to address the same problem?
4 comments
Dear Reviewer XtHb,
Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period ends in 48 hours, we kindly request that you review the rebuttal and confirm whether your concerns have been addressed. Thanks a lot.
Regards from the authors.
Dear Reviewer XtHb,
Thanks for your time and effort in reviewing this paper. As the author-reviewer discussion period ends in about 24 hours, we would be grateful if you could take a look at our rebuttal and let us know whether your concerns have been addressed. We are happy to clarify any remaining questions before the deadline.
Regards from the authors.
Thank you for your detailed rebuttal and additional experiments. While the additional experiments have improved the empirical evaluation, your clarification regarding novelty has not fully convinced me. In my view, the proposed framework appears to be primarily an integration of existing techniques rather than a sufficiently novel methodological contribution. Therefore, I do not believe the paper currently meets the acceptance standards of ICLR, and I have decided to maintain my original rating.
Dear Reviewer XtHb,
Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period will end in about 3 days, we gently request that you read the rebuttal and check whether your concerns are addressed. Thanks again for your time and help in improving the quality of this paper.
Regards from the authors.