acceptodds
Under review as a conference paper at ICLR 2027

Memory-Aware Routing for Cost-Efficient Language Agents

Abstract

Language-model agents must balance the reliability of strong models against the cost of invoking them at every decision step. Retrieved experience offers another way to manage this trade-off: a past trajectory may provide the support a smaller model needs to match a stronger reference. However, memory can also mislead, and its usefulness depends on both the current state and the model that consumes it. Routing therefore requires deciding not only which model to invoke, but also which memory can make that model sufficient. We formulate this problem as cost-constrained selection over memory-model pairs and develop a two-stage framework that separates relative memory benefit from absolute routing reliability. The first stage shortlists memories by predicted alignment gain; the second uses historical outcome evidence to estimate model-specific reliability, with a shared-bias-low-rank formulation for cross-model prediction. We evaluate an evidence-grounded approximation on OfficeBench, ScienceWorld, and -bench, alongside separate component studies of sharing and shortlisting. The evaluated router achieves up to 17.8% proxy cost savings over always invoking the reference, routing 25.7% of states to smaller models at 82.0% routed precision. These findings support treating episodic memory can effectively serve as part of model allocation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Routing, Agent Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_e3SaOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers problems related to model routing, basically determining which memories should be retrieved and which models should be invoked based on those memories. The goal is basically to try and avoid invoking models at every step and to achieve some cost savings, e.g. by effectively routing to smaller models when possible.

Critical strengths

The problem of memory-conditioned model routing is interesting, and the problem formulation here makes sense.

Reasonably detailed analysis, qualitative stuff, ablations, etc. Overall the analysis is reasonably thorough.

Easier to follow and more direct in its contribution compared to most papers in my batch.

Critical weaknesses

The biggest limitation here is just that the results seem really mixed. Even as presented in the abstract, the savings seem to be not very much (to the point that it would impact the likelihood of anyone reproducing the work here). And in the main experiments section the results do not seem all that compelling, as the comparison points are more likely model ablations etc. rather than a more convincing comparison against e.g. SotA models. Based on what's reported in the experiments I think it's doubtful that the paper would inspire enough confidence that others would be likely to follow up on this work.

The technical approach seems fine though perhaps is not a particularly large intellectual contribution. The paper is perhaps more a variation of existing work on model routing with some smaller specific technical innovations, but it's hard to spot a "big idea" here.

Questions for authors

see above.

Reviewer_cZwWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies memory-aware model routing for language agents, asking whether retrieved episodic experience can enable a cheaper model to safely replace a stronger reference model at a decision state. It formulates routing as cost-constrained selection over memory–model pairs and proposes a two-stage framework that first ranks memories by predicted alignment gain and then estimates model-specific reliability for cost-aware routing. Experiments show that memory can both rescue and harm smaller models.

Critical strengths
  1. The paper highlights an important interaction between memory usage and model routing.

  2. The gain–reliability decomposition is well motivated. By distinguishing relative gains from memory retrieval and the absolute reliability of routing decisions, the framework offers a coherent way to integrate memory retrieval with cost-aware model selection, rather than handling the two components separately.

  3. The experimental setup is reasonably rigorous within the chosen scope. The factorial rollouts vary memory and model choice while keeping the decision state and reference fixed, and the evaluation further includes task-disjoint splits, multiple routing baselines, and targeted ablations.

Critical weaknesses
  1. The evaluation does not sufficiently support the paper's broader claims about cost-efficient language agents. The experiments measure next-action agreement with a reference model rather than end-to-end task success. The “safety” is measured by agreement with a fixed GPT-5.4 reference action.

  2. The algorithmic novelty beyond the memory-conditioned routing formulation seems moderate. The implementation largely combines standard components, while the empirical benefit of the proposed structure is relatively small and not statistically conclusive.

Questions for authors
  1. End-to-end validity. Can the authors evaluate full agent episodes and report actual task success against realized inference cost, rather than relying primarily on next-action agreement with the reference model?

  2. Method necessity. How much of the overall improvement comes specifically from the gain–reliability decomposition and the shared low-rank, model-specific predictor?

Reviewer_WA5eOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies memory-aware model routing for LLM agents. The authors propose a two-stage framework that shortlists episodic memories by predicted gain and estimates model-specific reliability using historical outcomes. Experiments on OfficeBench, ScienceWorld, and TauBench examine the trade-off between reference-action agreement and inference cost.

Critical strengths
  • This work addresses a meaningful problem: reducing inference cost while preserving the reliability of language agents.

  • Since different models may benefit differently from the same retrieved memory, selecting the memory and model jointly is a reasonable idea.

Critical weaknesses
  • The writing could be improved. The method section is hard to follow, and the presentation could be clearer and more concise. Dense notation and repeated formulations of related objectives in Eqs. (1), (2), (5), and (13) make the core procedure difficult to identify. The notation also introduces ambiguity: CiC_i (Line 129) denotes the state, while Ci(k)C_i(k) (Line 140) denotes cost around Eq. (1). In addition, the reproducibility statement on page 10 refers to a “structural transition dataset,” a “docking protocol,” and a “reachability criterion,” which are not clearly connected to this work. The final sentences of the abstract and conclusion also contain grammatical errors, such as “support treating episodic memory can effectively serve…”. The paper needs a clearer account of the training and inference procedures and more careful editing. What does “execution adaptation” in Section 5.4 refer to, and how is it implemented?

  • The practical benefits are limited. Table 1 reports cost savings of only 4.7% on OfficeBench and 3.2% on TauBench, with most states still falling back to the reference model. The paper should discuss whether these limited savings justify the additional complexity of the memory and routing mechanisms. Moreover, quality is evaluated through agreement with reference actions at fixed decision states. The paper does not evaluate complete task success under actual routed execution, leaving the effects of subsequent state changes and accumulated errors unclear. This limits the evidence for the method’s benefits in practical agent execution.

Questions for authors
  • What does “evidence-grounded approximation” in the abstract mean? Does it mean that the evaluated method approximates the framework described in Section 4? If so, which components are approximated or replaced, and how does the experimental implementation differ from the proposed framework?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.