- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces routed continual learning, a system where an LLM router picks both a memory mechanism and a cluster of past experience for each new task an agent handles, instead of using one fixed memory setup for everything. They test it on a mix of CL-Bench and banking tasks across four models, and it beats both "no memory" and "one shared memory" baselines while being much cheaper to run.
Critical strengths
I really appreciated the setting that has been overlooked so far. The proposed approach is very simple and works well with several models on the adopted benchmark.
Critical weaknesses
- Single, author-constructed benchmark. All results rely on one benchmark the authors built themselves (CL-Bench's six tasks interleaved with τ³-Bench banking instances), so it's unclear whether the routing architecture's gains generalize beyond this specific constructed benchmark. This matters because three of the five memory systems (Hierarchical Regression, Hierarchical Discrete-Event State, Persistent-Entity State) are purpose-built for CL-Bench's structured-prediction tasks. On a stream drawn from other agentic benchmarks cited in the paper itself (e.g. WebArena, SWE-Bench-Verified, AppWorld, Mind2Web), Routed CL would most likely collapse to choosing between ICL and ReasoningBank only. The claim that "memory-system selection adds task-specific gains" is therefore demonstrated only on tasks engineered to make that claim true. This is compounded by CL-Bench itself being only an arXiv preprint rather than an established, peer-reviewed benchmark, which makes anchoring the entire evaluation on it more problematic.
- Router is purely prompt-based. System performance hinges entirely on the router's prompt, provided in the appendix, and the task descriptions it's given appear detailed enough to make routing comparatively easy on this particular benchmark. Given how central the router is to the whole architecture, it would strengthen the paper to measure sensitivity to less detailed or less informative prompts with a robustness/ablation study on prompt specificity.
- Limited conceptual novelty (less critical). Framing continual-learning memory as a routing problem over a library of heterogeneous representations is not new (cf. MemAgent, MESA, ALMA), and context injection via a prompt-level hook is also well established (MemGPT, Reflexion, ACE, ReasoningBank, Dynamic Cheatsheet). The paper should position its contribution more precisely against this line of work rather than presenting the routing framing itself as the core novelty.
Questions for authors
questions:
- in fig 2, I wonder if price is the right metric, as prices will change over time and across countries making the experiments not reproducible
suggestions:
- Calling it continual learning is questionable, as the system is prompt-based. In this sense, there is no learning, only memory management. It's important to clarify this point.
- include sonnet at it obtains surprisingly good results in CL-Bench
- I suggest basing evaluation on open-source models to achieve reproducibility
- It's not clear what part of the pipeline needs a hig/expensive model. Ablations combining different models. For example, taking the best model, opus 5 in this benchmark, we could change the LLM only for the routing to see if the choice of the model has impact on routing. We need to understand whether we need for the task itself or the routing.
We consider this construction of a composite benchmark a strength of the paper. A typical benchmark on which models or agentic harnesses are evaluated consists of independent tasks; the literature on continuous learning observes that in practice tasks arrive sequentially, which creates opportunities for amortizing effort and accumulating knowledge. We bring the evaluation closer to reality by interleaving tasks from disjoint streams of continuous learning challenge sets.
Not only is this setup more practically relevant, it advances our understanding of the original benchmark. Asawa et al., who introduced the CL Bench, report that “naive ICL outperforms systems dedicated to memory management.” In our evaluation, this advantage disappears when tasks from different streams are interleaved. These results suggest that the original finding is conditional on the task order preferencing in-context learning, rather than reflecting a general phenomenon.
We augment CL-Bench with τ³-Bench Banking and report combined results together with per-stream metrics (Appendix C.5). We use these benchmarks to empirically validate our proposal, rather than presenting benchmark performance improvements as a distinct contribution.
We follow the established terminology that considers in-context learning (ICL) a form of learning (see, for instance, https://arxiv.org/pdf/2301.00234, https://arxiv.org/pdf/2410.23042).
The price is a combined metric, aggregating unevenly priced input, cache read, cache write, and output tokens (reported in Table 7). Figure 2 is most useful for intra-model comparisons, as the relative costs of different models do fluctuate.
We appreciate the suggestions that provide useful directions for further exploration.
Our results suggest that stronger models make better routing decisions. Opus 5 and Sol match Ground-truth assignment, in both cluster and memory system, for 91% and 89% of source tasks across rollouts, compared with 71% for Opus 4.7 and 66% for Gemini 3.7 Flash (Table 3).
Since each routing decision is a choice among a handful of options (an existing cluster, or a new cluster with a selected memory system), a specialized model (such as Jev or Gemini Decision) may be enough to serve as the router.