acceptodds
Desk rejected at ICLR 2027

Promotion is Rent-or-Buy: Exact Ledgers and Competitive Limits for Memory Consolidation in Continual Agents

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

A continual learning agent must decide when to move an item from its memory into its model weights. Current systems make this choice by heuristic, with no guarantees. We cast the decision as a rent-or-buy problem: keeping an item in memory costs a small rent on every query, while promoting it into the weights costs a one-time price. The classical answer is the break-even rule (rent until the total rent paid equals the price, then buy). Promotion, however, has a side effect absent from the classical problem, which we call crowding: each promoted item makes the shared classifier slightly worse at separating nearby concepts. Our guarantees depend on how crowding is charged. If it is charged once, at the moment of promotion, the break-even rule stays within a constant factor of the optimal schedule. If it is charged on every query, as in deployed systems, no promote-only policy achieves a bounded competitive ratio, even with just two items. Allowing the agent to demote items back out of the weights restores a bounded ratio. We test these predictions in six pre-registered experiments on synthetic data and on features from real vision and text datasets, and we report the cases where the results went against our hypotheses.

Primary area:
learning theory
Keywords:
continual learning, lifelong learning, catastrophic forgetting, memory consolidation, online algorithms, competitive analysis, rent-or-buy, theory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UUwTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work studies when and how knowledge should be consolidated from external memory into model parameters. It formulates memory consolidation as an online rent-or-buy problem and provides theoretical results on promotion decisions, interference costs, and memory allocation. The analysis offers useful insights into the trade-offs involved in knowledge consolidation.

Critical strengths
  1. This work addresses an important question for pretrained models: when and how should external knowledge be consolidated into model parameters?
  2. This paper is clearly written and logically structured. The theoretical analysis is presented in a coherent manner.
Critical weaknesses
  1. The theoretical analysis primarily focuses on Gaussian classifiers with frozen features rather than sequential models. In sequential models, such as LLMs, knowledge interference can affect subsequent generation steps and lead to more complex interactions. It remains unclear whether the theoretical conclusions extend to these settings.
  2. The theoretical framework allows consolidated knowledge to be demoted at a specified cost. However, demotion in pretrained models may require model editing or unlearning, which can introduce additional interference and affect previously acquired knowledge. These effects are not explicitly modeled, limiting the practical applicability of the theoretical guarantees.
  3. The analysis relies on idealized assumptions, including Gaussian class distributions, shared covariance, and frozen feature representations. These assumptions may not hold in practical continual learning systems. Although the paper evaluates some deviations from these assumptions, the generalizability of the derived results to more complex models remains unclear.
Questions for authors

Some knowledge may become outdated or invalid as the environment changes. In such cases, previously consolidated knowledge may no longer be beneficial and could even become harmful. How does the proposed framework account for changes in knowledge validity over time, beyond changes in query frequency? Can the framework support updating or replacing outdated consolidated knowledge?

Reviewer_YEVKOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

The paper studies when a learning agent should move an item from its memory into its model weights, framing it as a "rent or buy" problem: keeping the item in memory costs a little on every query, promoting it into the weights costs a one-time price, and a promoted item also makes the classes already in the weights slightly worse ("crowding"). Main theory: if crowding is charged once, the classical buy-at-break-even rule keeps a constant-factor worst-case guarantee; if crowding is charged on every query (which the authors say is how deployed systems work), no policy that only promotes can have any constant-factor guarantee, even with two items, unless the agent is also allowed to demote items back out. Experiments: six pre-registered test programs on synthetic data and on frozen features (from off-the-shelf encoders like ViT-B/16, CLIP, ResNet-50) of 13 datasets, comparing the paper's own decision rules against each other.

Critical strengths
  1. Theorem 2 is a real result: with per-query crowding, no promote-only policy (deterministic or randomized) can have any constant-factor worst-case guarantee, even with just two items. The once-vs-per-query framing is the paper's genuine contribution.
  2. The experimental hygiene is unusual and honest: the failure tests were registered before the runs with fixed pass/fail rules, every loss is reported as a loss, and scripts regenerate every number. I would like to see this practice more often.
Critical weaknesses
  1. There is no comparison against any agent, memory, or RAG system. The whole pipeline uses off-the-shelf frozen encoders (ViT-B/16, CLIP, ResNet-50, sentence encoders) with simple heads, and every experimental comparison is between the paper's own decision rules (break-even, its demotion heuristic, the work-function run, random selection, uniform allocation). The systems the paper invokes as motivation - MemGPT, Mem0, A-MEM, RAG-style retrieval, replay-based continual learning like iCaRL - appear only in the related-work section and the bibliography; not one is run, and there is no end-to-end agent or retrieval task. The 13 "real datasets" are used only as feature sources, not as benchmarks where anyone else's numbers exist. The encoders themselves also skew old for a 2026 paper about LLM agent memory: the anchor setting is ViT-B/16 (2021), the list runs down through Swin-B and CLIP (2021) to ResNet-50 (2016), and the text side uses MiniLM and MPNet (2020); the newest entries are DINOv2 and BGE (2023-24), and no LLM-based embedder or post-2024 model appears anywhere. So the practical claim (deployed systems should make promotion reversible) is never checked against any deployed or competing system, nor on any current model.
  2. Apart from Theorem 2, most of the content is old material in new words, and the paper says so itself: Theorem 1 is "the textbook argument applied item by item" (their words in Section 1); the demotion result is a classical memory-management algorithm applied to this setting with a guarantee that is already 14x at two items and doubles with every added item, so not deployable; the stocking rule is a century-old inventory formula; the estimation constant dates from 1963. And the model is far from the systems used as motivation: the paper itself concedes its formulas "do not directly transfer to LLM agent architectures without adaptation", yet LLM agents are the opening pitch.
Questions for authors
  1. Why is there no comparison with any existing memory or retrieval system, even a small one (e.g. an iCaRL-style buffer, a RAG pipeline, or Mem0/MemGPT-style memory)? And why does no experiment use an LLM or any encoder newer than DINOv2/BGE, given the motivation is LLM agents?
  2. Is there any measurement from a real deployed system showing that crowding is really paid per query? If not, what does the theory say about practice?
  3. The introduction promises six tests lost; the main text enumerates about four. Where are the rest, and can the full list be in the main text?
  4. Test F-7: the cross-entropy head's interval [0.684, 0.716] sits mostly below your registered band [0.7, 1.3] and only escapes firing because the rule requires the whole interval outside. Is that not a real miss?
  5. What does Fig 1b's right panel add? All lines are flat, and the one-way curve overlaps the bound line; would one panel with a linear-scale axis make the point better?
1 comment

Thank you for the long reply. It basically covers my questions: the full list of the six lost tests, the F-7 explanation under the registered rule, the plan to redraw Figure 1b, I accept these. The larger experiments, the iCaRL buffer arm, the new embedder, the Pythia-160M run, are all planned for the revision. I will value them when I see them, but they were not part of the submission I reviewed. This is why my score moves only a little.

My main remaining problem is the readability. The paper borrows its core concepts from another field, rent-or-buy, competitive ratios, crowding, and applies them to memory consolidation. Borrowing ideas is fine, but then the paper owes the reader one framework figure that shows the whole setting in a single picture, who pays rent, who pays the one-time price, where crowding enters the cost. There is no such figure, so the reader has to assemble the model from text and formulas. The writing is also not academic enough, especially the style of the headings, and with so many formulas the main innovation, the contrast between Theorem 1 and Theorem 2, does not stand out, I had to dig for it.

One small thing: in Figure 2c the fonts of the axis labels are inconsistent with the other panels. This is simple, easy to find and easy to fix, exactly the kind of thing a careful pass should catch. Please also unify the figure and table styles in the revision.

With the rebuttal covering most of my questions I raise my score from 1 to 2. At the same time I lower my presentation score from 2 to 1. These two moves go together: the rating goes up because my questions were answered, the presentation goes down because, rereading the paper for this discussion, it is still difficult to follow, for the reasons above. My hope is that the authors revise the manuscript to make it easier to understand, with a framework figure and consistent figures.

Reviewer_PWv4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies when a continual learning agent should promote items from memory into its model weights by framing the decision as a rent-or-buy problem.

Critical strengths

None

Critical weaknesses

I find this paper very hard to read. Large parts of the text read like writing generated by AI and the logic is often disjointed. AI can certainly help with theoretical proofs, but a reasonable approach would be for the authors to understand the material themselves and rewrite the AI-generated exposition into something readers can follow. Otherwise, the authors' contribution is close to nonexistent.

Questions for authors

Please see Weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.