acceptodds
Under review as a conference paper at ICLR 2027

OMEGA: A Utility-Aware Lifecycle Co-Design for Long-Term Memory in LLM Agents

Abstract

Long-term memory systems for LLM agents face a fundamental tension: rich extraction and retrieval improve evidence coverage but incur prohibitive lifecycle costs. Controlled analyses on LoCoMo and LongMemEval reveal three design principles: (1) preserve complementary evidence without proliferating memory types; (2) decouple concise matching keys from coherent answer-bearing payloads; (3) expand retrieval coverage before folding evidence into a bounded solver context. We instantiate these principles in Omega, which combines session-aligned typed projection, key-payload separated maintenance, and bounded multi-route retrieval with late parent folding. Across three benchmarks, Omega achieves state-of-the-art performance, leading the strongest baseline by an average of 3.59 points while using 25–86% less solver context and cutting write-side calls by 81–88% on LoCoMo. Notably, Omega transfers to the held-out PersonaMem-128K, demonstrating robust cross-benchmark generalization.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Long-term Memory, Memory Lifecycle

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_tEN3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a retrieval framework for LLM agents, introducing a principle for managing memory-system complexity: "pay for on demand." Based on this principle, the authors derive three guidelines for memory writing, maintenance, and retrieval, respectively. They evaluate their proposed method, Omega, on LoCoMo, LongMemEval-S, and the held-out PersonaMem benchmark. The results show that Omega matches or outperforms strong baselines while reducing LLM calls by approximately 80%.

Critical strengths
  1. Practical motivation and reasonable design principles. The paper addresses a practical and important problem, and its guiding principle is intuitive and well-motivated. The overall analytical framework is sensible. In particular, the controlled experiments that isolate the writing, maintenance, and retrieval components provide a reasonable way to quantify their individual contributions.

  2. Insight into retrieval granularity. The authors observe that long memories suffer from signal dilution during cross-encoder reranking. This observation highlights the importance of separating document representation from the retrieval unit, a concept closely related to passage-level retrieval and document expansion in traditional information retrieval (IR).

  3. Extensive empirical evaluation. The paper provides comprehensive experimental evaluations and cost analyses, offering useful evidence of the framework's effectiveness and efficiency.

Critical weaknesses
  1. Limited technical novelty. Although the overarching principle is reasonable, it remains largely heuristic. The three proposed guidelines—compact type sets, key–payload separation, and recall-before-compression—have clear precedents in IR and retrieval-augmented generation (RAG). For instance, document expansion and doc2query have long explored generating short queries to improve the retrievability of longer documents. Similarly, late fusion and reciprocal rank fusion (RRF) are well-established IR techniques, while query rewriting has also been extensively studied.

  2. Primarily an integration of existing techniques. The main contribution lies in integrating established techniques into a unified memory lifecycle and using controlled experiments to attribute performance gains to individual components, rather than introducing fundamentally new retrieval or memory mechanisms.

  3. Venue fit. Given its emphasis on retrieval system design, empirical evaluation, and efficiency rather than methodological novelty, I believe this work would be a better fit for an information retrieval venue such as SIGIR.

Questions for authors

NA

Reviewer_bpnTOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies accuracy and resource trade-offs across memory construction, maintenance, and retrieval for LLM agents. Omega extracts episodic and factual memories at session boundaries, generates concise retrieval keys for long episodes while retaining their coherent payloads, and combines lexical and dense retrieval across query routes before folding matches into distinct parent payloads. The design is developed on LoCoMo and LongMemEval-oracle and evaluated on longer LongMemEval-S histories and the held-out PersonaMem-128K benchmark. With a shared Qwen solver, Omega reports gains of 8.00 and 2.05 points over the strongest evaluated baselines on the latter two settings, while reducing solver context. Stage-level ablations and LoCoMo write-cost measurements support the design choices.

Critical strengths
  1. Convincing evidence for the main accuracy results, including independent benchmark transfer. The Qwen comparisons share the solver and evaluation setup, and the primary PersonaMem result excludes answer choices from retrieval. Paired cluster-bootstrap intervals account for shared conversations or personas: the reported gains are +8.00 points on LongMemEval-S with a 95% interval of [+4.60, +11.60], and +2.05 on PersonaMem with [+0.24, +3.95]. The paper also correctly treats LoCoMo's smaller gain as statistically inconclusive and checks alternative judges and corrected references. These controls make the empirical case appreciably stronger than an aggregate-score comparison alone.

  2. The representation and retrieval analyses yield actionable design insight. Table 4 distinguishes children used as evidence from children used only to locate coherent parent memories, rather than simply comparing a larger store with a smaller one. The key-only design and the key-versus-payload reranking ablation support separating matching surfaces from answer-bearing content. The full-projection and full-history controls further illustrate that more available evidence can help or hurt depending on the task. The LoCoMo write-cost accounting includes child-key generation, so the reported savings over the multi-stage baselines are not obtained by omitting this maintenance expense.

Critical weaknesses
  1. The lifecycle-efficiency conclusion is broader than the measured comparison. The paper establishes lower solver-context use and substantially fewer write-side calls and input tokens on LoCoMo. However, Omega also uses query rewriting, multiple recall routes, and a 4B reranker; their query-time latency and computation are not included in the solver-context metric. Appendix B preserves each baseline's official retrieval configuration, while Omega's configuration is developed on the selected benchmarks. This is a useful comparison of released systems, but it does not show whether the advantage persists when the strongest baselines vary their context budgets under comparable resource constraints. The limitations section acknowledges that context length is only one component of cost. Since lifecycle utility is the central motivation, a joint accounting of construction, retrieval, and answering, together with a modest baseline budget sweep, is needed to establish the claimed practical trade-off.

  2. The construction study does not isolate the efficiency benefit of separate type-specific calls. Joint-3 and Joint-5 share one output budget, whereas Per-type-3 receives dedicated calls and produces approximately 2.5 times as many memory items. The observed accuracy gains therefore show that allocating more extraction capacity recovers useful evidence, but do not distinguish type separation from the increased total generation allowance. Moreover, the deployed system uses two types, while the joint-extraction comparison uses three or five; a joint episodic-plus-fact extractor is not evaluated. The paper explicitly recognizes output competition, which is helpful, but a joint two-type baseline with comparable total output capacity could materially change the accuracy-cost justification for this construction choice. This does not undermine the child-key results, but limits one of the main lifecycle recommendations.

Questions for authors
  1. Targeted validation of the efficiency claim. For Omega and the strongest relevant baselines, can you report retrieval-plus-answering latency or computation, including query rewriting and reranking, and compare a small range of solver-context budgets? Please separate one-time construction cost from per-query cost so that the trade-off can be assessed for different numbers of queries per history.

  2. Targeted construction control. How does a joint episodic-plus-fact extractor compare with the deployed two-call design when total output-token allowance is comparable? Reporting actual generated tokens, extraction cost, and downstream accuracy would clarify whether separate calls provide a benefit beyond additional generation capacity.

  3. Configuration-selection clarification. How was top-40 selected for PersonaMem without using its evaluation labels, and were the LongMemEval-S maintenance and retrieval ablations performed only after freezing the main configuration? A precise selection timeline would clarify the distinction between development evidence, context-extension evaluation, and the benchmark-family holdout.

Reviewer_mFfiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies which parts of a long-term memory system are worth the added complexity. Omega uses compact memory construction, short keys for retrieval, and fuller memory content for answering. It combines several retrieval routes and limits the context passed to the answering model. The goal is to reduce writing effort and reading context without giving up answer quality.

Critical strengths

1. The design addresses a clear difference between finding information and answering with it.
A short description can match a question well but still omit details needed for the answer. Omega uses short keys to find a memory and then supplies the fuller content, instead of asking one compressed representation to serve both purposes. This is a clear design choice with a relevant ablation. In Table 4, LongMemEval-S improves from 86.40 without child keys to 90.40 when keys are used for retrieval and duplicate parent memories are merged. The experiments help explain which added processing is useful, rather than only showing that the complete system performs well.

2. The main quality and resource results are supported across benchmarks.
On LoCoMo, Omega is statistically tied with HyperMem in quality while using about 0.6k rather than 4.3k context tokens. It scores 90.40 versus Mem0's 82.40 on LongMemEval-S, and 71.43 versus EverMemOS (A)'s 69.38 on held-out PersonaMem. The reported paired confidence intervals for the latter two differences are positive. The authors also test transfer with the main non-budget components fixed and account for the writing calls used to generate retrieval keys. I appreciate that the paper distinguishes a quality tie with lower context usage from a statistically supported quality improvement.

Critical weaknesses

The overall gains do not extend to all personalization tasks.
On PersonaMem, Omega scores 70.67 on preference evolution and 83.64 on user reasoning, below EverMemOS (A)'s 75.37 and 88.48. Providing all constructed memories in the Full Projection setting improves these categories further and raises the overall score by 3.19 points relative to Omega. This shows that the compact retrieval setting still loses useful quality on some tasks that matter for long-term user memory.

These results do not establish that retrieval errors are the only cause, since providing the full memory also changes the amount and organization of information. Still, they are a real limitation of the current system. A developer should not assume that the stronger overall average implies equally strong handling of changing preferences and user-level reasoning.

In addition, The tested models (GPT-4.1-mini backbone + GPT-4o-mini judge) in this paper are relatively outdated. It's not very clear how it works in frontier models.

Questions for authors

For the preference-evolution and user-reasoning errors, is the needed memory missing from the selected context, or is it present but not used correctly by the answering model? A small analysis of existing cases would help explain when the compact context is insufficient.

1 comment
AuthorsOpenReview ↗

## Response to Q1: When Compact Context Is Insufficient (P-Evol & U-Reas Analysis)

We thank the reviewer for this highly insightful observation. We agree that the overall average score should not obscure specific weaknesses. To address your core question-is the needed memory missing from the selected context, or is it present but not used correctly?—we conducted a manual diagnostic audit on a sampled subset of errors.

  1. Diagnostic Evidence: Retrieval Omission in the Sampled Cases

    We examined 20 diagnostic cases selected from the P-Evol and U-Reas error pools, tracing each from the source history to the constructed memory pool and the final selected context. As summarized below, retrieval omission was a prominent pattern in this sample.

    General PatternSpecific Failure PatternCases (%)Corrected by Full Projection*
    Retrieval OmissionMissing same-activity history5 (25%)5 / 5
    Retrieval OmissionMissing cross-activity history11 (55%)10 / 11
    Answer ErrorsRelevant evidence selected, but answer incorrect4 (20%)1 / 4
    Total20 (100%)16 / 20

    In this sample of 20 cases, 16 exhibited retrieval omission where the required historical evidence was already successfully extracted and stored in the memory pool, but missed by the compact selector. Manual tracing of the source history confirmed this presence. Among these 16 omission cases, 11 involved cross-activity history. The fact that Full Projection successfully corrected 15 of these 16 cases provides supplementary support that the deficit in these specific instances lies in the selection stage rather than initial construction.

  2. The Cross-Activity Boundary Challenge

    The most frequent pattern in this diagnostic sample (11 of the 20 diagnostic cases) is cross-activity retrieval omission. This occurs when the critical supporting evidence for a queried preference is embedded in the history of a different, seemingly unrelated activity. Compact selectors, which rely on surface-level semantic similarity to the current query, can struggle to bridge the logical dependencies across these activity boundaries (e.g., a query about current shopping preferences failing to retrieve a "gift-induced change in quality expectations" from a prior, distinct activity).

    Conversely, for the remaining 4 cases, the correct context was retrieved, but the model still produced an incorrect answer (answer errors). Full Projection only corrected 1 of these 4 cases, suggesting that for this subset, the issue is not a retrieval deficit.

This diagnostic explicitly identifies "improving cross-activity semantic bridging in the selector" as a concrete, high-priority direction for future system iterations.

Manuscript Revisions To ensure full transparency, we have added a dedicated diagnostic analysis in Section 4.5 and Appendix H.1 (including a summary table of these 20 cases), explicitly acknowledging the P-Evol/U-Reas deficits and documenting cross-activity evidence omissions alongside evidence-present answer errors in the diagnostic collection. Detailed records of all 20 diagnostic cases are provided in our anonymous supplementary artifact.

Comments on the paper

2 comments

Thank you for tracking down the PersonaMem errors. Of the 20 cases you examined, 16 were due to the selector missing evidence already present in memory. I'm still keeping the score at 2 points. Because under the default budget, OMEGA still lags behind EverMemOS (A) in tracking changes in user preferences and reasoning about the user.

AuthorsOpenReview ↗

Response to Further Official Comment by Reviewer mFfi: Clarification on System-level Trade-offs

Dear Reviewer mFfi,

Thank you for your response and continued engagement with our work.

Overall, Omega achieves a statistically significant performance advantage over EverMemOS (A) (71.43% vs. 69.38%), outperforming it in 4 of the 7 held-out subtasks (A-Rec, N-Sug, C-Gen, and P-Rec) and demonstrating broad effectiveness across diverse memory tasks.

We understand your concern regarding the performance gap in the two specific sub-categories (P-Evol and U-Reas) under the default budget. However, we respectfully suggest that evaluating our system solely based on absolute accuracy in these specific sub-tasks might overlook the core contribution of this paper: a holistic trade-off across performance, efficiency, and cost throughout the entire memory lifecycle.

To clarify this, we would like to highlight two key points from our detailed responses to Reviewer bpnT:

  1. Superior Lifecycle Efficiency at Default Settings: As detailed in our response to bpnT (Q1.1), Omega's default strategy introduces less than 50% of the context tokens compared to EverMemOS (A). More importantly, Omega achieves a significantly better Pareto frontier in terms of cost and latency:

Lifecycle Cost & Latency Comparison (from response to bpnT Q1.1)

MethodConsolidation (CNY/persona)Query (CNY/question)TTFT mean / p95 (s)Accuracy (%)
Omega (+G_rw)2.920.01432.57 / 3.0770.5
Omega (+G_ccr)2.920.00981.08 / 1.3272.5
EverMemOS7.240.00730.84 / 1.1066.0
EverMemOS (A)7.240.04009.75 / 11.9468.5

Specifically, compared to EverMemOS (A), Omega reduces consolidation costs by 60% (2.92 vs. 7.24 CNY), slashes query costs by up to 75% (0.0098 vs. 0.0400 CNY), and accelerates mean TTFT by 9× (1.08s vs. 9.75s). Omega is fundamentally designed as the most cost-effective and efficient memory strategy.

  1. Adjustable Performance with Maintained Efficiency: As shown in our response to bpnT (Q3), the performance in these specific subtasks improves markedly as additional context tokens are introduced, demonstrating that the observed gap is a deliberate design choice to prioritize efficiency under default settings, rather than a fundamental limitation:

Performance vs. Retrieval Budget on PersonaMem (from response to bpnT Q3)

Final retrieval budgetAvg. Tok.L-PrefP-EvolU-ReasA-RecN-SugC-GenP-RecOverall
top-20 (additional)0.8k72.7566.8682.9071.3552.1267.1485.3869.27
top-40 (primary frozen)1.5k74.3670.6783.6470.7756.5672.7783.6371.43
top-80 (additional)2.9k75.0675.0788.8576.7952.3276.5381.8772.86

By simply increasing the retrieval budget to top-80, Omega surpasses EverMemOS (A) in U-Reas (88.85 vs. 88.48) and trails by only ~ 0.3% in P-Evol (75.07 vs. 75.37). Crucially, even at top-80, Omega's average context (2.9k tokens) remains ~6% lower than EverMemOS (A)'s 3.1k tokens, and its query cost/latency stays well below those of EverMemOS (A).

Therefore, Omega's preset top-k configuration represents a deliberate, system-level trade-off: accepting a marginal accuracy difference in highly context-heavy reasoning tasks (P-Evol and U-Reas) in exchange for massive, quantifiable gains in lifecycle efficiency and cost-effectiveness. Such trade-offs are inevitable and essential for practical, scalable memory systems.

We hope this broader perspective clarifies the practical value and robustness of our design, and that our responses effectively address your concerns. We look forward to any further discussion with you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.