acceptodds
Under review as a conference paper at ICLR 2027

FOCUS: Managing Memory Retrieval Space with Reversible Forgetting in LLM Agents

Abstract

Long term memory enables LLM agents to retain information across interactions and reuse it in later responses. As interactions accumulate, memory stores grow, increasing the amount of information available for retrieval. However, the expanding retrieval space allows weakly related memories to compete with relevant evidence, which makes it harder to identify the information needed for each query. Removing memories can reduce this retrieval space, yet memories that have not been useful for an extended period may become relevant again in the future. To address this problem, we introduce FOCUS, a long term memory framework that uses a reversible memory lifecycle to control retrieval space growth. FOCUS assigns an activation value to each memory and organizes stored memories into an Active Set and a Dormant Set, with activation governing transitions between the two sets. Active memories participate directly in query matching, while Dormant memories remain stored in groups whose summaries are matched to the query to select relevant groups and guide retrieval of their full memory entries. Extensive experiments on LoCoMo and Memora with Qwen3-30B-A3B-Instruct and DeepSeek-V4-Flash show that FOCUS significantly improves long term memory performance over competing methods while maintaining the smallest retrieval space. Notably, in the longest Memora setting, FOCUS maintains a small retrieval space that is only 18.28% of all stored memories after 1,750 sessions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Long Term Memory, Memory Management, Retrieval Space, Reversible Forgetting

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_M6yZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates forgetting mechanisms for the memory of LLM agents. Long-term memory accumulates over the course of a conversation. However, the retrieval of weakly relevant memories can interfere with access to truly useful evidence. Simply deleting memories is problematic, as information that might prove useful in the future could be lost irretrievably.

The paper introduces FOCUS, a reversible forgetting mechanism that partitions memory into an "Active Set" and a "Dormant Set." Each memory entry is assigned an activation value that decays naturally over time. If an entry is retrieved and subsequently utilized, its activation value is reset to its maximum. If it is not used, it is assigned a temporary, progressively diminishing activation value. When an entry’s activation value falls below a specific threshold, it is moved to the Dormant Set. Following each response generation, the LLM evaluates whether the retrieved memories actually contributed to the response. Other components of the framework are based on LightMem.

Experiments conducted on the LoCoMo and Memora long-term conversational memory benchmarks, using the Qwen3-30B and DeepSeek-V4-Flash models, demonstrate that FOCUS outperforms all baselines in terms of accuracy. Ablation studies reveal that removing any component, such as lifecycle management, attribution-based evaluation, reactivation, or specific operational choices, leads to a decline in performance. While the computational cost of maintaining memory is higher than that of LightMem, it remains lower than that of other baselines.

Critical strengths
  1. The problem is clearly defined, and the design which decouples the retention of information from its participation in retrieval is sound. FOCUS distinguishes between whether a memory is stored and whether it is directly utilized during retrieval. This approach is both simple and versatile, it can be integrated into other memory systems and is highly practical for long-running, personal-assistant-style agents.

  2. The system performs exceptionally well in long-conversation scenarios, supported by robust statistical reporting. Experiments were conducted across two benchmarks with the longest configuration in Memora reaching an average of approximately 31,000 conversational turns and evaluated using two different backbones alongside five representative baselines, ensuring comprehensive coverage. The reported results represent the average of three independent runs, with the appendix providing standard deviations and bootstrap confidence intervals.

  3. The engineering design prioritizes efficiency and cost-effectiveness. Group summaries are updated in batches to avoid frequent LLM calls, while the use of two distinct thresholds prevents memories from oscillating back and forth between different sets. These design choices clearly reflect a consideration for real-world deployment scenarios.

Critical weaknesses
  1. The claims regarding novelty are overstated. The paper repeatedly asserts that FOCUS is the first long-term memory framework to manage the retrieval space through reversible memory state transitions. However, several cited works already feature similar mechanisms. For instance, MemoryBank employs a forgetting curve with intensity refreshed upon retrieval, while MemoryOS migrates items between hierarchical layers based on their "heat" (popularity/relevance). Furthermore, MemOS includes an archival state within the memory lifecycle from which items can be restored. The construction component draws from LightMem, and the selection of operations follows Mem0. The genuine innovations lie in distinguishing between updates triggered by active usage versus those resulting from simple retrieval, as well as a two-stage retrieval process that utilizes dual thresholds and groups dormant memories into summaries.

  2. The primary mechanism proposed in the paper lacks direct empirical validation. There is no direct evidence for the core claim of "reversibility", specifically, no analysis are provided on how many dormant memories are subsequently reactivated, whether these reactivated memories are actually utilized in generating correct responses, nor is there a comparison against a variant where such memories are simply deleted. In the ablation study, the w/o Reactivation condition still retains the retrieval of dormant memories, therefore it does not effectively test the distinction between reversible and irreversible processes. Furthermore, the accuracy of the attribution mechanism which relies entirely on the LLM to determine which memories were utilized remains unverified, even though the entire activation mechanism hinges on these judgments. Additionally, the problem identified in the motivation that weakly relevant memories compete with useful evidence is never directly measured. The paper fails to report metrics such as retrieval precision or recall, leaving it unclear whether FOCUS actually succeeds in mitigating such interference.

If the two points above can be clearly clarified, I might be inclined to accept.

Questions for authors
  1. Could you provide the results for the hard deletion variant?
  2. What proportion of dormant memories are reactivated, and of those reactivated, how many are actually put to use?
3 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

We appreciate your careful reading and comments on the contribution and empirical validation of FOCUS. We clarify the novelty claim and present results on reactivation and hard deletion, followed by a human audit of memory use attribution and an analysis of earlier evidence recovery.

W1. Concerns on novelty.

A: Thanks. We agree that the broad “first” claim overstates our contribution to some extent. In this paper, our main contribution is exactly reversible retrieval space management that balances a smaller direct retrieval space with continued access to individual atomic memory entries. Although the proposed FOCUS is somewhat similar to MemoryBank, MemoryOS, and MemOS in memory state management, it has large distinctions from these related methods. Specifically, MemoryBank uses probabilistic forgetting that can delete memories; MemoryOS can evict mid-term segments; MemOS supports archival restoration, but its documented Archived state is excluded from ordinary inference by default and accessed through special retrieval or backtracking. In contrast, FOCUS retains complete ormant entries within the normal retrieval process through group summaries, while allowing them to return to the Active Set. We will narrow the “first” claim and discuss these distinctions in the revised manuscript.

Furthermore, the proposed framework has two main parts: memory extraction and memory management. For memory extraction, we directly adopt LightMem’s buffering and topic segmentation for extraction and draw on Mem0’s explicit operations to incorporate extracted facts into the memory store, without any modifications. In this paper, our novelty actually lies in memory management mechanism, i.e., exploring the lifecycle and retrieval participation of stored entries, by combining activation-driven Active/Dormant transitions, memory use attribution, and retrieval through group summaries. We will revise the manuscript to emphasize our novelty in memory management but not in memory extraction.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

W2. Concerns on empirical validation.

A: Thanks for the suggestion. We complement the existing ablations with direct measurements of reactivation and retrieval, a hard deletion comparison, a human audit of memory use attribution, and an analysis of earlier evidence recovery.

2.1 Reactivation and retrieval. We first measured how often Dormant entries returned to the Active Set and how many returned following actual use. With Qwen as the backbone on Memora Quarterly, we tracked entries marked Dormant at session 500 through session 1500. Of these entries, 36.32% returned to the Active Set at least once. Among the reactivated entries, 47.60% returned following attributed use, corresponding to 17.29% of the original Dormant entries.

To examine whether relevant memories remained retrievable after entering the Dormant Set, we tracked the marked relevant entries during QA evaluation after memory construction. Specifically, we measured source coverage as the percentage whose groups were selected through summary matching, and final recall as the percentage that reached the answering context.

BenchmarkEligible questionsSource coverage (%)Final recall (%)
LoCoMo73694.7078.83
Quarterly17588.9553.73

These results show that FOCUS can recover relevant information through group summaries, even in Quarterly histories spanning nearly 2,000 sessions. However, memory entries differ across methods, so we retain end-to-end Accuracy and FAMA as the primary metrics for system comparison.

2.2 Hard deletion. To test whether retaining forgotten entries benefits later queries, we compared FOCUS with a hard deletion variant that deletes memory entries when their activation meets the forgetting condition, using Qwen on both benchmarks.

MethodLoCoMo AccuracyQuarterly FAMA
Hard deletion31.5513.47
FOCUS75.9738.50

According to the above table, hard deletion exhibits a substantial performance drop relative to FOCUS. This shows that, once deleted, forgotten information is unavailable to later queries, limiting the system’s ability to answer questions that depend on it. We will include these results in the ablation study section of the revised manuscript.

2.3 Use attribution audit. To assess the reliability of memory use attribution, we collected 92 judgments during memory construction, using 12 to establish annotation criteria and the remaining 80 for human auditing. Treating human labels as the reference and “used” as the positive class, precision is computed as TP/(TP+FP) and recall as TP/(TP+FN), where TP denotes correct “used” predictions, FP denotes predictions of “used” for memories labeled unused, and FN denotes predictions of “unused” for memories labeled used.

BackbonePrecision (%)Recall (%)
DeepSeek83.685.9
Qwen78.375.8

The human audit indicates that use attribution (with both backbones) provides informative but imperfect feedback. Although memories associated with FN judgments may receive only temporary activation and those associated with FP judgments may enter or remain in the Active Set, subsequent feedback from contextually related interactions can help limit their effects on memory management.

2.4 Recovery of earlier evidence. We further examined whether earlier information remains accessible as subsequent interactions accumulate. With Qwen as the backbone on Memora Quarterly, we marked questions and relevant memory entries whose evidence appeared before session 500, then continued memory construction through session 1500. We report FAMA at three checkpoints and recall of the marked entries at session 1500.

MethodFAMA: session 500FAMA: session 1000FAMA: session 1500Recall: session 1500 (%)
Mem027.4724.2322.1819.16
LightMem22.0716.5115.1218.84
FOCUS29.9031.2928.5138.66

FOCUS achieves higher FAMA at every checkpoint and higher final recall. Together, these results show that FOCUS preserves access to earlier evidence as history grows, helping sustain answer quality over long conversations.

Q1. Could you provide the results for the hard deletion variant?

A: Thanks. We have explained it in our response to Weakness 2.2.

Q2. What proportion of dormant memories are reactivated, and of those reactivated, how many are actually put to use?

A: Thanks. We have explained it in our response to Weakness 2.1.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

Thank you again for your thoughtful and constructive review. We have provided detailed responses to your comments and updated the manuscript accordingly.

As the rebuttal deadline (November 18, 11:59 PM AoE) is approaching, we would like to kindly follow up and ask whether our responses have addressed your questions and concerns.

We would greatly appreciate any further feedback you may have.

Best regards,
The Authors

Reviewer_JempOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FOCUS manages long-term memory through Active and Dormant sets. Active memories are searched directly, while Dormant memories remain accessible through group summaries. An activation mechanism uses time and attributed memory use to control transitions between these sets.

Critical strengths

I like the distinction between retaining information and keeping it directly searchable. This addresses a practical problem without permanently discarding potentially useful facts. The improvements across two benchmarks and two backbones are encouraging. I also appreciate the ablations, repeated runs, and clustered confidence intervals, which strengthen the empirical results.

Critical weaknesses

I have four main concerns:

  1. I am not convinced that the experiments isolate the benefit of the proposed activation mechanism. Removing lifecycle management also changes the retrieval structure and potentially the memory incorporation decisions. Comparisons with static hierarchical retrieval and a simple recency-based policy, using the same constructed memories, would help establish the contribution.

  2. The smaller “retrieval space” does not establish lower overall retrieval cost. The metric counts Active entries and Dormant summaries but excludes searching within selected Dormant groups. Moreover, on Memora Quarterly, FOCUS uses approximately 6.3× LightMem’s construction and maintenance tokens and 9.1× its LLM calls. I would like measured retrieval latency and a clearer accuracy–cost comparison.

  3. The central promise of recovering long-unused information is not directly tested. A fact may remain stored but become difficult to retrieve if its details are omitted from the group summary. The reactivation ablation is useful, but a controlled delayed-recall experiment would provide stronger evidence.

  4. I found the feedback protocol unclear. The paper should specify what constitutes a management step, how attribution operates during history ingestion, and whether answering one evaluation question changes the memory state used for subsequent questions. These details matter because the method adapts through usage.

Overall, I find the system promising and the reported gains credible. However, I currently lean toward weak reject because the specific contribution of reversible forgetting, its recovery reliability, and its practical efficiency remain insufficiently established.

Questions for authors

Please refer to weakness section

3 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

Thank you for the detailed review and questions about the evidence supporting FOCUS. Below, we present controlled comparisons of management policies, report practical costs, examine recovery of earlier information, and clarify the feedback and evaluation protocol.

W1: Concern on isolating the lifecycle contribution.

A: Thanks for your advice. To distinguish the contribution of lifecycle management from differences in memory content, we first compared FOCUS with Recency and Static while keeping memory entries and versions identical. Specifically, Recency assigns states according to the most recent write time, whereas Static retrieves all entries through fixed group summaries without lifecycle transitions. Although the stored content keeps the same in this comparison, retrieval also supplies candidates for memory incorporation during construction, meaning that the policy can affect subsequent updates. We therefore additionally rebuilt memory from the same histories under each policy to make extra comparison. Both comparisons used Qwen, with read-only evaluation QA.

SettingMethodLoCoMo AccuracyQuarterly FAMA
Fixed memoriesRecency74.4233.24
Fixed memoriesStatic73.6129.97
Rebuilt from historyRecency67.3232.68
Rebuilt from historyStatic71.6933.19
-FOCUS75.9738.50

FOCUS performs better in both settings. The improvements over Recency and Static with fixed memories shows that the gains persist when stored content is held constant, while the improvements over Recency and Static rebuilt from history further support applying its management policy throughout construction. We will include both comparisons in the revised manuscript to clarify the contribution of lifecycle management.

W2: Concern on retrieval cost.

A: Good suggestion. Compared with LightMem, which places greater emphasis on reducing construction and maintenance costs, FOCUS focuses on improving long-term memory through reversible management of retrieval space. Accordingly, we control retrieval space to improve access to relevant memories, without claiming lower retrieval costs. To assess the associated costs, we compared retrieval and answer generation across methods after memory construction, using Qwen on Memora Quarterly. Retrieval includes summary matching and search within Dormant groups, while generation is timed separately. QA input and output tokens are cumulative totals in millions (M).

MethodFAMARetrieval (s)Generation (s)QA input (M)QA output (M)
NativeRAG27.720.0352.4361.1280.079
Mem033.930.0543.5330.9830.119
MemoryOS35.720.1194.3412.5360.097
A-MEM34.400.0633.1805.7510.091
LightMem29.440.0603.4320.8940.101
FOCUS38.500.0943.1321.0580.064

According to the above table, FOCUS achieves the highest FAMA while maintaining retrieval/generation latency and QA token consumption comparable to competing methods. Together with the construction costs reported in Table 2(b) of the main paper, FOCUS is shown to achieve a favorable trade-off, improving memory performance while keeping construction and retrieval costs manageable.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

W3: Concern on recovering Dormant information.

A: Thanks. We examined memory recovery in two stages: retrieval of relevant Dormant entries during evaluation and continued access to earlier evidence as memory construction progresses. First, with Qwen, we marked the memory entries in the Dormant Set that were relevant to each evaluation question and tracked whether they were retrieved during QA. Based on this tracking, we measured source coverage as the percentage whose groups were selected through summary matching, and final recall as the percentage that reached the answering context.

BenchmarkEligible questionsSource coverage (%)Final recall (%)
LoCoMo73694.7078.83
Quarterly17588.9553.73

These results show that FOCUS can recover relevant information through group summaries, even in Quarterly histories spanning nearly 2,000 sessions. However, memory entries differ across methods, so we retain end-to-end Accuracy and FAMA as the primary metrics for system comparison.

Furthermore, we examined whether earlier information remains accessible as subsequent interactions accumulate. With Qwen as the backbone on Memora Quarterly, we marked questions and relevant memory entries whose evidence appeared before session 500, then continued memory construction through session 1500. We report FAMA at three checkpoints and recall of the marked entries at session 1500.

MethodFAMA: session 500FAMA: session 1000FAMA: session 1500Recall: session 1500 (%)
Mem027.4724.2322.1819.16
LightMem22.0716.5115.1218.84
FOCUS29.9031.2928.5138.66

FOCUS achieves higher FAMA at every checkpoint and higher final recall. Together, these results show that FOCUS preserves access to earlier evidence as history grows, helping sustain answer quality over long conversations.

W4: Concern on feedback and evaluation protocol.

A: Sorry for the confusion. FOCUS collects lifecycle feedback during memory construction and keeps memory fixed during evaluation QA. Specifically, FOCUS processes each dialogue batch after topic segmentation as one management step, indexed by tt. Within each step, it retrieves historical memories using the same configuration as evaluation QA and uses them as candidates for memory incorporation. It then assesses their use against the current batch through memory use attribution and updates activation values and lifecycle states accordingly. Thus, feedback comes from processing the dialogue history, while evaluation questions access the resulting memory without changing it. We will clarify this protocol in the manuscript.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

Thank you again for your thoughtful and constructive review. We have provided detailed responses to your comments and updated the manuscript accordingly.

As the rebuttal deadline (November 18, 11:59 PM AoE) is approaching, we would like to kindly follow up and ask whether our responses have addressed your questions and concerns.

We would greatly appreciate any further feedback you may have.

Best regards,
The Authors

Reviewer_uHpNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FOCUS introduces a reversible lifecycle for long-term agent memory. Rather than deleting information that has become less useful, it moves memories between Active and Dormant sets based on activation decay and estimated use in generated responses. Active memories are retrieved directly, while Dormant memories are accessed through group summaries and within-group search. Previously dormant information can subsequently regain direct retrieval access. The framework combines this lifecycle with buffered extraction and explicit memory-update operations. Experiments on LoCoMo and Memora, using two language-model backbones, show improved task performance and a smaller first-stage retrieval index. The evaluation includes component ablations, sensitivity analysis, repeated runs, and paired statistical comparisons.

Critical strengths
  1. The framework addresses a practical limitation of growing memory stores without relying on permanent deletion.

  2. The empirical improvements are consistent and supported by more than point estimates.

Critical weaknesses
  1. The practical efficiency benefit remains unclear because the main metric measures only the first-stage index. FOCUS counts Active entries and Dormant summaries as retrieval units, although accessing a selected Dormant group still requires searching its individual memories. The reported reduction therefore does not directly measure total retrieval work, latency, or storage consumption. Moreover, Table 2 shows substantially higher construction and maintenance costs than LightMem: 286.22K versus 31.60K LLM calls on Memora Quarterly. Without end-to-end latency and amortized cost measurements, it is difficult to determine when the smaller index compensates for this additional overhead.

  2. The experiments do not fully isolate the proposed lifecycle from hierarchical retrieval. The “w/o Lifecycle” ablation makes all entries directly searchable, removing both dynamic state transitions and the two-stage access structure. Its performance decline therefore cannot be attributed exclusively to the activation policy.

  3. The evaluation protocol is underspecified for a method that adapts to previous queries and responses. The paper states that histories are processed chronologically, but does not clearly define the management-step schedule or explain how retrieval and usage attribution are obtained during ingestion of prerecorded conversations. It is also unclear whether evaluation questions share an evolving memory state or begin from independent snapshots.

Questions for authors
  1. What constitutes one lifecycle-management step, and how are retrieval and attribution signals collected during history ingestion? Are memory states reset between evaluation questions? If not, how does changing question order affect performance?

  2. What are the total retrieval latency, number of examined memory vectors, and storage requirements, including within-group retrieval? At what query volume do savings offset the additional construction and maintenance calls relative to LightMem?

  3. How does FOCUS compare with a static grouped index and a simple recency-based lifecycle under the same extraction, update, summary, and retrieval settings? What activation rule is used when attribution is removed?

  4. What is the routing recall for relevant Dormant entries, particularly after long inactivity or topic shifts? How accurate is the usage-attribution model, and can mistaken response content cause obsolete memories to remain active?

  5. Can Memory Presence Accuracy and Forgetting Absence Accuracy be reported separately? This would clarify whether the FAMA improvements primarily reflect better recall of valid information or reduced use of obsolete information.

3 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/3)

Thank you for the careful questions. Below, we clarify practical costs, compare memory management policies, and explain the construction and evaluation protocol.

W1 & Q2. Concerns on retrieval, storage, and maintenance cost.

A: Thanks. Compared with LightMem, which places greater emphasis on reducing construction and maintenance costs, FOCUS focuses on improving long-term memory through reversible management of retrieval space. To assess the associated costs, we measured retrieval and generation separately with Qwen on Memora Quarterly after memory construction. Retrieval includes summary matching and within-group search. QA tokens are cumulative totals in millions (M).

MethodFAMARetrieval (s)Generation (s)QA input (M)QA output (M)
NativeRAG27.720.0352.4361.1280.079
Mem033.930.0543.5330.9830.119
MemoryOS35.720.1194.3412.5360.097
A-MEM34.400.0633.1805.7510.091
LightMem29.440.0603.4320.8940.101
FOCUS38.500.0943.1321.0580.064

According to the above table, FOCUS achieves the highest FAMA while maintaining retrieval/generation latency and QA token consumption comparable to competing methods. Together with the construction costs reported in Table 2(b) of the main paper, the retrieval and QA measurements demonstrate a favorable trade-off between memory performance and construction and retrieval costs.

Regarding retrieval space, Figure 3 compares the maximum first-stage retrieval space across methods. To account for additional search within selected Dormant groups, we report the number of entries searched relative to first-stage retrieval units:

BenchmarkWithin-group entries / first-stage units (%)
LoCoMo17.75
Memora Weekly48.36
Memora Monthly23.85
Memora Quarterly12.13

The ratios show that this additional search involves fewer entries than first-stage matching across all four settings, particularly on Quarterly. Furthermore, to compare storage costs, we measured the space required for memory text, summaries, and metadata in memory stores constructed with Qwen as the backbone. The results are reported below in MB:

MethodLoCoMoWeeklyMonthlyQuarterly
NativeRAG0.2590.3621.4134.650
Mem00.2700.4660.8391.677
MemoryOS0.7051.0843.83512.322
A-MEM0.4720.7632.9669.733
LightMem0.4970.4891.9435.242
FOCUS0.5850.5932.3897.831

FOCUS stores complete entries alongside summaries and metadata, requiring more storage than LightMem but less than A-MEM and MemoryOS on Quarterly. Combined with the retrieval measurements above, these storage results demonstrate the additional costs associated with FOCUS’s gains in memory performance.

W2 & Q3. Concerns on ablation study.

A: Thanks for the suggestion. To compare lifecycle management with static grouping and recency, we rebuilt memory from the same dialogue histories under each policy using Qwen. Specifically, Static retrieves all entries through fixed group summaries without lifecycle transitions, whereas Recency assigns states according to the most recent write time. Each policy operated throughout memory construction, and evaluation QA was read-only.

MethodLoCoMo AccuracyQuarterly FAMA
Static71.6933.19
Recency67.3232.68
FOCUS75.9738.50

FOCUS outperforms both alternatives on both benchmarks, supporting the benefit of its memory management policy over static grouped retrieval and state assignment based on the most recent write time. We will include these comparisons in the revised manuscript.

In addition, the w/o Attribution ablation treats retrieval as use and refreshes activation whenever a memory is retrieved, whereas memory use attribution provides more selective feedback based on actual use.

W3 & Q1. Concerns on lifecycle-management step and evaluation protocol.

A: Sorry for the confusion. FOCUS collects lifecycle feedback during memory construction and keeps memory fixed during evaluation QA. Specifically, FOCUS processes each dialogue batch after topic segmentation as one management step, indexed by tt. Within each step, it retrieves historical memories using the same configuration as evaluation QA and uses them as candidates for memory incorporation. It then assesses their use against the current batch through memory use attribution and updates activation values and lifecycle states accordingly. Thus, lifecycle feedback comes from the dialogue history, while evaluation QA leaves memory unchanged, eliminating the need for resets between questions and preventing question order from affecting subsequent memory states. We will clarify this protocol in the manuscript.

AuthorsOpenReview ↗

Official Comment by Authors (2/3)

Q4. Concerns on dormant routing and use attribution.

A: Thanks for raising these points. We assess access to Dormant memories first, then examine the reliability of memory use attribution and the effects of incorrect feedback.

We examined memory recovery in two stages: retrieval of relevant Dormant entries during evaluation and continued access to earlier evidence as memory construction progresses. First, with Qwen, we marked the memory entries in the Dormant Set that were relevant to each evaluation question and tracked whether they were retrieved during QA. Based on this tracking, we measured source coverage as the percentage whose groups were selected through summary matching, and final recall as the percentage that reached the answering context.

BenchmarkEligible questionsSource coverage (%)Final recall (%)
LoCoMo73694.7078.83
Quarterly17588.9553.73

These results show that FOCUS can recover relevant information through group summaries, even in Quarterly histories spanning nearly 2,000 sessions. However, memory entries differ across methods, so we retain end-to-end Accuracy and FAMA as the primary metrics for system comparison.

Furthermore, we examined whether earlier information remains accessible as subsequent interactions accumulate. With Qwen as the backbone on Memora Quarterly, we marked questions and relevant memory entries whose evidence appeared before session 500, then continued memory construction through session 1500. We report FAMA at three checkpoints and recall of the marked entries at session 1500.

MethodFAMA: session 500FAMA: session 1000FAMA: session 1500Recall: session 1500 (%)
Mem027.4724.2322.1819.16
LightMem22.0716.5115.1218.84
FOCUS29.9031.2928.5138.66

FOCUS achieves higher FAMA at every checkpoint and higher final recall. Together, these results show that FOCUS preserves access to earlier evidence as history grows, helping sustain answer quality over long conversations.

In addition to these retrieval analyses, we examined the feedback used to manage memories. To assess the reliability of memory use attribution, we collected 92 judgments during memory construction, using 12 to establish annotation criteria and the remaining 80 for human auditing. Treating human labels as the reference and “used” as the positive class, precision is computed as TP/(TP+FP) and recall as TP/(TP+FN), where TP denotes correct “used” predictions, FP denotes predictions of “used” for memories labeled unused, and FN denotes predictions of “unused” for memories labeled used.

BackbonePrecision (%)Recall (%)
DeepSeek83.685.9
Qwen78.375.8

The human audit indicates that use attribution (with both backbones) provides informative but imperfect feedback. Although memories associated with FN judgments may receive only temporary activation and those associated with FP judgments may enter or remain in the Active Set, subsequent feedback from contextually related interactions can help limit their effects on memory management.

Additionally, regarding “can mistaken response content cause obsolete memories to remain active?”, this does not occur during evaluation, as evaluation QA does not change memory states.

AuthorsOpenReview ↗

Official Comment by Authors (3/3)

Q5. Can Memory Presence Accuracy and Forgetting Absence Accuracy be reported separately?

A: Good suggestion. To distinguish retention of valid information from avoidance of obsolete information, we report memory presence accuracy (MP) and forgetting absence accuracy (FA) separately alongside FAMA.

Qwen Backbone

MethodWeekly MPWeekly FAWeekly FAMAMonthly MPMonthly FAMonthly FAMAQuarterly MPQuarterly FAQuarterly FAMA
NativeRAG60.1367.1135.6753.3774.6429.3555.1786.9627.72
A-MEM63.2670.4839.6152.2579.6431.2554.2982.4834.40
LightMem58.0460.5541.8848.1375.4229.6144.8381.7429.44
Mem064.9367.1945.7753.0078.7535.6452.0283.2233.93
MemoryOS65.5565.6242.8751.8780.0333.9950.3183.5035.72
FOCUS72.2369.8051.8259.2481.7240.6057.1987.3838.50

DeepSeek Backbone

MethodWeekly MPWeekly FAWeekly FAMAMonthly MPMonthly FAMonthly FAMAQuarterly MPQuarterly FAQuarterly FAMA
NativeRAG62.0082.0335.3154.1280.5430.1956.5182.8933.70
A-MEM66.8172.2741.5158.2476.0634.0359.1576.9637.45
LightMem66.6050.0043.7248.3176.3130.4946.2877.0528.00
Mem075.3764.4556.4360.8677.2144.5550.1675.2833.55
MemoryOS65.7657.0339.9450.0075.8036.3246.1279.0229.74
FOCUS78.0872.6661.9868.3578.0852.5059.3679.8839.63

FOCUS achieves the highest MP across all settings with both backbones while maintaining competitive FA. Notably, high FA accompanied by low MP can indicate limited memory recall, highlighting the need to assess both metrics together.

Reviewer_aXU7OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces FOCUS, a memory framework that disentangles active and dormant memories in order to control the retrieval space in direct retrieval. Specifically, FOCUS keeps frequently-used memories in active memory, which will be searched individually, and move those memories not commonly used to dormant, which organized by semantic-group summaries. In empirical results, FOCUS improves LoCoMo and Memora noticably compared with the best baseline.

Critical strengths
  1. The method design is an meaningful integration of common memory management methods and move beyond in details. The summary-driven retrieval, transferrable between active and dormant memories, and attribution for use are reasonable and sensible. Though not complicated, I think it's a sensible idea to distinguish "retrieved" from "actually used", since former might introduce some unused memories that merely semantically relevant, but not useful. Also, introducing memory lifecycle and transfer between active and dormant instead of relying on deletion is a novelty that simulates human's memory, which is convincing and natural.

  2. The evaluation and verification are overall solid. On top of normal evaluation with multiple models and benchmarks, the direct retrieval space size is also illustrated across model and compared with dormant memory size, which shows FOCUS's advantages, while the benchmark evaluation also doesn't show any degradation on performance. Thus, it's relatively convincing about its memory efficiency, capacity, and advantages.

Critical weaknesses
  1. My main concern is that when hitting memories that is not in active memory, would the accuracy and latency at retrieval have clear disadvantage from dormant memories? Also, since this method is online, we don't yet establish its latency at updating the memory.

  2. Also, there are some precedents works on multi-tier memory (like MemoryOS, which is included in baseline comparison), and other multi-decay rate in different memory section, like FadeMem (https://arxiv.org/abs/2601.18642). I think overall novelty of newly introduced semantic-group summaries and dynamic distribution between active and dormant memories is still strong, but more baseline comparison can further establish the advantages and make argument stronger.

Questions for authors
  1. Clearly, FOCUS has a higher maintenance cost than typical memory framework, could you report end-to-end retrieval latency and total token count at matched answer quality (if it has difference), including dormant-group search, summary regeneration, and use attribution?
3 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for reviewing FOCUS and raising questions about its practical performance. Below, we report Dormant retrieval performance and costs during evaluation and memory construction, followed by the additional comparison with FadeMem.

W1 & Q1. Concerns on Dormant retrieval and memory construction costs.

A: Thanks. To examine whether Dormant memories remain accessible through group summaries, we measured how reliably relevant entries were retrieved during evaluation QA. With Qwen, we marked the memory entries in the Dormant Set that were relevant to each evaluation question and tracked whether they were retrieved during QA. Based on this tracking, we measured source coverage as the percentage whose groups were selected through summary matching, and final recall as the percentage that reached the answering context.

BenchmarkEligible questionsSource coverage (%)Final recall (%)
LoCoMo73694.7078.83
Memora Quarterly17588.9553.73

These results show that FOCUS can recover relevant information through group summaries, even in Quarterly histories spanning nearly 2,000 sessions. However, memory entries differ across methods, so we retain end-to-end Accuracy and FAMA as the primary metrics for system comparison.

Furthermore, to assess the associated costs, we compared retrieval and answer generation across methods after memory construction, using Qwen on Memora Quarterly. Retrieval includes summary matching and search within Dormant groups, while generation is timed separately. QA input and output tokens are cumulative totals in millions (M).

MethodFAMARetrieval (s)Generation (s)QA input (M)QA output (M)
NativeRAG27.720.0352.4361.1280.079
Mem033.930.0543.5330.9830.119
MemoryOS35.720.1194.3412.5360.097
A-MEM34.400.0633.1805.7510.091
LightMem29.440.0603.4320.8940.101
FOCUS38.500.0943.1321.0580.064

According to the above table, FOCUS achieves the highest FAMA while maintaining retrieval/generation latency and QA token consumption comparable to competing methods.

It is worth noting that memory updates occur only during memory construction, while evaluation QA leaves memory unchanged. Table 2(b) of the main paper already reports input and output token consumption for construction and maintenance, including group summary generation and memory use attribution. To complement these token measurements with a runtime analysis, we additionally measured construction and maintenance latency. With Qwen as the backbone on Memora Quarterly, we recorded construction and maintenance time over approximately 2,000 sessions and report the average time per session below.

MethodTime/session (s)
Mem022.482
MemoryOS26.844
A-MEM40.930
LightMem18.036
FOCUS24.292

The results show that FOCUS requires more construction and maintenance time than LightMem and Mem0, but less than MemoryOS and A-MEM. Together with the retrieval and QA costs reported above, these construction and maintenance measurements demonstrate a favorable trade-off between memory performance and the costs of construction and retrieval.

W2. Concern on FadeMem comparison.

A: Thanks for the suggestion. We have added the comparison with FadeMem, using the same backbones and experimental settings as FOCUS.

BackboneMethodLoCoMo ACCWeekly FAMAMonthly FAMAQuarterly FAMA
QwenFadeMem53.7447.3434.6229.20
QwenFOCUS75.9751.8240.6038.50
DeepSeekFadeMem56.5954.9646.3334.16
DeepSeekFOCUS77.8661.9852.5039.63

The results show that FOCUS outperforms FadeMem across both backbones and all evaluated settings. This advantage may stem from how the two methods handle forgotten memories. FadeMem uses two memory layers with different decay rates and prunes low-priority memories, potentially removing information needed later. In contrast, FOCUS retains complete Dormant entries and keeps them accessible through group summaries. We will discuss this distinction and include the full comparison in the revised manuscript.

Thank you for your additional experiments and I believe your followup addresses most of my concern on latency, resource demanding of FOCUS, and how large it's better than baselines. Now I think it's a solid work with good intuition-grounding and good performance. I would raise my rating to strong accept (4).

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for taking the time to review our follow-up and reassess the paper. We appreciate the helpful feedback, which has strengthened the experimental evaluation and clarified the presentation of FOCUS.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.