- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Learn to Remember (LeRe), a memory-augmented framework in which a frozen LLM cycles through Planner, Solver, and Curator roles and learns which memory entries to retrieve instead of accumulating more memory. Its core component, the Contrastive Contextual Memory Encoder (CCME), trains lightweight heads on a frozen encoder online, using the Curator's attribution of which entries actually helped as the training signal. A Guard–Consolidate–Maintain (GCM) gate filters answer leakage, merges near-duplicates, and quarantines unreliable entries.
Critical strengths
- The question is clear, the limitations indeed exist in related work.
- The methods have great transferabilities, can be used with different LLMs
- Comprehensive experiments on 12 streams and 3 backbones
Critical weaknesses
- Can the author add one ablation variant that uses the solver (without memories)? I'd like to see how much of the performance will drop under such circumstances.
- Many results drop on MATH and MMLU Eng. The gain is not stable across tasks. Even compared with other baselines, LeRe is weaker on these two datasets.
- In the related work section, the authors talk about many related methods, but very few of them are compared to LeRe in the experimental section.
- More hyperparameter analysis needs to be added, as there are so many hyperparameters in Table 4.
minor:
- Abstract can be revised, some words, for example, entry, do not have a clear meaning under such a semantic background.
Questions for authors
Please refer to the weakness section
3 comments
Response to Reviewer LkDF
Baseline selection criteria clarification
Thank you for raising this question. We appreciate the opportunity to clarify our baseline selection. Our evaluation focuses on frozen, API-served models that adapt online to a single-pass stream of reasoning problems, without access to ground-truth labels or token log-probabilities. Dynamic Cheatsheet (both variants) and ACE are directly applicable to this setting, while RAG-ICL provides a standard retrieval-based baseline.
Several other methods discussed in Section 2 are closely related but operate under different assumptions:
- Weight-based test-time adaptation (Wang et al., 2021; Niu et al., 2022; Zuo et al., 2025) requires gradient access to update model parameters, which is unavailable for the API-served models considered in our experiments.
- Learned retrievers (EPR, LLM-R, REPLUG) typically rely on offline training using gold outputs or model likelihoods. These requirements are not compatible with our setting, where ground-truth labels and token likelihoods are unavailable.
- Interactive agent memory methods (e.g., Reflexion and workflow memory) generally involve repeated attempts at the same task or interactions within multi-turn environments. Our evaluation instead considers independent reasoning problems processed in a single pass (Pass@1).
- Planning and query-reformulation methods (ReAct, Plan-and-Solve, HyDE) focus on improving reasoning or retrieval rather than maintaining and updating cross-query memory. LeRe incorporates planning through its Planner, whose contribution is separately evaluated in the
w/o Plannerablation (Table 2).
We agree that a clearer comparison with these methods would help readers understand our baseline selection. In the revised manuscript, we will add an appendix table summarizing the related methods, their key requirements, and how they differ from our experimental setting.
Hyperparameter analysis
Thank you for this helpful suggestion. Our current evaluation focuses on LeRe's overall effectiveness, including accuracy comparisons across backbones, cost-efficiency analysis, component ablations, and analyses of the memory mechanism, with additional experiments provided in the Appendix.
For hyperparameter sensitivity, we evaluated the number of retrieved entries (Appendix C.3.1, Table 8). The results show that accuracy varies by at most 3.2 percentage points across these settings. All other hyperparameters listed in Table 4, including retrieval weights, consolidation thresholds, quarantine rules, and the CCME training schedule, were fixed in advance and kept consistent across all backbones and task streams, without task-specific tuning.
We agree that a more comprehensive sensitivity analysis would provide further insight into LeRe's behavior. We have acknowledged this limitation in the revised manuscript and identified a systematic study of these hyperparameters as a direction for future work.
Minor Comment: Abstract Clarity
Thank you for pointing out this wording issue. We have revised the Abstract (with changes highlighted in red) to improve clarity, replacing ambiguous terms such as "entry" with more specific descriptions, including "retrieved reasoning strategies" and "memory items," where appropriate.
Thank you for your response. After reading the full rebuttal, I find that some of my concerns have been addressed. I am inclined to keep my current score.
Response to Reviewer LkDF
Dear Reviewer LkDF,
Thank you for your careful evaluation of our work and your constructive suggestions for improving the manuscript's clarity. We have addressed your comments in the revised manuscript, with all corresponding changes highlighted in red. Below, we respond to each of your comments and questions point by point:
Performance of the ablation variant without memories
The ablation results, including the baseline that isolates the contribution of memory, are presented below:
Table B: Ablation accuracy comparison with newly added memory isolating baseline
As shown in Table B, disabling cross-query memory consistently reduces accuracy across all backbones and benchmarks, with an average drop of 4.8 percentage points and a maximum drop of 11.8 points on AIME. These results show that cross-query memory provides substantial additional gains beyond those achieved by the Solver and code execution alone.
LeRe’s performance on specific datasets
Thank you for this observation. We agree that LeRe does not improve performance on every stream, and we recognize this as a limitation of the current framework. Our results suggest two possible explanations:
Despite these limitations, LeRe achieves the highest average improvement across all 12 streams on each backbone (+6.9, +9.0, and +4.6 percentage points, compared with +3.7, +2.4, and +3.2 for the strongest competing method, DC RS). The improvements are particularly clear on challenging reasoning tasks, where LeRe outperforms the baselines in all 9 AIME settings and every HLE-Exact evaluation.
These results suggest that LeRe is effective for complex reasoning, although its performance depends in part on the backbone's verification capability and the difficulty of the task. Improving self-verification to make memory updates more reliable, particularly on weaker backbones, is an important direction for future work.