acceptodds
Under review as a conference paper at ICLR 2027

RETAIN AND REVEAL: MULTI-CONTINUATION TRAINING THROUGH AN OPTICAL MEMORY BOTTLENECK

Abstract

Large language model agents must preserve evidence for decisions whose requirements become clear only as new observations arrive. Long-context question answering provides a concrete setting: later passages can determine which earlier relations are needed. Under limited memory, these relations must survive both selective retention and compression into a readable representation. We introduce R&R, which trains one committed memory to support multiple source-grounded continuations. The writer commits without access to the continuation probes. Its memory is rendered once, and the reader answers the same question using the identical image paired with different continuations. Continuation rewards encourage retention of unresolved alternatives, while reader supervision trains evidence recovery through the optical channel. Paired memory skills coordinate evidence organization and interpretation, with failure-driven revisions validated on held-out histories under frozen model weights. Experiments on long-context QA benchmarks demonstrate competitive accuracy across settings under matched answer-time memory-input token allowances. Gains over textual memory approaches are more pronounced at smaller budgets, supporting the value of training bounded memory to preserve and recover evidence for subsequent reasoning. Controlled comparisons show mean-accuracy gains from continuation diversity, joint probe supervision, and paired skill revision.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agent memory, Self evolution, Long-context QA, Optical compression.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QobyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Retain and Reveal (R&R) for bounded memory in long-context QA. A writer commits a textual memory before seeing later continuations, and the memory is rendered once as an image. A reader then answers multiple continuations from that same image. Writer rewards, reader supervision and validated skill revision are used for training.

Critical strengths
  1. Valuable core idea. Testing one committed memory against multiple later uses directly targets delayed relevance, and the ablation shows that distinct continuations help beyond repeated reads of a single continuation.
  2. Joint treatment of retention and readability. The method trains both what the writer preserves and whether it can be recovered after rendering, and the experiments include useful ablations.
Critical weaknesses
  1. Training and inference are misaligned. Training uses probe reads on intermediate memory, whereas inference continues recurrent writing and reads only the final memory. This makes it hard to tell which component produces the gains.
  2. The contribution of optical memory cannot be attributed. No new visual encoder or component is trained, and layout, skills and training signals change together. The results do not show that image memory itself is better than well-structured text memory.
Questions for authors
  1. Answer-match accuracy (SEM) does not show that the method retains and correctly uses evidence relations. Could you add direct evidence-level metrics?
  2. The motivation is general agent memory, but the experiments are long-context QA. Could you add an agent-memory setting, or narrow the claims to long-document memory compression?
  3. The evaluation uses 128 examples per dataset, and several gains over strong baselines are small.
  4. Figure 1 stacks several memory paradigms and does not highlight R&R's full mechanism.
Reviewer_jR6cOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Retain and Reveal (R&R), which trains one committed memory to support multiple source-grounded continuations. The writer commits without access to the continuation probes. Its memory is rendered once, and the reader answers the same question using the identical image paired with different continuations. Continuation rewards encourage retention of unresolved alternatives, while reader supervision trains evidence recovery through the optical channel.

Critical strengths

The experiments cover four long-context QA benchmarks, multiple context lengths, and different memory budgets.

Critical weaknesses
  1. The technical novelty of R&R is not sufficiently established. Mem-α and UMA (Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning) already explore optimizing shared memory through multiple downstream QA rewards, which is closely related to the core idea of Multi-continuation Training. Although R&R differs by evaluating intermediate memory under a fixed question with alternative future evidence, the paper does not clearly demonstrate the unique benefit of this design. Furthermore, optical memory and failure-driven skill revision have been explored in prior works such as MemOCR and MemSkill. The authors should better clarify the novel contributions beyond combining existing techniques.

  2. The construction of Multi-continuation Probes is not sufficiently documented, despite being central to the proposed training objective. Appendix B specifies validity constraints but does not provide a complete generation procedure. The authors should clarify the construction algorithm or prompts, the proportions of natural suffixes and controlled selectors, probe validity rates, the distribution of K, and the verification process. Results across different probe types would also help assess the generality of the approach.

Questions for authors
  1. The necessity of computing the writer's continuation reward through the optical channel is not fully established. In Table 3, text-scored writer rewards outperform image-scored rewards at both memory budgets, while the reader and final inference remain optical. Although these settings have different information budgets, the results do not demonstrate a clear advantage of optical reward computation.

  2. The performance advantage of R&R over MemOCR becomes smaller at 100K context length and disappears in the 256-token setting. However, the continuation-diversity ablation in Table 2 is conducted only at 30K, making it unclear whether Multi-continuation Training itself scales effectively to longer contexts.

  3. The overall framework combines several existing ideas, including optical memory, reinforcement learning, reader supervision, and failure-driven skill revision. In particular, the skill-revision mechanism is closely related to prior work such as MemSkill and Reflexion.

  4. The paper claims that Multi-continuation Training helps prevent the premature deletion of potentially useful evidence. Although the experiments show improvements in final QA accuracy, they do not directly demonstrate whether the writer actually retains more unresolved relations in intermediate memories.

  5. The related work also overlooks important studies: 1. Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning. 2. Language Modelling with Pixels 3. Vision-centric Token Compression in Large Language Model

Reviewer_L2CFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper trains a recurrent text-memory writer and a vision-language reader (both Qwen3-VL-8B). The final memory is rendered as an image under a visual-token budget. During training, an intermediate memory is rendered once and read with several "continuations", meaning later evidence that selects different earlier relations. The mean probe score is a GRPO (Shao et al., 2024) reward for the writer (with step-limited credit) and an NLL target for the reader. A paired writer/reader instruction "skill" is also revised between training intervals. Validation: HotpotQA, 2Wiki, NQ and TriviaQA at 10K–100K tokens and 64/256-token budgets, compared with Mem0 (Chhikara et al., 2025), Mem-α\alpha (Wang et al., 2025), MemAgent (Yu et al., 2026) variants and MemOCR (Shi et al., 2026) in Table 1, plus ablations (Tables 2–3).

Critical strengths
  1. A clear "delayed relevance" problem statement (pp.1–4) with a sensible design: the writer "commits to a memory before seeing the continuation probes used to evaluate it" (p.2), continuation credit goes "only to steps that can affect the tested state" (Eq. 8, p.13), and Single repeat vs Multi make the same number of probe reads with matched writer rollouts and optimization steps (Table 2, p.7).
Critical weaknesses
  1. The headline results cannot be checked from the paper.
    • Table 1's R&R entries are "averaged over three runs" (p.7), but "updated task-level scores, the new baselines' breakdowns, and run-level dispersion remain unreported" (p.17). Tables 6–9 and Fig. 4 come from an "earlier run series" or "earlier diagnostic series" (p.17, p.8).
    • Table 1's MemOCR row equals the "R&R (earlier)" macro-averages of Tables 6–7 in five of six entries (72.6/69.5/67.1 at 256 tokens; 67.3/64.7 at 64 tokens); the sixth is 60.7 in Table 1 against 61.8 recomputed in the Table 7 footnote. Beyond "an adaptation of the method" (p.16), the paper does not say how MemOCR, MemAgent and Mem-α\alpha were obtained on Qwen3-VL-8B.
    • With 128 instances per dataset (p.6), the four-dataset macro standard error is about 2 points, comparable to the margins over MemOCR (1.3–3.0 points in five of six settings, −0.3-0.3 at 100K/256). The paper itself does "not infer statistical significance from the observed gaps" (p.6).

The central claim, an advantage over the strongest baseline at matched memory budgets, cannot be checked from the submission.

  1. The gains are not attributed to the proposed mechanisms.
    • The largest single-component gain in Table 3 is skill revision (+4.6/+3.9 at B=64/256B = 64/256, p.8), whose proposer model, schedule and final skill are not given (pp.15–16). Without revision, Joint probes (62.8/67.8) score below Table 1's MemOCR row at 30K (64.7/69.5).
    • Text-scored writer reward beats image-scored reward (64.1/68.5 vs 62.8/67.8, Table 3), and the paper concedes these results "do not establish an advantage for scoring writer rewards after rendering" (p.8), which undercuts the optical-coupling contribution.
    • KK, λW\lambda_W, λR\lambda_R, λval\lambda_{\mathrm{val}}, ϵ\epsilon and the revision interval have no reported values ("fixed within a run and selected without test data", p.16), and the ablations are 30K-only means "without run-level SD" (Table 3, p.8).

Even taken at face value, the gains are not shown to come from multi-continuation training through the optical channel.

Questions for authors
  1. Table 1's MemOCR row equals the "R&R (earlier)" macro-averages of Tables 6–7 in five of six entries. Which system produced each of these rows, what differs between the earlier and the updated series, and how were MemOCR, MemAgent and Mem-α\alpha trained or adapted on Qwen3-VL-8B?
  2. Please report per-task results and the run-level standard deviation (Table 1 averages three runs) for every updated Table 1 entry and for Tables 2–3.
  3. How are probes and controlled selectors generated (p.4), what fraction of training boundaries has K≥2K \geq 2 valid probes, and what are KK, λW\lambda_W, λR\lambda_R, λval\lambda_{\mathrm{val}}, ϵ\epsilon and the revision interval? How does the objective differ from UMA (Zhang et al., 2026), which also trains a memory agent end to end with reinforcement learning for long-context reasoning and is not cited?
  4. Please compare against MemOCR or Joint probes under the same skill-revision budget, and report the edit-proposer model, the number of accepted revisions and the final skill.

References:

  • Chhikara, P., Khant, D., Aryan, S., Singh, T., and Yadav, D. Mem0: Building production-ready AI agents with scalable long-term memory. arXiv:2504.19413, 2025.
  • Shao, Z., et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv:2402.03300, 2024.
  • Shi, Y., et al. MemOCR: Layout-aware visual memory for efficient long-horizon reasoning. arXiv:2601.21468, 2026.
  • Wang, Y., et al. Mem-α\alpha: Learning memory construction via reinforcement learning. arXiv:2509.25911, 2025.
  • Yu, H., et al. MemAgent: Reshaping long-context LLM with multi-conv RL-based memory agent. ICLR, 2026.
  • Zhang, K., Gui, S., Yang, S., Chen, W., and Feng, Y. Learning to remember: End-to-end training of memory agents for long-context reasoning. arXiv:2602.18493, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.