acceptodds
Under review as a conference paper at ICLR 2027

ReMemorize: Reward-Guided Adaptive Memory Management for Faithful Clinical Text Summarization

Abstract

Large language models show strong potential for clinical summarization, yet factual inconsistency remains a major concern in long-context generation. Existing approaches largely rely on retrieval, verification, or post-hoc refinement, while leaving autoregressive memory dynamics fixed and allowing supporting evidence to be progressively overwritten during generation. We propose ReMemorize, a framework that introduces explicit control over information retention through a recurrent memory state with gated updates during decoding. The interface is trained using supervised memory-consistency objectives, group-relative reward-weighted candidate learning, and a clipped score-weighted auxiliary gate objective based on sequence-level signals for factual alignment, coherence, completeness, and hallucination suppression. Across two popular summarization benchmarks, ReMemorize improves factual consistency over strong baselines, with gains concentrated on faithfulness rather than surface-level lexical overlap. Our results suggest that explicit control over information retention can improve factual grounding in clinical summarization. Our code is publicly available at https://anonymous.4open.science/r/ReMemorize-467B.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
factual consistency / hallucination mitigation, memory-augmented language models, reward-guided learning, group-relative policy optimization (GRPO), long-context generation, clinical text summarization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VqUBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies adding trainable recurrent memory to a train-frozen language model. Continuous gates control memory updates, while a separate gate controls memory injection into decoding. Training combines reference-summary supervision, reward-weighted candidate learning, memory consistency, and an auxiliary write-gate objective. Experiments on MIMIC-IV-Ext-BHC and SOAP report ROUGE, BERTScore and factuality scores with baseline SFT/direct-GRPO controls, two backbones, and a human evaluation.

Critical strengths

S1. A worthwhile problem and an inspectable mechanism. Faithful clinical summarization is valuable. The paper specifies a compact recurrent state, memory-conditioned updates, and decoding-time injection, providing a concrete approach to study evidence retention

S2. Relevant controls and explicit qualifications. SFT, direct-GRPO, and a second backbone provide useful comparisons (Tables 1, 4, 7). The authors also acknowledge the gate objective's limitation and the restricted interpretation of inference-time ablations

Critical weaknesses

W1. Reliability concerns about the factuality comparison

W1.1. The baseline scoring error requires tracing the reported results. Paper mentions it use MiniCheck for factuality scoring but supplemental code evaluate_baselines.py uses DeBERTa for factuality scoring

Although, entailment is index 1, some baseline code evaluate_baselines.py reports index 2 (neutral) as entailment.

W1.2. The metric definition differs from the Full evaluator. Equation 19 reports the claim support probability as 1 when claim support probability is above 0.5. However, evaluate.py averages probabilities. These measures can rank systems differently

W2. Insufficient evidence for reward-learned write selectivity

W2.1. The auxiliary objective does not connect sampled decisions to summary quality The continuous write gate balances retention of the previous memory with incorporation of new content. However, the binary variables in Equation 10 do not control the evaluated rollouts, which use continuous gates. Their auxiliary reward term has zero expected gate gradient at the behavior policy, consistent with the limitation acknowledged on page 9. Section 3.3 also states that the write policy is optimized only through the auxiliary gate objective. This leaves the claimed contribution of reward-learned write selectivity insufficiently supported.

W2.2. The gate ablation does not isolate the value of learning selectivity. Table 7 replaces the trained gate with 0.5 only at inference; Table 8 reports a Full mean source gate of 0.097. This intervention changes both input dependence and update strength. As Appendices G/L acknowledge, the 74.3→55.6 drop shows checkpoint sensitivity but it does not support the advantage of a trained gating network over a separately trained fixed-gate model. Freezing the initialized write policy while retaining all other training would directly test the value of gate learning.

Questions for authors

Q1. Evaluation. Which commands and scorer versions produced Tables 1, 4, and 7? Can predictions or permitted per-example scores establish a common evaluation and reconcile W1.1–W1.2?

Q2. Purpose of the binary gate variables. What motivates the binary variables in Equation 10, given that they do not control candidate generation and their auxiliary reward term lacks an informative expected gate gradient, as acknowledged on page 9? What benefit does this term provide beyond reward-weighted candidate learning and entropy regularization?

Reviewer_L5f3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a system for clinical summarization entitled REMEMORIZE. This framework that regulates memory updates during autoregressive generation through gated recurrent memory and reward-guided memory-interface training. On very small LLMs, using two clinical summarization benchmarks, they achieve slightly better results than baselines. They perform some ablation studies.

Critical strengths

The strengths lie in the fact that the paper shows that on these small LLMs using this gated memory architecture helps improve factuality and other aspects of summarization to a small degree.

Critical weaknesses

The critical weakness is that this work makes very little impact. If I want to use clinical summarization, will I use this system? The answer is an unequivocal "no".

  1. The major drawback of this work is that it uses very small, weak LLMs. The SOTA (check LM Arena) is much bigger LLMs. Even if we want open-weight LLMs for privacy reasons, there are much larger open-weight LLMs, which a medical application would use in reality. The only case I can think of is edge devices. But, there are other resource constraints there and why should I want a on-client summarization anyway? I can do it on a larger open-weight LLM on a server. So, these slight improvements do not really matter in the real world. The authors have shown no evidence that this technique solves a real problem on SOTA LLMs.

  2. The model architecture has a small novelty but it is not major. Using a memory snapshot, training via rewards, etc. are minor adaptations and applications of core ideas that have been used in the field.

  3. The presentation of the paper can be substantially improved. The main algorithm is in the Appendix. I have no idea what it is. The paper must be self-sufficient according to guidelines. If a reviewer has to read the appendix, we are giving in essence an extra page to this paper and not the others, which is unfair. It is indeed an art to bring the most important things in the 9 pages.

  4. With so many things being sent to the Appendix, the paper is unreproducible standalone. I think the authors can think what is standard and just refer to them and not explain that in the paper or using equations, etc. Only the changes to standard architectures can be mentioned while summarizing the parts of the system that are standard, etc.

Therefore, this paper will not really impact the SOTA in medical summarization that much - at least we do not have that evidence in this paper.

Questions for authors

Address the weaknesses please.

Reviewer_g4HmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LLMs often miss important information or generate factual errors in long clinical text summarization. Existing methods do not directly keep and update information during text summary generation. Based on this , the authors add a dynamic memory module to LLMs to help memorize and generate. A learnable gate controls is also designed for memory updates, and a projector adds memory information to the LLM's hidden/latend states. The memory module is trained while keeping LLM backborn frozen using rewards based on factuality, coherence, completeness, and hallucination.

Critical strengths
  1. The combination of dynamic memory, adaptive gating, and reward-guided learning to improve factual consistency without fine-tuning the LLM backbone is a very interesting idea. Specifically, this work use a learnable gate to control memory updates and reward signals to train the memory module. This helps the model better retain and use information from the source text. The overall framework is simple and well organized, with clear roles for each component.

  2. Unlike methods that build a fixed memory representation from the input, ReMemorize allows memory to be updated during both source encoding and summary generation. During encoding, the gate controls how much new information is added based on the current token and existing memory. And the memory is added to the LLM's hidden states and continues to update as new tokens are generated. This design provides an interesting view to retain and use important information throughout long text generation.

Critical weaknesses
  1. Looks that the proposed method does not use any clinical-specific knowledge or constraints. So it seems to me it should be a general LLM summarization method rather than a method specifically designed for clinical summarization. The authors should compare it with recent LLM-based general summarization methods, such as EntityBB (EMNLP 2026) and prior works in related works of this paper, on clinical datasets. More importantly, they should also test ReMemorize on general benchmarks. These two directions of evaluation would help show whether the method is better suited for clinical summarization or can generalize to other domains.

  2. The memory module update is actually a weighted combination of all tokens. As more tokens are processed, information from earlier tokens may gradually decay or be overwritten by new information. This seems to conflict with the main goal of preserving important information in long documents.

  3. The authors use a PPO-style objective to train the memory gate with rewards from generated summaries. However, the sampled gate actions are not actually used to generate these summaries. Although the ablation results show that the gate-related components are useful, it remains unclear whether the reward signals can effectively teach the gate to retain important information.

Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.