- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies adding trainable recurrent memory to a train-frozen language model. Continuous gates control memory updates, while a separate gate controls memory injection into decoding. Training combines reference-summary supervision, reward-weighted candidate learning, memory consistency, and an auxiliary write-gate objective. Experiments on MIMIC-IV-Ext-BHC and SOAP report ROUGE, BERTScore and factuality scores with baseline SFT/direct-GRPO controls, two backbones, and a human evaluation.
Critical strengths
S1. A worthwhile problem and an inspectable mechanism. Faithful clinical summarization is valuable. The paper specifies a compact recurrent state, memory-conditioned updates, and decoding-time injection, providing a concrete approach to study evidence retention
S2. Relevant controls and explicit qualifications. SFT, direct-GRPO, and a second backbone provide useful comparisons (Tables 1, 4, 7). The authors also acknowledge the gate objective's limitation and the restricted interpretation of inference-time ablations
Critical weaknesses
W1. Reliability concerns about the factuality comparison
W1.1. The baseline scoring error requires tracing the reported results. Paper mentions it use MiniCheck for factuality scoring but supplemental code evaluate_baselines.py uses DeBERTa for factuality scoring
Although, entailment is index 1, some baseline code evaluate_baselines.py reports index 2 (neutral) as entailment.
W1.2. The metric definition differs from the Full evaluator. Equation 19 reports the claim support probability as 1 when claim support probability is above 0.5. However, evaluate.py averages probabilities. These measures can rank systems differently
W2. Insufficient evidence for reward-learned write selectivity
W2.1. The auxiliary objective does not connect sampled decisions to summary quality The continuous write gate balances retention of the previous memory with incorporation of new content. However, the binary variables in Equation 10 do not control the evaluated rollouts, which use continuous gates. Their auxiliary reward term has zero expected gate gradient at the behavior policy, consistent with the limitation acknowledged on page 9. Section 3.3 also states that the write policy is optimized only through the auxiliary gate objective. This leaves the claimed contribution of reward-learned write selectivity insufficiently supported.
W2.2. The gate ablation does not isolate the value of learning selectivity. Table 7 replaces the trained gate with 0.5 only at inference; Table 8 reports a Full mean source gate of 0.097. This intervention changes both input dependence and update strength. As Appendices G/L acknowledge, the 74.3→55.6 drop shows checkpoint sensitivity but it does not support the advantage of a trained gating network over a separately trained fixed-gate model. Freezing the initialized write policy while retaining all other training would directly test the value of gate learning.
Questions for authors
Q1. Evaluation. Which commands and scorer versions produced Tables 1, 4, and 7? Can predictions or permitted per-example scores establish a common evaluation and reconcile W1.1–W1.2?
Q2. Purpose of the binary gate variables. What motivates the binary variables in Equation 10, given that they do not control candidate generation and their auxiliary reward term lacks an informative expected gate gradient, as acknowledged on page 9? What benefit does this term provide beyond reward-weighted candidate learning and entropy regularization?