- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors introduce Revisable Loop Memory (RLM), a fixed-size associative memory that lets looped Transformers preserve and revise intermediate computations across loops. The main contribution of the paper is a gated delta-write mechanism that stores only the difference between a block’s proposed value and the memory’s current prediction, allowing targeted corrections leaving untouched unrelated associations. On Ouro-1.4B base model, matched fine-tuning and ablations show that RLM outperforms native fine-tuning variants, improving GSM8K and MATH500 by 2.3 and 2.2 points with only 0.2% additional computation.
Critical strengths
Strengths:
-
The authors propose Revisable Loop Memory (RLM), a gated delta-write memory that allows looped models to preserve and selectively revise intermediate computations across loops.
-
Under matched fine-tuning, RLM improves over native fine-tuning by 2.3% on GSM8K and 2.2% on MATH500, while adding only about 0.2% computation.
Critical weaknesses
Weaknesses:
-
The evaluation is limited to Ouro-1.4B. Although Ouro is also available at 2.6B and in reasoning-tuned variants, these settings are not evaluated, making it unclear whether RLM generalizes across model scales, checkpoint types, architectures, and broader training domains.
-
The backbone and memory interface are jointly fine-tuned, so the experiments do not isolate whether RLM benefits an existing frozen looped model. Consequently, it is difficult to determine how much of the improvement comes from the memory mechanism itself rather than from the backbone adapting to it.
-
The empirical gains are relatively modest. RLM improves over matched native fine-tuning by 2.3% on GSM8K and 2.2% on MATH500, while gains on most other benchmarks are below one point. This raises questions about whether the improvement is substantial enough to justify the added architectural complexity.
Questions for authors
Q1: How does RLM perform on Ouro-2.6B, reasoning-tuned Ouro checkpoints, or another looped backbone such as Nanbeige4.2-3B? These experiments would help establish whether the gains generalize across model scales, checkpoint types, and architectures. Moreover, a larger set of benchmarks spanning different domains is needed to fully assess the benefits of the propsoed approach
Q2: Could you evaluate RLM on a frozen backbone, training only the memory interface? This would help isolate the contribution of RLM from improvements enabled by jointly adapting the backbone.
Q3: Could you provide statistical significance estimates for RLM versus Native–FT? Confidence intervals are reported for selected ablations, but not for the main comparison against native fine-tuning.
Q4: Why are the improvements over Native–FT relatively small on most non-mathematical benchmarks, and do you expect larger gains with broader training data?
4 comments
Response to Reviewer jCXf [2/2]
Q3. Could you provide statistical significance estimates for RLM versus Native–FT? Confidence intervals are reported for selected ablations, but not for the main comparison against native fine-tuning.
A4. The paired bootstrap 95% interval for RLM minus Native–FT is [0.71, 3.89] on GSM8K and [−0.27, 4.67] on MATH500. We use the procedure of Table 3: 10,000 replicates that resample the three training seeds and all problems, with the same draws applied to every interface. The GSM8K interval excludes zero. On MATH500 the mean gain is 2.20 points, and the wider interval mainly reflects the 500-problem test set, where one problem moves accuracy by 0.2 points.
| Contrast (3 seeds, paired bootstrap 95% CI) | GSM8K | MATH500 |
|---|---|---|
| RLM − Native–FT | 2.27 [0.71, 3.89] | 2.20 [−0.27, 4.67] |
| RLM − Add–State | 1.74 [0.53, 3.01] | 1.60 [−0.20, 3.47] |
| RLM − Delta–Input | 1.36 [0.13, 2.63] | 1.20 [−0.73, 3.20] |
We will add the Native–FT interval to the Table 3 caption.
Q4. Why are the improvements over Native–FT relatively small on most non-mathematical benchmarks, and do you expect larger gains with broader training data?
A5. The adaptation data contain only the GSM8K and MATH training splits, so every non-mathematical benchmark is out of domain, and RLM still improves all six general benchmarks. The interface is trained only through the final-loop loss on 32M mathematics tokens, so it learns which block outputs to keep and revise for mathematical reasoning. Its largest gains are on the two benchmarks that match this data, and it transfers positively to the general benchmarks (0.55 to 0.98 points) and to HumanEval, HumanEval+, and MBPP. Training on a broader mixture would supervise the memory on those domains directly, and we expect larger gains there.
Dear authors, Thank you for your answers.
I still have concerns about the benefit of the proposed approach. Gains seem to be quite modest. Especially, concerning models such as Ouro-2.6B.
I'll keep my score.
Thank you
Dear Reviewer jCXf,
Thank you for reading our rebuttal and for the follow-up.
We agree that the absolute gains are modest, including on Ouro-2.6B. We reported these numbers so the improvement is easy to judge. Two points may still be useful: the comparison is against a baseline that already received the same fine-tuning, and the extra cost is only 0.2% more compute. In our ablations, an additive memory of the same size does not give this gain; only the corrective write does.
We appreciate the time you have spent on the review :)
Best regards, The authors
Response to Reviewer jCXf [1/2]
W1. The evaluation is limited to Ouro-1.4B. Although Ouro is also available at 2.6B and in reasoning-tuned variants, these settings are not evaluated, making it unclear whether RLM generalizes across model scales, checkpoint types, architectures, and broader training domains.
Q1. How does RLM perform on Ouro-2.6B, reasoning-tuned Ouro checkpoints, or another looped backbone such as Nanbeige4.2-3B? These experiments would help establish whether the gains generalize across model scales, checkpoint types, and architectures. Moreover, a larger set of benchmarks spanning different domains is needed to fully assess the benefits of the propsoed approach
A1. Thank you for the suggestion. With the same interface hyperparameters and matched fine-tuning, RLM improves over Native–FT on Ouro-2.6B by 1.44 points on GSM8K and 1.40 points on MATH500. Both new backbones use the zero-shot greedy protocol of Table 3 with T=4 and a 2,048-token limit. Ouro-1.4B-Thinking is already tuned for reasoning and reaches 85.44 on GSM8K and 88.00 on MATH500 after Native–FT. On this checkpoint, RLM adds 0.23 points on GSM8K and is within one problem of Native–FT on MATH500.
Across architectures, RLM interacts with the backbone only by reading block outputs and adding a scaled residual at loop boundaries. It leaves attention, the feed-forward blocks, the tokenizer, and the prediction head unchanged, so it attaches to any looped Transformer. On benchmark coverage, Table 2 already spans twelve benchmarks in knowledge (MMLU, MMLU-Pro), reasoning (BBH, ARC-C), commonsense (HellaSwag, Winogrande), mathematics (GSM8K, MATH500), and code (HumanEval, HumanEval+, MBPP, MBPP+), and Table 4 adds ARC-E and CommonsenseQA. RLM is higher than Native–FT on eleven of the twelve benchmarks in Table 2.
W2. The backbone and memory interface are jointly fine-tuned, so the experiments do not isolate whether RLM benefits an existing frozen looped model. Consequently, it is difficult to determine how much of the improvement comes from the memory mechanism itself rather than from the backbone adapting to it.
Q2. Could you evaluate RLM on a frozen backbone, training only the memory interface? This would help isolate the contribution of RLM from improvements enabled by jointly adapting the backbone.
A2. With the backbone frozen, training only the 1.07M interface parameters improves Ouro-1.4B by 0.61 points on GSM8K and 0.60 points on MATH500. The interface is trained on the same data and 32M-token budget with checkpoint selection by development NLL, and all rows below use the few-shot protocol of Table 2. The frozen blocks were never trained to produce memory keys or to read the boundary injection, and the interface still raises accuracy on both benchmarks.
Joint fine-tuning gives larger gains because the backbone and the interface adapt together, and Table 3 isolates the memory mechanism within this joint setting. Native–FT, RLM, and the four memory ablations all fine-tune the same backbone with the same data and budget. The additive variants stay between 79.68 and 80.21 on GSM8K, and only the corrective write with the memory-aware gate reaches 81.80, so backbone adaptation alone does not produce the gain.
W3. The empirical gains are relatively modest. RLM improves over matched native fine-tuning by 2.3% on GSM8K and 2.2% on MATH500, while gains on most other benchmarks are below one point. This raises questions about whether the improvement is substantial enough to justify the added architectural complexity.
A3. The gains are measured against a baseline that already receives the same fine-tuning, and they are comparable in size to the gain from fine-tuning the whole 1.4B backbone. In Table 2, full fine-tuning raises Ouro from 78.62 to 80.89 on GSM8K and from 81.80 to 83.40 on MATH500, and the 1.07M-parameter interface then raises these to 82.64 and 85.60. This gain costs about 0.2% more computation per token (10.90 vs. 10.87 GFLOPs in Table 5), while one more native pass adds about a quarter (13.54 GFLOPs). The interface keeps attention, the tokenizer, and the output head unchanged, its state does not grow with the number of loops, and a zero residual scale recovers the native computation.