acceptodds
Under review as a conference paper at ICLR 2027

From History to State: Revisable Memory for Looped Transformers

Abstract

Looped Transformers reuse one layer stack across several passes, and each pass hands only its final hidden state to the next pass. The outputs of the blocks inside a pass are not kept, so a later pass cannot revise what an earlier block computed. Keeping these outputs in an additive memory does not allow revision either: the value retrieved for a key is a decayed sum of every value written under that key, so a block that contradicts an earlier block can only add to the earlier value. We propose Revisable Loop Memory, \method, which gives each token a fixed-size associative memory that persists across passes. At each block, the memory predicts the value for the block's key, and \method writes only the gated difference between the block's value and this prediction. The size of each write therefore depends on what the memory already holds: a block that agrees with the memory writes little, and a block that disagrees can replace the stored value for its key while leaving associations under orthogonal keys unchanged, which an additive write can do only by erasing the whole memory. The next pass reads the memory at the loop boundary. On Ouro-1.4B under matched fine-tuning, \method improves GSM8K by 2.3 points and MATH500 by 2.2 points over native fine-tuning with 0.2% more computation per token, and it outperforms additive writing with the same memory size.

Primary area:
foundation or frontier models, including LLMs
Keywords:
looped transformers, recurrent computation, attention residuals, neural network architectures, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jCXfOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce Revisable Loop Memory (RLM), a fixed-size associative memory that lets looped Transformers preserve and revise intermediate computations across loops. The main contribution of the paper is a gated delta-write mechanism that stores only the difference between a block’s proposed value and the memory’s current prediction, allowing targeted corrections leaving untouched unrelated associations. On Ouro-1.4B base model, matched fine-tuning and ablations show that RLM outperforms native fine-tuning variants, improving GSM8K and MATH500 by 2.3 and 2.2 points with only 0.2% additional computation.

Critical strengths

Strengths:

  • The authors propose Revisable Loop Memory (RLM), a gated delta-write memory that allows looped models to preserve and selectively revise intermediate computations across loops.

  • Under matched fine-tuning, RLM improves over native fine-tuning by 2.3% on GSM8K and 2.2% on MATH500, while adding only about 0.2% computation.

Critical weaknesses

Weaknesses:

  • The evaluation is limited to Ouro-1.4B. Although Ouro is also available at 2.6B and in reasoning-tuned variants, these settings are not evaluated, making it unclear whether RLM generalizes across model scales, checkpoint types, architectures, and broader training domains.

  • The backbone and memory interface are jointly fine-tuned, so the experiments do not isolate whether RLM benefits an existing frozen looped model. Consequently, it is difficult to determine how much of the improvement comes from the memory mechanism itself rather than from the backbone adapting to it.

  • The empirical gains are relatively modest. RLM improves over matched native fine-tuning by 2.3% on GSM8K and 2.2% on MATH500, while gains on most other benchmarks are below one point. This raises questions about whether the improvement is substantial enough to justify the added architectural complexity.

Questions for authors

Q1: How does RLM perform on Ouro-2.6B, reasoning-tuned Ouro checkpoints, or another looped backbone such as Nanbeige4.2-3B? These experiments would help establish whether the gains generalize across model scales, checkpoint types, and architectures. Moreover, a larger set of benchmarks spanning different domains is needed to fully assess the benefits of the propsoed approach

Q2: Could you evaluate RLM on a frozen backbone, training only the memory interface? This would help isolate the contribution of RLM from improvements enabled by jointly adapting the backbone.

Q3: Could you provide statistical significance estimates for RLM versus Native–FT? Confidence intervals are reported for selected ablations, but not for the main comparison against native fine-tuning.

Q4: Why are the improvements over Native–FT relatively small on most non-mathematical benchmarks, and do you expect larger gains with broader training data?

4 comments
AuthorsOpenReview ↗

Response to Reviewer jCXf [1/2]

W1. The evaluation is limited to Ouro-1.4B. Although Ouro is also available at 2.6B and in reasoning-tuned variants, these settings are not evaluated, making it unclear whether RLM generalizes across model scales, checkpoint types, architectures, and broader training domains.

Q1. How does RLM perform on Ouro-2.6B, reasoning-tuned Ouro checkpoints, or another looped backbone such as Nanbeige4.2-3B? These experiments would help establish whether the gains generalize across model scales, checkpoint types, and architectures. Moreover, a larger set of benchmarks spanning different domains is needed to fully assess the benefits of the propsoed approach

A1. Thank you for the suggestion. With the same interface hyperparameters and matched fine-tuning, RLM improves over Native–FT on Ouro-2.6B by 1.44 points on GSM8K and 1.40 points on MATH500. Both new backbones use the zero-shot greedy protocol of Table 3 with T=4 and a 2,048-token limit. Ouro-1.4B-Thinking is already tuned for reasoning and reaches 85.44 on GSM8K and 88.00 on MATH500 after Native–FT. On this checkpoint, RLM adds 0.23 points on GSM8K and is within one problem of Native–FT on MATH500.

BackboneNative–FT GSM8KRLM GSM8KNative–FT MATH500RLM MATH500
Ouro-1.4B (Table 3, 3 seeds)79.53 ± 0.3081.80 ± 0.2383.00 ± 0.5385.20 ± 0.40
Ouro-2.6B81.2782.7185.8087.20
Ouro-1.4B-Thinking85.4485.6788.0087.80

Across architectures, RLM interacts with the backbone only by reading block outputs and adding a scaled residual at loop boundaries. It leaves attention, the feed-forward blocks, the tokenizer, and the prediction head unchanged, so it attaches to any looped Transformer. On benchmark coverage, Table 2 already spans twelve benchmarks in knowledge (MMLU, MMLU-Pro), reasoning (BBH, ARC-C), commonsense (HellaSwag, Winogrande), mathematics (GSM8K, MATH500), and code (HumanEval, HumanEval+, MBPP, MBPP+), and Table 4 adds ARC-E and CommonsenseQA. RLM is higher than Native–FT on eleven of the twelve benchmarks in Table 2.


W2. The backbone and memory interface are jointly fine-tuned, so the experiments do not isolate whether RLM benefits an existing frozen looped model. Consequently, it is difficult to determine how much of the improvement comes from the memory mechanism itself rather than from the backbone adapting to it.

Q2. Could you evaluate RLM on a frozen backbone, training only the memory interface? This would help isolate the contribution of RLM from improvements enabled by jointly adapting the backbone.

A2. With the backbone frozen, training only the 1.07M interface parameters improves Ouro-1.4B by 0.61 points on GSM8K and 0.60 points on MATH500. The interface is trained on the same data and 32M-token budget with checkpoint selection by development NLL, and all rows below use the few-shot protocol of Table 2. The frozen blocks were never trained to produce memory keys or to read the boundary injection, and the interface still raises accuracy on both benchmarks.

ModelTrained paramsGSM8KMATH500
Ouro-1.4Bnone78.6281.80
RLM, frozen backbone1.07M79.2382.40
Native–FT1.4B80.8983.40
RLM, joint1.4B + 1.07M82.6485.60

Joint fine-tuning gives larger gains because the backbone and the interface adapt together, and Table 3 isolates the memory mechanism within this joint setting. Native–FT, RLM, and the four memory ablations all fine-tune the same backbone with the same data and budget. The additive variants stay between 79.68 and 80.21 on GSM8K, and only the corrective write with the memory-aware gate reaches 81.80, so backbone adaptation alone does not produce the gain.


W3. The empirical gains are relatively modest. RLM improves over matched native fine-tuning by 2.3% on GSM8K and 2.2% on MATH500, while gains on most other benchmarks are below one point. This raises questions about whether the improvement is substantial enough to justify the added architectural complexity.

A3. The gains are measured against a baseline that already receives the same fine-tuning, and they are comparable in size to the gain from fine-tuning the whole 1.4B backbone. In Table 2, full fine-tuning raises Ouro from 78.62 to 80.89 on GSM8K and from 81.80 to 83.40 on MATH500, and the 1.07M-parameter interface then raises these to 82.64 and 85.60. This gain costs about 0.2% more computation per token (10.90 vs. 10.87 GFLOPs in Table 5), while one more native pass adds about a quarter (13.54 GFLOPs). The interface keeps attention, the tokenizer, and the output head unchanged, its state does not grow with the number of loops, and a zero residual scale recovers the native computation.

AuthorsOpenReview ↗

Response to Reviewer jCXf [2/2]

Q3. Could you provide statistical significance estimates for RLM versus Native–FT? Confidence intervals are reported for selected ablations, but not for the main comparison against native fine-tuning.

A4. The paired bootstrap 95% interval for RLM minus Native–FT is [0.71, 3.89] on GSM8K and [−0.27, 4.67] on MATH500. We use the procedure of Table 3: 10,000 replicates that resample the three training seeds and all problems, with the same draws applied to every interface. The GSM8K interval excludes zero. On MATH500 the mean gain is 2.20 points, and the wider interval mainly reflects the 500-problem test set, where one problem moves accuracy by 0.2 points.

Contrast (3 seeds, paired bootstrap 95% CI)GSM8KMATH500
RLM − Native–FT2.27 [0.71, 3.89]2.20 [−0.27, 4.67]
RLM − Add–State1.74 [0.53, 3.01]1.60 [−0.20, 3.47]
RLM − Delta–Input1.36 [0.13, 2.63]1.20 [−0.73, 3.20]

We will add the Native–FT interval to the Table 3 caption.


Q4. Why are the improvements over Native–FT relatively small on most non-mathematical benchmarks, and do you expect larger gains with broader training data?

A5. The adaptation data contain only the GSM8K and MATH training splits, so every non-mathematical benchmark is out of domain, and RLM still improves all six general benchmarks. The interface is trained only through the final-loop loss on 32M mathematics tokens, so it learns which block outputs to keep and revise for mathematical reasoning. Its largest gains are on the two benchmarks that match this data, and it transfers positively to the general benchmarks (0.55 to 0.98 points) and to HumanEval, HumanEval+, and MBPP. Training on a broader mixture would supervise the memory on those domains directly, and we expect larger gains there.

Dear authors, Thank you for your answers.

I still have concerns about the benefit of the proposed approach. Gains seem to be quite modest. Especially, concerning models such as Ouro-2.6B.

I'll keep my score.

Thank you

Dear Reviewer jCXf,

Thank you for reading our rebuttal and for the follow-up.

We agree that the absolute gains are modest, including on Ouro-2.6B. We reported these numbers so the improvement is easy to judge. Two points may still be useful: the comparison is against a baseline that already received the same fine-tuning, and the extra cost is only 0.2% more compute. In our ablations, an additive memory of the same size does not give this gain; only the corrective write does.

We appreciate the time you have spent on the review :)

Best regards, The authors

Reviewer_jKfsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Revisable Loop Memory for looped Transformers. Each block writes a gated correction to a fixed-size memory, which is read at loop boundaries.

Critical strengths

The method is clear and reports improvements over matched fine-tuning baselines. The controlled ablations support the benefit of delta updates and memory-aware gating.

Critical weaknesses

The motivation needs stronger support. The native model is trained end to end, so the final loss can already encourage it to preserve information needed by later loops. The lack of direct access to intermediate outputs does not establish that useful information is lost. Table 1 shows diminishing gains across loops, but this could also reflect ordinary convergence.

The experiments support the method's effectiveness, but do not establish an information bottleneck in the native interface. Please provide evidence for this explanation or present it more clearly as a hypothesis.

Questions for authors

The experiments support the method's effectiveness, but do not establish an information bottleneck in the native interface. Please provide evidence for this explanation or present it more clearly as a hypothesis.

2 comments
AuthorsOpenReview ↗

Response to Reviewer jKfs [1/1]

W1 (part 1). The motivation needs stronger support. The native model is trained end to end, so the final loss can already encourage it to preserve information needed by later loops. The lack of direct access to intermediate outputs does not establish that useful information is lost.

A1. Thank you for the careful reading. Table 3 tests exactly this case: the native interface trained end to end, against memories trained end to end under the same loss. Native–FT uses the same data, budget, four passes, and final-loop loss as RLM, so its final hidden state has the same incentive to preserve what later loops need, while it also serves as the input to the next pass. Adding a memory of the same size trained the same way does not by itself close the gap: the additive memories stay between 79.68 and 80.21 on GSM8K, against 79.53 for Native–FT, while only the corrective write with the memory-aware gate raises GSM8K to 81.80 and MATH500 from 83.00 to 85.20. This matches Proposition 1: a delta write can replace the value stored for one key while leaving orthogonal keys unchanged, and an additive write can do this only by erasing the whole memory.


W1 (part 2). Table 1 shows diminishing gains across loops, but this could also reflect ordinary convergence.

W1 (part 3) & Q1. The experiments support the method's effectiveness, but do not establish an information bottleneck in the native interface. Please provide evidence for this explanation or present it more clearly as a hypothesis.

A2. Section 1 presents Table 1 as a design question rather than as proof of information loss, and a boundary intervention shows that later passes read content from the memory. Section 1 reads: "Ordinary convergence explains part of this pattern, but the measurement raises a design question: what should one pass hand to the next, and how should a later pass revise what earlier passes computed?" To test directly whether later passes read useful content from the memory, we intervene once at loop boundary t=2 of the trained RLM model. One condition replaces each memory matrix with a random matrix of the same Frobenius norm. The control keeps the memory and adds noise to the boundary input, scaled to match the shift in boundary input caused by the replacement. We measure response-token NLL against the unmodified model at the same depth. Replacing the memory raises NLL 2.4 times as much as the matched noise at T=4, and 2.7 times as much at T=8.

Final depthMatched residual noise, ΔNLLMemory replacement, ΔNLL
T=4+0.010+0.024
T=8+0.007+0.019

Following your suggestion, we will state the motivation explicitly as a hypothesis in Section 1: the native interface passes only the final hidden state between loops, and a key-addressed, revisable memory lets later passes use earlier block outputs more effectively. Table 3 and this intervention test that hypothesis.

Dear Reviewer jKfs, we would like to follow up on whether our response has fully addressed your concerns. We are also happy to answer any further questions.

Reviewer_YvoKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Revisable Loop Memory quips looped/latent-recurrent Transformers with fixed-size associative matrices updated via delta-rule (corrective, not additive) writes. At each block, the hidden state produces a key/candidate value; memory predicts the value; a gate controls residual write-back. At loop boundaries, memory is injected via scaled residuals — leaving causal attention and prediction heads unchanged.

Critical strengths

Timely; Looped transformer are getting focused those days

Sharp problem focus; mechanism rigorously grounded. Proposition 1 formalizes delta writes as truly local and orthogonal-preserving, with clear contrast to additive writes.

Budgeted ablations and honest cost reporting: ~0.2% FLOP overhead, ~1.07M extra params, no change to memory scaling.

Critical weaknesses

Core claims lack robust stats: main results use one seed; MATH500 confidence intervals include zero — support is solid only on GSM8K. Depth extrapolation confounds RLM with protocol/adaptation effects.

Questions for authors

Could you please supplement the bootstrap intervals for Table 2 with at least one additional training seed (or a second backbone model), and include a matched-protocol Native-FT baseline at T=5–8? This would let us verify the statistical robustness of the core improvements and the claim that RLM specifically supports depth extrapolation.

4 comments
AuthorsOpenReview ↗

Response to Reviewer YvoK [1/1]

W1. Core claims lack robust stats: main results use one seed; MATH500 confidence intervals include zero — support is solid only on GSM8K.

Q1 (part 1). Could you please supplement the bootstrap intervals for Table 2 with at least one additional training seed (or a second backbone model)

A1. Thank you for the question. The 2.3 and 2.2 point gains over Native–FT come from Table 3, which already averages three training seeds (17, 29, 43). Only the twelve-benchmark evaluation in Table 2 and the depth study in Table 4 use the seed-17 checkpoint. The MATH500 intervals in the Table 3 caption compare RLM with the two strongest memory ablations, Add–State and Delta–Input, rather than with Native–FT. Applying the same paired bootstrap, which resamples both seeds and problems (Reproducibility Statement), gives the RLM minus Native–FT intervals below. The GSM8K interval excludes zero. On MATH500 the mean gain is 2.20 points with a lower bound of −0.27, which reflects the 500-problem test set, where one problem moves accuracy by 0.2 points.

Contrast (3 seeds, paired bootstrap 95% CI)GSM8KMATH500
RLM − Native–FT2.27 [0.71, 3.89]2.20 [−0.27, 4.67]
RLM − Add–State1.74 [0.53, 3.01]1.60 [−0.20, 3.47]
RLM − Delta–Input1.36 [0.13, 2.63]1.20 [−0.73, 3.20]

Following your suggestion, we also evaluate a second backbone, Ouro-2.6B, together with the reasoning-tuned Ouro-1.4B-Thinking, under the zero-shot greedy protocol of Table 3 with T=4 and a 2,048-token limit. On Ouro-2.6B, RLM improves over Native–FT by 1.44 points on GSM8K and 1.40 points on MATH500. On Ouro-1.4B-Thinking, which is already tuned for reasoning, RLM is 0.23 points higher on GSM8K and within one problem of Native–FT on MATH500.

BackboneNative–FT GSM8KRLM GSM8KNative–FT MATH500RLM MATH500
Ouro-1.4B (Table 3, 3 seeds)79.53 ± 0.3081.80 ± 0.2383.00 ± 0.5385.20 ± 0.40
Ouro-2.6B81.2782.7185.8087.20
Ouro-1.4B-Thinking85.4485.6788.0087.80

W2. Depth extrapolation confounds RLM with protocol/adaptation effects.

Q1 (part 2). and include a matched-protocol Native-FT baseline at T=5–8? This would let us verify the statistical robustness of the core improvements and the claim that RLM specifically supports depth extrapolation.

A2. Against matched Native–FT, RLM is higher on every benchmark at every depth from T=5 to T=8, and the gap grows with depth, from 0.62 points at T=4 to 1.41 points at T=8. Native–FT starts from the same Ouro-1.4B checkpoint and uses the same GSM8K and MATH data, 32M-token budget, T=4 training, seed 17, checkpoint selection, and few-shot evaluation as RLM in Table 4, so only the loop count T changes at test time. From T=4 to T=8, the Native–FT average falls by 0.99 points, while RLM changes by 0.20 points. The extrapolation therefore comes from the memory interface rather than from the adaptation protocol.

BenchmarkT=4 Native–FT / RLMT=5T=6T=7T=8
ARC-C62.12 / 62.8062.20 / 63.2361.95 / 63.0561.52 / 62.8860.84 / 62.63
ARC-E84.68 / 85.4084.81 / 85.6984.43 / 85.7784.01 / 85.4883.59 / 85.27
C-QA76.58 / 77.2376.66 / 77.6476.25 / 77.5675.84 / 77.3175.35 / 77.07
HellaSwag75.32 / 75.9175.38 / 76.0875.10 / 75.9974.78 / 75.7674.45 / 75.48
MMLU68.93 / 69.4869.02 / 69.8168.84 / 69.9468.55 / 69.7268.21 / 69.36
Winogrande73.64 / 74.1973.88 / 74.5173.56 / 74.7473.24 / 74.3572.93 / 74.03
Average73.55 / 74.1773.66 / 74.4973.36 / 74.5172.99 / 74.2572.56 / 73.97

We will add this matched Native–FT to Table 4.

Dear Reviewer YvoK, we hope that our response and additional results address your concerns. Please let us know your thoughts, and we would be happy to answer any further questions.

Thanks for your reply. I have kept my score as positive.

Dear Reviewer YvoK,

We sincerely appreciate your thoughtful feedback and constructive engagement throughout the review process, which have helped us improve the clarity and presentation of our work.

Cheers,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.