- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper explores test-time training (TTT) when the model learns from its own generated text. It mainly compares Closed Loop: uses updated model to generate next round training chunk and Writes Off: puts training chunk into context without any model update. This paper found the root cause of degradation in TTT comes from the feedback loop in which updated weights determine the text used for later updates. In the end, the authors proposed Settlement that each update is held pending and committed only if it does not increase loss on independent real text.
Critical strengths
This paper conduct a complete and well-defined experiment control. By comparing different model scale, books, seeds, generator update, and schedule, this paper achieves state restoration and generator isolation carefully. This carefully defined experiment rules out Fixed Generation and real-text learning that update model weights with less harm.
Critical weaknesses
W1: First, this paper restrict the scope on simple self-writing TTT that apply next-token updates base on the model's own unconditioned, generation without any reward, verifier, or interleaved external input. Under this setting, the Closed Loop collapses into a repetition attractor (repeated-4 = .99) and it kind of push the model weight into a state showing abnormal behaviours. This is a well known problem of TTT and this paper just designed a set of experiment to reveal this already known problem without any insight about why using updated weights determine the text used for later updates will decrease performance.
More fundamentally, the influence of TTT self-writing is in two side: it can help or harm for the following generation. This paper only captures the harm of self-writing: all language-model outcomes are Negative Log Likelihood on independent human-written text, which generated chunks cannot help by construction. This makes the Write-Off works as performance celling because it do not update model weight in a negative direction.
W2. Limited novelty of Settlement. Settlement commits an update only if it does not increase loss on held-out real text. This is a standard validation-based accept/reject rule. It should work as baseline instead of proposed solution. This Settlement also shows less practical as it requires identifiable real text for validation and needs the real deployment of the model update to assess its quality.
Questions for authors
-
In Page 9, line 316, the authors claimed that "The write effect is confirmed at both 125M and 3B." However, the table 4 shows a significant 12x less NLL than 125M scale. The same effect also appears in table 1 as 3B scale with significant less H difference. Could you please justify why?
-
Could the authors provide some qualitative inspection of the closed-loop generated training chunks that also degrade the performance of replay recorded so significantly.
-
The proposed Settlement is defined at very late section 6 and used before it in section 5. The authors would be better to change the order of presentation.
-
The author would be better to evaluate at least one setting where self-writing could help, such as recall of self-generated content beyond the window
4 comments
Response to Reviewer XeCQ: W2
W2. Settlement: Motivation, Design, and Practical Use
Per-update validation is not common in continuous test-time training for language models. Earlier methods address unstable adaptation through sample selection (EATA [11]), optimisation and recovery (SAR [12]), or weight restoration (CoTTA [13]). These methods do not use independent real-text prediction to decide whether to retain each proposed update. Concurrent work also explores selective commitment: VANE [14] uses future visual observations, while CGPI [15] uses navigation-specific interaction evidence. These methods operate in different settings and do not study persistent feedback from self-generated language.
Settlement goes beyond a standard accept/reject rule by validating the accumulated model state before committing an update. Settlement is motivated by a specific failure mechanism established in our experiments. An update can improve the model's fit to generated text while hurting its prediction on real text. More importantly, two updates can each pass validation but become harmful when combined.
Table 26 tests this problem directly. Joint Commit checks two updates separately. It tests and , but then commits without testing the combined model. In our additional analysis, we examined 2,226 pairs of updates that passed validation individually. In 238 pairs (10.7%), adding the second update increased validation loss relative to keeping the first alone. This shows that an update that looks safe on its own may no longer help after another update has been accepted.
Settlement avoids this problem by checking the model after combining the proposed update with all previously accepted updates. If the first update is kept, the second is tested on top of it. The second update is accepted only if the resulting model does not increase validation loss relative to the current model. We also compared this approach with a standard fixed-reference validation baseline. Both methods apply updates sequentially and use the same scheduled validation passages and evaluation budget. Fixed-Reference accepts an update as long as the resulting model remains better than the original model. Sequential Validation instead requires it to be no worse than the model currently in use.
| Policy | Real-text B ↑ |
|---|---|
| Joint Commit | −1.093 |
| Scheduled Fixed-Reference | −0.027 |
| Sequential Validation | +0.019 |
| Settlement | +0.038 |
B measures improvement in real-text NLL change relative to No Updates. Joint Commit performs poorly despite validating individual updates. Under the same scheduled validation protocol, Sequential improves B over Fixed-Reference by 0.046 nats, with positive differences across all three offset groups. This matters because an update can still outperform the original model while making the current model worse.
The claim that Settlement requires deploying an update before evaluating it is incorrect. Algorithm 1 evaluates the accepted state and the candidate state on the same validation text, starting from the same restored context. Both evaluations are read-only. The candidate enters the live model only after passing validation, so evaluating it does not require exposing subsequent live generation to an unvalidated update.
Settlement is not restricted to settings with identifiable real text. Real-text loss provides the validation signal in our language-model experiments; in agent workflows, candidate updates can be evaluated using task reward or success. Our submission includes WebShop, where Settlement selects updates using validation reward and improves exact task success over Writes Off. We also added ALFWorld results, where Settlement uses task-success validation.
| Experiment | Writes Off | Settlement | Difference |
|---|---|---|---|
| WebShop exact success | 16.00% | 18.53% | +2.53 pp |
| ALFWorld unseen-task success, rank (8) | 86.8% | 91.0% | +4.2 pp |
In WebShop, Settlement retains 10 of 180 generated updates. Its exact-success improvement over Writes Off is 2.53 percentage points. The ALFWorld results are means over three paired seeds; the archived records do not establish separation between validation and outcome evaluation, so we treat them as supporting observations.
[10] Wang, et al. "No Time Like the Present: Agentic Test-Time Training for LLM Agents." arXiv, 2026.
[11] Niu et al. Efficient Test-Time Model Adaptation without Forgetting. ICML, 2022.
[12] Niu et al. Towards Stable Test-Time Adaptation in Dynamic Wild World. ICLR, 2023.
[13] Wang et al. Continual Test-Time Domain Adaptation. CVPR, 2022.
[14] Ji et al. VANE: Reliable Test-Time Training for Vision-Language-Action Models via Future Visual Representation Prediction. arXiv, 2026.
[15] Li et al. Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigation. arXiv, 2026.
Response to Reviewer XeCQ: Q1 and Q2
Q1. Effect smaller at 3B
The write effect is consistently positive, but its magnitude differs substantially across models. Our claim concerns the presence of the effect, not a uniform effect size across scales.
In Table 4, retaining updates adds 2.268 nats of excess NLL at 125M, compared with 0.1927 nats [0.1043, 0.3272] at 3B. Both effects are positive, although the latter is approximately twelve times smaller.
Importantly, the results do not follow a monotonic scaling trend. Table 1 reports excess NLL increases of 3.01, 6.00 and 0.40 nats at 125M, 760M and 3B, respectively. The smaller models are trained reproductions, whereas 3B uses a released checkpoint with a different training lineage. Model size and training history are therefore not independently controlled, so these experiments cannot establish why the effect is smaller at 3B.
Our conclusion is consequently narrower and well supported: retaining self-generated updates can harm independent prediction at multiple scales, but the severity is model-dependent. We will revise the statement to read: “Retaining generated-text updates adds a positive replay cost at both 125M and 3B, although the effect is substantially smaller at 3B.”
Q2. Qualitative inspection of the generated training text
Thank you for this suggestion. We inspected the actual Closed Loop generations and compared them with Fixed Generation, where the learner continues to update but the generator remains frozen. The following excerpts come from the stored 125M outputs (Book 3, seed 1) at corresponding stream positions.
| Chunk | Closed Loop | Distinct-2 / Repeated-4 | Fixed Generation | Distinct-2 / Repeated-4 |
|---|---|---|---|---|
| 1 | "I never once heard anybody call it any other of his Perrieries..." | 0.93 / 0.00 | "an ingenuity which could understand so much of that genial little intelligence..." | 0.89 / 0.00 |
| 12 | "the gentleman--the author, the colonel--the colonel!" | 0.64 / 0.07 | "Harder--harder than when I was under your cares. And, you say..." | 0.85 / 0.01 |
| 30 | "I could not write--I could not write--I could not write..." | 0.01 / 0.99 | "Mr. Giles was going home to London, and even could stand the sight of her fancy-paper..." | 0.89 / 0.01 |
| 105 | "I could not write--I could not write--I could not write..." | 0.01 / 0.99 | "an extended plunge,--a retreat,--a slip step,--jumping crocodiles with a flute..." | 0.91 / 0.00 |
Closed Loop progressively degenerates into repeated short phrases, while Fixed Generation maintains lexical diversity. By chunk 30, the Closed Loop text has almost entirely collapsed. Across all 30 inspected Closed Loop streams (six books, five seeds), 29 end in exact cycles of at most 25 tokens. This agrees with Table 8, where final repeated-4 reaches 0.9902 for Closed Loop, compared with 0.0082 for Fixed Generation.
We also examined the actual recordings used in the fixed-receiver Replay experiment and matched the text to its measured write cost. Two passages from the same source trajectory provide a particularly clear contrast:
| Recorded source | Generated text | Distinct-2 / Repeated-4 | Replay write cost (nats) |
|---|---|---|---|
| Book 2, seed 7, stage 81 | "to have anything to have done with you about it..." | 0.02 / 0.98 | +0.02 |
| Book 2, seed 7, stage 97 | "ononononononon..." | 0.00 / 1.00 | +5.38 to +6.97 |
These examples come from a separate recording used for Table 19. Diversity scores describe the displayed chunks; write costs are measured over the corresponding 8,192-token passages, with ranges across three receiver configurations.
Despite similarly extreme repetition, the two passages have markedly different write costs. Table 19 confirms that damage is concentrated rather than uniform: the mean passage-level write cost rises from 0.0439 to 0.7512 nats, while the median falls from 0.0422 to 0.0064. At the final measured position, only 12 of 72 comparisons exceed 0.5 nats, all originating from just four of the 24 recorded source sequences. A small subset of source trajectories therefore accounts for the large passage-level losses.
Why does replaying fixed text still cause harm? Table 4 isolates the effect of retaining updates from identical recorded text. Reading alone increases real-text NLL by 1.606 nats relative to the receiver control; reading and writing increases it by 3.874 nats. The additional 2.268 nats comes from persistent weight updates. Table 5 provides complementary evidence of a transfer conflict: after Closed Loop adaptation, a single update lowers source-chunk NLL by 0.0354 nats on average while increasing independent real-text NLL by 0.0381 nats.
The key finding is that repetition alone does not identify harmful updates. The qualitative examples show how Closed Loop changes its training text. Replay and single-update experiments show that learning from this text can improve source fit while harming independent prediction.
Response to Reviewer XeCQ: Q3 and Q4
Q3. Presentation order
Thank you for this helpful suggestion. Sections 4 and 5 establish that self-generated feedback degrades prediction and explain why an update can fit its source while harming new text. These findings motivate the simple validation rule introduced as Settlement in Section 6: retain an update only after checking its effect on independent text.
The WebShop comparison in Section 5 was the only place in Sections 4–5 that reported a Settlement result before its definition. We have moved this result to Section 6.2, page 8, lines 423–426, after the method definition and real-text results, and before the comparison with Source Masking. The revised passage compares Settlement with Closed Loop and Fixed Generation and explains how selective retention supports task performance. These changes are highlighted in blue. We thank the reviewer again for helping us improve the clarity of the presentation.
Q4. Can self-writing provide a benefit?
Yes. Our new experiments show that self-writing can help a model remember its own generated answers beyond the available context, as well as improve language modelling and downstream task performance.
1. Recalling self-generated answers.
We tested the reviewer's suggestion directly using 64 questions from 2WikiMultihopQA, two generation seeds, and a frozen Qwen3.8-27B model [16] with trainable rank-16 LoRA adapters. The model first generated answers with the supporting passages in context. We then trained the adapters on these answers, conditioned on the corresponding questions and record IDs. The training schedule was chosen on separate calibration examples and fixed before testing (16 Adam steps per four answers, learning rate .
All conditions then processed the same 32K tokens of real text and eight real-text updates. For evaluation, we cleared the KV cache and asked the original questions again, providing neither the supporting passages nor the generated answers.
| Condition | QA F1 ↑ | Exact recall ↑ |
|---|---|---|
| No generated-answer update | 17.37 | 18.75% |
| Write-All | 61.56 | 93.75% |
| Shuffled-answer update | 16.29 | — |
Self-writing raises QA F1 by 44.19 points (95% paired bootstrap CI [29.40, 57.89]), while exact recall of the model's own answers increases from 18.75% to 93.75%. When we shuffle the same generated answers across questions, keeping the training budget unchanged, F1 drops to 16.29. The benefit therefore comes from learning the question–answer associations, not simply from additional exposure to generated text.
2. What happens with a single update? We also examined a single-update setting with the 3B TTT-E2E model, using one update on a generated person–location record followed by 64K tokens of real text, well beyond its 8K attention window.
| Condition | Answer NLL ↓ |
|---|---|
| No update | 3.9241 |
| Correct record update | 3.7082 |
| Permuted record update | 3.7070 |
Writing the correct record reduces answer NLL by 0.2159 nats (95% CI [0.1809, 0.2534]). However, permuting the person–location associations produces almost the same improvement. This suggests that a single update can improve answer likelihood without reliably preserving the underlying associations. The Qwen experiment offers stronger evidence of recall, particularly through its shuffled-answer control.
3. Benefits beyond recall.
We further find that self-generated updates can improve independent language modeling. In our new 3B PG19 experiment, covering 16 books and three seeds per book, Settlement retains 154/384 generated updates and improves real-text NLL over Gen-Off by 0.001337 nats/token (95% CI [0.000255, 0.002466]). This improvement remains positive after correction for three primary comparisons.
We see similar benefits in agent tasks:
| Setting | Writes Off | Settlement | Retained updates |
|---|---|---|---|
| WebShop exact success ↑ | 16.00% | 18.53% | 10/180 |
| WebShop mean reward ↑ | 0.3309 | 0.3823 | 10/180 |
| ALFWorld, LoRA rank 8 ↑ | 86.8% | 91.0% | 12/36 |
| ALFWorld, LoRA rank 16 ↑ | 86.8% | 90.0% | 7/36 |
The agent results are encouraging, although the WebShop comparison uses five seeds and ALFWorld uses three paired seeds, so the precision of these gains remains limited.
The generated-answer experiment directly addresses the suggested setting: the model remembers its generated answers better after training on them, even after processing 32K tokens of other text and clearing its context. TShuffling the answers across questions removes the benefit, indicating that the model learns which answer belongs to which question. In a separate 3B experiment, retaining selected generated updates also lowers NLL on held-out real text compared with rejecting all generated updates.
[16] Qwen Team. Qwen3.8-27B. Hugging Face model card, 2026. https://huggingface.co/Qwen/Qwen3.8-27B
Response to Reviewer XeCQ: W1
Thank you for your time and your constructive suggestions. We appreciate your recognition of our “complete and well-defined experiment control” and that our paper “achieves state restoration and generator isolation carefully.”
W1.1: "paper restrict the scope on .... TTT that ....base on the model's own unconditioned, generation without a..... interleaved external input."
Thanks for your comments. We suppose that the reviewer may have misunderstood our scope. Generation starts from an 8K real-text prefix (Sec. 3). We also test interleaved real text (Sec. 4; Appendix B.3), real-text validation through Settlement (Sec. 6.2), and task-reward validation in WebShop (Appendix G).
In Table 12, real text makes up 31% of each stream, either spread evenly or delivered in blocks. The model always learns from real text. Learning from generated text, compared with simply reading it, adds 0.0694 nats of NLL with evenly spaced real text and 0.7155 nats with blocks. Real-text training therefore does not necessarily eliminate the harm, and its timing matters. Unfiltered Closed Loop is a baseline, not our entire study.
W1.2. "paper just designed a set of experiment to reveal this already known problem"
Thanks for the feedback. We respectfully disagree. Earlier studies examined collapse from repeated training on synthetic data [1–3]. Recent work has developed test-time learning in other settings [4–9]. Our contribution is not another observation of collapse. We show how retained updates harm prediction and how feedback amplifies that harm.
First, generated text alone does not explain the degradation. In Fixed Generation, the learner still trains on generated text, but its updates cannot affect future generations. Excess NLL falls from 2.8308 to 0.0707 nats, a 97.5% reduction (Table 8). Removing this feedback eliminates nearly all the degradation in this setting, although the learner still trains on generated text.
Second, an update can fit its training text better while making other predictions worse. In Recorded Replay, learning from the same recorded text adds 2.268 nats of excess NLL compared with reading it without retaining updates (Table 4). Single-update experiments show this tradeoff directly: an update can lower loss on generated text while raising loss on independent real text (Table 5). Greater conflict between the two gradients is associated with greater harm (Table 6).
Third, repetition is a warning sign, but it does not tell us which updates to retain. aTTT downweights repeated tokens [10]. We replay recorded passages on fixed receiver models and measure how much learning from each passage changes real-text NLL. Mean write cost rises from 0.0439 to 0.7512 nats, while the median falls from 0.0422 to 0.0064 (Figure 4; Table 19). Large write costs are concentrated in a few passages. Q2 shows that similarly repetitive passages can have very different write costs.
These findings motivate Settlement: it checks an update’s effect on independent real-text loss before retaining it. PWUP likewise recognizes “a more specific explanation of the failure than parameter drift or poor generated text alone,” while 9HjW finds our interventions “more persuasive than correlations between drift and performance.”
W1.3. "generated chunks cannot help by construction".
You have raised an important point. However, our new experiments show that selected generated updates can improve prediction on held-out real text. At 3B, Settlement and No-Generated have access to the same real-text passages and run the same number of model calls. Settlement selects generated updates using same-book validation; No-Generated discards every generated update. Settlement lowers NLL by 0.001337 nats/token across 16 PG19 books (Appendix H.1). On 32 additional books, the reduction is 0.001098 nats/token.
These results show that self-writing can improve independent real-text prediction. Q4 also tests recall of self-generated answers after they are removed from context.
[1] Shumailov et al. AI Models Collapse When Trained on Recursively Generated Data. Nature, 2024.
[2] Alemohammad et al. Self-Consuming Generative Models Go MAD. ICLR, 2024.
[3] Gerstgrasser et al. Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. COLM, 2024.
[4] Sun et al. Learning to (Learn at Test Time): RNNs with Expressive Hidden States. ICML, 2025.
[5] Akyürek et al. The Surprising Effectiveness of Test-Time Training for Few-Shot Learning. ICML, 2025.
[6] Hu et al. Test-Time Learning for Large Language Models. ICML, 2025.
[7] Tandon et al. End-to-End Test-Time Training for Long Context. arXiv, 2025.
[8] Bansal et al. Let's (not) just put things in Context: Test-time Training for Long-context LLMs. ICLR, 2026.
[9] Zhu et al. Self-Guided Test-Time Training for Long-Context LLMs. arXiv, 2026.