acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Training for Pretrained LLMs: An Empirical Study of Limited Fast-Weight Utilization

Abstract

Test-time training (TTT) enables a model to learn from its input context during inference by updating a subset of its parameters, effectively turning them into a form of dynamic memory. In-Place TTT extends this approach to pretrained LLMs by repurposing existing MLP parameters as fast weights through continued pretraining. Unlike dedicated memory trained from scratch, these parameters already encode useful computation, creating a tension between contextual adaptation and pretrained capabilities. We study this tension through a detailed analysis across multiple fast-weight learning rules, varying the update objective as well as write and forget gating mechanisms. Our central finding is that performance gains can arise largely from changes learned by the underlying model during TTT-aware training, while the direct contribution of inference-time fast-weight updates is often surprisingly small. To understand how this behavior emerges, we examine training dynamics and find that TTT-aware training progressively limits how strongly fast-weight updates alter the model’s computation, through mechanisms that depend on the learning rule. Amplifying updates at inference time can leave long-context retrieval intact while degrading pretrained capabilities, suggesting that capability preservation may partly explain this suppression. We further analyze how TTT-aware training shapes context utilization: the underlying model learns to attend more strongly to relevant evidence even without fast-weight updates, whereas the write gates governing those updates become more selective and focused on query-relevant tokens despite their limited direct contribution. Together, our results highlight the challenges of turning pretrained parameters into dynamic memory without disrupting the computation they already encode.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Test-time training, parametric memory, continual learning, meta-learning, large language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_b3zeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper studies In-Place Test-Time Training (TTT), where pretrained MLP parameters are updated online as fast weights. It compares Frobenius, Delta, gated, and decay-based update rules on Qwen3-4B and LLaMA-3.1-8B across RULER, HELMET, and pretrained-capability benchmarks. The main finding is that TTT-aware training often improves the underlying model even when inference-time fast-weight updates are disabled; the direct TTT effect is usually small and sometimes negative. The authors also show that training suppresses update magnitude, likely because larger updates interfere with pretrained capabilities, while query-aware prompting makes writing more selective.

Critical strengths
  1. The on/off disentanglement is a simple, valuable protocol, and it exposes a real gap in how TTT-for-LLM results are reported. Comparing the same trained model with fast-weight updates on and off separates the effect of inference-time adaptation from everything else that TTT-aware training changes. Prior In-Place TTT work does not report this comparison. Across six learning rules, two backbones, two training stages and two benchmark families, TTT-Effect stays within about ±1 point almost everywhere. That is notable for models trained explicitly with active updates. Because it is a paired comparison on the same model and the same examples, it is the most robust result in the paper. It should change how future work in this area evaluates and attributes improvements, and this is the main reason I see value in publishing.
  2. Thia paper provides a systematic, mechanism-level look at how training controls update magnitude across learning rules. The paper goes beyond benchmark tables. It tracks the relative output change, ‖V‖, the β gates and β selectivity over training.
Critical weaknesses
  1. The headline claim, that TTT-Benefit comes largely from the outer loop, rests on differences that are probably within run-to-run noise. Tables 2–5 report many differences of less than one percentage point, but the paper does not provide results across multiple independent training or evaluation repetitions. This makes it difficult to assess whether the small TTT-Benefit and TTT-Effect values are reliable.
  2. The proposed explanation for suppression (preserving pretrained capabilities) is not tested against an obvious alternative: full attention already makes the fast weights redundant. In-Place TTT keeps full causal attention over the whole context, and every evaluation fits inside the attention window. Attention already has lossless access to every needle, so outer-loop training has little reason to rely on a lossy fast-weight memory, whatever the cost to pretrained computation.
  3. Narrow experimental scope for general claims about pretrained parameters as dynamic memory. All conclusions come from one framework with one fixed set of hyperparameters: MLP down-projections in 6 of 36 layers, 4K chunks, η=3, β and α initializations taken from or near the original. There is no sweep over the number or placement of TTT layers, chunk size, inner-loop learning rate, or which parameters act as fast weights. Small TTT-Effect may well be a property of this configuration rather than of fast weights in pretrained LLMs in general. The LLaMA replication is partial: Stage 1 only, three rules, up to 64K. The capability-preservation test (Figure 2b) covers only GDN and GDN-Residual. The paper also does not reconcile its results with the original In-Place TTT paper. As I understand it, that paper reports clear long-context gains over its baseline under a nominally matched setup. Here, the per-sample fast-weight reset with padding to chunk boundaries is at least one visible deviation. Without that reconciliation, readers cannot tell whether this is a negative finding about the method or a reproduction gap. This limits how far the conclusions in the title and abstract can be generalized.
Questions for authors
  1. How much do scores vary across seeds for standard-CPT and for at least one TTT variant? Can you report bootstrap confidence intervals for TTT-Benefit, TTT-Effect and the outer-loop effect in Tables 2, 3 and 5? If the outer-loop effect does not exceed CPT seed variance, the central claim should be reframed.
  2. Did the authors reproduce the In-Place TTT paper’s reported RULER numbers with the Frobenius variant?
  3. For the delta variants, what are TTT-Effect and the relative output change at step 0, is the decline in Figure 1a mostly undoing the large initial erasure caused by V≈0, E≈−O and η=3?
4 comments

Dear Reviewer b3ze,

We sincerely thank the reviewer for the thoughtful and constructive assessment of our work. We especially appreciate the reviewer's recognition of the value and robustness of our TTT-ON/OFF comparison protocol, its potential impact on future TTT research, and our systematic, mechanism-level analyses of fast-weight updates.

1. Are the small TTT-Benefit and TTT-Effect values reliable given training and evaluation variance? (W1, Q1)

We agree that small performance differences relative to standard-CPT should be interpreted cautiously, and we conducted two additional analyses to quantify variability.

Training-seed variability. We trained additional independent runs of standard-CPT (2 seeds) and Frobenius (3 seeds), and evaluated all checkpoints on the same fixed evaluation set. The table below reports mean ± std across training seeds.

BenchmarkLengthCPT Perf.Frobenius Perf.TTT-BenefitTTT-Effect
RULER16K89.7 ± 0.189.8 ± 0.5−0.2 ± 0.0+0.1 ± 0.0
32K84.8 ± 0.284.2 ± 0.6−1.0 ± 0.3−0.1 ± 0.3
64K79.0 ± 0.779.3 ± 0.7−0.1 ± 0.90.0 ± 0.4
128K69.7 ± 0.568.3 ± 0.7−1.5 ± 1.4−1.6 ± 0.4
HELMET16K56.1 ± 0.256.7 ± 0.7+0.4 ± 0.7+0.3 ± 0.1
32K60.3 ± 0.860.6 ± 0.3+0.4 ± 0.5+0.3 ± 0.3
64K61.7 ± 0.263.2 ± 0.7+1.8 ± 0.9+0.5 ± 0.3
128K55.6 ± 0.655.3 ± 0.9+0.1 ± 0.1−0.4 ± 0.7

Across independent training runs, TTT-Benefit is task- and context-dependent: it can yield clear improvements over standard-CPT (e.g., +1.8 ± 0.9 on HELMET at 64K) while remaining negligible or negative elsewhere. In contrast, TTT-Effect is consistently small: its positive values are at most +0.5 ± 0.3, even where TTT-Benefit is largest. The only notable TTT-Effect is negative, on RULER at 128K (−1.6 ± 0.4), where it matches TTT-Benefit (−1.5 ± 1.4), so disabling the updates recovers standard-CPT performance. Thus, in no setting does TTT-aware training induce substantial positive reliance on inference-time fast-weight updates.

Evaluation-seed variability. We additionally re-evaluated all trained checkpoints on three independently sampled evaluation sets (seeds 42, 43, 44). Unlike bootstrapping, which can only resample examples from a single fixed set, this draws new evaluation instances and therefore also captures variability that a single draw cannot reveal. The table below reports results averaged across the six TTT variants (mean ± std across evaluation seeds), for the standard (CQ) and query-aware (QCQ) RULER formats and HELMET.

BenchmarkLengthCPT Perf.TTT Perf.TTT-BenefitTTT-Effect
RULER (CQ)16K89.0 ± 0.989.0 ± 1.00.0 ± 0.20.0 ± 0.1
32K84.6 ± 0.284.3 ± 0.1-0.3 ± 0.3+0.1 ± 0.0
64K78.6 ± 0.278.2 ± 0.7-0.4 ± 0.60.0 ± 0.1
128K69.5 ± 0.568.3 ± 0.6-1.2 ± 0.3-1.1 ± 0.2
RULER (QCQ)16K87.4 ± 0.687.6 ± 0.7+0.3 ± 0.10.0 ± 0.1
32K82.8 ± 0.883.3 ± 0.5+0.5 ± 0.5+0.1 ± 0.1
64K78.2 ± 0.878.3 ± 0.8+0.1 ± 0.00.0 ± 0.2
128K73.6 ± 1.973.3 ± 1.4-0.2 ± 0.4-0.2 ± 0.1
HELMET16K55.6 ± 0.955.6 ± 1.20.0 ± 0.4-0.2 ± 0.2
32K58.5 ± 1.559.0 ± 1.6+0.5 ± 0.1+0.1 ± 0.1
64K60.6 ± 1.161.6 ± 1.5+1.0 ± 0.40.0 ± 0.3
128K54.8 ± 1.354.9 ± 1.3+0.1 ± 0.2-0.1 ± 0.1

The same pattern holds: TTT-Effect is within ±0.2 points in every setting, including under query-aware prompting (QCQ), except RULER (CQ) at 128K (−1.1 ± 0.2), while TTT-Benefit varies across tasks and lengths. We have updated Tables 2 and 3 to report mean performance across evaluation seeds, with the corresponding standard deviations in Appendix C.2 (Table 5) and C.5 (Table 8).

Given that TTT-Benefit varies across tasks and is not consistently positive, we agree with the reviewer that the central claim should be reframed, and we apologize for the lack of clarity in the original manuscript: our wording around "gains" from TTT-aware training overstated what the results support. Our central claim is that TTT-aware training can change model performance without inducing corresponding reliance on inference-time fast-weight updates, rather than that outer-loop training yields large or consistent gains. This claim is further supported beyond benchmark differences: the training dynamics in Section 4.2 directly show that outer-loop training progressively attenuates the functional effect of fast-weight updates. We have revised the Abstract, Contributions, and Conclusion accordingly.

2. Are fast weights unused only because full attention already sees everything? (W2)

We agree that full attention may reduce reliance on fast-weight updates by keeping relevant information directly accessible. To test this, we constructed three hybrid models by replacing 25%, 50%, and 75% of the full-attention layers in Qwen3-4B-Base with sliding-window attention (SWA, 4K window), and trained both a standard-CPT baseline and a Frobenius TTT variant for each, following the Stage 1 recipe (32K context, 23B tokens).

As shown in the table below, restricting attention substantially limits the backbone's long-context performance: at 32K, standard-CPT drops from 81.5 with full attention to 56.9 with 75% SWA. In this setting, the fast weights are far less redundant with attention, and TTT-Benefit increases to +2.1/+2.4 at 16K/32K. However, TTT-Effect remains small (+0.7/−0.1), so the improvement still arises primarily from changes learned by the underlying model rather than from inference-time fast-weight updates. Thus, redundancy with full attention alone does not explain the limited fast-weight utilization.

SWA Ratio(16K) Standard-CPT(16K) Frobenius(16K) TTT-Benefit(16K) TTT-Effect(32K) Standard-CPT(32K) Frobenius(32K) TTT-Benefit(32K) TTT-Effect
Qwen3-4B-Base88.5———82.2———
0%90.089.8-0.2-0.181.582.0+0.50.0
25%88.088.2+0.3+0.373.571.5-2.0-1.1
50%86.485.7-0.7-0.173.874.2+0.4+0.6
75%81.283.2+2.1+0.756.959.3+2.4-0.1

Furthermore, our update-amplification experiments (Section 4.3, Figure 2) show that stronger fast-weight updates can degrade pretrained capabilities even when long-context retrieval remains intact. Together, these results suggest that capability preservation may also constrain fast-weight utilization, beyond the availability of full attention. We have added the SWA results and this discussion to Appendix C.6 (Table 9).

3. Experimental scope and generalizability. (W3, Q2)

We address these concerns through additional experiments and clarifications.

(1) Training configurations. To examine sensitivity to the configuration, we conducted additional ablations on inner-loop learning rate, number of TTT layers, and chunk size under stage-1 training (32K context, 23B tokens).

Reducing η from 3 to 0.3 yields TTT-Effects of 0.0, −0.1, and −0.5 points on RULER at 16K, 32K, and 64K, comparable to the original configuration (−0.1, 0.0, −0.3). Reducing the chunk size from 4,096 to 1,024 yields TTT-Effects of 0.0, +0.2, and −1.0, despite a +1.0-point TTT-Benefit at 32K.

Doubling the number of TTT layers from 6 to 12 (every third layer instead of every sixth) yields TTT-Effects of +0.1, −0.4, and −0.9 points, compared with −0.1, 0.0, and −0.3 for the original 6-layer configuration. Adding more writable layers thus makes the effect of inference-time updates slightly more negative rather than more useful, consistent with interference with pretrained computation.

Variant(16K) Perf.(16K) TTT-Benefit(16K) TTT-Effect(32K) Perf.(32K) TTT-Benefit(32K) TTT-Effect(64K) Perf.(64K) TTT-Benefit(64K) TTT-Effect
standard-CPT90.0——81.5——68.1——
Frobenius (η=3, 6 layers, chunk-size=4,096)89.8-0.2-0.182.0+0.50.067.4-0.7-0.3
Frobenius (η=0.3)90.00.00.081.50.0-0.167.2-0.9-0.5
Frobenius (chunk-size=1,024)90.1+0.10.082.8+1.00.268.3+0.1-1.0
Frobenius (12 layers)89.6-0.4+0.180.9-0.6-0.468.0-0.1-0.9

Together with the six learning rules spanning update objectives, write gating, and decay, these ablations show limited or negative TTT-Effects across configurations, suggesting that the finding is not tied to a single setting.

(2) Extended LLaMA replication. We trained LLaMA-3.1-8B through Stage 2 (128K context, 17B tokens) and evaluated on three data seeds. The results remain consistent with our central observation: TTT-Effect is generally small (positive effects ≤+0.5), while reaching −1.8 ± 1.0 on RULER 128K. This further supports that limited fast-weight utilization beyond Qwen3-4B.

BenchmarkLengthCPT Perf.TTT Perf.TTT-BenefitTTT-Effect
RULER16K93.3 ± 0.893.1 ± 0.2−0.2 ± 0.6+0.1 ± 0.3
32K90.2 ± 0.889.8 ± 0.7−0.4 ± 0.10.0 ± 0.2
64K82.3 ± 0.581.9 ± 0.5−0.4 ± 0.1−0.7 ± 0.5
128K65.2 ± 1.565.4 ± 1.3+0.2 ± 0.8−1.8 ± 1.0
HELMET16K50.8 ± 0.950.0 ± 1.2−0.9 ± 0.30.0 ± 0.2
32K51.9 ± 1.050.9 ± 1.4−1.0 ± 0.4+0.2 ± 0.2
64K54.6 ± 1.054.5 ± 0.6−0.1 ± 0.8+0.5 ± 0.3
128K52.0 ± 0.951.6 ± 0.2−0.3 ± 0.70.0 ± 0.0

(3) Capability-preservation analysis. Figure 2b originally focused on GDN and GDN-Residual because these variants maintain retrieval performance under substantial update amplification, allowing us to examine capability interference without concurrent retrieval degradation. Following the reviewer's suggestion, we have updated Figure 2b to include all six learning rules and the standard-CPT baseline in the revised manuscript. The extended results show that Frobenius degrades substantially more under amplification than GDN and GDN-Residual, further illustrating how learning rules differ in their ability to preserve pretrained capabilities.

(4) Reproduction gap (Q2). Our Frobenius variant implements the original In-Place TTT learning rule using the publicly released codebase and follows its training configuration as closely as possible, but is trained on our own long-context corpus, as the original training data is not publicly available. Our corpus consists of 25B unique tokens curated from publicly available instruction-following, math, code, and heavily filtered web documents, with approximately 75% long documents. Standard continued pretraining on this corpus substantially improves long-context performance of Qwen3-4B-Base (RULER 65.8→78.6 at 64K and 50.5→69.5 at 128K), suggesting the data is suitable for continued pretraining.

Regarding per-sample fast-weight resets and chunk-boundary padding, these choices were made to prevent updates from carrying over between independent samples. Since the original data preprocessing details are unavailable, we cannot confirm whether these choices differ from the original setup. Thus, while our implementation closely follows the original method, differences in data and preprocessing may account for some of the discrepancy.

Importantly, our central finding does not depend on matching the original gains. The original work reports gains over its baseline, which corresponds to TTT-Benefit, whereas TTT-Effect is a paired comparison within the same trained model. A larger TTT-Benefit therefore does not by itself imply greater reliance on inference-time updates, as our SWA experiments illustrate (response to W2).

(5) Scope of conclusions. With these additions, the small TTT-Effect holds across six learning rules varying the update objective, write gating, and decay; two backbones (Llama and Qwen), both trained up to 128K context; different inner-loop learning rates, numbers of TTT layers, and chunk size; hybrid models with sliding-window attention (response to W2); standard and query-aware prompting; and both training- and evaluation-seed variation (response to W1). Thus, within the setting where pretrained parameters are repurposed as fast weights, limited fast-weight utilization is consistent across all configurations we examined.

4. Is the decline in update magnitude just recovery from a disruptive initialization? (Q3)

We agree that the initial updates in prediction-error rules can be disruptive, since V ≈ 0 at initialization leads to a large residual E ≈ −O.

The table below extends the relative MLP output changes shown in Figure 1a to include initialization (step 0), with exact values reported at each training step.

StepDeltaDelta + βGDN
00.3480.0980.097
15000.2920.0690.068
30000.2840.0580.058
45000.2790.0540.054
60000.2770.0530.051
75000.2750.0520.050
90000.2750.0520.050
110770.2750.0520.050

At step 0, the relative output change is 0.348 for Delta, 0.098 for Delta+β, and 0.097 for GDN, decreasing to 0.292 / 0.069 / 0.068 by step 1.5K; the corresponding step-0 TTT-Effects on RULER 16K/32K are −0.6/−7.4 points for Delta, +0.2/+1.0 for Delta+β, and +0.1/+1.2 for GDN. For these rules, a substantial part of the decline indeed occurs early in training, consistent with the reviewer's hypothesis, particularly for ungated Delta, whose initial updates substantially hurt retrieval.

However, recovery from initialization does not fully explain the attenuation in the gated variants. For Delta+β and GDN, the fast-weight updates at initialization (step 0) actually improve retrieval (TTT-Effect on RULER of +1.0 and +1.2 at 32K), yet training reduces this effect to +0.2 and +0.5, respectively, and their write gates continue to decrease throughout training (Figure 1c). If outer-loop training optimized only for retrieval, there would be little reason to suppress updates that already help it. This suggests that the attenuation is driven by another pressure, consistent with our update-amplification results (Figure 2b), where stronger updates degrade pretrained capabilities even when retrieval is preserved.

Dear reviewer b3ze, Thank you again for your thoughtful and constructive feedback. As we near the end of the discussion period, we hope our responses and additional experiments have addressed your concerns. We are happy to discuss any aspects of the paper that may require further clarification.

Reviewer_MVvWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the effects and benefits of applying test-time training (TTT) to a pretrained LLM. The paper focuses on In-Place TTT, which turns the MLP down-projection in a few layers into fast weights that are updated while the model reads the input. The authors continue to pretrain LLMs with six update rules that differ in their objective, write gate, and decay. They then separate two quantities: TTT-Benefit, the gain over a standard CPT baseline, and TTT-Effect, the difference between turning the updates on and off in the same model. The main finding is that the updates themselves contribute little at test time, and that whatever gain TTT-aware training brings comes mostly from changes to the base model. The paper further shows that the size of the updates decreases during training and that amplifying them at test time hurts pretrained capabilities more than retrieval. They report that TTT-aware training increases attention to the relevant evidence even with updates turned off.

Critical strengths
  1. I think the research question the paper studies is quite important and interesting. TTT is often assumed to help because of the test-time updates, and the paper directly tests this assumption.
  2. The main analytical mechanism of splitting between TTT-Benefit and TTT-Effect is simple and clean. Future TTT studies can easily reuse it.
  3. The study is systematic. It compares six update rules, two backbones, and three types of evaluation.
  4. The analysis looks beyond final accuracy. It studies training dynamics, update amplification, attention to the needle, and write-gate selectivity, which gives a much more complete picture of how the test-time updates behave.
Critical weaknesses

1. Issues on novelty and impact:

  • The main finding may be expected, which weakens the insight. In every experiment, the model keeps full attention over the whole input. In RULER and HELMET, all the information needed is inside the context that attention can see, and the standard CPT model without any updates already does well on them. So, it’s not surprising that the model learns to not depend on a second memory that gives it no new information. The paper presents the small effect of the updates as surprising and suggests that it may partly come from a conflict between storing new information and keeping pretrained knowledge. For this to be a real insight, the paper must show that its explanation holds and that the simpler alternative does not. The separate memory layer baseline I suggest under W3 would test this. Without it, the paper's main lesson may simply be that extra memory goes unused when attention is already enough, which contributes less.
  • The conclusions are broader than the experiments, limiting the impact. The paper speaks about test-time training for pretrained language models in general, but the study uses only In-Place TTT. Apart from the update rule, it keeps In-Place TTT's default settings: only 6 layers are updated, one matrix per layer is used, the chunk size is 4K, and the inner learning rate is fixed. The finding of limited use could be coupled with these choices. We therefore cannot tell whether the finding applies to other TTT designs. To support the general claim, the paper should include at least one study that varies the chunk size or the number and position of the updated layers. I think the paper should include a discussion of other TTT frameworks (Atlas: memorize the context by optimizing the memory based on the current and past tokens, PERK: meta-learned LoRA TTT, Doc-to-LoRA: hypernet LoRA updates, and Cartridges: learnable KV-cache, as a few examples). Even with no empirical results, at least a theoretical comparison would be informative.
  • The paper gives little practical direction for the field. A diagnostic paper has the most impact when it tells others what to do differently. The paper concludes with the challenge of turning pretrained weights into memory, but apart from GDN-Residual, it does not test changes suggested by its findings. For example, it does not test whether a separate memory module or a different choice of layers leads to useful updates that the model actually utilizes. I think the bigger opportunity here is for the paper to move from describing a problem to a step toward solving it.

2. Concerns regarding sufficient support of claims and significance:

  • The abstract says that performance gains can arise largely from changes learned by the underlying model. However, Table 2 shows very little gain to support that. On RULER and HELMET, the average TTT-Benefits at the four lengths are within a marginal range, and the sign changes across lengths. For LLaMA-3.1-8B, all three TTT variants are worse than the baseline on HELMET. The outer-loop effect is therefore also a small number that may be within the run noise. The paper needs to show how much two normal CPT runs differ by chance. Without that, we cannot conclude whether a difference of half a point means anything. A more accurate summary is that TTT-aware training moves performance relative to standard CPT by up to about 2–3 points in either direction, while turning the updates on adds almost nothing.
  • The shrinking of the updates during training is not clearly shown to be a learned trade-off. For the prediction-error rules, the shrinking may partly be recovery from the initialization. For the prediction-error rules, the paper indicates that the early updates push the MLP output toward zero, and training would naturally reduce them. For the Frobenius rules, the update is zero at the start and must first grow before it shrinks, but the value stays near zero on the scale of Figure 1b. In addition, the appendix reports a single learning rate, 5e-6, for all runs, which may be too low for modules that are trained from scratch.

3. Issues on experimental rigor and clarity:

  • The benchmarks test retrieval, which is already a known strength of full attention. RULER and the HELMET QA tasks mainly ask the model to retrieve the right information in a context, and full attention is already very good at this. The HELMET ICL tasks ask the model to learn a label mapping from examples, but all the examples are still visible to attention. On such tasks, TTT has little room to help, so a marginal TTT effect is expected. To show the real benefit or effect of TTT, the paper should use settings where TTT can access the hidden information better than attention can. Examples include removing the context from the KV cache after it is written into the fast weights, using sliding-window attention, or testing beyond the trained 128K length. Another example is tasks where the model must learn a rule spread across more context than attention can cover.
  • I think the following baselines test alternative explanations but are missing from the paper. The first is a model trained with random weight changes of the same size in the same layers. TTT training may act as a kind of noise-based regularization, and if random changes also increase attention to the needle, the attention change in Section 5.2 would not be specific to TTT. The second is a dedicated memory separate from the pretrained MLP, for example, a fast-weight module with its own key, value, and output projections and a learned output gate. This comparison would test whether the problem comes from writing into pretrained weights, against the simpler explanation that the memory is redundant with full attention.
Questions for authors
  1. What’s your insight and hypothesis on whether the reported challenge of In-place TTT also applies to other TTT frameworks like PERK, Atlas, Doc-to-LoRA, and Cartridges?
  2. Why does Figure 2b show only GDN and GDN-Residual and not the other four rules?
  3. Could you explain why its results differ from the original In-Place TTT work? The paper follows the In-Place TTT setup closely, so what causes these differences?
  4. Could you add the standard CPT model to Figure 2b, to show whether TTT training itself changes these scores?
  5. Does TTT-Effect become more negative as the number of processed chunks grows?
11 comments

We sincerely thank the reviewer for the thoughtful and detailed evaluation of our work. We appreciate the reviewer's recognition of the importance of our main research question, the simplicity and reusability of the TTT-Benefit and TTT-Effect decomposition for future TTT studies, the systematic nature of our experiments, and the value of our analyses beyond final accuracy. We address the reviewer's concerns in detail below.

1. Is limited fast-weight utilization an expected finding? (W1-1)

We respectfully disagree that this finding is expected. If TTT-aware training yielded no benefit at all, limited utilization would indeed be unsurprising given full attention. However, TTT-aware training does change performance, and in several settings improves it, both in prior work (In-Place TTT) and in our experiments. For example, in Table 2, Frobenius+β and Frobenius improve over standard-CPT by +2.3 and +1.8 points on HELMET at 64K. Our finding is that such gains, when they occur, arise from changes learned by the underlying model rather than from the inference-time updates: in these same settings, TTT-Effect is only +0.9 and +0.1, respectively. Moreover, at 128K on RULER, the updates are not merely unused but actively harmful (average TTT-Effect −1.1), which redundancy with attention alone would not predict, but which is consistent with interference with pretrained computation (Section 4.3). Whether TTT-aware training helps thus varies across tasks, but in no setting does it induce substantial positive reliance on the updates. This distinction is not visible from final performance alone, which is why we introduce the TTT-Benefit and TTT-Effect decomposition.

2. Does redundancy with full attention explain the limited utilization? (W1-1, W3-1)

We agree that full attention may reduce reliance on fast-weight updates by keeping relevant information directly accessible. To test whether this alone explains our findings, we constructed three hybrid models by replacing 25%, 50%, and 75% of the full-attention layers in Qwen3-4B-Base with sliding-window attention (SWA, 4K window), and trained both a standard-CPT baseline and a Frobenius TTT variant for each, using our first-stage training setup (32K context, 23B tokens).

As shown in the table below, restricting attention substantially limits the backbone's long-context performance: at 32K, standard-CPT drops from 81.5 with full attention to 56.9 with 75% SWA, as most layers can no longer see evidence beyond the 4K window. In this setting, the fast weights are far less redundant with attention, and TTT-Benefit increases to +2.1/+2.4 at 16K/32K. However, TTT-Effect remains small (+0.7/−0.1), so the improvement still arises primarily from changes learned by the underlying model rather than from inference-time fast-weight updates. Thus, limited fast-weight utilization persists even when attention can no longer access the full context, and redundancy with full attention alone does not explain it.

SWA Ratio(16K) Standard-CPT(16K) Frobenius(16K) TTT-Benefit(16K) TTT-Effect(32K) Standard-CPT(32K) Frobenius(32K) TTT-Benefit(32K) TTT-Effect
Qwen3-4B-Base88.5———82.2———
0%90.089.8-0.2-0.181.582.0+0.50.0
25%88.088.2+0.3+0.373.571.5-2.0-1.1
50%86.485.7-0.7-0.173.874.2+0.4+0.6
75%81.283.2+2.1+0.756.959.3+2.4-0.1

Instead, our update-amplification experiments (Section 4.3, Figure 2) support the hypothesis that the updates remain small to preserve pretrained capabilities: stronger fast-weight updates degrade knowledge and commonsense reasoning performance even when long-context retrieval remains intact. Because the fast weights are pretrained MLP parameters, larger writes interfere with the computation they already encode.

We have added the SWA results and this discussion to Appendix C.6 (Table 9).

3. Are the findings specific to the default In-Place TTT configuration? (W1-2)

Following the reviewer's suggestion, we varied the number and position of updated layers, the inner-loop learning rate, and the chunk size using our first-stage training setup (32K context, 23B tokens); results are summarized in the table below.

Doubling the number of TTT layers from 6 to 12 (every third layer instead of every sixth) yields TTT-Effects of +0.1, −0.4, and −0.9 points on RULER at 16K, 32K, and 64K, compared with −0.1, 0.0, and −0.3 for the original 6-layer configuration. Adding more writable layers thus makes the effect of inference-time updates slightly more negative rather than more useful, consistent with interference with pretrained computation. Reducing the inner-loop learning rate η from 3 to 0.3 yields TTT-Effects of 0.0, −0.1, and −0.5 points, comparable to the original configuration. Reducing the chunk size from 4,096 to 1,024 yields TTT-Effects of 0.0, +0.2, and −1.0 points at 16K, 32K, and 64K, respectively, despite a +1.0-point TTT-Benefit at 32K.

These results, together with the six learning rules already studied in the paper, which vary the update objective, write gating, and decay, consistently show small TTT-Effect, suggesting that limited fast-weight utilization is not an artifact of a single inner-loop design, learning rate, the number of updated layers, or chunk size.

Variant(16K) Perf.(16K) TTT-Benefit(16K) TTT-Effect(32K) Perf.(32K) TTT-Benefit(32K) TTT-Effect(64K) Perf.(64K) TTT-Benefit(64K) TTT-Effect
standard-CPT90.0——81.5——68.1——
Frobenius (η=3, 6 layers, chunk-size=4,096)89.8-0.2-0.182.0+0.50.067.4-0.7-0.3
Frobenius (η=0.3)90.00.00.081.50.0-0.167.2-0.9-0.5
Frobenius (12 layers)89.6-0.4+0.180.9-0.6-0.468.0-0.1-0.9
Frobenius (chunk-size=1,024)90.1+0.10.082.8+1.00.268.3+0.1-1.0

4. Include a discussion of other TTT methods (W1-2, Q1)

Whether the challenges observed in In-Place TTT extend to other TTT methods likely depends on whether the memory shares parameters with pretrained computation.

Atlas is closest to In-Place TTT in its update mechanism: its memory is updated as the context is processed by minimizing a gated squared-error key–value objective over recent tokens, similar to our Delta and GDN variants. However, Atlas uses dedicated memory trained from scratch rather than pretrained parameters that already encode useful computation, so the tension between contextual adaptation and pretrained capabilities does not arise in the same way.

PERK, Doc-to-LoRA, and Cartridges instead encode the context into additional parameters attached to a frozen pretrained model. PERK encodes the context into a LoRA adapter through a few gradient steps of next-token prediction, with the adapter initialization meta-learned. Doc-to-LoRA uses a hypernetwork trained with context distillation to generate, in a single forward pass, a LoRA adapter applied to the MLP down-projections, the same matrices used as fast weights in In-Place TTT. Since a LoRA adapter modifies the effective weights additively (W + AB), as In-Place TTT does (W0+ΔWW_0 + \Delta W), the tension we identify may also arise in PERK and Doc-to-LoRA. Indeed, recent work on Doc-to-LoRA (Cheng et al., 2026) finds that generated adapters are often too weak to override pretrained knowledge, while globally amplifying them degrades unrelated capabilities, mirroring our update-amplification results (Section 4.3). Cartridges, in contrast, trains a compact KV cache offline via context distillation without modifying model weights, and should therefore be least affected by this tension.

While our findings may not generalize to all TTT methods, In-Place TTT offers an attractive approach by enabling efficient test-time updates of existing pretrained LLMs without introducing separate memory modules, making it important to understand how its inference-time updates contribute to performance. We have added this discussion to the related work section (Appendix A.1) of the revised manuscript.

References.

[Cheng et al., 2026] Cheng et al. The Override Gap: A Magnitude Account of Knowledge Conflict Failure in Hypernetwork-Based Instant LLM Adaptation. arXiv:2604.23750, 2026.

5. Does the paper offer practical direction to the field beyond describing a problem? (W1-3)

We respectfully disagree that the paper only describes a problem. Rather, it establishes through controlled experiments that the problem exists, identifies where the gains from TTT-aware training actually come from, and provides a tool for future work to detect it.

First, the TTT-Benefit / TTT-Effect decomposition separates gains from TTT-aware training from the direct contribution of inference-time updates, which final performance alone cannot reveal. As the reviewer notes, "future TTT studies can easily reuse it"; Reviewer i8PW similarly states that "other TTT papers should run the same check," and Reviewer b3ze that it "should change how future work in this area evaluates and attributes improvements."

Second, using this decomposition, we show that even when TTT-aware training improves performance (e.g., +2.3 and +1.8 on HELMET at 64K for Frobenius+β and Frobenius), the gain arises largely from changes learned by the underlying model rather than from the updates, and that at long contexts the updates can even hurt (RULER at 128K, average TTT-Effect −1.1). Our update-amplification experiments further reveal a trade-off that retrieval benchmarks alone would miss: stronger updates can preserve retrieval while degrading pretrained capabilities (Figure 2), and update rules that preserve the pretrained initialization, such as GDN-Residual, mitigate this degradation (Figure 2b).

Finally, these findings hold not only for the Frobenius rule of In-Place TTT, but across six learning rules varying the update objective, write gating, and decay, and, in our new experiments, across different numbers of TTT layers and inner-loop learning rates (response to W1-2) and under restricted attention (response to W1-1/W3-1). Notably, following the reviewer's suggestion to vary the choice of layers, doubling the number of TTT layers does not increase utilization of TTT fast weights.

We agree that designing TTT mechanisms that overcome this limitation, for example by efficiently separating writable memory from pretrained computation (response to Q1), is an important next step, and we view our diagnostic protocol and findings as a necessary foundation for it.

6. Are the performance differences statistically meaningful? (W2-1)

To quantify run-to-run variability, we trained standard-CPT with two seeds and Frobenius with three seeds, and evaluated all checkpoints on the same fixed evaluation set (table below).

Across standard-CPT runs, performance varies by 0.1–0.8 points (standard deviation). Against this variation, TTT-Benefit is task- and context-dependent: it can yield clear improvements over standard-CPT (e.g., +1.8 ± 0.9 on HELMET at 64K) while remaining negligible or negative elsewhere. In contrast, TTT-Effect is consistently small: its positive values are at most +0.5 ± 0.3 points, even where TTT-Benefit is largest. The only notable TTT-Effect is negative, on RULER at 128K (−1.6 ± 0.4), where it matches TTT-Benefit (−1.5 ± 1.4), so disabling the updates recovers standard-CPT performance. We have also updated Table 2 to report the mean over three independently sampled evaluation sets.

BenchmarkLengthCPT Perf.Frobenius Perf.TTT-BenefitTTT-Effect
RULER16K89.7 ± 0.189.8 ± 0.5−0.2 ± 0.0+0.1 ± 0.0
32K84.8 ± 0.284.2 ± 0.6−1.0 ± 0.3−0.1 ± 0.3
64K79.0 ± 0.779.3 ± 0.7−0.1 ± 0.90.0 ± 0.4
128K69.7 ± 0.568.3 ± 0.7−1.5 ± 1.4−1.6 ± 0.4
HELMET16K56.1 ± 0.256.7 ± 0.7+0.4 ± 0.7+0.3 ± 0.1
32K60.3 ± 0.860.6 ± 0.3+0.4 ± 0.5+0.3 ± 0.3
64K61.7 ± 0.263.2 ± 0.7+1.8 ± 0.9+0.5 ± 0.3
128K55.6 ± 0.655.3 ± 0.9+0.1 ± 0.1−0.4 ± 0.7

We therefore agree with the reviewer's summary: TTT-aware training can move performance relative to standard-CPT by up to about 2–3 points in either direction, while turning the updates on adds almost nothing, and at long contexts can even hurt (RULER at 128K, average TTT-Effect −1.1 in Table 2). We apologize for the lack of clarity in the original manuscript, where our wording around "gains" from TTT-aware training overstated what the results support, and we have revised the Abstract, Contributions, and Conclusion to state our central finding as limited fast-weight utilization: TTT-aware training can change model performance without inducing corresponding reliance on inference-time fast-weight updates.

Reviewer_i8PWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper is an empirical study of how much fast weights are actually used when TTT is applied to pretrained LLMs. It focuses on In-Place TTT, which treats the MLP down-projection in some layers as fast weights and uses continued pretraining to teach the model to update them chunk by chunk at inference time. The question is whether the gains from TTT training come from the fast-weight updates at inference time or from changes to the backbone during training. To answer it, the authors compare three models: one trained with standard continued pretraining, a TTT model with updates on, and the same TTT model with updates off. From these they define two metrics, TTT-Benefit and TTT-Effect. Besides the additive update used in In-Place TTT, they test variants with prediction-error updates, write gates and decay, six inner-loop rules in total. The paper reports three main findings. First, TTT training gives no consistent gain over standard continued pretraining, and turning off the updates at inference barely changes the results. When the question is also given before the context, the gain from TTT grows, but it mostly comes from changes in the backbone, which attends more strongly to the relevant evidence. Second, the outer loop gradually shrinks the updates during training. When the updates are scaled up at inference time, retrieval with the prediction-error rules is mostly unaffected, but accuracy on commonsense tasks drops clearly. The authors argue from this that keeping the updates small is tied to preserving pretrained capabilities. Third, these patterns are largely consistent across Qwen3-4B-Base and LLaMA-3.1-8B and across several kinds of long-context tasks.

Critical strengths

1.The question is clear and worth asking. Most TTT papers only report that adding TTT helps. This one checks whether the gain comes from the fast-weight updates at inference time or from training itself. The authors compare the TTT model with updates on, the same model with updates off, and a model trained with ordinary continued pretraining, and define TTT-Benefit and TTT-Effect from these. The idea is simple and easy to reuse, and I think other TTT papers should run the same check.

2.The study is also broad and well controlled. It covers six inner-loop rules (additive, prediction-error, write gating, decay, and residual-only decay) on two backbones, and evaluates them on RULER, HELMET and commonsense tasks. All variants and the continued-pretraining baseline use the same data and training recipe. The authors also track how the output change, the value norm and the write gates evolve during training. So the low use of fast weights is not specific to one rule.

Critical weaknesses

1.In this setting the fast weights have little to do, so the conclusion may not generalize. The backbone uses full attention extended to 128K with YaRN, and every evaluation fits inside the attention window, so attention alone is enough for retrieval. The fast weights may be used little simply because they are not needed, not because training suppresses them to protect pretrained abilities. The paper does not discuss this, and it has no experiment where attention cannot see the evidence, such as sliding-window attention, lengths beyond the window, or a truncated KV cache. Those are exactly the cases where TTT should matter. This limits how far the “limited fast-weight utilization” claim in the title can be taken.

2.The reproduced gains are far below those reported for In-Place TTT, and the remaining differences are too small to support the decomposition. With the same additive rule and an almost identical two-stage recipe, the original In-Place TTT paper reports gains of +4.4 at 64K and +2.2 at 128K, while this paper gets +0.6 and −0.4. The paper does not discuss this gap, and it does not name its training corpus. If the reproduction never recovers the original gains, the finding that fast weights are barely used may only describe this particular reproduction. TTT-Benefit and TTT-Effect are also mostly within ±1 point, and the claim that the gains mainly come from changes to the backbone rests on splitting differences of less than one point into two parts. Without confidence intervals or significance tests, I cannot tell signal from noise.

Questions for authors
  1. Could you measure TTT-Effect in settings where attention cannot see the evidence, for example with sliding-window attention, at lengths beyond the attention window, or with a truncated KV cache?
  2. With the same additive rule and an almost identical training recipe, the original In-Place TTT paper reports gains of +4.4 at 64K and +2.2 at 128K, while this paper gets +0.6 and −0.4. Where does this gap come from? Is it the corpus, the chunk size, or some other setting? Could you also give the name and composition of the training corpus?
  3. Could you report confidence intervals for TTT-Benefit and TTT-Effect, for example by bootstrapping over the RULER evaluation samples?
4 comments

Dear reviewer i8PW,

We sincerely thank the reviewer for the thoughtful and constructive feedback. We appreciate the recognition that disentangling the contributions of inference-time fast-weight updates and TTT-aware training is a question worth asking, and that our proposed metrics should be used for evaluating future TTT methods. We are also grateful that the reviewer found our study broad and well-controlled.

1. Are fast weights unused only because full attention already sees everything? (W1, Q1)

Following the reviewer’s suggestion, we conducted additional experiments replacing Attention layers in the Qwen3-4B-Base model with Sliding Window Attention (SWA) layers with 4K window. Specifically, we construct three Hybrid models by replacing 25%, 50%, and 75% of the full Attention layers with SWA. For each hybrid, we train both a standard-CPT baseline and a Frobenius TTT variant at 32K context length on 23B tokens, following the same recipe as Stage 1 in our paper.

The table below reports RULER accuracy for all models. Replacing full Attention with SWA substantially limits the backbone's long-context performance: at 32K, standard-CPT drops from 81.5 with full Attention to 56.9 with 75% SWA. In this setting, the fast weights are no longer redundant with attention, and TTT-aware training yields clearly larger gains, with TTT-Benefit increasing to +2.1/+2.4 at 16K/32K. However, TTT-Effect remains small (+0.7/−0.1), indicating that these gains still arise primarily from changes learned by the underlying model rather than from inference-time fast-weight updates. Thus, redundancy with full attention alone does not explain the limited fast-weight utilization.

We have added these results to Appendix C.6 (Table 9) of the revised manuscript.

SWA Ratio(16K) Standard-CPT(16K) Frobenius(16K) TTT-Benefit(16K) TTT-Effect(32K) Standard-CPT(32K) Frobenius(32K) TTT-Benefit(32K) TTT-Effect
Qwen3-4B-Base88.5———82.2———
0%90.089.8-0.2-0.181.582.0+0.50.0
25%88.088.2+0.3+0.373.571.5-2.0-1.1
50%86.485.7-0.7-0.173.874.2+0.4+0.6
75%81.283.2+2.1+0.756.959.3+2.4-0.1

2. Why are the TTT gains smaller than in the original In-Place TTT paper? (W2, Q2)

We implemented our experiments using the publicly released In-Place TTT codebase and followed its training configuration as closely as possible, but trained on our own long-context corpus, as the original training data is not publicly available. Our corpus consists of 25B unique tokens curated from publicly available instruction-following, math, code, and heavily filtered web documents, with approximately 75% comprising long documents. Standard continued pretraining on this corpus substantially improves long-context performance over the pretrained base model (RULER 65.8→78.6 at 64K and 50.5→69.5 at 128K; Table 2 in the main paper), indicating that the data is suitable for long-context continued pretraining.

Importantly, our goal is not to challenge the gains reported by the original work, and our central finding does not depend on matching them. The original work reports gains over its baseline, which corresponds to TTT-Benefit, whereas TTT-Effect is a paired comparison within the same trained model. Hence, a larger TTT-Benefit does not by itself imply greater reliance on inference-time updates. Our 75% SWA experiment illustrates this directly: TTT-Benefit reaches +2.1/+2.4 at 16K/32K, while TTT-Effect is only +0.7/−0.1.

3. Are the small TTT-Benefit and TTT-Effect values reliable? (W2, Q3)

We agree that small performance differences should not be overinterpreted. To assess evaluation variability, we re-evaluated all trained checkpoints on three independently sampled evaluation sets (seeds 42, 43, 44). Unlike bootstrapping, which can only resample examples from a single fixed set, this draws new evaluation instances and therefore also captures variability that a single draw cannot reveal.

The table below reports results averaged across the six TTT variants (mean ± std over three evaluation seeds). The two quantities behave very differently. TTT-Effect is consistently small: it is within ±0.2 points in every setting except RULER at 128K, with standard deviations of at most 0.3, and it is never meaningfully positive. Because TTT-Effect compares the same model with updates enabled and disabled on identical examples, much of the evaluation noise cancels in this paired comparison (as also noted by Reviewer b3ze). In contrast, TTT-Benefit is task- and length-dependent: it is negligible on RULER at 16K–64K (−0.4 to 0.0), but reaches +1.0 ± 0.4 on HELMET at 64K, where TTT-Effect is 0.0 ± 0.3, so this gain persists entirely with updates disabled. The one setting where TTT-Effect departs from zero is RULER at 128K, where it is negative (−1.1 ± 0.2) and accounts for nearly all of the TTT-Benefit (−1.2 ± 0.3); here, enabling updates measurably hurts, consistent with the interference between fast-weight writing and pretrained computation analyzed in Section 4.3. Thus, whether TTT-aware training helps depends on the task, but in no setting does it induce reliance on inference-time fast-weight updates.

BenchmarkContextCPT Perf.TTT Perf.TTT-BenefitTTT-Effect
RULER16K89.0 ± 0.989.0 ± 1.00.0 ± 0.20.0 ± 0.1
32K84.6 ± 0.284.3 ± 0.1−0.3 ± 0.3+0.1 ± 0.0
64K78.6 ± 0.278.2 ± 0.7−0.4 ± 0.60.0 ± 0.1
128K69.5 ± 0.568.3 ± 0.6−1.2 ± 0.3−1.1 ± 0.2
HELMET16K55.6 ± 0.955.6 ± 1.20.0 ± 0.4−0.2 ± 0.2
32K58.5 ± 1.559.0 ± 1.6+0.5 ± 0.1+0.1 ± 0.1
64K60.6 ± 1.161.6 ± 1.5+1.0 ± 0.40.0 ± 0.3
128K54.8 ± 1.354.9 ± 1.3+0.1 ± 0.2−0.1 ± 0.1

We have updated Table 2 to report the mean over three evaluation seeds and added the corresponding standard deviations in Appendix C.5.

We also apologize for the lack of clarity in the original manuscript: our wording around "gains" from TTT-aware training overstated what the results support. Our central claim is that TTT-aware training can change model performance without inducing corresponding reliance on inference-time fast-weight updates, rather than that outer-loop training yields large or consistent gains. We have revised the Abstract, Contributions, and Conclusion to make this clear.

Thank you for the response and the new experiments. The results over several evaluation seeds show that the near-zero TTT-Effect is stable. The SWA hybrid experiments also show that when attention is limited, the TTT gains still come mainly from changes to the backbone. The revised central claim now matches the evidence better. I hope the revision will include these results. I will keep my score.

Thank you for your thoughtful feedback and for taking the time to consider our additional experiments and clarifications. We appreciate you recognizing our revised central claim, and we have incorporated this into our revised manuscript.

Comments on the paper

1 comment

We thank all reviewers for recognizing the importance of our research question, the simplicity and reusability of the TTT-Benefit / TTT-Effect decomposition, and the value of our systematic, mechanism-level analyses beyond final accuracy.

The main concerns were statistical reliability, redundancy with full attention, and the generality and interpretation of our findings. We addressed them with new experiments, summarized below.

1. Statistical reliability. We conducted additional training runs and evaluations with different seeds to assess the consistency of our observations. TTT-Effect remains consistently small, with positive values of at most +0.5 points averaged across training seeds, while TTT-Benefit varies across tasks.

2. Redundancy with full attention. We systematically varied the proportion of SWA layers (25%, 50%, 75%, and 100%) and evaluated performance on RULER and HELMET across three independently sampled evaluation sets. At 75% SWA, RULER TTT-Benefit reaches +2.5 at 16K, while TTT-Effect is only +0.3. Even with 100% SWA, which substantially reduces baseline performance (RULER 32K: 81.3 → 21.6), TTT-Effect remains small on RULER and negative on HELMET. Thus, redundancy with full-attention layers alone cannot explain limited fast-weight utilization.

3. Generality of our claims. We ablated different algorithmic choices, including inner-loop learning rate, number of TTT layers, chunk sizes, and extended our evaluation to Llama-3.1-8B trained at 128K context. Across all settings, TTT-Effect is at most slightly positive and often negative, supporting limited utilization of inference-time fast-weight updates.

4. Claims and scope. We revised the Abstract, Contributions, and Conclusion to state our central finding precisely and limit the scope of our claims: TTT-aware training can change performance without inducing corresponding reliance on inference-time updates. We also expanded Section 6.2 to compare In-Place TTT with related memory approaches (Atlas, PERK, Doc-to-LoRA, and Cartridges), highlighting differences in access to the original context and potential interference with pretrained computation. Finally, we clarified the reproduction gap with the original In-Place TTT setup.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.