- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the sensitivity of vision test-time training (TTT) models to the magnitude of their inner updates. It finds that existing models perform well only within a narrow range around the training magnitude, and attributes this fragility to updates compensating for harmful outputs at the initial weights. The authors propose η-ViT³, which samples a shared update magnitude from U(0, 2) during each training forward pass. Experiments on classification, segmentation, and image generation show that this simple change broadens the usable adaptation range with little impact on standard-setting performance and no additional inference cost. The increased tolerance persists under distribution shifts, resolution changes, and downstream fine-tuning, while retaining dependence on input-specific update directions.
Critical strengths
-
An insightful diagnosis of adaptation fragility. The paper shows that vision TTT models can be highly sensitive to update magnitude, with skipping adaptation sometimes causing more damage than removing the TTT branch entirely. Its output decomposition and intervention experiments provide evidence that adaptation compensates for harmful initial-weight outputs. This analysis goes beyond reporting sensitivity and offers a useful explanation of why adaptation fails, making it a substantive contribution to understanding TTT models.
-
A simple intervention with convincing empirical support. Sampling the update magnitude during training substantially broadens the usable adaptation range without adding parameters or inference computation. For the tiny classifier, accuracy drops by at most 0.5 points over magnitudes from 1/8 to 2, compared with up to 16.8 points for the matched baseline. The persistence of this benefit across model sizes, resolution changes, distribution shifts, and downstream fine-tuning strengthens its practical significance, while direction-replacement experiments support that the model still uses input-specific adaptation.
Critical weaknesses
The practical value of a wider adaptation range is unclear. What practical benefit does changing the update magnitude provide? In Table 3, all three models perform worse than their standard-training baselines at the default setting. What useful capability does varying the update magnitude provide that justifies this loss in performance?
Questions for authors
Could the authors provide a concrete use case where varying the update magnitude offers a measurable benefit over the standard fixed-magnitude baseline, sufficient to justify the lower default-setting performance reported in Table 3?