- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper find that existing TTT methods work unevenly across tasks. To address this, the authors combine five existing TTT methods as callable tools and maintain a skill library. This capability is implemented by SFT and RL process. On a curated benchmark, the trained model improve performance significantly.
Critical strengths
- This paper identify several key problems in TTT. When to explore, when to exploit, and how to do it.
- This paper propose a unified framework to combine all the 5 TTT methods.
- Beyond final acc, the paper also considers the exact cost.
Critical weaknesses
- I think the main limitation of this paper is the lack of novelty. It merely combines several existing methods, without solving any new problems.
- The evaluation benchmark deliberately selects queries that benefit from TTT, limiting the generalizability of the proposed method.
- Ablation studies do not sufficiently isolate the cause of improvement. Moreover, it could be better to test what happens without skill reuse.
- The cost analysis does not include response time or else metrics which related to real world application. Besides, the paper should compare with methods that use the same amount of compute.
Questions for authors
Could the authors clarify the position of this paper? ( Something more than simple combination of existing methods)
6 comments
Response to Reviewer naaw (Split 2 of 5)
W2. Evaluating on benchmark distribution where TTT benefits are smaller.
W2. The evaluation benchmark deliberately selects queries that benefit from TTT, limiting the generalizability of the proposed method.
We thank the reviewer for this constructive suggestion. We constructed an unfiltered benchmark and evaluated both the original Agentic-TTT checkpoint without retraining and a policy retrained on the new training distribution.
New benchmark construction: we uniformly sampled queries from the same source datasets as our original benchmark, without filtering by TTT benefits. Using the same heuristic assignment of reference TTT methods, we obtained the following distribution:
| Split | Queries | Direct already correct | Reference TTT -> still wrong | Reference TTT -> beneficial |
|---|---|---|---|---|
| Train | 244 | 89 | 130 | 25 (10.2%) |
| Dev | 244 | 95 | 130 | 19 (7.8%) |
| Eval | 242 | 92 | 124 | 26 (10.7%) |
Only 10.7% of evaluation queries benefit from reference TTT, compared with approximately 65% in the original evaluation set. These categories describe outcomes under the assigned reference method, rather than all possible TTT methods. Choosing an unsuitable TTT method can harm performance, as illustrated by the negative gains from applying SEAL to math and coding tasks (c.f. Table 1 of our paper, the "Always-SEAL" column), where the method is poorly matched to the task.
Agentic-TTT retains utility gains without retraining.
We compare utility, TTT build counts, and total evaluation FLOPs below. Compute includes TTT sampling and learning, policy reasoning, and answer generation, normalized by the direct-answer baseline’s total FLOPs ().
| Method | Utility | TTT Build count | FLOPs / Direct |
|---|---|---|---|
| Direct Answer | 0.380 | 0 | 1.00 |
| Prompted Policy | 0.397 | 63 | 21.33 |
| Semantic Router | 0.413 | 205 | 22.80 |
| Agentic-TTT (original) | 0.438 | 116 | 24.79 |
| Agentic-TTT (retrained) | 0.426 | 83 | 18.66 |
| Oracle | 0.483 | 33 | 8.00 |
Transferable benefits. The original checkpoint improves utility by 5.8 percentage points over direct answering and outperforms prompted and semantic routing baselines. This suggests that our policy trained on a benefit-enriched distribution can transfer its knowledge of suitable TTT methods to settings where beneficial opportunities are much less frequent, rather than applying TTT indiscriminately.
Behavioral differences. The training distribution also appears to shape the policy’s behavioral tendency. The original policy, trained on a benefit-enriched distribution, uses TTT more aggressively; while the policy trained on the lower-benefit distribution produces more conservative behavior. During evaluation, build operations decrease from 116 to 83, and total compute falls from 24.79 to 18.66 times the direct-answer baseline.
These results show that Agentic-TTT retains substantial utility gains on an unfiltered, lower-benefit distribution without retraining. However, the original policy may still adapt too aggressively, spending compute on attempts that do not improve outcomes. While this checkpoint remains effective, training on the distribution that reflects the expected deployment setting can help achieve a better utility–compute trade-off by encouraging a more conservative adaptation strategy. We have added this evaluation to the revised paper (Appendix B.3) and summarize it in the Introduction.
Response to Reviewer naaw (Split 3 of 5)
W3. Ablation study: without skill reuse.
W3. Ablation studies do not sufficiently isolate the cause of improvement. Moreover, it could be better to test what happens without skill reuse.
We thank the reviewer for these constructive comments. We examine the advantage of skill reuse through both a breakdown of the trained policy's behavior and a retrained policy variant without reuse.
Skill reuse provides substantial gains at low marginal cost. As shown in the table below, reuse accounts for only 17.1% of evaluation events and requires just 4.3 minutes in total for loading skills and answering queries. It consumes only 1.5% of the overall evaluation time while contributing 44.8% of the observed utility gain, yielding substantial benefits at low marginal cost. Once a useful skill has been built, reuse can provide an efficient and economical way to benefit future queries without repeating adaptation cost.
| Action category | Triggers/evaluation (action %) | Evaluation time (time %) | Utility contribution (gain %) |
|---|---|---|---|
| Build / Update | 97.3 (49.9%) | 251.4 min (84.9%) | +0.140 (55.2%) |
| Reuse | 33.3 (17.1%) | 4.3 min (1.5%) | +0.114 (44.8%) |
| Direct Answer | 64.3 (33.0%) | 5.7 min (1.9%) | 0.000 (0.0%) |
Removing reuse increases building frequency but reduces utility. Following the reviewer’s suggestion, we retrained Agentic-TTT with skill reuse removed from its available actions:
| Variant | Utility | Build / Reuse / Direct Answer count | Total evaluation time |
|---|---|---|---|
| Agentic-TTT | 0.510 | 97.3 / 33.3 / 64.3 | 296.20 min |
| Without skill reuse | 0.439 | 126.7 / 0 / 68.3 | 473.15 min |
Without reuse, the policy relies more heavily on new adaptations. It performs 30.2% more build operations, with evaluation time increasing from 296.2 to 473.15 minutes; yet the utility falls from 0.510 to 0.439. More frequent adaptation therefore does not compensate for the loss of skill reuse. This ablation supports the importance of reusing acquired capabilities for effective self-improvement across queries. We have added both analyses to the revised paper: Section 3.1 reports the time and utility shares of skill reuse, Section 3.5.3 presents the ablation without reuse, and Appendix F.3 provides the full breakdown.
Response to Reviewer naaw (Split 1 of 5)
We thank the reviewer for your careful reading and constructive comments. We appreciate your recognition of the challenges we identify in TTT and our consideration of cost-utility trade-off. We address the reviewer's concerns below.
W1&Q1. The position of this paper.
Agentic-TTT is not just trivial combination of TTT methods. Its core is a deployment-time decision problem: how a model can autonomously decide whether to adapt, which TTT method to use, and when to reuse previously acquired capabilities for self-improvement. Our contribution lies in formulating, training and evaluating a policy for these decisions.
1. Existing TTT methods do not autonomously solve deployment-time method selection and application.
We show that TTT methods cannot be unconditionally applied or universally beneficial across all cases. They rely on various prerequisites and have unique failure modes: for example, TTT-FS [1] requires in-context demonstrations; TTRL [2] explicitly discusses failure when the model lacks prior knowledge (in Section 4.3 of their paper).
Our experiment in Table 1 further shows that fixed-method baselines produce uneven or negative benefits across domains. These findings motivate an explicit decision layer: deployment requires determining which TTT adaptation, if any, is appropriate for the given test query. Our Agentic-TTT makes this decision problem a learning objective, rather than fixed rules.
2. Agentic-TTT supports capability accumulation and reuse through stateful sequential decisions.
We view model deployment as a stateful sequential decision process over evolving sample pools and skill library. Our policy must organize adaptation material, determine when a method's prerequisites are satisfied, and identify when existing skills can serve future queries. Method selection is only one part of this broader capability-management problem: the model grows its capability set over time. This is very different from many previous TTT works, as they mostly treat each query in isolation, and focus on immediate rewards without considering long-term self-improvement.
Our additional experiments support the value of these designs. As reported in our responses to
Reviewer XYgN's Q1on routing baselines andyour W3on skill reuse, both learned routers based on Qwen3-Embedding-8B and the framework variant without skill reuse substantially underperform our framework. These results highlight the importance of management over the whole skill system.3. Agentic-TTT is scalable to additional TTT methods.
Agentic-TTT treats all TTT methods as callable tools. The harness and policy-learning pipeline operate over this interface, separating the decision process from individual TTT algorithms. Additional future compatible TTT methods can therefore be incorporated into our framework, expanding the repertoire available for deployment-time self-improvement.
In conclusion, our work
Offers a new perspective: No single TTT method is universally beneficial, and a model needs wise decision-making to achieve general self-improvement.
Formulates a new research problem: How to achieve autonomous decision-making over callable TTT tools and evolving skill library?
Delivers a new methodology: an executable harness, guidance distillation followed by outcome-driven RL, and a training objective that controls the utility-computation trade-off.
Reference
[1] Akyürek, Ekin, et al. "The surprising effectiveness of test-time training for few-shot learning." International Conference on Machine Learning (ICML), 2025.
[2] Zuo, Yuxin, et al. "Ttrl: Test-time reinforcement learning." The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025).