- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Test-Time Adaptation via Stability-Aware Confidence Optimization (TASCO) for LLM reasoning. Motivated by an observed association between local confidence stability and answer correctness, TASCO optimizes a continuous prefix shared across unlabeled queries from a target task while keeping the backbone frozen. It introduces two alternative strategies: Random Perturbation (RP) combines entropy minimization with a penalty on the variance of trajectory-average token log-probabilities across generations under Gaussian-perturbed prefixes; Sharpness-Aware Perturbation (SAP) minimizes predictive entropy at a gradient-guided perturbed prefix while holding the generated trajectory fixed. The learned prefix is held fixed during final answer generation. Experiments on five LLM variants across reasoning benchmarks report improved average accuracy and shorter final generations relative to zero-shot CoT, alongside comparisons with existing adaptation methods. The paper supports its approach with ablations, behavioral analyses, and local sensitivity analysis.
Critical strengths
-
The stability-aware perspective is supported by complementary diagnostic and behavioral analyses. Appendix B reports significant confidence-controlled associations between local stability and correctness in four of the five examined models, providing empirical motivation for the optimization objective. Behavioral analyses compare TASCO with confidence-only prefix optimization through perturbation robustness (Table 3), early decoding distributions (Figure 6), and an external LLM judge's assessment of step correctness, informativeness, redundancy, and premature commitment (Table 9). The qualitative example in Table 10 illustrates a different solution path that preserves a valid root missed by the confidence-only baseline. Together, these analyses connect the optimization design to both probabilistic and semantic aspects of generation, helping interpret the reported accuracy gains.
-
The reported accuracy gains are supported by informative controls and transfer experiments. SAP improves average accuracy over TTSV by 2.9, 4.8, and 4.4 percentage points on the three backbones in Table 1. The unoptimized-prefix control and RP's same-rollout Perturbed Confidence control directly examine the contributions of prefix optimization and the stability objective (Figure 4). Furthermore, prefixes transferred between mathematical benchmarks improve over zero-shot CoT on all 12 evaluated source-target pairs without target-side re-optimization on Qwen2.5-Math-7B (Figure 8), supporting generalization to mathematical queries beyond the source adaptation set.
Critical weaknesses
-
The efficiency evaluation could more fully account for adaptation cost and task-startup latency. The reported token savings concern final answer generation, while obtaining the prefix also requires confidence-only warm-up and iterative optimization. RP introduces additional perturbed rollouts, and SAP requires two gradient computations per update; distinct perturbed prefixes also limit exact prompt-KV reuse across branches. Reporting the time and compute needed to obtain a usable prefix, together with the number of queries required to amortize this cost, would clarify the practical resource trade-off. This concern applies to prefix adaptation, while final answer generation uses the fixed learned prefix.
-
Continual adaptation in deployment could be discussed more explicitly. Real-world deployment can involve a continuing stream of queries whose task distribution evolves over time. The authors' cross-benchmark transfer experiments provide useful evidence that learned prefixes generalize across the evaluated mathematical benchmarks (Figure 8). Yet these experiments apply a fixed prefix to each target benchmark, leaving open how the prefix should be maintained as deployment queries evolve. It would be better if the authors discussed a continual adaptation strategy, clarifying when to reuse an existing prefix, update it incrementally using incoming unlabeled queries, or optimize a new prefix.
Questions for authors
-
Adaptation cost and deployment readiness. On the same hardware, how much wall-clock time is required for confidence-only warm-up and subsequent prefix optimization for each TASCO variant? Please report the initial delay before the adapted prefix is ready and the amortized end-to-end cost, including final generation, across different query counts relative to zero-shot CoT and confidence-only adaptation.
-
Transfer baseline. In Figure 8, how does a source-trained TASCO prefix compare with a confidence-only prefix trained on the same source queries and transferred without target-side adaptation? This comparison would help clarify the contribution of stability optimization to the observed transfer gains.
-
Adaptation data requirements. How sensitive is TASCO to the number of unlabeled queries used for prefix adaptation? Can a prefix learned from a small subset retain its benefits on held-out queries from the same task?
4 comments
W2: Continual adaptation in deployment could be discussed more explicitly. Real-world deployment can involve a continuing stream of queries whose task distribution evolves over time. The authors' cross-benchmark transfer experiments provide useful evidence that learned prefixes generalize across the evaluated mathematical benchmarks (Figure 8). Yet these experiments apply a fixed prefix to each target benchmark, leaving open how the prefix should be maintained as deployment queries evolve. It would be better if the authors discussed a continual adaptation strategy, clarifying when to reuse an existing prefix, update it incrementally using incoming unlabeled queries, or optimize a new prefix.
Response to weakness 2: We thank the reviewer for recognizing the generalization of TASCO and for raising the question of how prefixes should be maintained as deployment queries evolve. We agree that this is an interesting direction and conduct a small prototype experiment to explore possible maintenance strategies.
We construct a query stream consisting of MATH500, Minerva Math, and GPQA, with 100 queries sampled from each dataset. Starting from a prefix optimized on MATH500, we use the first 50 queries in each stage to observe the incoming distribution and perform adaptation, and the remaining 50 for evaluation. We consider a semantic similarity-based strategy that chooses between prefix reuse, incremental update, and re-optimization according to the similarity between incoming queries and the existing adaptation data.
| Query Stream | Fixed Reuse | Incremental Update | Re-optimization |
|---|---|---|---|
| MATH500 | 78.0 | 74.0 | 68.0 |
| Minerva Math | 32.0 | 40.0 | 30.0 |
| GPQA | 30.0 | 34.0 | 38.0 |
The results show that reusing the existing prefix works well when the query distribution remains similar, while further updates may overfit the small set of new queries. Incremental updates are more effective for related but different mathematical problems, whereas re-optimization performs best after the shift to a more distinct domain. These findings provide preliminary evidence for selecting a prefix maintenance strategy based on changes in the query distribution.
Q2: Transfer baseline. In Figure 8, how does a source-trained TASCO prefix compare with a confidence-only prefix trained on the same source queries and transferred without target-side adaptation? This comparison would help clarify the contribution of stability optimization to the observed transfer gains.
Rseponse to question 2: We thank the reviewer for the suggestion. To evaluate the contribution of stability optimization to cross-benchmark transfer, we compare TASCO with the confidence-only baseline TTSV by directly transferring their learned prefixes to other benchmarks for evaluation. As shown below, TASCO consistently outperforms TTSV, further demonstrating the benefits of stability-aware optimization for cross-benchmark transfer.
| Source | Target | TTSV | TASCO | Δ (pp) |
|---|---|---|---|---|
| MATH500 | AMC23 | 56.1 | 64.4 | +8.3 |
| Minerva Math | 29.4 | 33.4 | +4.0 | |
| AIME24 | 8.9 | 21.2 | +12.3 | |
| Minerva Math | MATH500 | 66.2 | 71.2 | +5.0 |
| AMC23 | 56.7 | 61.2 | +4.5 | |
| AIME24 | 13.5 | 19.1 | +5.6 | |
| AIME24 | MATH500 | 68.4 | 74.0 | +5.6 |
| AMC23 | 59.1 | 63.1 | +4.0 | |
| Minerva Math | 28.7 | 34.2 | +5.5 | |
| AMC23 | MATH500 | 64.4 | 70.4 | +6.0 |
| Minerva Math | 31.6 | 35.3 | +3.7 | |
| AIME24 | 13.3 | 25.4 | +12.1 | |
| Overall mean | 41.36 | 47.74 | +6.38 |
Q3: Adaptation data requirements. How sensitive is TASCO to the number of unlabeled queries used for prefix adaptation? Can a prefix learned from a small subset retain its benefits on held-out queries from the same task?
Rseponse to question 3: We thank the reviewer for raising this question. To evaluate the sensitivity of TASCO-SAP to adaptation data size, we sample 25, 50, 100, 200, 300, and 500 unlabeled problems from the MATH training split to optimize separate prefixes, and evaluate each prefix on the held-out MATH500 benchmark across three backbone models.
As shown in the table, performance generally improves with more adaptation queries. Notably, using only 300 unlabeled queries, all three models achieve accuracy comparable to or higher than that obtained with 500 queries. These results suggest that TASCO-SAP does not require a large adaptation set to achieve competitive performance on unseen queries from the same task.
| Model | n = 25 | n = 50 | n = 100 | n = 200 | n = 300 | n = 500 |
|---|---|---|---|---|---|---|
| Qwen2.5-Math-1.5B | 59.0 | 61.4 | 63.6 | 69.6 | 70.6 | 72.4 |
| Qwen2.5-Math-7B | 45.6 | 58.8 | 64.6 | 68.2 | 75.0 | 74.0 |
| LLaMA-3.1-8B-Instruct | 41.2 | 44.2 | 46.6 | 45.8 | 48.8 | 49.8 |
Thank you for the detailed clarfications and additional experiments. The adaptation-time breakdown and amortization analysis substantially address my efficiency concerns, while the transfer baseline and held-out evaluation strengthen the evidence for generalization. Overall, these additions clarify the method’s practical trade-offs, so I will maintain my positive score.
Dear Reviewer 4to7,
Thank you for your positive feedback and for taking the time to consider our response. Your suggestions helped us clarify the method’s practical trade-offs and strengthen our evaluation of generalization. We also appreciate your continued support of our work.
We will incorporate the additional analyses and experimental results from our discussion into the final manuscript.
Authors 35531
Response to Reviewer 4to7
We are grateful to reviewer 4to7 for the positive assessment of our stability-aware perspective, empirical findings, and generalization results. We also appreciate the constructive feedback and have conducted additional experiments to address the reviewer's concerns.
Response to weakness 1 and question 1: We thank the reviewer for raising concerns about adaptation costs and task-startup latency. To address these concerns, we provide three analyses: comparing per-question prefix optimization and inference times; breaking down the two optimization stages to quantify startup latency; and evaluating amortized end-to-end costs across query volumes, accounting for prefix reuse.
Optimization and Inference Efficiency. First, we have added an Optimization and Inference Efficiency subsection reporting prefix optimization and inference times on MATH500 and AIME24 using Qwen2.5-Math-7B on a single NVIDIA A800 GPU. Both TASCO variants undergo five epochs of confidence-only warm-up followed by five epochs of stability-aware optimization. As shown in the table below, TASCO-RP incurs 2.3–2.7 times the optimization cost of TTSV due to repeated perturbed rollouts, whereas TASCO-SAP reuses the generated trajectory with only one additional forward and backward pass, increasing optimization time by less than 5%.
Task-Startup Latency. To further clarify task-startup latency, we report the time breakdown of the two optimization stages on MATH500. As shown below, both variants require 26.7 minutes for confidence-only warm-up, followed by 116.0 minutes of stability optimization for TASCO-RP and 28.4 minutes for TASCO-SAP. The total time to obtain an adapted prefix is therefore 142.7 minutes for TASCO-RP and 55.1 minutes for TASCO-SAP.
Amortized End-to-End Efficiency. Since the learned prefix can be reused, we further examine how adaptation costs are amortized across different query volumes. On MATH500, TASCO-SAP requires approximately 55.1 minutes for initial adaptation but saves 10.31 seconds per query relative to Zero-Shot CoT, requiring about 321 subsequent queries to offset this cost. Relative to TTSV, only 32 queries are needed to recover SAP's additional adaptation cost. The table below reports the amortized total time per query, including initial adaptation and final generation, across different query volumes. As query volume increases, both TASCO variants progressively amortize their upfront costs. TASCO-RP requires more queries due to its higher optimization cost but benefits from faster inference at larger scales, whereas TASCO-SAP achieves lower overall costs with fewer queries.
Revised PDF: Section 4.6 ("OPTIMIZATION AND INFERENCE EFFICIENCY").