- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how model-internal confidence signals can improve multi-trajectory aggregation for test-time reasoning. To handle high-confidence but incorrect or stagnating trajectories, it introduces the inter-layer confidence difference between the final and penultimate layers as a complementary signal to static confidence. The method independently ranks trajectories by tail-averaged final-layer confidence and tail-averaged inter-layer confidence dynamics, retains the top- under each criterion, takes their intersection, and then performs weighted majority voting using the inter-layer confidence dynamics as weights.
Critical strengths
- The method is simple and requires no additional training or external verifier. It uses the model’s own inter-layer confidence differences to support trajectory filtering and weighted voting.
Critical weaknesses
- The paper lacks comparisons with vanilla performance and with state-of-the-art training-free baselines such as Certaindex [1], DEER [2], and ASAG [3]. It also does not report generation-length or efficiency metrics, such as average generated tokens or end-to-end inference cost.
- The method requires multiple sampled trajectories per problem: with N=1, joint filtering and weighted aggregation degenerate, so it cannot be applied in single-sample settings. In contrast, Certaindex [1], DEER [2], and ASAG [3] do not require multi-sample comparison. This is a significant limitation of the proposed method. In addition, Comparing N-sample and single-sample settings is also unfair.
- The paper ignores computational overhead: the cost of sampling N trajectories is not accounted for, and the extra vocabulary distribution computed from the penultimate layer is not discussed.
- The evaluation primarily focuses on mathematical and scientific reasoning, but do not include code generation. The method’s applicability to these tasks therefore remains unverified.
- The retention ratio is set to 60% for Qwen3 and 50% for Gemma4. Why are these values chosen? The need to tune this parameter limits the effectiveness and generality of the method.
- can also be negative, which would assign a negative voting weight to that trajectory. The paper does not discuss or address this issue.
[1] Efficiently Scaling LLM Reasoning Programs with Certaindex. NeurIPS, 2025.
[2] Dynamic early exit in reasoning models. ICLR, 2026.
[3] Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models. ICML, 2026.
Questions for authors
Could the authors provide an ablation using the same trajectory pool that filters trajectories solely by inter-layer confidence differences while retaining the same dynamics-weighted voting, to assess whether adding final-layer confidence filtering provides additional benefits?
4 comments
Response to Reviewer TiMu [part 2]
3. Retention Ratio and Generality (W5)
Our revised experiments remove dynamics-specific filtering thresholds while maintaining consistent improvements across different trajectory-pool settings.
As detailed in our General Response, we have redesigned the main experiments to use confidence dynamics directly as a voting weight, without requiring an additional dynamics-based filtering threshold. We compare standard, confidence-weighted, and dynamics-weighted voting on identical trajectory pools under three shared settings: the full pool (100%) and the top-80% and top-50% confidence-filtered subsets. The revised results show that dynamics-weighted voting consistently outperforms its confidence-weighted counterparts across these settings, demonstrating that its effectiveness does not depend on a particular retention threshold. These experiments also clarify the complementary roles of confidence filtering and dynamics weighting.
4. Handling Negative Confidence Dynamics (W6)
We apply a non-negative shift to confidence-dynamics scores before using them as voting weights.
We thank the reviewer for identifying this missing implementation detail. For each sampling pool , we compute:
This ensures non-negative voting weights while preserving the relative ordering and pairwise differences of the original scores. We have clarified this implementation in Section 3.2 of the revised manuscript.
5. Ablation of Confidence and Dynamics Filtering (Q1)
Our ablations show that confidence is more effective for filtering trajectories, while confidence dynamics provides a stronger voting weight.
Our revised main experiments (see General Response) already demonstrate that dynamics-weighted voting generally outperforms confidence-weighted voting on identical trajectory pools, including confidence-filtered subsets. To directly address the question, we additionally compare confidence-based and dynamics-based filtering while keeping tail dynamics-weighted voting fixed, using the same top-50% retention ratio.
| Model | CF-DᵀW | CᵀF-DᵀW | DF-DᵀW | DᵀF-DᵀW |
|---|---|---|---|---|
| Qwen3-1.7B | 49.84 | 49.24 | 48.96 | 49.06 |
| Qwen3-8B | 68.71 | 69.09 | 68.87 | 68.95 |
| Nemotron-Nano-4B | 55.25 | 56.16 | 54.64 | 55.45 |
Overall accuracy (%) under top-50% retention. CF/DF denote global confidence/dynamics filtering, while CᵀF/DᵀF denote tail-based filtering. All methods use tail dynamics-weighted voting (DᵀW).
Confidence-based filtering achieves the highest accuracy on all three models: 49.84%, 69.09%, and 56.16%, compared with 49.06%, 68.95%, and 55.45% using tail dynamics filtering. Together with our weighting ablations, these results support our revised design: confidence-based filtering selects more reliable candidates, while confidence dynamics provides a complementary signal for weighting their contributions during aggregation.
Thank you for the detailed rebuttal and additional experiments. Some of my concerns have been addressed, but I still have the following concerns:
-
The authors only report an additional 3%–8% in theoretical FLOPs, without providing evaluations of actual inference latency, memory consumption, or end-to-end computational cost.
-
The proposed method requires generating multiple reasoning trajectories, but the authors do not provide accuracy–compute curves or demonstrate whether comparable accuracy can be achieved with fewer sampled trajectories. Therefore, it remains unclear whether the modest accuracy improvements justify the additional computational cost.
We thank the reviewer for this follow-up.
Response to Concern 1: Actual Inference Latency and Memory Overhead
To complement our theoretical FLOPs analysis, we conducted an additional end-to-end inference benchmark using Qwen3-1.7B on a single NVIDIA H100 GPU. We evaluated 10 AIME problems with 10 trajectories per problem, using identical random seeds, generated sequences, and sequence lengths for both settings.
| Metric | Standard (C) | Ours (C + ΔC) | Overhead |
|---|---|---|---|
| Generation Time | 878 s | 935 s | +6.5% |
| Throughput | 1,949 tok/s | 1,830 tok/s | −6.1% |
| Peak GPU Memory | 62,464 MiB | 62,584 MiB | +0.19% |
The results show that computing inter-layer confidence dynamics introduces only 6.5% additional inference time and 0.19% peak memory overhead, consistent with our theoretical FLOPs estimates (3%–8%). These measurements confirm that confidence dynamics can be extracted with modest additional computational cost, without requiring extra trajectory generation.
Response to Concern 2: Accuracy–Compute Tradeoff
Our Section 4.5 and additional experiments demonstrate that confidence dynamics improves sample efficiency, achieving higher accuracy with substantially fewer trajectories.
We extend our evaluation in Section 4.5 to sampling budgets across three models and five reasoning benchmarks. For each problem, we draw 20 random subsets of distinct trajectories without replacement from a fixed pool of 50, reporting the mean accuracy. All methods use identical subsets for fair comparison.
As shown in the new accuracy–sampling-budget curves (Appendix E, Figure 7), dynamics-weighted voting consistently outperforms standard majority voting across all evaluated budgets, with an increasingly pronounced advantage over confidence-weighted voting at larger budgets. Notably, tail dynamics-weighted voting with only surpasses both standard and confidence-weighted voting with across all three models, even without filtering. These results demonstrate that confidence dynamics enables more effective use of the sampling budget, and offsets its modest additional computational overhead (3%–8% theoretical FLOPs per token).
Response to Reviewer TiMu [part 1]
1. Baseline Comparability and Computational Efficiency (W1–W3)
1.1 Methodological Scope and Baseline Comparability (W1, W2)
Our method focuses on post-hoc aggregation under a fixed sampling budget, rather than single-sample reasoning or adaptive generation.
Multi-trajectory aggregation is an established test-time scaling paradigm that improves reasoning reliability by combining complementary reasoning paths. Like self-consistency and majority voting, our method requires N>1 trajectories, which is inherent to the aggregation setting. Importantly, we do not compare N-sample aggregation against single-sample inference to claim improvements. Instead, all methods operate on exactly the same N sampled trajectories, enabling fair comparisons with standard majority voting, confidence-weighted voting, and confidence-based filtering.
The suggested methods, Certaindex, DEER, and ASAG, primarily address complementary problems, including adaptive sampling, early stopping, and generation-time intervention to reduce redundant reasoning. Our method instead improves the aggregation of completed trajectories without modifying generation. These approaches could potentially be combined with ours to construct and aggregate more efficient trajectory pools. We have clarified this distinction in the revised manuscript.
1.2 Computational Overhead (W1, W3)
Our method incurs only modest additional computation beyond standard N-sample generation.
All aggregation methods share the same trajectory generation cost. Our additional overhead comes primarily from applying the LM head to the penultimate-layer hidden states to obtain an extra vocabulary distribution. Final-layer logits are already available during decoding, while the additional confidence calculations are comparatively inexpensive. We estimate the per-token computational cost as follows:
These theoretical FLOPs estimates indicate only 3%–8% additional computation per generated token. Computing confidence dynamics only over tail tokens can further reduce this overhead when implemented without full-sequence intermediate-layer projections. We emphasize that our contribution is improving aggregation accuracy under the same multi-sample generation budget, rather than reducing the number of generated tokens. We have clarified this distinction and the associated computational overhead in the revised manuscript.
2. Effectiveness on Coding Tasks (W4)
Our additional evaluation demonstrates that confidence-dynamics-weighted voting also improves code generation performance.
We thank the reviewer for this suggestion. We extend our evaluation to LiveCodeBench v6 using Qwen3-1.7B and Qwen3-8B under sampling budgets N∈{20,25,30} (Appendix F).
Voting accuracy (%) on LiveCodeBench v6. CW and DW denote global confidence- and dynamics-weighted voting; CᵀW and DᵀW use their tail-based counterparts.
Dynamics-weighted voting consistently outperforms standard majority voting and confidence-weighted voting across all six settings, improving over standard voting by 3.05–3.82 percentage points on Qwen3-1.7B and 4.43–5.35 points on Qwen3-8B. These results provide additional evidence that confidence dynamics is useful beyond mathematical and scientific reasoning, extending to code generation.