- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the joint optimization of rollout allocation and voting rules for LLM test-time compute under token budgets. The authors propose PRICE, which adaptively selects the number of rollouts and voting temperature for each query. The paper provides theoretical characterizations of the accuracy–cost Pareto frontier and evaluates the method on MATH-500.
Critical strengths
- The problem of optimizing test-time compute efficiency is relevant and practically meaningful.
- The paper provides a relatively comprehensive theoretical analysis.
Critical weaknesses
- Incomplete computation cost accounting. My main concern is that the reported token costs exclude self-evaluation scoring, query embeddings, predictor inference, and calibration. Since PRICE relies on these additional components, the reported 2.8–3.5× budget improvement may not translate into comparable end-to-end efficiency gains. A fair comparison accounting for all computational overhead is necessary.
- Insufficient ablation of the deployed method. PRICE-deployed combines adaptive voting, learned accuracy prediction, sequential stopping, and budget-aware allocation. However, the experiments do not sufficiently isolate the contribution of adaptive voting in the deployed setting. A baseline using the same predictors and stopping mechanism but a fixed voting rule would be particularly important.
- Limited experimental generalization. The main experiments focus on MATH-500 with relatively small language models. It remains unclear whether the improvements generalize to stronger reasoning models, different task domains, or settings with more capable verifiers.
Questions for authors
See weakness.
4 comments
Weakness 1. Score flexibility and computational overhead
Self-evaluation is one choice of score. Our framework accepts any per-response scores, including verifier or reward-model outputs and intrinsic generation statistics. BoN and score-weighted voting methods such as CISC also require scores; the cost of obtaining them depends on the chosen scoring procedure.
Our theoretical analysis uses a token-wise budget: the entropic risk of generated rollout tokens, b = γ⁻¹ log E[exp(γL)]. The 2.8–3.5× result compares the budgets needed to match the submitted SC baseline’s accuracy. P(True) scoring and embedding produce no additional generated tokens; their computation is measured separately below.
For the P(True) implementation, we additionally measure scoring and other online overhead using Qwen2.5-1.5B and Llama3.2-3B on all 500 MATH-500 questions in vLLM with FP16 on an H200 NVL per query. Scoring and all other non-generation work together account for at most 1.48% of PRICE’s cumulative compute time.
Table 1. Compute breakdown (seconds per question). Other online work includes embedding, controller, parsing, clustering/voting, and request preparation.
| Component / seconds per question | Qwen | Llama |
|---|---|---|
| Generation | 23.099 | 12.790 |
| P(True) scoring | 0.206 | 0.105 |
| Other online work | 0.112 | 0.058 |
| Total | 23.417 | 12.953 |
| Non-generation share | 1.36% | 1.26% |
At matched accuracy, PRICE uses less cumulative computation than SC and CISC. Relative to SC, the reduction is 21.3–32.7% on Qwen and 19.8–30.6% on Llama, including scoring and controller costs.
Table 2. Accuracy-matched mean cumulative compute time (seconds per question).
| Model | Matched accuracy | PRICE (s) | SC (s) | CISC (s) |
|---|---|---|---|---|
| Qwen2.5-1.5B | 60.8% | 9.02 | 11.87 | 10.92 |
| Qwen2.5-1.5B | 62.6% | 11.45 | 14.55 | 18.90 |
| Qwen2.5-1.5B | 63.4% | 12.54 | 18.63 | 23.22 |
| Llama3.2-3B | 55.0% | 12.95 | 16.15 | 15.76 |
| Llama3.2-3B | 57.0% | 21.42 | 30.85 | 27.25 |
| Llama3.2-3B | 59.0% | 34.89 | 46.48 | 38.74 |
The gains also hold with intrinsic scores. We refit PRICE-deployed for each score and use that score for BoN and calibration-tuned CISC. Each intrinsic score beats the strongest evaluated baseline at all six Qwen budgets, with positive paired-bootstrap 95% confidence intervals. Likelihood and mean top-1 use generation logs without an extra verifier pass.
Table 3. Qwen2.5-1.5B accuracy (%) under generated-token risk budget b. Parentheses show gains over the best of SC, BoN, CISC, AC, ESC, and DeepConf, with matched scores for score-dependent methods.
| Budget b | P(True) | Likelihood | Mean top-1 | LGC |
|---|---|---|---|---|
| 3679.55 | 61.30 | 61.48 (+5.18) | 61.44 (+5.32) | 61.47 (+5.28) |
| 5290.65 | 63.33 | 63.58 (+5.13) | 63.48 (+5.03) | 63.53 (+5.08) |
| 6658.19 | 64.31 | 64.78 (+4.40) | 64.85 (+4.60) | 64.72 (+4.39) |
| 11552.59 | 66.17 | 66.85 (+2.93) | 66.77 (+2.85) | 66.77 (+2.85) |
| 14515.83 | 66.58 | 67.37 (+2.75) | 67.33 (+2.72) | 67.28 (+2.77) |
| 31982.79 | 67.56 | 68.57 (+1.31) | 68.71 (+1.47) | 68.36 (+1.32) |
Weakness 2. Voting potential with deployed allocation and stopping fixed
We fix PRICE-deployed’s allocation, rollout trajectories, stopping counts, and token costs, and replace only the final vote. The adaptive voting improves over majority-only and the best global fixed temperature at all twelve budgets, with positive pointwise 95% confidence intervals.
Table 4. Adaptive voting accuracy gains (pp; 95% CI) at generated-token risk budgets.
| Model | Token-risk budget b | Adaptive voting − majority-only | Adaptive voting− best fixed voting (pp; 95% CI) |
|---|---|---|---|
| Qwen | 3,679.55 | +2.06 [1.47, 2.76] | +2.04 [1.46, 2.69] |
| Qwen | 5,290.65 | +2.61 [1.90, 3.42] | +2.60 [1.88, 3.37] |
| Qwen | 6,658.19 | +2.87 [2.06, 3.77] | +2.87 [1.04, 3.73] |
| Qwen | 11,552.59 | +2.60 [1.64, 3.74] | +2.60 [1.60, 3.64] |
| Qwen | 14,515.83 | +2.84 [1.81, 3.06] | +2.84 [1.80, 3.97] |
| Qwen | 31,982.79 | +2.26 [1.12, 3.59] | +2.25 [1.10, 3.43] |
| Llama | 4,426.10 | +1.40 [0.99, 1.87] | +1.38 [0.97, 1.82] |
| Llama | 6,234.63 | +1.77 [1.26, 2.39] | +1.76 [1.24, 2.35] |
| Llama | 8,072.20 | +1.94 [1.35, 2.66] | +1.91 [1.32, 2.58] |
| Llama | 13,783.96 | +2.34 [1.65, 3.16] | +2.33 [1.63, 3.11] |
| Llama | 17,425.72 | +2.50 [1.77, 3.39] | +2.48 [1.74, 3.30] |
| Llama | 40,399.43 | +2.56 [1.69, 3.59] | +2.53 [1.66, 3.41] |
Weakness 3. Newer models, harder tasks, and score choice
We added Qwen3-4B and Qwen3-8B on Minerva and OlympiadBench benchmarks. PRICE shows consistent improvements over the baselines, with confidence intervals excluding zero across the evaluated settings. On Qwen3-8B, PRICE improves over the baseline by 4.2–5.0 pp over Minerva and 2.1–5.6 pp on OlympiadBench over all six budgets.
Tables 5–6 use likelihood scores; Tables 7–8 use P(True). For Tables 7–8, the budget includes generated-output tokens and actual scoring-input tokens. Parentheses report accuracy gains (percentage points) over the stronger of SC and BoN.
Table 5. Qwen3-4B on Minerva with intrinsic likelihood scores. Accuracy (%) under generated-token risk budgets.
| Method / budget | 21.4k | 29.0k | 33.0k | 44.8k | 52.2k | 70.6k |
|---|---|---|---|---|---|---|
| SC | 54.2 | 54.8 | 55.6 | 56.4 | 56.6 | 56.6 |
| BoN | 52.5 | 52.5 | 52.4 | 52.1 | 51.8 | 51.1 |
| PRICE | 60.2 (+6.1) | 60.3 (+5.4) | 60.3 (+4.7) | 60.4 (+4.0) | 60.5 (+3.9) | 60.5 (+3.3) |
Table 6. Qwen3-4B on OlympiadBench with intrinsic likelihood scores. Accuracy (%) under generated-token risk budgets.
| Method / budget | 23.2k | 31.3k | 39.4k | 63.9k | 80.2k | 129.3k |
|---|---|---|---|---|---|---|
| SC | 57.6 | 58.8 | 59.7 | 61.0 | 61.5 | 62.3 |
| BoN | 55.2 | 55.1 | 55.1 | 55.0 | 55.0 | 55.0 |
| PRICE | 63.1 (+5.5) | 64.4 (+5.6) | 64.6 (+4.9) | 64.7 (+3.7) | 65.0 (+3.5) | 65.0 (+2.7) |
Table 7. Qwen3-8B on Minerva with P(True) scores. Accuracy (%) under processed-token risk budgets, including scoring inputs.
| Method / budget | 22.7k | 30.6k | 37.9k | 50.5k | 59.2k | 64.8k |
|---|---|---|---|---|---|---|
| SC | 53.7 | 54.1 | 54.5 | 54.4 | 54.9 | 55.0 |
| BoN | 52.8 | 53.2 | 53.5 | 53.9 | 54.0 | 54.2 |
| PRICE | 58.7 (+5.0) | 59.0 (+4.9) | 59.1 (+4.6) | 59.3 (+4.9) | 59.3 (+4.4) | 59.3 (+4.2) |
Table 8. Qwen3-8B on OlympiadBench with P(True) scores. Accuracy (%) under processed-token risk budgets, including scoring inputs.
| Method / budget | 44.4k | 56.4k | 69.7k | 97.5k | 117.5k | 168.8k |
|---|---|---|---|---|---|---|
| SC | 59.9 | 61.2 | 61.9 | 63.1 | 63.5 | 63.8 |
| BoN | 57.8 | 59.0 | 59.5 | 59.9 | 60.4 | 60.6 |
| PRICE | 65.5 (+5.6) | 65.6 (+4.4) | 65.6 (+3.7) | 65.8 (+2.7) | 65.9 (+2.4) | 65.9 (+2.1) |
Authors addressed my concern and thanks for the rebuttal. I raised my score to 3.
Thank you for recognizing the practical relevance of test-time compute efficiency and the theoretical analysis. This paper is primarily a theoretical study of LLM test-time compute. Its goal is to provide a rigorous understanding of how rollout count and voting rule jointly determine the accuracy vs. token-cost Pareto frontier, and why voting must be part of compute allocation.
We have fully addressed the weaknesses and questions raised in your review through clearer score and cost accounting, a comparison experiment to illustrate the benefit of adaptive voting which fixes deployed allocation and stopping, and evaluations on newer models and harder mathematics tasks. We would appreciate a renewed assessment centered on the framework, theoretical contributions, and this additional evidence.
Summary of weaknesses and questions