- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper examines per-instance scoring signal selection for test-time scaling, a mechanism that decides whether to apply low-cost internal confidence or an external process reward model (PRM) after candidate generation. It verifies the complementary strengths of the two scoring signals and shows that simple disagreement-based heuristics, with answer entropy being the most representative, achieve competitive accuracy-efficiency trade-offs when compared against learned selectors.
Critical strengths
- The paper identifies an interesting and practically relevant direction for improving test-time scaling efficiency, with insights supported by empirical evidence of complementarity between internal and external scoring signals.
- The experiments compare different selection strategies and investigate their behavior across models, scoring signals, and sampling budgets. The finding that simple heuristics remain competitive with learned selectors is particularly interesting.
Critical weaknesses
The proposed scoring signal selection is only evaluated under Best-of-N, although Appendix A.2 shows that alternative aggregation methods, such as weighted majority voting, substantially reduce the complementarity between scoring signals. It remains unclear whether the proposed approach would provide similar benefits under these alternative aggregation strategies.
Questions for authors
The current framework invokes the PRM on all candidates whenever external scoring is selected. Could selectively evaluating only a subset of candidates further improve the accuracy–efficiency tradeoff?
4 comments
Gentle Follow-Up on Our Responses
Thank you again for your time and thoughtful comments on our manuscript. We have carefully addressed the concerns and questions raised in your review and provided detailed responses, including additional analyses and experimental results.
As the discussion period is approaching its end, we would like to kindly follow up in case you have not yet had a chance to review our responses. We appreciate your time and thoughtful feedback, and we hope our responses help clarify the concerns raised in your reviews.
Thank you again for your time and consideration.
Dear Reviewer DZv3,
As the discussion period nears its end, we would like to follow up on our rebuttal responses. We have provided detailed explanations and additional experimental results to address the concerns raised in your review, and we hope these have helped clarify the points you raised.
Thank you again for your time and thoughtful feedback.
Thank you for the response. My concerns are mostly addressed. I will maintain my positive rating.
W1. Does routing remain useful beyond BoN?
We agree that the aggregation rule affects the benefit of scoring-signal selection. As shown in Appendix A.2, switching from BoN to WMV reduces the PRM-only rescue fraction from 13.65% to 5.88% on MMLU-Pro and from 14.00% to 6.80% on MATH500. To examine whether routing remains effective, we evaluated actual entropy-routing frontiers using the same aggregation rule on both internal and external branches.
Excess AUC measures the full-frontier improvement over random routing matched by PRM call rate. The fixed-point comparison uses a no-choice-aware random baseline matched by expected PRM scoring FLOPs.
Under WMV, entropy routing retains positive excess AUC on both benchmarks. The paired changes in excess AUC from BoN to WMV are −0.0023 [−0.0074, 0.0027] on MMLU-Pro and +0.0049 [−0.0029, 0.0126] on MATH500, neither of which is statistically significant. Under the stricter FLOPs-matched comparison, the improvement is statistically supported on MMLU-Pro but not on MATH500.
We also evaluated Borda aggregation, obtaining smaller excess AUCs of 0.0026 and 0.0034, respectively. These results show that routing can remain beneficial beyond BoN, while its effectiveness depends on the aggregation rule and computational budget. We will extend the aggregation analysis to include these routing results.
Q1. Can scoring only a candidate subset improve the frontier?
Yes. We evaluated candidate pruning by selecting the top-k candidates according to internal confidence and applying PRM scoring only to this subset (see Reviewer X749, W4-2). We also compared against randomly selected subsets and combined pruning with a free no-choice check that skips external scoring when all candidates yield the same answer. Computation is estimated using per-candidate token-weighted analytical FLOPs, including generation and PRM scoring.
Total FLOPs are normalized to full PRM scoring. No-choice + Top-16 achieves accuracy close to full PRM while reducing total computation by 31.01% on MMLU-Pro and 25.97% on MATH500. The accuracy differences from full PRM are not statistically significant, although this does not establish equivalence. At k=4, confidence-based selection significantly outperforms random subset selection on both benchmarks, indicating that internal ranking is useful under smaller scoring budgets.
Compared with the original entropy gate, no-choice + Top-16 achieves higher observed accuracy with lower computation on both benchmarks. Candidate pruning therefore provides a stronger accuracy–compute operating point among the evaluated strategies. We will include these results as an additional baseline and clarify that the original binary routing policy studies question-level scoring-signal selection, whereas pruning also controls which candidates receive external scores. These comparisons are based on offline evaluations and analytical FLOPs rather than measured inference latency.