- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a simple workflow that treats test-time scaling as an instance-level strategy routing problem. It uses answer consistency across stages to choose early stopping, voting, or rewriting for each problem. The method needs no extra training and is easy to integrate. Experiments across multiple models and math benchmarks report average accuracy gains of about 3–7 percentage points over strong baselines, with lower inference costs.
Critical strengths
- The workflow is simple, needs no extra training, and is easy to integrate. This gives the method practical value.
- Experiments across multiple models and math benchmarks report average accuracy gains over other strong baselines, with lower inference costs. The results suggest practical benefits in both accuracy and cost.
Critical weaknesses
-
I find the novelty limited. The method combines consistency checks, voting, and rewriting in a staged routing workflow. Its added contribution over existing methods needs clarification. Table 4 tests budgets of 4, 6, and 8 generations. It remains unclear how further sampling of the original problems compares with rewriting at larger budgets on a fixed high-disagreement subset.
-
The benefits of rewriting and the reliability of the comparisons need clarification. In Table 3, OnlyRewrite scores 90.4, 59.0, and 56.7 on Math500, Olympiad, and AIME24, respectively. These scores exceed SCoP's 88.2, 52.6, and 40.0 in Table 1. Table 1 reports the standard deviation of the average score across three random seeds, but the evaluation protocol remains unclear. It is unclear whether these experiments use consistent evaluation settings.
Also, some text in Figures 2–4 is too small to read clearly.
Questions for authors
-
Do the main experiments and ablation studies adopt consistent evaluation settings (particularly for AIME24/25)? Does the gain stem from fluctuations in the evaluation? Will a higher number of sampling iterations n (e.g., 16, 32, 64) yield different results?
-
Appendix C requires “remove unnecessary descriptions” and "simplify", yet the author attributes some of the errors to over-compression. How does the method perform with a paraphrasing prompt that does not require compress text?
Dear Reviewer UGNa,
Thank you for your careful review. We clarified that direct selection and plurality voting are equivalent in the default two-round case, while the second pair still provides the consistency check that determines whether an instance is resolved or routed to rewriting. We also explained that Tables 6–9 are independent diagnostic runs, reported exact route-level statistics from an end-to-end run, and added a matched random-routing baseline controlling candidate outputs, route proportions, strategies, and computation. Disagreement-guided routing improves average accuracy by 6.7 points under this control. These clarifications and analyses have been incorporated into the revised PDF.
We would appreciate your assessment of whether these results address the concerns and would be grateful if you could reconsider the rating. We remain happy to clarify any outstanding issues.
Thank you for your time.