- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how noisy process-reward scores can cause search to discard useful partial solutions permanently. It keeps historical prefixes available for later expansion. Subpool Selection chooses promising prefixes from a random part of this history, while Power Backtrack SMC combines new and historical samples with corrected weights. The paper derives a consistency result for a score-based target and compares accuracy under token and runtime budgets on mathematical reasoning tasks.
Critical strengths
-
The paper connects the problem in the Introduction to both an explicit search mechanism and experimental evidence. Appendix C.5 gives a concrete case where a useful earlier prefix can be revisited, and Table 5 shows benefits from retaining history and from stochastic access to it. With the same general evaluation protocol, Backtrack SMC improves average accuracy from 50.88 to 52.43, and Subpool Selection improves over persistent greedy selection from 51.89 to 55.00. This supports the claim that simply storing history is not enough.
-
The evaluation is fairly thorough. It includes repeated runs for Qwen2.5, separate token and runtime comparisons, another generator family, and another reward-model family. The mixture-weight derivation makes the sampling target explicit, and the theorem is correctly described as targeting a reward-score proxy rather than true answer correctness.
Critical weaknesses
-
The component studies establish the value of backtracking and random subpools more clearly than the value of every part of Power Backtrack SMC. I did not find an ablation that keeps the same historical pool and power schedule but removes or replaces only the mixture correction. The power and mixture sensitivity studies are useful, but they do not isolate that new correction. The direct evidence for the proposed failure also relies heavily on one trace-level example; the overall gains support the method, but do not quantify how often recovery from a previously discarded useful prefix explains those gains. The Qwen3 results are point estimates, as acknowledged, so the ranking on that model remains less certain.
-
The theorem needs a few implementation details made explicit. Algorithms 1 and 4 initialize weights with the reward score, while Appendix B.5 uses the reward score raised to the initial power. These agree if the initial power is one, but this should be stated. The proof also assumes strictly positive scores and a normalized extension rule for every prefix; the paper should explain score flooring and what happens when a prefix is already complete. In the adaptive-power argument, the schedule uses individual particle counts, so its limiting step deserves more explanation than calling it a continuous function of the weighted empirical measure. Appendix B.4 also says low score variance slows the power increase, while Equation 23 gives a larger increase when variance is low. These are fixable points, but they matter for checking that the implemented algorithm is covered by the stated result.
Questions for authors
-
Is the initial power fixed to one? Please reconcile the initial weights in Algorithm 4 with those in Appendix B.5, and explain how zero scores and completed prefixes are handled.
-
Could you make the limiting argument for the adaptive power schedule explicit, including the role of particle multiplicities? Please also resolve the difference between Equation 23 and the following explanation of low score variance.
-
Is there an existing matched-budget ablation that changes only the mixture correction? If recovery statistics over several problems, or repeated Qwen3 runs, are already available, please provide them; these would affect confidence in the component and accuracy claims.
Thanks for the detailed response, I'm increasing my score accordingly.