- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
To reduce the difficulty of selecting the correct answer from a large number of candidates and to alleviate the issue of slow candidate generation, this paper proposes the Multi-Sequence Verifier (MSV), a lightweight verifier that predicts each candidate’s correctness by jointly attending over the hidden representations of all sampled candidates, rather than processing each one in isolation. MSV has shown effectiveness in mathematical reasoning tasks, in some cases outperforming the best baseline's "best-of-64" performance by approximately 6 percent, and a streaming variant of MSV improves the estimated accuracy-latency tradeoff.
Critical strengths
- The method proposed in the paper serves multiple functions: it guides the selection of answers while simultaneously facilitating early stopping.
- The method designed in the paper is lightweight.
- The paper is well written.
Critical weaknesses
- It may be beneficial to discuss whether model improvements stem from comparing the content of different responses or from leveraging information about the ordinal positions of the candidates. In other words, given that the model employs per-sequence embeddings, it might be beneficial to verify whether the model's final predictions remain stable with respect to the numbering of the candidates.
- Can a verifier trained on a specific generalize better to different values? (This is distinct from simply reusing grouped segments in integer multiples, the two concepts seem to differ slightly). One possible example is to train on N=64, and then evaluate and test on N<64, rather than retraining or using grouping.
- The majority of experiments are based on models within the 1.x billion parameter range, and the baselines used in the paper are generally not the most up-to-date (the newest is GenSelect, from 2025, in the appendix). The authors are encouraged to test generalizability using SOTA baselines and models (for baselines, some relevant baesline such as TrajSelector (arXiv:2510.16449), some recent baselines such as CAPS (arXiv:2605.15513) and One-Token Verification (arXiv:2603.01025). NOTE: These are merely examples of baselines. They are not intended as mandatory points of comparison. However, the baselines currently included in the paper do not seem sufficiently comprehensive.), which helps to show the potential and novelty of the proposed method in the manuscript. Moreover, the tested datasets are mainly mathematical tasks. The performance of the proposed method on other datasets remains unclear.
- Is MSV still useful for tasks where reliable tools like SymPy are not available?
Questions for authors
Please see the Critical Weaknesses.
5 comments
W3. Model scale, baseline coverage, and task diversity
Regarding the lack of experiments with models larger than 1.x billion parameter models, Section 4.5 and Appendix D.4 report experiments with three additional base models, including DeepSeek-R1-Distill-Llama-8B. MSV consistently achieves the highest mean best-of-64 accuracy among the compared methods.
Regarding the lack of baselines, the paper compares MSV against independent outcome verification with Probe and MSV$_1$, the Qwen2.5-Math-PRM-7B process reward model, self-consistency, weighted-voting variants, the generative joint selector GenSelect, and the parallel early-stopping methods ST-BoN and Parallel-Probe. These baselines cover independent verification, process verification, voting, generative joint selection, trajectory pruning, and parallel early stopping.
We provide two additional baselines, CAPS (arXiv:2605.15513) and TrajSelector (arXiv:2510.16449) on the five math reasoning benchmarks.
Table R.5 Best-of-16 accuracy compared with recent test-time scaling baselines
| Dataset | Pass@1 | CAPS | TrajSelector | Probe best-of-16 | MSV16 best-of-16 |
|---|---|---|---|---|---|
| AIME | 27.48% | 33.48% | 37.89% | 44.70% | 46.15% |
| MATH | 67.55% | 61.38% | 70.93% | 73.45% | 74.72% |
| OlympiadBench | 41.05% | 44.98% | 45.54% | 50.39% | 51.54% |
| Omni-MATH | 24.32% | 26.90% | 28.29% | 31.76% | 31.98% |
| AMC12 | 39.65% | 44.22% | 47.57% | 51.98% | 53.99% |
MSV16 achieves the highest accuracy on all five benchmarks, outperforming both CAPS and TrajSelector. These results provide a broader comparison against more recent verification and test-time scaling methods. We thank the reviewer for this suggestion, and will make sure to incorporate these results into our revised manuscript.
Regarding task diversity, Appendix B.13 trains the verifiers on 224 ARC-Challenge questions and evaluates them on 448 MMLU test questions. MSV16 achieves the highest best-of-16 accuracy of 54.0%, compared with 49.0% for MSV$_1$ and 48.5% for Probe.
As detailed in our response to Reviewer DMgB, we have also added experiments on MBPP+ code generation and TL;DR free-form summarization. MSV16 achieves the highest best-of-16 performance in both settings, reaching 66.15% accuracy on MBPP+ and a mean reward of 3.6391 on TL;DR. Together with the MMLU results, these experiments extend our evaluation beyond mathematical reasoning to multiple-choice knowledge, complete program generation, and free-form text generation.
W4. Reliance on symbolic equivalence
We agree this is an important question, and we address this concern in Section 4.4 and Appendix D.3. We replaced SymPy with approximate textual and semantic equivalence checkers and retrained MSV with the resulting signals. With Jaccard or embedding similarity, MSV remains within 0.4 percentage points of the matched SymPy variant, showing that it can retain its advantage with approximate agreement signals.
As detailed in our response to Reviewer DMgB, our new MBPP+ and TL;DR experiments provide further evidence. Both experiments replace symbolic equivalence with a continuous agreement feature based on Jaccard similarity between candidate outputs. This allows MSV to model agreement among complete programs and free-form summaries, for which exact symbolic equivalence is unavailable. MSV16 with continuous agreement achieves the highest best-of-16 performance on both tasks.
Together, these results show that MSV does not depend on SymPy or exact answer equivalence. Its agreement mechanism can instead use approximate, domain-appropriate similarity measures for outputs with multiple valid forms.
We hope these address the critical weaknesses and that the reviewer will reconsider the assessment.
Dear reviewer idSY, we thank you again for the insightful questions and feedbacks. We have posted a response to your concerns and a revised version of our manuscript. We would greatly appreciate any further feedback or remaining concerns you might have.
We thank the reviewer for the feedback and questions.
W1. Sensitivity to candidate ordering
We tested 100 permutations on each of the problems on AIME, and found that the mean change in best-of-64 relative to the original ordering had 95% confidence interval [−0.18%, +1.22%]. This suggests that the ordering was not a key contributing factor to MSV’s performance. A remaining question is how much variance there is in MSV’s predictions, subject to permutations. We found that the standard deviation of best-of-64 accuracy across the 100 permutations was 0.24%, indicating a relatively small variance. For context, the standard deviation of best-of-64 accuracy across five MSV checkpoints was 0.73%. Moreover, across pairs of permutations, MSV selected different answer classes on only 2% of problems on average.
W2. Generalization across candidate-set sizes
We agree this is an important question, and it is addressed in Appendix B.7. We copy the weights of an MSV trained with N=64 into an N=16 model, retain the first 16 sequence embeddings, and apply it to groups of 16 candidates without retraining or partitioning the candidates into fixed-size subgroups. Across the five benchmarks, the difference between direct transfer and separate N=16 training ranges from −0.46 to +0.36 percentage points. These results show that the candidate-set size used during training does not fix the inference candidate-set size, and that MSV transfers directly from N=64 to N=16 without a meaningful loss in observed accuracy.