- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Decoupled Marginal Sharpening (DMS), a training-free inference-time scaling framework that separately sharpens reasoning-path and answer-marginal distributions, using Fully Adapted Annealed Sequential Monte Carlo (FAA-SMC) to mitigate particle degeneracy and preserve reasoning diversity. Experiments across three language models and four reasoning benchmarks show that DMS achieves the highest mean pass@1 among the compared methods in all 12 settings, while reducing inference latency relative to Marginal Sharpening.
Critical strengths
-
The paper provides a well-motivated and mathematically clear framework for decoupling reasoning-path sharpening from answer-level sharpening. By aggregating evidence across different reasoning paths before sharpening the answer distribution, DMS addresses an important limitation of sequence-level power sampling while requiring neither additional training nor external reward models.
-
The experimental results are strong across multiple models and reasoning tasks. DMS achieves the highest mean pass@1 in all 12 model–benchmark settings, with substantial improvements on MATH500 and HumanEval. The paper also provides extensive ablation studies, sensitivity analyses, and runtime comparisons, demonstrating practical effectiveness and improved inference efficiency over Marginal Sharpening.
Critical weaknesses
-
In Table 2, Answer Sharpening Only achieves the same pass@1 (77.22%) as Island-DMS, with slightly higher expected accuracy (76.89% vs. 76.83%). Appendix D further shows that the best expected accuracy on AIME24/25 is achieved with , without additional reasoning sharpening. These results weaken the evidence that the proposed two-stage sharpening is consistently more effective than answer sharpening alone.
-
The contributions of individual components and the fairness of baseline comparisons remain unclear. DMS differs from Marginal Sharpening not only in its target distribution but also in its sampling and answer-selection procedures. Moreover, the methods use different reasoning budgets and decoding strategies, while DMS employs model- and task-specific sharpening exponents. Without more controlled comparisons and a clear hyperparameter selection protocol, it is difficult to determine how much of the improvement comes from the proposed method itself.
Questions for authors
-
Table 2 and Appendix D suggest that answer sharpening alone can match or outperform the full DMS method on AIME24/25. Could the authors provide similar ablations across the other models and benchmarks to demonstrate when reasoning-level sharpening provides additional benefits? In particular, how does DMS compare with the same framework using under identical computational budgets?
-
How were the sharpening exponents in Table 3 selected? Were they tuned on separate validation sets or directly on the evaluation benchmarks? Additionally, could the authors provide a more controlled comparison with Marginal Sharpening using matched inference budgets and comparable answer-generation procedures to better isolate the benefits of the proposed target distribution and sampling algorithm?
3 comments
Q3: Additionally, could the authors provide a more controlled comparison with Marginal Sharpening using matched inference budgets and comparable answer-generation procedures to better isolate the benefits of the proposed target distribution and sampling algorithm?
A3: We provide a controlled comparison on MATH500 problems using Qwen3-8B. All methods use the same 32 unsharpened reasoning traces, , and DMS’s reasoning–answer boundary handling, including the original Marginal Sharpening token decoder. At , DMS and Marginal Sharpening share the same ideal answer target, allowing us to compare their answer-sampling procedures with the reasoning traces fixed.
The original Marginal Sharpening decoder generates one answer token by token. We also evaluate a variant that selects from 32 complete, cross-scored answers while retaining the grouped approximation used by Marginal Sharpening. This variant and DMS use exactly the same answer candidates and cross-scoring results, so their difference comes from how these scores are combined to assign answer-selection weights.
| Answer-sampling method | pass@1, mean ± SD | Expected accuracy |
|---|---|---|
| Marginal Sharpening: original token decoder | 85.93 ± 0.50 | — |
| Marginal Sharpening: selection from 32 cross-scored answers | 85.93 ± 0.31 | 85.72 |
| DMS: selection from 32 cross-scored answers | 86.07 ± 0.31 | 85.93 |
Under this controlled comparison, DMS achieves the highest mean pass@1 at 86.07%, while both Marginal Sharpening variants achieve 85.93%. For the two methods using the same complete answer candidates, DMS also increases expected accuracy from 85.72% to 85.93%. With the answer candidates and cross-scoring results fixed, DMS’s weighting scheme assigns more probability to correct answers on average.
We also provide majority-voting results on MATH500, GPQA-Diamond, and AIME24/25 using Qwen3-8B. MV16 and MV32 perform majority voting over 16 and 32 independently generated trajectories, respectively. Accuracy is reported as mean ± standard deviation over three seeds.
| Benchmark | Method | pass@1 | Time (s/problem) |
|---|---|---|---|
| MATH500 | MV16 | 80.20 ± 0.72 | 64.53 |
| MATH500 | MV32 | 82.40 ± 0.92 | 128.50 |
| MATH500 | DMS | 85.80 ± 0.20 | 117.86 |
| GPQA-Diamond | MV16 | 60.94 ± 1.05 | 189.23 |
| GPQA-Diamond | MV32 | 59.76 ± 1.05 | 375.01 |
| GPQA-Diamond | DMS | 59.93 ± 1.77 | 237.56 |
| AIME24/25 | MV16 | 71.11 ± 0.96 | 509.78 |
| AIME24/25 | MV32 | 73.33 ± 1.67 | 1007.94 |
| AIME24/25 | Island-DMS | 77.22 ± 0.96 | 874.87 |
For the runtime comparison, MV32 generates its 32 independent trajectories in two batches of 16, keeping the generation batch size identical to MV16 and limiting peak memory demand. Compared with MV32, DMS improves pass@1 by 3.40 percentage points on MATH500 and 3.89 points on AIME24/25, while requiring less time on both benchmarks. On GPQA-Diamond, DMS achieves similar accuracy to MV32 while reducing runtime from 375.01 to 237.56 seconds per problem; MV16 achieves the highest mean accuracy on this benchmark.
We hope we have addressed all of your questions. If you have any further questions, we would be happy to answer them and if you don’t, would you kindly consider increasing your score?
Dear Reviewer oomt,
Thank you again for your constructive feedback. In our response, we report new ablations on MATH500 and GPQA-Diamond comparing full DMS with answer-only sharpening under the same sampling budgets. We also clarify how we selected the sharpening exponents and compare DMS with Marginal Sharpening using the same 32 reasoning traces and the same reasoning–answer boundary protocol.
As the discussion period draws to a close, we would appreciate hearing whether our response addresses your concerns or whether there is anything else we can clarify.
Thank you for your time and consideration.
Response to Reviewer oomt
We thank the reviewer for your valuable time and effort in providing detailed feedback on our work. We hope our response will fully address all your questions.
Q1: Table 2 and Appendix D suggest that answer sharpening alone can match or outperform the full DMS method on AIME24/25. Could the authors provide similar ablations across the other models and benchmarks to demonstrate when reasoning-level sharpening provides additional benefits? In particular, how does DMS compare with the same framework using (αZ=1) under identical computational budgets?
A1: We provide additional ablations on MATH500 and GPQA-Diamond using Qwen3-8B. The structured-only variant uses αZ=αA=1. The other variants apply reasoning sharpening, answer sharpening, or both. Expected accuracy measures the probability that the final answer selection returns a correct answer.
Compared with answer-only sharpening, full DMS improves pass@1 from 84.80% to 85.53% on MATH500 and from 57.91% to 59.93% on GPQA-Diamond. Expected accuracy also increases on both benchmarks. These comparisons demonstrate a benefit from reasoning sharpening under the main DMS sampling procedure.
Appendix F provides further evidence using a higher-budget replica-exchange reasoning sampler. On Qwen3-4B/HumanEval, increasing αZ from 1 to 1.05 improves expected accuracy from 95.06% to 96.59%, with the same answer-sampling procedure and αA=4.
Together with the AIME ablation, these results support controlling reasoning and answer sharpening independently. In particular, the GPQA-Diamond and HumanEval results show that reasoning sharpening can further improve accuracy beyond answer sharpening alone.
Q2: How were the sharpening exponents in Table 3 selected? Were they tuned on separate validation sets or directly on the evaluation benchmarks?
A2: We would like to mention that Table 3 reports the DMS exponents for each model–benchmark setting, and Appendix C specifies the baseline configurations. Appendix D further compares performance across different combinations of αZ and αA.
For each model–benchmark setting, we randomly selected 20% of the questions as a pilot set and performed a grid search over (αZ,αA) using a single tuning seed. We then fixed the selected configuration and evaluated it on the full benchmark using independent three different seeds reported in the paper.