acceptodds
Under review as a conference paper at ICLR 2027

Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling

Abstract

We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-sampling methods define the sequence-level power target , where , which increases the relative probability of completions that are already likely under the base model. Existing methods use Metropolis–Hastings or Sequential Monte Carlo (SMC) to approximately sample from this target. However, two challenges arise at different levels. At the target level, full-sequence sharpening can concentrate probability on a few exact reasoning–answer trajectories, suppressing alternative paths that support the same answer. At the sampler level, finite-particle SMC approximations can suffer from path collapse when resampling repeatedly duplicates a few high-weight prefixes. To address these challenges, we propose Decoupled Marginal Sharpening (DMS), which separately controls how strongly the model favors particular reasoning paths and how strongly it favors final answers after aggregating support across those paths. At the target level, DMS first samples from a sharpened reasoning distribution, then marginalizes the model's answer probabilities over the resulting weighted reasoning population and sharpens the induced answer marginal. Separate exponents allow mild reasoning sharpening to encourage exploration of diverse paths, followed by stronger answer-marginal sharpening to concentrate probability on answers with greater aggregate support across those paths. At the sampler level, we develop Fully Adapted Annealed SMC, which combines an annealed sequence of power targets with locally fully adapted proposals and predictive ancestor resampling. Experiments on MATH500, GPQA-Diamond, HumanEval, and AIME24/25 across three base models show that DMS achieves the highest mean pass@1 among the compared methods in all tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
sequential monte carlo, inference time scaling, LLM reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oomtOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Decoupled Marginal Sharpening (DMS), a training-free inference-time scaling framework that separately sharpens reasoning-path and answer-marginal distributions, using Fully Adapted Annealed Sequential Monte Carlo (FAA-SMC) to mitigate particle degeneracy and preserve reasoning diversity. Experiments across three language models and four reasoning benchmarks show that DMS achieves the highest mean pass@1 among the compared methods in all 12 settings, while reducing inference latency relative to Marginal Sharpening.

Critical strengths
  • The paper provides a well-motivated and mathematically clear framework for decoupling reasoning-path sharpening from answer-level sharpening. By aggregating evidence across different reasoning paths before sharpening the answer distribution, DMS addresses an important limitation of sequence-level power sampling while requiring neither additional training nor external reward models.

  • The experimental results are strong across multiple models and reasoning tasks. DMS achieves the highest mean pass@1 in all 12 model–benchmark settings, with substantial improvements on MATH500 and HumanEval. The paper also provides extensive ablation studies, sensitivity analyses, and runtime comparisons, demonstrating practical effectiveness and improved inference efficiency over Marginal Sharpening.

Critical weaknesses
  • In Table 2, Answer Sharpening Only achieves the same pass@1 (77.22%) as Island-DMS, with slightly higher expected accuracy (76.89% vs. 76.83%). Appendix D further shows that the best expected accuracy on AIME24/25 is achieved with αZ=1\alpha_Z = 1, without additional reasoning sharpening. These results weaken the evidence that the proposed two-stage sharpening is consistently more effective than answer sharpening alone.

  • The contributions of individual components and the fairness of baseline comparisons remain unclear. DMS differs from Marginal Sharpening not only in its target distribution but also in its sampling and answer-selection procedures. Moreover, the methods use different reasoning budgets and decoding strategies, while DMS employs model- and task-specific sharpening exponents. Without more controlled comparisons and a clear hyperparameter selection protocol, it is difficult to determine how much of the improvement comes from the proposed method itself.

Questions for authors
  • Table 2 and Appendix D suggest that answer sharpening alone can match or outperform the full DMS method on AIME24/25. Could the authors provide similar ablations across the other models and benchmarks to demonstrate when reasoning-level sharpening provides additional benefits? In particular, how does DMS compare with the same framework using αZ=1\alpha_Z = 1 under identical computational budgets?

  • How were the sharpening exponents in Table 3 selected? Were they tuned on separate validation sets or directly on the evaluation benchmarks? Additionally, could the authors provide a more controlled comparison with Marginal Sharpening using matched inference budgets and comparable answer-generation procedures to better isolate the benefits of the proposed target distribution and sampling algorithm?

3 comments

Response to Reviewer oomt

We thank the reviewer for your valuable time and effort in providing detailed feedback on our work. We hope our response will fully address all your questions.


Q1: Table 2 and Appendix D suggest that answer sharpening alone can match or outperform the full DMS method on AIME24/25. Could the authors provide similar ablations across the other models and benchmarks to demonstrate when reasoning-level sharpening provides additional benefits? In particular, how does DMS compare with the same framework using (αZ=1)(\alpha_Z=1) under identical computational budgets?

A1: We provide additional ablations on MATH500 and GPQA-Diamond using Qwen3-8B. The structured-only variant uses αZ=αA=1\alpha_Z=\alpha_A=1. The other variants apply reasoning sharpening, answer sharpening, or both. Expected accuracy measures the probability that the final answer selection returns a correct answer.

VariantMATH500 Expected accuracyMATH500 pass@1GPQA-Diamond Expected accuracyGPQA-Diamond pass@1
Structured only82.9883.6757.2957.91
Answer sharpening only85.5184.8058.7357.91
Reasoning sharpening only83.9583.8759.4560.27
Full DMS85.9785.5359.6159.93

Compared with answer-only sharpening, full DMS improves pass@1 from 84.80% to 85.53% on MATH500 and from 57.91% to 59.93% on GPQA-Diamond. Expected accuracy also increases on both benchmarks. These comparisons demonstrate a benefit from reasoning sharpening under the main DMS sampling procedure.

Appendix F provides further evidence using a higher-budget replica-exchange reasoning sampler. On Qwen3-4B/HumanEval, increasing αZ\alpha_Z from 1 to 1.05 improves expected accuracy from 95.06% to 96.59%, with the same answer-sampling procedure and αA=4\alpha_A=4.

Together with the AIME ablation, these results support controlling reasoning and answer sharpening independently. In particular, the GPQA-Diamond and HumanEval results show that reasoning sharpening can further improve accuracy beyond answer sharpening alone.


Q2: How were the sharpening exponents in Table 3 selected? Were they tuned on separate validation sets or directly on the evaluation benchmarks?

A2: We would like to mention that Table 3 reports the DMS exponents for each model–benchmark setting, and Appendix C specifies the baseline configurations. Appendix D further compares performance across different combinations of αZ\alpha_Z and αA\alpha_A.

For each model–benchmark setting, we randomly selected 20% of the questions as a pilot set and performed a grid search over (αZ,αA)(\alpha_Z,\alpha_A) using a single tuning seed. We then fixed the selected configuration and evaluated it on the full benchmark using independent three different seeds reported in the paper.

Q3: Additionally, could the authors provide a more controlled comparison with Marginal Sharpening using matched inference budgets and comparable answer-generation procedures to better isolate the benefits of the proposed target distribution and sampling algorithm?

A3: We provide a controlled comparison on MATH500 problems using Qwen3-8B. All methods use the same 32 unsharpened reasoning traces, αA=4\alpha_A=4, and DMS’s reasoning–answer boundary handling, including the original Marginal Sharpening token decoder. At αZ=1\alpha_Z=1, DMS and Marginal Sharpening share the same ideal answer target, allowing us to compare their answer-sampling procedures with the reasoning traces fixed.

The original Marginal Sharpening decoder generates one answer token by token. We also evaluate a variant that selects from 32 complete, cross-scored answers while retaining the grouped approximation used by Marginal Sharpening. This variant and DMS use exactly the same answer candidates and cross-scoring results, so their difference comes from how these scores are combined to assign answer-selection weights.

Answer-sampling methodpass@1, mean ± SDExpected accuracy
Marginal Sharpening: original token decoder85.93 ± 0.50—
Marginal Sharpening: selection from 32 cross-scored answers85.93 ± 0.3185.72
DMS: selection from 32 cross-scored answers86.07 ± 0.3185.93

Under this controlled comparison, DMS achieves the highest mean pass@1 at 86.07%, while both Marginal Sharpening variants achieve 85.93%. For the two methods using the same complete answer candidates, DMS also increases expected accuracy from 85.72% to 85.93%. With the answer candidates and cross-scoring results fixed, DMS’s weighting scheme assigns more probability to correct answers on average.

We also provide majority-voting results on MATH500, GPQA-Diamond, and AIME24/25 using Qwen3-8B. MV16 and MV32 perform majority voting over 16 and 32 independently generated trajectories, respectively. Accuracy is reported as mean ± standard deviation over three seeds.

BenchmarkMethodpass@1Time (s/problem)
MATH500MV1680.20 ± 0.7264.53
MATH500MV3282.40 ± 0.92128.50
MATH500DMS85.80 ± 0.20117.86
GPQA-DiamondMV1660.94 ± 1.05189.23
GPQA-DiamondMV3259.76 ± 1.05375.01
GPQA-DiamondDMS59.93 ± 1.77237.56
AIME24/25MV1671.11 ± 0.96509.78
AIME24/25MV3273.33 ± 1.671007.94
AIME24/25Island-DMS77.22 ± 0.96874.87

For the runtime comparison, MV32 generates its 32 independent trajectories in two batches of 16, keeping the generation batch size identical to MV16 and limiting peak memory demand. Compared with MV32, DMS improves pass@1 by 3.40 percentage points on MATH500 and 3.89 points on AIME24/25, while requiring less time on both benchmarks. On GPQA-Diamond, DMS achieves similar accuracy to MV32 while reducing runtime from 375.01 to 237.56 seconds per problem; MV16 achieves the highest mean accuracy on this benchmark.


We hope we have addressed all of your questions. If you have any further questions, we would be happy to answer them and if you don’t, would you kindly consider increasing your score?

Dear Reviewer oomt,

Thank you again for your constructive feedback. In our response, we report new ablations on MATH500 and GPQA-Diamond comparing full DMS with answer-only sharpening under the same sampling budgets. We also clarify how we selected the sharpening exponents and compare DMS with Marginal Sharpening using the same 32 reasoning traces and the same reasoning–answer boundary protocol.

As the discussion period draws to a close, we would appreciate hearing whether our response addresses your concerns or whether there is anything else we can clarify.

Thank you for your time and consideration.

Reviewer_duBvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DMS separately controls sharpening of the reasoning-path distribution and the answer marginal. It uses sequential Monte Carlo to generate weighted reasoning paths, combines candidate-answer probabilities across paths, and then sharpens and samples from the answer distribution. The method requires neither training nor an external reward model and reports the highest mean accuracy among the compared methods across three models and four benchmarks.

Critical strengths
  1. The formulation separates sharpening of the reasoning distribution from sharpening of the answer marginal and provides the corresponding sampling and importance corrections. I did not identify a clear error in the core weight derivation; the use of an answer-weight exponent of α_A−1 has a reasonable justification.

  2. The complete method produces substantial gains in several settings. For Qwen3-8B, MATH500 accuracy rises from the strongest baseline's 75.47% to 85.53%, and HumanEval from 86.99% to 95.93%. The paper reports three random runs, includes cross-scoring in runtime measurements, and shows ablations where reasoning sharpening does not help.

Critical weaknesses
  1. At α_Z=1, DMS's ideal target reduces to Marginal Sharpening, but its answer sampler still differs. On Qwen3-8B/AIME24/25, answer-only sharpening gets 77.22% pass@1, versus 74.44% for standard DMS and 77.22% for Island-DMS (Table 2). Appendix F finds benefits with a different, higher-budget MCMC sampler; evidence for reasoning sharpening under the main FAA-SMC configurations remains limited.

  2. Majority voting is missing on math and multiple-choice tasks. Appendix G includes cross-scoring time, but fixed-configuration timings do not show whether comparable computation spent on more independent traces and voting would achieve similar accuracy.

  3. DMS and Marginal Sharpening reserve an answer budget, insert a thought-ending marker when needed, and use a fixed non-code answer prefix; ordinary decoding and Power-SMC do not. With both exponents at 1, Qwen3-8B/AIME pass@1 is already 67.78%, versus 55.56% for ordinary decoding. Protocol effects remain confounded with sharpening, except against Marginal Sharpening, which shares the protocol.

  4. Table 3 uses two model- and task-specific exponents, while Marginal Sharpening uses 4 throughout. Despite Appendix D's sensitivity analysis, independent validation and comparable baseline tuning remain unclear. The 0.34 point lead on DeepSeek/GPQA-Diamond also does not by itself establish a stable advantage.

  5. On Qwen3-8B/AIME24/25, reasoning sharpening lowers oracle pass@32 from 82.78% to 78.33%; Island-DMS recovers it to 81.67% (Table 2). The paper acknowledges this coverage trade-off. More surviving ancestors do not establish more distinct correct solutions, and the diversity analysis lacks a direct comparison with Power-SMC.

  6. Island-DMS uses two particles per island, but its ESS threshold is not explicitly stated. If the standard half-population threshold is applied within each island, ESS < 1 cannot trigger resampling. The actual island threshold and resampling counts are needed; Table 4 reports only non-island DMS.

  7. The related-work section should be in the main text, with a concise comparison to Power-SMC and Marginal Sharpening. after all, reviewers are under no obligation to read the appendix.

Questions for authors
  1. Does answer-only sharpening (α_Z=1) retain most of the gains on MATH500, GPQA-Diamond, and HumanEval under the main DMS pipeline? Table 2 covers Qwen3-8B/AIME24/25, while Appendix F changes the reasoning sampler.

  2. How were the two exponents in Table 3 selected for each model and task? Was an independent validation set used, and why was Marginal Sharpening fixed at 4?

  3. With two particles per island, what ESS threshold does Island-DMS actually use? Did within-island resampling occur in the reported experiments?

  4. In Table 2, reasoning sharpening reduces oracle pass@32 from 82.78% to 78.33%. Is this mainly due to losing correct reasoning paths or to candidate-answer generation? Can the saved reasoning and answer samples distinguish these explanations?

  5. Table 2 reports only mean results for the key ablations. What are the single seed results or variability for standard DMS, answer-only sharpening, and Island-DMS? Are the observed gaps consistent across seeds?

4 comments

Response to Reviewer duBv

We thank the reviewer for your valuable time and effort in providing detailed feedback on our work. We hope our response will fully address all your questions.


Q1: Does answer-only sharpening αZ=1\alpha_Z=1 retain most of the gains on MATH500, GPQA-Diamond, and HumanEval under the main DMS pipeline? Table 2 covers Qwen3-8B/AIME24/25, while Appendix F changes the reasoning sampler.

A1: We provide additional ablations on MATH500 and GPQA-Diamond using Qwen3-8B. The structured-only variant uses αZ=αA=1\alpha_Z=\alpha_A=1. The other variants apply reasoning sharpening, answer sharpening, or both. We compute expected accuracy by summing the selection probabilities of all correct answer candidates and averaging over problems.

VariantMATH500 Expected accuracyMATH500 pass@1GPQA-Diamond Expected accuracyGPQA-Diamond pass@1
Structured only82.9883.6757.2957.91
Answer sharpening only85.5184.8058.7357.91
Reasoning sharpening only83.9583.8759.4560.27
Full DMS85.9785.5359.6159.93

Compared with answer-only sharpening, full DMS improves pass@1 from 84.80% to 85.53% on MATH500 and from 57.91% to 59.93% on GPQA-Diamond. Expected accuracy also increases on both benchmarks. These comparisons demonstrate a benefit from reasoning sharpening under the main DMS sampling procedure.

Appendix F provides further evidence using a higher-budget replica-exchange reasoning sampler. On Qwen3-4B/HumanEval, increasing αZ\alpha_Z from 1 to 1.05 improves expected accuracy from 95.06% to 96.59%, with the same answer-sampling procedure and αA=4\alpha_A=4.

Together with the AIME ablation, these results support controlling reasoning and answer sharpening independently. In particular, the GPQA-Diamond and HumanEval results show that reasoning sharpening can further improve accuracy beyond answer sharpening alone.


Q2: How were the two exponents in Table 3 selected for each model and task? Was an independent validation set used, and why was Marginal Sharpening fixed at 4?

A2: We would like to mention that Table 3 reports the DMS exponents for each model–benchmark setting, and Appendix C specifies the baseline configurations. Appendix D further compares performance across different combinations of αZ\alpha_Z and αA\alpha_A.

For each model–benchmark setting, we randomly selected 20% of the questions as a pilot set and performed a grid search over (αZ,αA)(\alpha_Z,\alpha_A) using a single tuning seed. We then fixed the selected configuration and evaluated it on the full benchmark using independent three different seeds reported in the paper.

For Marginal Sharpening, we use the configuration with sharpening exponent 4 and eight Monte Carlo groups, giving 32 reasoning traces in total. This is one of the configurations reported in the original paper [2].


Q3: With two particles per island, what ESS threshold does Island-DMS actually use? Did within-island resampling occur in the reported experiments?

A3: Island-DMS computes ESS separately within each island and uses a threshold of 0.75×0.75 \times the island’s particle count. With two particles, resampling is triggered when ESS<0.75×2=1.5\mathrm{ESS}<0.75\times2=1.5. ESS is checked every 64 tokens using predictive weights that include the annealing correction and fully adapted proposal normalizers. When triggered, systematic resampling is performed within that island, and its importance weights are reset to equal values. No particles are exchanged between islands.

Within-island resampling does occur. We provide the recorded counts for Qwen3-8B on all 60 AIME24/25 problems, using eight islands with two particles each and three seeds:

SeedDuring generationTerminal annealing correctionTotal eventsMean events per island
100804101165261.10
100813811044851.01
100823681124801.00
Overall1,1593321,4911.04

Each event denotes one resampling operation in one island. Terminal annealing correction updates weights and resamples completed paths without generating new tokens. Across the three seeds, 1,053 of 1,440 islands (73.13%) resampled at least once.

Q4: In Table 2, reasoning sharpening reduces oracle pass@32 from 82.78% to 78.33%. Is this mainly due to losing correct reasoning paths or to candidate-answer generation? Can the saved reasoning and answer samples distinguish these explanations?

A4: Table 2 shows an exploration–exploitation trade-off between average candidate correctness and correct-answer coverage. Reasoning sharpening increases avg@32 from 66.48% to 71.77%, while reducing oracle pass@32 from 82.78% to 78.33%. Here, avg@32 measures the average correctness of the 32 candidates.

Sharpening concentrates generation toward more likely reasoning paths. In this setting, the resulting candidates are more often correct on average, while reduced exploration can make less likely but 'lucky' correct completions harder to discover on some problems. Thus, average candidate correctness and oracle coverage can move in opposite directions. Island-DMS raises oracle pass@32 to 81.67% with the same total reasoning-particle budget.


Q5: Table 2 reports only mean results for the key ablations. What are the single seed results or variability for standard DMS, answer-only sharpening, and Island-DMS? Are the observed gaps consistent across seeds?

A5: We provide the individual seed results for the Table 2 ablations below. All variants use Qwen3-8B on the full 60-question AIME24/25 benchmark with K=16K=16 and M=32M=32. Results are percentages, and SD is the sample standard deviation across the three seeds. The categorical pass@1 results correspond to the same answer draws used in Table 2.

Expected accuracy

VariantSeed 10080Seed 10081Seed 10082Mean ± SD
Standard DMS74.8579.6969.1374.55 ± 5.29
Answer-only sharpening76.9877.6975.9876.89 ± 0.86
Island-DMS77.7876.1476.5776.83 ± 0.85

Categorical pass@1

VariantSeed 10080Seed 10081Seed 10082Mean ± SD
Standard DMS75.0078.3370.0074.44 ± 4.19
Answer-only sharpening78.3378.3375.0077.22 ± 1.92
Island-DMS78.3376.6776.6777.22 ± 0.96

AIME24/25 involves challenging, extended mathematical reasoning, where maintaining diverse reasoning paths is particularly important. Reasoning sharpening can concentrate particles on a few paths, and repeated resampling can further reduce diversity. Island-DMS confines resampling to individual islands, preventing a high-weight path in one island from replacing particles in other islands and helping preserve diversity across the population.

The observed results are consistent with this motivation. Compared with standard DMS, Island-DMS improves mean expected accuracy from 74.55% to 76.83% and reduces the across-seed SD from 5.29 to 0.85. It also matches answer-only sharpening in mean pass@1 at 77.22%, while showing lower pass@1 variability (0.96 vs. 1.92).


Q6: Comparison with majority voting under comparable computational budgets

A6: We provide majority-voting results on MATH500, GPQA-Diamond, and AIME24/25 using Qwen3-8B. MV16 and MV32 perform majority voting over 16 and 32 independently generated trajectories, respectively. Accuracy is reported as mean ± standard deviation over three seeds.

BenchmarkMethodpass@1Time (s/problem)
MATH500MV1680.20 ± 0.7264.53
MATH500MV3282.40 ± 0.92128.50
MATH500DMS85.80 ± 0.20117.86
GPQA-DiamondMV1660.94 ± 1.05189.23
GPQA-DiamondMV3259.76 ± 1.05375.01
GPQA-DiamondDMS59.93 ± 1.77237.56
AIME24/25MV1671.11 ± 0.96509.78
AIME24/25MV3273.33 ± 1.671007.94
AIME24/25Island-DMS77.22 ± 0.96874.87

For the runtime comparison, MV32 generates its 32 independent trajectories in two batches of 16, keeping the generation batch size identical to MV16 and limiting peak memory demand. Compared with MV32, DMS improves pass@1 by 3.40 percentage points on MATH500 and 3.89 points on AIME24/25, while requiring less time on both benchmarks. On GPQA-Diamond, DMS achieves similar accuracy to MV32 while reducing runtime from 375.01 to 237.56 seconds per problem; MV16 achieves the highest mean accuracy on this benchmark.

Q7: Related-work comparison in the main text

A7: We have moved the Related Work discussion into the main text of the revised paper and include a concise comparison with Power-SMC and Marginal Sharpening.

Power-SMC [1] sharpens the distribution over complete generations, including both reasoning and the final answer. Marginal Sharpening [2] samples reasoning traces from the base model and sharpens the answer distribution obtained by combining these traces.

DMS controls reasoning sharpening and answer sharpening with separate exponents. FAA-SMC samples the reasoning paths, and complete answer candidates are evaluated across these paths before final selection. At αZ=1\alpha_Z=1, DMS recovers the same ideal answer target as Marginal Sharpening, while using a different answer-sampling procedure.


Q8: Separating the effects of the generation protocol and sharpening

A8: The structured-only ablation in Table 2 sets αZ=αA=1\alpha_Z=\alpha_A=1 while retaining the DMS generation protocol: separate reasoning and answer budgets, the reasoning–answer boundary, and the answer prefix. It also retains the same number of reasoning particles and answer candidates.

Under this same protocol, standard DMS improves pass@1 from 67.78% to 74.44% on Qwen3-8B/AIME24/25. Answer-only sharpening achieves 77.22%, and Island-DMS also achieves 77.22%.

The additional ablations in Q1 provide the same control on two other benchmarks. Compared with the structured-only variant, full DMS improves pass@1 from 83.67% to 85.53% on MATH500 and from 57.91% to 59.93% on GPQA-Diamond.

These comparisons demonstrate improvements from sharpening while keeping the generation format and sampling budgets fixed.


We hope we have addressed all of your questions. If you have any further questions, we would be happy to answer them and if you don’t, would you kindly consider increasing your score?

References:

[1] Azizi et al. Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning. 2026.

[2] Arzhantsev, Sakhi, and Chopin. Self-Consistency via Marginal Sharpening. 2026.

Dear Reviewer duBv,

Thank you again for your constructive feedback. In our response, we report new ablations on MATH500 and GPQA-Diamond that test whether reasoning sharpening improves on answer sharpening alone under the same generation protocol. We also compare DMS with majority voting in terms of accuracy and runtime. For AIME24/25, we provide results for each seed and report how often resampling occurred within the two-particle islands.

As the discussion period draws to a close, we would appreciate hearing whether our response addresses your concerns or whether there is anything else we can clarify.

Thank you for your time and consideration.

Reviewer_H9wTOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes DMS, which splits power sampling into two stages. In the first stage, reasoning paths are sampled from a mildly sharpened distribution with a new SMC sampler (FAA-SMC) based on annealing, fully adapted proposals and resampling before propagation. In the second stage, answer candidates are generated, each one is scored under all reasoning paths to estimate the answer marginal, and that marginal is sharpened with a separate exponent αA. Experiments on MATH500, GPQA-Diamond, HumanEval and AIME24/25 with three models show that DMS has the best pass@1 in all 12 settings and is faster than Marginal Sharpening.

Critical strengths

The motivation is clear, and the small example in the footnote makes it easy to see why sharpening each trajectory before marginalizing can flip the ranking of answers.

The reported gains are large in several settings, for example 8 to 10 points over the best baseline on MATH500 for the Qwen3 models, although they are not compute matched. Results are averaged over three seeds, and the paper reports wall-clock time and a good set of ablations.

Critical weaknesses

I don't think that the paper's own ablation supports its main idea. Sharpening only the answer, with no extra sharpening of the reasoning, does as well as the full method, and the reasoning exponents used in practice are all very close to 1. To me this suggests that most of the gain comes from answer-level marginal sharpening, not from using two separate exponents, which is the main contribution of the paper.

I was also surprised that this answer-only variant clearly beats the Marginal Sharpening baseline, since the paper says the two target the same distribution. The gap probably comes from how the answer is estimated, not from the decoupling.

I could not find how the exponents were picked for each model and benchmark, or whether a validation set was used. The comparison is also not compute matched. Standard decoding uses one sample, while DMS uses 16 reasoning paths and 32 answers, so majority voting with a similar number of samples is the obvious baseline and is missing. DMS is also up to about 3 times slower than standard decoding.

Questions for authors

How were (αZ, αA) and the Power-SMC exponents chosen, and on which data?

Can you add majority voting over 16 or 32 standard samples, together with its runtime?

Is the gap to Marginal Sharpening explained by the answer estimator described in Appendix C, and could you run that baseline with the same 32 cross-scored answers?

4 comments

Response to Reviewer H9wT

We thank the reviewer for your valuable time and effort in providing detailed feedback on our work. We understand your main concern to be whether reasoning sharpening provides benefits beyond answer sharpening alone and whether DMS’s advantages are supported by fair comparisons. To address these questions, we provide additional component ablations, controlled comparisons with Marginal Sharpening, and comparisons with majority voting. We hope these additional results and clarifications address your concerns and would appreciate your reconsideration of our paper’s evaluation.


Q1: Contribution of reasoning sharpening and support from the ablation study

A1: We provide additional ablations on MATH500 and GPQA-Diamond using Qwen3-8B. The structured-only variant uses αZ=αA=1\alpha_Z=\alpha_A=1. The other variants apply reasoning sharpening, answer sharpening, or both. We compute expected accuracy by summing the selection probabilities of all correct answer candidates and averaging over problems.

VariantMATH500 Expected accuracyMATH500 pass@1GPQA-Diamond Expected accuracyGPQA-Diamond pass@1
Structured only82.9883.6757.2957.91
Answer sharpening only85.5184.8058.7357.91
Reasoning sharpening only83.9583.8759.4560.27
Full DMS85.9785.5359.6159.93

Compared with answer-only sharpening, full DMS improves pass@1 from 84.80% to 85.53% on MATH500 and from 57.91% to 59.93% on GPQA-Diamond. Expected accuracy also increases on both benchmarks. These comparisons demonstrate a benefit from reasoning sharpening under the main DMS sampling procedure.

Appendix F provides further evidence using a higher-budget replica-exchange reasoning sampler. On Qwen3-4B/HumanEval, increasing αZ\alpha_Z from 1 to 1.05 improves expected accuracy from 95.06% to 96.59%, with the same answer-sampling procedure and αA=4\alpha_A=4.

Together with the AIME ablation, these results support controlling reasoning and answer sharpening independently. In particular, the GPQA-Diamond and HumanEval results show that reasoning sharpening can further improve accuracy beyond answer sharpening alone.


Q2: Is the gap to Marginal Sharpening explained by the answer estimator described in Appendix C, and could you run that baseline with the same 32 cross-scored answers?

A2: At αZ=1\alpha_Z=1 and the same αA\alpha_A, DMS and Marginal Sharpening share the same ideal answer target. We provide the requested comparison on MATH500 problems using Qwen3-8B. All methods use the same 32 unsharpened reasoning traces, αA=4\alpha_A=4, and DMS’s reasoning–answer boundary handling, including the original Marginal Sharpening token decoder.

The original Marginal Sharpening decoder generates one answer token by token. We also evaluate a variant that selects from 32 complete, cross-scored answers while retaining the grouped approximation used by Marginal Sharpening. This variant and DMS use exactly the same answer candidates and cross-scoring results, so their difference comes from how these scores are combined to assign answer-selection weights.

Answer-sampling methodpass@1, mean ± SDExpected accuracy
Marginal Sharpening: original token decoder85.93 ± 0.50—
Marginal Sharpening: selection from 32 cross-scored answers85.93 ± 0.3185.72
DMS: selection from 32 cross-scored answers86.07 ± 0.3185.93

Under this controlled comparison, DMS achieves the highest mean pass@1 at 86.07%, while both Marginal Sharpening variants achieve 85.93%. For the two methods using the same complete answer candidates, DMS also increases expected accuracy from 85.72% to 85.93%. With the answer candidates and cross-scoring results fixed, DMS’s weighting scheme assigns more probability to correct answers on average.

Q3: How were (αZ,αA)(\alpha_Z,\alpha_A) and the Power-SMC exponents chosen, and on which data?

A3: For DMS, we selected (αZ,αA)(\alpha_Z,\alpha_A) separately for each model–benchmark setting. We randomly sampled 20% of the questions as a pilot set and performed a grid search using a single tuning seed. We then fixed the selected configuration and evaluated it on the full benchmark using three independent evaluation seeds.

For Power-SMC, we use the original paper’s default exponent α=4\alpha=4 on MATH500 and HumanEval. On GPQA-Diamond, which was not evaluated in the original paper, we further tune α\alpha because the default value performs poorly in this setting.

Table 3 reports the selected DMS exponents, and Appendix C specifies the Power-SMC configurations. Appendix D further compares performance across different combinations of αZ\alpha_Z and αA\alpha_A.


Q4: Can you add majority voting over 16 or 32 standard samples, together with its runtime?

A4: We provide majority-voting results on MATH500, GPQA-Diamond, and AIME24/25 using Qwen3-8B. MV16 and MV32 perform majority voting over 16 and 32 independently generated trajectories, respectively. Accuracy is reported as mean ± standard deviation over three seeds.

BenchmarkMethodpass@1Time (s/problem)
MATH500MV1680.20 ± 0.7264.53
MATH500MV3282.40 ± 0.92128.50
MATH500DMS85.80 ± 0.20117.86
GPQA-DiamondMV1660.94 ± 1.05189.23
GPQA-DiamondMV3259.76 ± 1.05375.01
GPQA-DiamondDMS59.93 ± 1.77237.56
AIME24/25MV1671.11 ± 0.96509.78
AIME24/25MV3273.33 ± 1.671007.94
AIME24/25Island-DMS77.22 ± 0.96874.87

For the runtime comparison, MV32 generates its 32 independent trajectories in two batches of 16, keeping the generation batch size identical to MV16 and limiting peak memory demand. Compared with MV32, DMS improves pass@1 by 3.40 percentage points on MATH500 and 3.89 points on AIME24/25, while requiring less time on both benchmarks. On GPQA-Diamond, DMS achieves similar accuracy to MV32 while reducing runtime from 375.01 to 237.56 seconds per problem; MV16 achieves the highest mean accuracy on this benchmark.


We hope we have addressed all of your questions. If you have any further questions, we would be happy to answer them and if you don’t, would you kindly consider increasing your score?

Dear Reviewer H9wT,

Thank you again for your constructive feedback. In our response, we report new ablations testing whether reasoning sharpening improves on answer sharpening alone. We also compare the DMS and Marginal Sharpening answer estimators using the same 32 cross-scored answers, clarify how we selected the sharpening exponents, and report accuracy and runtime for majority voting with 16 and 32 samples.

As the discussion period draws to a close, we would appreciate hearing whether our response addresses your concerns or whether there is anything else we can clarify.

Thank you for your time and consideration.

Thank you for the new results. Majority voting is now covered, but DMS and Marginal Sharpening are almost equal once the setup is matched, and the exponents were tuned on part of the test set, so I am raising my score to 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.