acceptodds
Under review as a conference paper at ICLR 2027

Is Sharper Always Better? Rethinking the Role of Sharpness in Power Sampling for Language Models

Abstract

Power Sampling (PS) is a training-free inference-time method that improves language model reasoning by sharpening the trajectory-level distribution. While recent studies have developed effective methods for sampling from the power distribution with a fixed sharpening strength , the impact of remains largely unexplored. In this work, we first systematically investigate the role of the sharpening strength in PS across diverse models and reasoning tasks, demonstrating that reasoning performance varies non-monotonically with across models and tasks. We further analyze the effect of through a preservation–recovery decomposition, which shows that larger improves the preservation of correct solutions favored by the base model, but reduces the recovery of alternative correct trajectories when these dominant trajectories are misleading. These opposing trends provide empirical evidence for a concentration–exploration trade-off governed by . Based on these observations, we propose Adaptive Concentration–Exploration Power Sampling (ACE), a training-free decoding framework built upon the Sequential Monte Carlo method that adaptively controls for each problem instance according to its evolving particle population during inference. By adapting the sharpening strength throughout generation, ACE dynamically balances concentration on promising trajectories with the preservation of diverse reasoning paths. Experiments across diverse reasoning benchmarks demonstrate that ACE consistently improves reasoning performance through adaptive sharpening without additional training or external reward signals.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Test-Time Scaling, Power Sampling, Distribution Sharpening

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ybKrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Adaptive Concentration–Exploration Power Sampling (ACE), a decoding method that adapts the sharpening strength alpha along the response generation trajectory for a problem instance. It frames power sampling as a trade-off between exploration and concentration, via low and high alpha in inference-time, and introduces a preservation-recovery decomposition of the overall accuracy performance to describe the behavior of a fixed power sampling relative to a high power sampling reference. The authors demonstrated that ACE could enable consistent directional evolution of αt\alpha_t values across problems of each task on the two classes of Qwen and Phi models used.

Critical strengths
  • ACE adapts alpha at inference-time along fixed token lengths during generation, via alpha maximization constrained by a minimum effective sampling size on weighted particles.
  • Similar to the power sampling methods TMC and SMC, the efficiency of ACE is better than MH. The efficiency of ACE is comparable to TMC and while being slightly worse than SMC.
  • Excluding margins of error, ACE is competitive with or performs better than other power sampling methods when used on the Qwen models (it seems inconclusive for Phi models except for the averaged accuracy).
  • ACE has another lever to adjust alpha; it counts the number of occurrences of token-level entropy exceeding a predefined level for potential additional alpha updates.
Critical weaknesses
  • There seems to be no clear trend that demonstrates an optimal alpha for Qwen2.5-7B on HumanEval in Figure 1; that its accuracy is increasing and decreasing for a few cycles with increasing alpha seems to be counterintuitive. What is the standard deviation of the accuracy at each alpha?
  • More rigorous baselines needed: As ACE is an adaptive method over alpha, it would be fairer to compare ACE to the baseline methods with their respective hyperparameters tuned to each model and task combination, i.e. temperature for temperature sampling and alpha for the other power sampling methods (instead of only using the recommended value of 4 for alpha in Table 1). Indeed, as shown in Figure 7, TMC produces higher accuracies when alpha is tuned away from the value of 4; and in Appendix D2, the ablation on tuning alpha is performed for only SMC on only a seed without the margins of error for discerning accuracies that are quite close to each other.
  • Gap in the variety of tasks and evaluation metrics: The paper is quite narrowly focused on ACE vs other untuned power sampling methods on the accuracy metric. What is the effect of power sampling on other tasks such as creative writing, and on other qualities such as token diversity, especially when particle diversity is a core mention in the paper?
  • Some inconclusive results: In Table 1, ACE is not conclusively better than the other baseline methods on the Phi models, and it seems MH is arguably stronger even without tuning its alpha. Having another model class would present more generalizable results than the two (Qwen and Phi) currently used. Considering the margins of error in Table 8, a number of the accuracies of ACE are unlikely to be statistically significant.
  • Fixing the diagnostic reference: L205 and L241 - Since the sharpening effect is measured using preservation-recovery relative to strong sharpening, using alpha=10 as the diagnostic reference is still arbitrary. As alpha increases, we expect stronger sharpening to the limit of a greedy behavior within this decoding regime as the fixed reference.
  • There is no analysis of ACE generated examples; please provide analysis of positive and negative ACE examples on the used datasets to show when exploration and concentration is good or bad. There are also no discussions to connect the empirical alpha_t distribution with the tasks. Please also highlight positive ACE examples relative to negative examples of other power sampling methods, and vice versa, for a more complete analysis.
  • As the method requires a number of hyperparameters, such as τESS\tau_{ESS}, Δup\Delta_{up}, Δdown\Delta_{down}, block size B, τmat\tau_{mat}, etc, what are the effects of altering critical hyperparameters like B?
  • Provide clearer result for comparison: Table 1 to include the per model per task accuracy of ACE without the token-level entropy feature, so as to be fairer in comparison with the baseline methods?
Questions for authors

Please refer to questions, clarifications and feedback in weaknesses.

8 comments

We appreciate the reviewer's careful evaluation of our work and the constructive feedback for further improvement. Below, we address the concerns regarding the interpretation of our findings, baseline comparisons, experimental generalizability, diagnostic analysis, and hyperparameter sensitivity.

1. Interpretation of Figure 1 and Variability Across Sharpening Strengths

The purpose of Figure 1 is not to suggest that accuracy follows a smooth or unimodal relationship with α\alpha, but to illustrate that different reasoning tasks may favor different sharpening strengths even for the same model. The fluctuations observed on HumanEval are consistent with our broader finding that stronger sharpening does not necessarily improve performance.

We have provided additional standard-deviation analyses for preservation and recovery in Appendix D.1 (Figure 9, p. 22). We also clarify that the highlighted α\alpha values represent the best observed performance within the evaluated sweep, rather than statistically established unique optima.

2. Comparison with Tuned Baselines

We thank the reviewer for this suggestion. While tuning a fixed sharpening strength can improve performance, it requires an additional offline search over multiple α\alpha values for each model–benchmark pair and cannot account for instance-level variation during generation. In contrast, ACE automatically adapts α\alpha without requiring such a search.

In addition to the tuned SMC comparison in Appendix D.2 (Table 5, p. 24), we have added a tuned TMC comparison on Qwen3-8B-Base in Appendix D.2 (Table 6, p. 25). For each benchmark, the best fixed α\alpha is selected using seed 42 and evaluated on five independent seeds. ACE outperforms tuned TMC on all three benchmarks, achieving an average accuracy of 63.5%, compared with 62.4% for tuned TMC and 61.5% for default TMC (α=4\alpha=4). These results further support the effectiveness of adaptive sharpening without requiring costly offline hyperparameter searches.

MethodHumanEvalGPQA-DiamondMATH-500Avg.
TMC (α=4\alpha=4)67.139.378.061.5
TMC (tuned)68.340.478.662.4
ACE70.041.079.563.5

3. Generalization to Creative Writing

We thank the reviewer for suggesting an evaluation beyond reasoning accuracy. We conduct an additional experiment on EQ-Bench Creative Writing v3, using Qwen3-8B-Base and 96 writing samples per method. We follow the official EQ-Bench Creative Writing v3 evaluation rubric, using DeepSeek-V4.1-Flash-1017 (deepseek-flash) as the LLM judge to assess writing quality on a 0–100 scale.

MethodWriting Score (0–100)
Standard22.37
Low-temp22.67
SMC20.79
TMC20.98
ACE27.54

ACE achieves the highest writing score of 27.54, outperforming Standard by 5.17 points, SMC by 6.75 points, and TMC by 6.56 points. These results suggest that the benefits of adaptive sharpening may extend beyond tasks evaluated by exact-match accuracy to open-ended text generation. We have included this experiment in Appendix D.9 (Table 14, p. 33).

4. Generalization Across Model Families

We thank the reviewer for highlighting the need for broader model coverage. To further evaluate the generalizability of ACE beyond the Qwen and Phi model families, we conduct additional experiments on SmolLM3-3B, covering HumanEval, GPQA-Diamond, and MATH-500. All results are averaged over five independent random seeds.

MethodHumanEvalGPQA-DiamondMATH-500Avg.
Standard58.2930.6166.8051.90
Low-temp64.5134.1471.5356.73
TMC65.7332.7374.2057.55
SMC68.5431.6274.6758.28
ACE69.5135.9675.0760.18

ACE achieves the highest accuracy across all three benchmarks, outperforming SMC by 0.97, 4.34, and 0.40 percentage points on HumanEval, GPQA-Diamond, and MATH-500, respectively. Its average accuracy reaches 60.18%, compared with 58.28% for SMC and 57.55% for TMC. We have included these additional results in Appendix D.9 (Table 13, p. 32), providing further evidence that ACE's effectiveness is not limited to the Qwen and Phi model families.

5. Robustness to the Diagnostic Reference

We thank the reviewer for raising this concern. To examine whether our preservation–recovery findings depend on the choice of α=10\alpha=10 as the diagnostic reference, we conduct an additional experiment using a much stronger sharpening strength of α=1000\alpha=1000 as the reference.

We evaluate Qwen3-8B-Base and Phi-4-mini across MATH-500, GPQA-Diamond, and HumanEval, reporting averages over three random seeds. As shown in the new figure, preservation generally increases with α\alpha, while recovery generally decreases, consistent with our original findings. Importantly, all evaluated values α∈{2,…,10}\alpha\in \{2,\ldots,10\} are now distinct from the reference, so neither trend is determined by the reference endpoint.

These results suggest that the observed preservation–recovery trade-off is not specific to choosing α=10\alpha=10 as the diagnostic reference. We have included this additional analysis in Appendix D.1 (Figure 11, p. 23).

6. Case Studies and Task-Dependent Sharpening Behavior

We thank the reviewer for this valuable suggestion. We have added a case study in Appendix D.8 (pp. 31–32) to illustrate how ACE's adaptive sharpening can affect the generated reasoning process.

In a MATH-500 problem involving rotational symmetry, ACE correctly obtains the answer 1008010080, whereas SMC, TMC, and MH incorrectly account for reflectional symmetry and return 50405040. During generation, ACE dynamically adjusts its sharpening strength from α0=4\alpha_0=4 to αT=3.097\alpha_T=3.097, with eight downward and three upward updates. This example illustrates how adaptive sharpening may help preserve alternative reasoning paths rather than prematurely concentrating on misleading solutions.

We further relate this behavior to the task-dependent sharpening trajectories in Figure 5 and Appendix D.6 (Figure 14). Across the evaluated models, αt\alpha_t generally increases on HumanEval, decreases on GPQA-Diamond, and remains relatively close to its initialization on MATH-500, indicating distinct adaptation patterns across reasoning tasks.

7. Sensitivity to Block Size

We thank the reviewer for suggesting additional hyperparameter sensitivity analyses. In addition to the studies on the ESS threshold and initial sharpening strength in Appendix D.5, we have conducted a new block-size ablation on Qwen3-8B-Base using HumanEval.

Block Size BBAccuracy (%)
6470.73
12870.12
19270.12
38470.12
76870.73

The accuracy varies by only 0.61 percentage points across the evaluated block sizes, suggesting that ACE is relatively insensitive to this hyperparameter in the tested setting. We have included these results in Appendix D.5 (Table 11, p. 29).

8. Effect of Entropy-Triggered Updates

We thank the reviewer for suggesting a more detailed comparison. We have added the per-model, per-benchmark accuracy of ACE without entropy-triggered updates in Appendix D.4 (Table 10, p. 27). This variant retains adaptive sharpening at regular block boundaries while disabling entropy-triggered updates. These results provide a more detailed assessment of the contribution of entropy-triggered updates to ACE.

Dear Reviewer ybKr,

Thank you again for your constructive feedback. We have carefully addressed your concerns point by point and provided additional clarifications and experimental results where appropriate. We would greatly appreciate any further feedback and look forward to your response and continued discussion.

Best regards, The Authors

Reviewer_R3CYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at how the sharpening strength alpha affects power sampling. The authors sweep alpha on several models and benchmarks and find that a bigger alpha is not always better, and the best value depends on the model and the task. They split problems based on whether alpha=10 solves them and use this to argue there is a trade-off between concentration and exploration. Then they propose ACE, which runs SMC power sampling but adjusts alpha during generation based on ESS, with extra updates at high-entropy tokens. ACE is compared with MH, TMC and SMC with alpha=4, and it is better on average in most cases.

Critical strengths

The question makes sense to me. Most power sampling papers just fix alpha=4 and don't look at it much, and this paper shows pretty clearly that the best alpha depends on the model and the task. The method is easy to add on top of SMC and doesn't cost much more to run.

Critical weaknesses

My biggest issue is that I'm not convinced the gains come from adapting alpha. The improvements over SMC are around one or two points, which is not much given the variance on these benchmarks, especially GPQA. In the Figure 6 ablation, ACE without the entropy trigger is about the same as plain SMC. And in Table 7, the version that keeps alpha fixed at 4 still gets most of the gain. So the improvement may come from the extra resampling at high-entropy tokens or other implementation differences, not from the alpha rule itself.

I'm not sure the preservation/recovery analysis says that much either. The groups are defined by the alpha=10 runs with the same seeds, so it is not very surprising that alpha values close to 10 agree more with them. The ESS rule doesn't really connect to this analysis anyway, because ESS has no idea whether the dominant trajectories are right or wrong. Proposition 1 assumes no resampling and the base model as the proposal, which is not how the method is actually run.

The novelty of the method feels limited to me. Picking the next exponent so that ESS stays above a threshold is basically adaptive tempering, which has been used in SMC for a long time (the paper itself cites Jasra et al. and Zhou et al. for this). The authors say the difference is that there is no fixed final alpha here, but that also means it is not clear what distribution ACE is actually sampling from in the end. I would like to see this discussed in the main paper and not only in the appendix.

Questions for authors
  • Could you add a baseline that uses fixed alpha=4 but keeps the same entropy-triggered resampling and the other settings of ACE? This would show whether adapting alpha really matters.
  • Does the preservation/recovery trend still hold if the alpha=10 reference comes from a different seed?
  • It would be good to see a compute-matched baseline like self-consistency with the same number of samples.
6 comments

We appreciate the reviewer's careful assessment of our work and the constructive suggestions for improvement. Below, we address the concerns regarding the effectiveness of adaptive sharpening, the preservation–recovery analysis, the theoretical justification, and the relationship to existing adaptive SMC methods.

1. Effectiveness of Adaptive Sharpening

We appreciate the reviewer's concern regarding whether the performance gains genuinely arise from adapting α\alpha.

First, we additionally compare ACE with a tuned fixed-α\alpha SMC baseline (Appendix D.2, Table 5), where the fixed sharpening strength is selected separately for each model–benchmark pair. ACE outperforms tuned SMC in five out of six settings, achieving an average accuracy of 60.8%, compared with 60.0% for tuned SMC and 58.9% for the default SMC (α=4\alpha=4). These results highlight the importance of sharpening strength α\alpha, as tuning α\alpha improves the performance of fixed-α\alpha SMC. More importantly, ACE achieves better overall performance through automatic, instance-level adaptation, without requiring an expensive offline search over α\alpha for each model–benchmark pair.

Second, the improvements are not uniformly marginal relative to the observed variance. For example, on HumanEval, ACE improves accuracy from 58.1 ± 1.5% to 62.3 ± 0.8% on Qwen2.5-7B and from 67.7 ± 2.0% to 70.0 ± 0.8% on Qwen3-8B-Base (Appendix D.7, Table 8).

Third, our ablation study (Appendix D.4, Table 7) further distinguishes adaptive sharpening from adaptive proposal temperature. The α\alpha-only variant, which adapts α\alpha while keeping the proposal temperature fixed, outperforms the temperature-only variant in most settings. For instance, on Phi-4-mini with GPQA-Diamond, α\alpha-only achieves 33.6%, compared with 30.1% for temperature-only. These results suggest that adapting the trajectory-level sharpening strength makes an important contribution to ACE's performance gains, beyond the effect of proposal-temperature adaptation alone.

Furthermore, to directly address the concern about entropy-triggered resampling, we introduce an additional fixed-α\alpha SMC baseline with the same entropy-triggered ESS checks and resampling mechanism as ACE. As detailed in Appendix D.4, Table 9 (p. 26), ACE outperforms this baseline on all three evaluated benchmarks, providing further evidence that the improvements cannot be attributed solely to additional entropy-triggered resampling.

To further assess the generalizability of ACE, we additionally evaluate SmolLM3-3B, extending our experiments beyond the Qwen and Phi model families. As reported in Appendix D.9, Table 13 (p. 32), ACE achieves the highest average accuracy across all three benchmarks, reaching 60.18%, compared with 58.28% for SMC and 57.55% for TMC. In particular, ACE improves GPQA-Diamond accuracy from 31.62% to 35.96% (+4.34 percentage points). These results provide additional evidence that adaptive sharpening remains effective across different model families.

MethodHumanEvalGPQA-DiamondMATH-500Avg.
Standard58.2930.6166.8051.90
Low-temp64.5134.1471.5356.73
TMC65.7332.7374.2057.55
SMC68.5431.6274.6758.28
ACE69.5135.9675.0760.18

2. Motivation and Theoretical Justification of ESS-Based Adaptation

We agree with the reviewer that ESS does not directly indicate whether dominant trajectories are correct. However, trajectory correctness is generally unavailable during generation without external verification. Our preservation–recovery analysis reveals that excessive sharpening can suppress alternative correct trajectories, motivating the use of ESS as an observable measure of particle weight concentration to regulate the concentration–exploration trade-off. We do not assume that ESS can predict correctness; rather, it provides a practical signal for controlling concentration during inference.

We also acknowledge that Proposition 1 is established under simplified assumptions that do not fully match the implementation of ACE. Its purpose is to establish a theoretical relationship between α\alpha and ESS under these conditions, thereby motivating the ESS-based adaptation rule, rather than guaranteeing ESS monotonicity throughout the actual sampling process. In practice, ACE explicitly computes the ESS for each candidate α\alpha and selects the next sharpening strength based on the resulting ESS values. We have further clarified the motivation for ESS-based adaptation in Section 5.2 (p. 6, immediately after Proposition 1), emphasizing that ESS measures particle weight concentration rather than trajectory correctness. The theoretical assumptions and proof of Proposition 1 are provided in Appendix A (pp. 13–14).

3. Relationship to Adaptive Tempering and Target Distribution

We respectfully disagree that ACE is merely a direct application of conventional adaptive tempering. We acknowledge that ESS-based adaptation has been extensively studied in SMC. However, Jasra et al. (2011)[1] and Zhou et al. (2016)[2] primarily use ESS or CESS to adapt intermediate tempering schedules toward a prespecified terminal target distribution. The adaptation determines how to reach that target efficiently, rather than changing the target itself.

In contrast, ACE is motivated by our empirical finding that the appropriate sharpening strength varies across models and reasoning tasks, with different problem instances potentially requiring different levels of sharpening. Rather than following a schedule toward a fixed exponent, ACE adaptively controls the trajectory-level sharpening strength itself, allowing αt\alpha_t to increase or decrease according to the evolving particle population. Thus, our contribution is not the ESS criterion alone, but its use to dynamically regulate the sharpening target for language-model reasoning.

Regarding the final distribution, ACE defines an intermediate target proportional to pθ(y≤t∣x)αtp_\theta(y_{\leq t}\mid x)^{\alpha_t} and adjusts particle weights when αt\alpha_t changes. The terminal exponent αT\alpha_T depends on the problem instance and the stochastic adaptive sampling process, rather than being fixed in advance. Thus, ACE constructs a particle-based approximation under an adaptively selected sharpening target, without claiming exact sampling from a single prespecified power distribution. Figure 5 and Appendix D.6 (Figure 14) illustrate the distinct sharpening trajectories across models and tasks.

We have clarified the relationship to conventional adaptive tempering and the interpretation of ACE's adaptive target distribution in Section 5.2 (p. 7).

[1]Jasra, A., Stephens, D. A., Doucet, A., & Tsagaris, T. (2011). Inference for Lévy‐driven stochastic volatility models via adaptive sequential Monte Carlo. Scandinavian Journal of Statistics, 38(1), 1-22.

[2]Zhou, Y., Johansen, A. M., & Aston, J. A. (2016). Toward automatic model comparison: an adaptive sequential Monte Carlo approach. Journal of Computational and Graphical Statistics, 25(3), 701-726.

4. Response to Q1: Contribution of Adaptive Sharpening

We thank the reviewer for this suggestion. To isolate the contribution of adaptive sharpening, we additionally evaluate SMC + Entropy, which keeps α=4\alpha=4 fixed while incorporating the same entropy-triggered mechanism as ACE, with all other settings unchanged.

The following table compares the accuracy and average inference latency per problem on Qwen3-8B-Base, averaged over five random seeds.

MethodHumanEvalGPQA-DiamondMATH-500
SMC67.6837.3779.40
SMC + Entropy64.3939.2978.28
ACE70.0041.0079.52

As shown in the table, ACE outperforms SMC + Entropy across all three benchmarks, with improvements of 5.61, 1.71, and 1.24 percentage points on HumanEval, GPQA-Diamond, and MATH-500, respectively. In contrast, adding entropy-triggered ESS checks and resampling to fixed-α\alpha SMC does not consistently improve accuracy. These results suggest that the gains of ACE cannot be explained solely by entropy-triggered resampling, supporting the contribution of adaptive sharpening. The full experimental results have been added to Appendix D.4, Table 9 (p. 26).

5. Response to Q2: Cross-Seed Preservation–Recovery Analysis

We thank the reviewer for this valuable suggestion. To examine whether the preservation–recovery trends depend on using the same random seed to define the reference partition, we conduct an additional cross-seed analysis using three seeds: 42, 1062, and 2082.

Specifically, for each seed used to evaluate α∈{2,…,9}\alpha \in \{2,\ldots,9\}, we independently construct the reference partition using the α=10\alpha=10 outcomes from each of the other two seeds. This yields six cross-seed combinations, and we report their average results.

As shown in Appendix D.1, Figure 10 (p. 22), preservation generally increases with α\alpha, while recovery generally decreases across the evaluated models and benchmarks. Importantly, the reference partition and evaluation outcomes are obtained from different random seeds, avoiding the deterministic endpoint agreement induced by using the same seed.

These results further support that the observed preservation–recovery trends are not merely an artifact of the same-seed reference partition.

6. Response to Q3: Comparison with Self-Consistency and Best-of-N

Following Ji et al. (2026)[3], we additionally compare ACE with Self-Consistency (SC) and Best-of-N (BoN) on SmolLM3-3B using GPQA-Diamond. Results are averaged over three random seeds.

MethodAccuracy (%)
SC35.19 ± 1.54
BoN33.33 ± 1.34
ACE36.87 ± 2.53

As shown in the table, ACE achieves the highest average accuracy of 36.87%, outperforming SC and BoN. These results suggest that adaptive sharpening provides a competitive alternative to conventional test-time sampling and aggregation strategies.

[3]Ji, X., Tutunov, R., Zimmer, M., & Ammar, H. B. (2026). Scalable power sampling: Unlocking efficient, training-free reasoning for llms via distribution sharpening. arXiv preprint arXiv:2601.21590.

Reviewer_rKcqOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This work analyzes the concentration–exploration trade-off in Power Sampling methods which were proposed in recent works, the main idea of which is power sharpening across the whole future trajectory. Based on the observation of this trade-off, the work proposes a particle population method to adjust the sharpness hyperparameter during sampling, to achieve better performance with Power Sampling.

Critical strengths

The concentration–exploration trade-off is well analyzed and demonstrated properly. The existence of such a trade-off is in line with expectations, so I cannot say that this point has great novelty, but it is still a clear analytical work.

The ACE method proposed is well motivated by the analysis and shows competitive performance on the benchmarks HumanEval, GPQA-Diamond, and MATH-500.

The writing is clear and easy to follow.

Critical weaknesses

The novelty and contribution of the ACE method are relatively limited considering the related work and the performance gain. The method adds bidirectional alpha adjustment based on the SMC method to better control the hyperparameter, which is not interesting to me. More importantly, the performance gain is marginal and not stable, meaning that the problem addressed by the method may not be critical.

The ACE method also involves higher inference cost. This makes it more subtle to attribute the performance gains to the trade-off problem, rather than consider them as inference-time scaling, i.e., a larger sampling budget.

Questions for authors

Can you provide detailed inference FLOPs and token counts for ACE and the baseline methods, including naive decoding?

3 comments

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. We address the concerns regarding novelty, performance, the significance of the problem, and inference cost below.

1. Novelty and Contributions

We thank the reviewer for recognizing that our concentration–exploration analysis is well conducted and that ACE is well motivated by our empirical findings. We agree that the general concentration–exploration trade-off is intuitive. However, this aspect has not been explicitly studied in existing Power Sampling literature. Prior works have primarily focused on developing efficient sampling algorithms for the power distribution, while largely overlooking how the sharpening strength α\alpha itself affects reasoning performance and the underlying mechanisms.

To the best of our knowledge, our work provides the first systematic investigation of the concentration–exploration trade-off induced by sharpening strength in Power Sampling. Specifically, we reveal a preservation–recovery trade-off: stronger sharpening better preserves solutions obtained under strong sharpening but reduces the ability to recover alternative correct solutions. This analysis provides a new perspective on understanding the role of α\alpha in Power Sampling. Building on this finding, we propose ACE to dynamically regulate the sharpening strength during generation.

2. Performance and Stability

We acknowledge that ACE does not outperform all baselines in every setting. Nevertheless, it achieves the highest average accuracy on four out of five models and outperforms fixed-α\alpha SMC in 13 out of 15 model–benchmark combinations.

For example, ACE improves GPQA-Diamond accuracy from 37.4% to 41.0% (+3.6 percentage points) on Qwen3-8B-Base, HumanEval accuracy from 58.1% to 62.3% (+4.2 points) on Qwen2.5-7B, and MATH-500 accuracy from 63.0% to 66.2% (+3.2 points) on Phi-4-mini. These results demonstrate that ACE provides meaningful improvements across different models and reasoning tasks, rather than benefiting only from a particular model–benchmark setting. Regarding stability, we additionally report standard deviations over five random seeds in Appendix D.7, Table 12 (p. 31). For example, on HumanEval with Qwen2.5-7B, ACE achieves 62.3 ± 0.8%, compared with 58.1 ± 1.5% for SMC. On GPQA-Diamond with Qwen3-8B-Base, ACE achieves 41.0 ± 1.8%, compared with 37.4 ± 0.7% for SMC. These results provide further evidence of performance improvements across independent runs.

To further assess the generalizability of ACE, we additionally evaluate SmolLM3-3B, extending our experiments beyond the Qwen and Phi model families. As reported in Appendix D.9 (Table 13, p. 32), ACE achieves the highest mean accuracy across HumanEval, GPQA-Diamond, and MATH-500, with an average of 60.18%, compared with 58.28% for SMC and 57.55% for TMC. In particular, ACE improves GPQA-Diamond accuracy from 31.62% to 35.96% (+4.34 percentage points). These results provide additional evidence that the benefits of adaptive sharpening extend beyond the original model families.

MethodHumanEvalGPQA-DiamondMATH-500Avg.
Standard58.2930.6166.8051.90
Low-temp64.5134.1471.5356.73
TMC65.7332.7374.2057.55
SMC68.5431.6274.6758.28
ACE69.5135.9675.0760.18

3. Significance of the Problem

The appropriate sharpening strength varies not only across models and benchmarks but also across individual problem instances. As shown in Section 4 and Appendix D.6, a single fixed α\alpha cannot fully accommodate such variation.

This is further supported by our comparison with tuned fixed-α\alpha SMC in Appendix D.2 (Table 5, p. 24). Tuning α\alpha improves the average accuracy of SMC from 58.9% to 60.0%, confirming the importance of selecting an appropriate sharpening strength. Nevertheless, ACE achieves a higher average accuracy of 60.8%, outperforming tuned SMC in five out of six settings. More importantly, such offline tuning requires searching over multiple candidate values for each model–benchmark pair and cannot capture instance-level changes during generation.

ACE addresses this limitation by automatically adapting the trajectory-level sharpening strength according to the evolving particle population, without requiring an offline search for each model–benchmark pair. Thus, its contribution is not simply to obtain marginal performance gains, but to provide a mechanism for adapting the sampling distribution to the evolving reasoning process.

4. Inference Cost

To provide a concrete comparison of inference costs, we conduct detailed profiling on Qwen3-8B-Base using HumanEval, comparing the decoding methods under the same hardware and evaluation settings. We report the average output tokens, total generated tokens, estimated model FLOPs, and wall-clock latency per problem over 164 problems (seed 42). Accuracy values are taken from the five-run averages in Table 1. Due to the substantially higher computational cost of MH, we defer its detailed profiling to a future revision.

MethodAccuracy (%)Output TokensTotal Generated TokensFLOPs (TFLOPs)Latency (s)
Standard42.6236.09236.095.525.27
Low-temp65.6799.24799.2414.6517.96
TMC67.1621.2630,067.27503.1936.89
SMC67.7452.7638,569.76636.5226.02
ACE70.0430.4736,705.85649.1328.41

ACE and SMC use the same particle number (64), block size (192), and maximum generation length (3,072 tokens). ACE does not introduce additional independent particle rollouts; its additional overhead mainly comes from adaptive sharpening and entropy-triggered updates.

As shown in the table, ACE improves accuracy over SMC by 2.3 percentage points (70.0% vs. 67.7%), while requiring only 1.98% more model FLOPs (649.13 vs. 636.52 TFLOPs) and 9.19% more inference time (28.41 vs. 26.02 seconds). Moreover, ACE generates slightly fewer total tokens than SMC (36,705.85 vs. 38,569.76). These results indicate that ACE achieves better reasoning performance with modest additional computational overhead, rather than relying on a substantially larger sampling budget.

Dear Reviewer rKcq,

Thank you again for your constructive feedback. We have carefully addressed your concerns point by point and provided additional clarifications and experimental results where appropriate. We would greatly appreciate any further feedback and look forward to your response and continued discussion.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.