acceptodds
Under review as a conference paper at ICLR 2027

Distributional Position Shift for Reasoning: Shared-Future Polar Sampling

Abstract

Reasoning systems can solve problems that their base models cannot solve with standard decoding. Reinforcement learning (RL) and multi-step inference have emerged as powerful approaches for improving the reasoning capabilities of large language models by shaping their policy distributions and exploiting information from multi-step trajectories. More recently, power sampling has been proposed as a training-free approach to improve reasoning by sharpening the sequence distribution during generation. However, local sharpening can only amplify the model’s existing preferences: it preserves the relative ranking of candidate segments and therefore cannot promote a lower-ranked candidate that may lead to better reasoning when followed by later steps. This motivates a different question: can sampling use evidence from future reasoning to revise the model’s local candidate preferences? In this work, we introduce *Shared-Future Polar Sampling* (SFPS), a training-free approach that enables *position shifts* in the candidate distribution by using evidence from subsequent reasoning to re-rank candidate segments. Inspired by RL’s use of rollout comparisons, SFPS uses short continuations to estimate how future reasoning supports each candidate segment. These continuations are shared across candidates, allowing their future support to be compared efficiently. Our polar sampling rule uses both the magnitude and direction of this support to update the candidate distribution. SFPS outperforms MCMC Power Sampling on all ten model–task pairs and GRPO on eight. Across four Qwen2.5-Math-7B tasks, it uses 20.13% of Power’s generated tokens and 9.31% of its inference time on average.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM reasoning, distributional position shift, polar sampling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TmpUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Shared-Future Polar Sampling (SFPS), a training-free decoding method for reasoning. At each step, it samples several candidate segments and generates a few short continuations shared across them. An importance-weighted candidate–continuation matrix estimates how well each candidate is supported by its future, and continuation tokens are drawn with a "polar" proposal. The authors prove consistency for a finite candidate pool and horizon, and compare SFPS with base sampling, low-temperature decoding, MCMC Power, and public GRPO checkpoints on math, science QA, and code tasks.

Critical strengths
  1. The paper expresses local candidate likelihoods and the conditional probabilities of short continuations within a single importance-weighted comparison matrix, and analyzes the estimation properties of the polar proposal. Compared with a purely heuristic sampling rule, this makes the objective, sources of approximation, and scope of applicability clearer.

  2. On the reported model–task pairs, SFPS outperforms MCMC Power while using substantially fewer generated tokens and less wall-clock time. The appendix also provides useful mechanism analyses of the matrix readout and the proposal.

Critical weaknesses
  1. End-to-end accuracy supports the empirical effectiveness of the method, but does not verify the mechanistic claim that future evidence identifies better reasoning candidates. The paper describes SFPS as using evidence from future reasoning to revise local preferences. However, by definition, its future evidence is the sum of the base model's powered probabilities over future continuations; it measures the model's confidence in a continuation rather than correctness. Thus, at an individual decision point, this signal can shift probability mass toward a more confident but incorrect candidate. The final accuracies in Table 1 show that the method is effective overall in the evaluated settings, but this end-to-end metric also conflates candidate generation, repeated block commitments, test-time computation, and a preference for low-entropy continuations. It therefore cannot identify whether the gain is caused by future evidence promoting more correct candidates. The observation in the appendix that responses concentrate in high-likelihood, low-entropy regions is also consistent with the alternative explanation that the main benefit comes from a stronger confidence preference. Consequently, the paper's account of “future reasoning support” is currently better viewed as an explanatory hypothesis than as a directly supported conclusion.

  2. The experiments are insufficient to support the claim that SFPS is efficient and superior to existing training-free methods. Scalable Power Sampling and Power-SMC are efficient power-based samplers cited in Appendix A, but the main table compares only with MCMC Power. The computational costs of SFPS and MCMC Power differ substantially, yet the paper reports only one operating point and does not show performance under comparable compute budgets. Moreover, shared continuations require repeated scoring in multiple candidate contexts, but the actual computational cost is not fully characterized. The main table also lacks confidence intervals or significance tests; some gaps are small, while several tasks, especially AI-MO and GPQA, have limited sample sizes, making the stability of the results difficult to assess. Finally, the GRPO checkpoints are trained on mathematics data but are also evaluated on GPQA and HumanEval, and the paper does not fully specify prompt formats and tuning procedures for the baselines. Taken together, the relative performance across methods and tasks remains difficult to interpret reliably.

Questions for authors
  1. Could the paper more directly test whether a position shift improves correctness? For example, across tasks, what is the final accuracy of candidates for which SFPS changes the choice away from the initial leader, relative to the original leader? This would help distinguish beneficial future comparison from merely preferring low-entropy continuations.

  2. Could the authors compare with Scalable Power Sampling and Power-SMC? If reproduction is not feasible, please explain why. For MCMC Power, could the paper report performance-versus-budget curves under comparable generated-token counts, actual scoring cost, or wall-clock time, rather than only one configuration point?

  3. Please provide the prompt formats, answer extraction procedures, MCMC hyperparameters, and tuning procedures for all baselines, and report confidence intervals for the main results.

1 comment

W1 & Q1: Does future-based selection improve correctness beyond increasing model confidence?

We now test final correctness after changing only the opening. The future score is likelihood-based; its usefulness is evaluated through a controlled outcome comparison. At the first shift in 267 MATH500 problems, both branches use the same saved state and Base continuation policy (Table 4(b); Appendix J.4):

OpeningFinal accuracy
Initial bb-leader60.67%
Future-selected63.67%

The +3.0 pp gain measures the effect of choosing a different opening under a fixed downstream policy. Full-decoder ablations provide complementary evidence: removing future aggregation or column weighting reduces accuracy from 77.6% to 74.0% or 74.2%, with paired SFPS gains of +3.6 pp [1.2, 6.0] and +3.4 pp [1.2, 5.6] (Table 5).

W2 & Q2: How does SFPS compare across sampling budgets, including its scoring cost?

We added efficient baselines, SFPS–SPS compute comparisons, and an MCMC step sweep. The four-task results and MATH500 costs are:

Qwen2.5-Math-7B scores (%) and MATH500 costs are from Tables 1–2; HumanEval uses pass@1. Parentheses normalize to Base: 8.25 s and 672.6 tokens.

MethodMATH500AMCGPQAHumanEvalTime (s; × Base)Tokens (T = Base)
MCMC Power71.6045.7835.3555.49179.45 (21.75×)14,544.6 (21.63T)
SPS76.2050.6036.8755.4977.08 (9.34×)34,582.3 (51.42T)
Power-SMC, cap 2,04870.2046.9925.7650.6128.55 (3.46×)36,959.2 (54.95T)
Power-SMC, cap 3,07270.8046.9926.7750.6131.70 (3.84×)37,666.4 (56.00T)
SFPS77.6053.0142.4260.9816.55 (2.01×)3,381.1 (5.03T)

SPS: L=K=M=8L=K=M=8 (HF adaptation); Power-SMC: 64 particles (paper-based HF reference), as specified in Appendix J.2. Tokens include all candidates, particles, and futures.

At similar latency, SFPS improves accuracy by 5.8 pp near 17 s (about 2× Base time) and 3.4 pp near 27 s (about 3.3× Base time) over SPS (Figure 5). The new MCMC sweep (Table 17) gives:

MCMC steps0246810
Accuracy (%)69.070.870.873.470.871.6

Its best tested accuracy is 73.4%, versus SFPS's 77.6%. Figure 5 reports measured time/token budgets; Table 17 varies MCMC steps.

Shared-future scoring is included in the measured runtime. A full default round samples KL+nH=640KL+nH=640 tokens and requires up to mnH=3,072mnH=3,072 candidate–future token evaluations for eight live candidates (Appendix H.2). These evaluations are batched and build the comparison matrix during generation; selecting from the completed matrix requires no additional model pass (Section 4.4).

W2 & Q3: Are the evaluation protocol, baseline settings, and statistical uncertainty specified?

All methods use Power Sampling's task templates, answer extraction, and grading, with inference on H100 GPUs (Appendix H.3). Table 7 and Appendix J document configurations and sensitivity tests; Table 17 reports the MCMC step sweep. Primary comparisons retain the original SFPS default. For all main comparisons, we use the original default configurations of SFPS and each baseline, without additional hyperparameter tuning.

Table 14 gives the following SFPS-minus-baseline MATH500 differences, with paired question-level 95% intervals:

BaselineAccuracy gain (pp)95% paired CI
SPS+1.4[−1.8, 4.6]
Power-SMC, cap 2,048+7.4[3.6, 11.2]
Power-SMC, cap 3,072+6.8[3.0, 10.6]

Tables 5 and 15 also report paired intervals for readout and sensitivity comparisons. Table 9 lists benchmark sizes; the GRPO rows measure transfer of the specific math-trained checkpoints identified in Table 8. We will release all experimental code, baseline configurations, and evaluation scripts.

Reviewer_p2U9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Shared-Future Polar Sampling (SFPS), a training-free decoder that compares candidate openings using shared short continuations. It constructs a matrix of importance-weighted contributions, selects an opening based on its row sum, and commits to a continuation from that row. A weighted-norm token proposal is used to generate the continuations. The analysis characterizes candidate rank reversals, establishes one-step optimality for a specified weighted MSE, and proves consistency for the finite-horizon comparison. Across ten model-task pairs, SFPS improves on the tested MCMC Power baseline. Table 2 also reports substantially lower wall-clock latency on the four Qwen2.5-Math-7B tasks.

Critical strengths
  1. Sharing short continuations across candidate openings is an effective way to compare them without generating a separate full response for each. Sections 3.2 and 4.4 explain how the importance weights accumulated during generation produce the comparison matrix in Eq. (5). The connection between sampling and candidate selection is made explicit.

  2. On Qwen2.5-Math-7B MATH500, SFPS reaches 77.6%, compared with 71.6% for MCMC Power, with reported times of 16.55 versus 179.45 seconds per problem (Tables 1 and 2). Table 3 also shows gains of 3.6 and 3.4 percentage points over Initial-b and Equal-column; both paired confidence intervals exclude zero. These ablations provide evidence for using aggregate future evidence and column weights.

Critical weaknesses
  1. Figure 4(b) compares Polar, Mass, and Euclidean using one-step weighted MSE, while Table 3 compares readout rules. These results do not indicate whether the lower estimation error translates into better final answers. Comparing the three proposals on MATH500, with the remaining settings fixed, would address this concern. Please report accuracy and latency. Since the Polar proposal is a central contribution, its practical benefit is important to my overall assessment.

  2. Table 1 does not include Scalable Power Sampling or Power-SMC, both of which address the inference cost of power sampling. The reported speedup over MCMC Power is substantial, but it does not establish how SFPS compares with these more efficient alternatives. A comparison with at least one of them on Qwen2.5-Math-7B MATH500, using the same prompts and environment, would help assess the practical gain.

Questions for authors
  1. With the other settings fixed, how do MATH500 accuracy and latency change when the Polar proposal is replaced with Mass or Euclidean?

  2. How does SFPS compare with Scalable Power Sampling or Power-SMC on Qwen2.5-Math-7B MATH500 under the same prompts and in the same environment?

1 comment

W1 & Q1: Does Polar improve final accuracy over Mass and Euclidean at similar cost?

Yes: Polar improves MATH500 accuracy by 3.2 pp over Mass and 2.6 pp over Euclidean at similar latency (Table 4(a); Base time = 8.25 s):

ProposalMATH500 accuracyTime (s; × Base)
Polar77.6%16.55 (2.01×)
Mass74.4%16.85 (2.04×)
Euclidean75.0%16.82 (2.04×)

We keep the candidate and future budgets, exponent, output cap, and readout fixed; each proposal uses its corresponding importance correction (Appendix J.3). Together with Polar's 78.6% lower local MSE than Mass (Figure 4(b)), this provides both estimation and final-answer evidence for the proposal.

W2 & Q2: How does SFPS compare with efficient power samplers under the same evaluation setup?

We added SPS and Power-SMC on all four tasks, alongside MCMC Power. SFPS leads all four tasks among these samplers:

Qwen2.5-Math-7B scores (%) and MATH500 costs are from Tables 1–2; HumanEval uses pass@1. Parentheses normalize to Base: 8.25 s and 672.6 tokens.

MethodMATH500AMCGPQAHumanEvalTime (s; × Base)Tokens (T = Base)
MCMC Power71.6045.7835.3555.49179.45 (21.75×)14,544.6 (21.63T)
SPS76.2050.6036.8755.4977.08 (9.34×)34,582.3 (51.42T)
Power-SMC, cap 2,04870.2046.9925.7650.6128.55 (3.46×)36,959.2 (54.95T)
Power-SMC, cap 3,07270.8046.9926.7750.6131.70 (3.84×)37,666.4 (56.00T)
SFPS77.6053.0142.4260.9816.55 (2.01×)3,381.1 (5.03T)

SPS: L=K=M=8L=K=M=8 (HF adaptation); Power-SMC: 64 particles (paper-based HF reference), as specified in Appendix J.2. Tokens include all candidates, particles, and futures.

On MATH500, SFPS reaches 77.6% versus SPS's 76.2%, while running 4.66× faster with 90.2% fewer sampled tokens. At similar latency, SFPS improves accuracy by 5.8 pp near 17 s (about 2× Base time) and 3.4 pp near 27 s (about 3.3× Base time) over SPS (Figure 5).

All methods use Power Sampling's task templates, answer extraction, and grading protocol, and all inference experiments run on H100 GPUs (Appendix H.3). We will release all experimental code, baseline configurations, and evaluation scripts.

Reviewer_Ru4JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Shared-Future Polar Sampling (SFPS), a training-free decoding method to improve LLM reasoning. The proposed method uses short future continuations to reassess candidate preferences and approximate a sequence-level power sampling objective. Specifically, it uses shared continuations and importance weighting to estimate candidate-specific future support, while introducing a polar sampling rule to improve estimation efficiency. The method iteratively selects and commits the best-supported candidate–continuation pair. Experiments across three models and four benchmarks show improvements over MCMC Power Sampling with lower inference costs.

Critical strengths
  1. The paper presents a well-motivated perspective on improving reasoning through future-aware candidate selection, offering a training-free alternative to modifying model parameters.

  2. The proposed shared-future sampling framework is supported by theoretical analyses of importance sampling, estimation efficiency, and finite-horizon consistency.

  3. SFPS consistently outperforms MCMC Power Sampling across the evaluated model–task combinations.

Critical weaknesses
  1. Insufficient comparisons with recent baselines. The main experiments primarily compare SFPS with basic decoding strategies and MCMC Power Sampling, while GRPO serves as a post-training reference. Several closely related and more recent training-free sampling methods, such as Scalable Power Sampling, Power-SMC, and APPS, are not included. This limits the evidence for SFPS's relative effectiveness and efficiency, particularly given the substantial computational overhead of MCMC Power Sampling.

  2. Insufficient ablations and sampling stability analysis. The method involves several important sampling choices, yet the paper provides limited sensitivity analyses of key parameters, such as the number and length of shared continuations and the power exponent. In particular, using very few shared continuations may lead to high-variance importance estimates and unstable candidate selection, which is not directly examined. Moreover, the contribution of Polar Sampling is primarily demonstrated through local estimation error rather than end-to-end comparisons against simpler sampling proposals. These missing analyses make it difficult to assess the robustness of the chosen configuration and the claimed accuracy–efficiency trade-off.

  3. Limited justification of the reasoning improvements. Although the paper analyzes candidate ranking changes and improvements in estimation error, their connection to actual reasoning performance remains insufficiently established. In particular, stronger future likelihood concentration does not necessarily imply more correct reasoning, and the observed position shifts are not directly connected to improvements in solution correctness. It also remains unclear why SFPS achieves better accuracy than MCMC Power Sampling, given their closely related power-sampling objectives.

  4. Motivation for model selection. While the experiments cover three models and multiple benchmarks, the choice of evaluation models could be better motivated. In particular, two of the three models are from the Qwen2.5-Math family, despite the method being presented as a general training-free decoding approach. It would be helpful to clarify the rationale behind these choices and whether there are particular considerations regarding the applicability of SFPS to more general instruction-tuned or newer models.

Questions for authors

Please refer to the concerns raised in the Weaknesses section.

4 comments

W1: How does SFPS compare with recent sampling baselines?

We now compare with MCMC Power, SPS, and Power-SMC on all four tasks. SFPS achieves the highest accuracy among these samplers, with lower measured time and sampled-token cost at the reported settings.

Qwen2.5-Math-7B scores (%) and MATH500 costs are from Tables 1–2; HumanEval uses pass@1. Parentheses normalize to Base: 8.25 s and 672.6 tokens.

MethodMATH500AMCGPQAHumanEvalTime (s; × Base)Tokens (T = Base)
MCMC Power71.6045.7835.3555.49179.45 (21.75×)14,544.6 (21.63T)
SPS76.2050.6036.8755.4977.08 (9.34×)34,582.3 (51.42T)
Power-SMC, cap 2,04870.2046.9925.7650.6128.55 (3.46×)36,959.2 (54.95T)
Power-SMC, cap 3,07270.8046.9926.7750.6131.70 (3.84×)37,666.4 (56.00T)
SFPS77.6053.0142.4260.9816.55 (2.01×)3,381.1 (5.03T)

SPS: L=K=M=8L=K=M=8 (HF adaptation); Power-SMC: 64 particles (paper-based HF reference), as specified in Appendix J.2. Tokens include all candidates, particles, and futures.

On MATH500, SFPS is 4.66× faster than SPS and uses 90.2% fewer sampled tokens. At similar latency, SFPS improves accuracy by 5.8 pp near 17 s (about 2× Base time) and 3.4 pp near 27 s (about 3.3× Base time) over SPS (Figure 5).

All methods follow Power Sampling's task templates and evaluation protocol on H100 GPUs (Appendix H.3). APPS has not released its code publicly, so we could not include it in our comparison. We will release all experimental code, configurations, and evaluation scripts.

W2: How do the proposal, hyperparameters, and sampling stability affect performance?

The complete-decoder ablation shows that Polar improves accuracy at similar latency. With the remaining settings fixed, Table 4(a) reports:

ProposalMATH500 accuracyTime (s; × Base)
Polar77.6%16.55 (2.01×)
Mass74.4%16.85 (2.04×)
Euclidean75.0%16.82 (2.04×)

Polar gains 3.2 pp over Mass and 2.6 pp over Euclidean. Each proposal uses its corresponding importance correction (Appendix J.3).

Table 15 varies n=1,2,4,8n=1,2,4,8, H=32,96,192H=32,96,192, and α=2,4,6\alpha=2,4,6 one factor at a time: accuracy remains within 75.4%–78.2%, and we retain the original default. Repeated sampling at 100 fixed states gives selection agreement of 63.89%, 75.00%, 80.61%, and 83.39% as nn increases from 1 to 8; log-margin RMS falls from 62.28 to 35.90 (Table 16). These results show improved repeatability with more futures and support the default n=4n=4 cost–stability tradeoff.

W3: Do position shifts improve correctness, and why does SFPS differ from MCMC Power?

We added a direct test of the opening choice. At the first shift in 267 MATH500 problems, both branches use the same saved state and Base continuation policy. Changing only the opening from the initial bb-leader to the future-selected candidate increases accuracy from 60.67% to 63.67% (Table 4(b)).

Across complete decoding, removing future aggregation or column weighting lowers accuracy from 77.6% to 74.0% or 74.2%. SFPS's paired gains are +3.6 pp [1.2, 6.0] and +3.4 pp [1.2, 5.6] (Table 5). These interventions connect the selection mechanism to final-answer correctness.

SFPS uses the likelihood signal to compare candidates over a finite horizon and commit the best-supported opening–continuation pair (Eqs. (5)–(8)), whereas MCMC Power samples toward a sequence-level power target. The proposal and readout ablations identify the choices that improve SFPS's decisions. Unlike MCMC Power's sequence-level MCMC updates, SFPS generates shared futures token by token, with every sampled future token contributing to the comparison and weight updates across all candidate openings.

W4: Why these models, and does SFPS apply beyond math-specialized models?

We use Qwen2.5-Math-1.5B and 7B to test different sizes, and Llama-3.2-3B-Instruct to test a general-purpose model family. On Llama, SFPS improves MATH500, AMC, and HumanEval accuracy from 49.00/25.30/49.39% to 51.60/27.71/50.61% over MCMC Power (Table 3). SFPS uses next-token probabilities while keeping model weights fixed (Algorithm 1), so the decoding procedure also applies to general instruction-tuned models.

Thank you for the additional experiments and detailed responses, which have helped address several of my concerns, particularly W2. Given that your response was posted with only about one day remaining in the discussion period, I understand that there may be limited time for further experiments. I would nevertheless appreciate any brief clarifications the authors could provide on the following points:

  1. Baseline comparisons (W1). The reported Power-SMC results differ considerably from those in the original paper, particularly in latency (1.46× vs. 3.46× Base) and accuracy (76.1% vs. 70.2% on MATH500). The reported SPS latency also appears substantially higher. I recognize that implementation details and evaluation settings may contribute to such differences, and my intention is not to question the fidelity of the implementations. Rather, it would be helpful to understand the likely reasons for these discrepancies and whether they affect the interpretation of the comparisons.

  2. Future evidence and correctness (W3). The new intervention provides useful evidence on MATH500, though I am still uncertain whether future likelihood concentration is a reliable indicator of reasoning quality more generally. On harder or less familiar problems, confidently incorrect reasoning paths might also exhibit highly concentrated future distributions. I would be interested in the authors' thoughts on this potential limitation, particularly whether SFPS's effectiveness might depend on the model's familiarity with the task.

  3. Model selection (W4). To clarify my earlier concern, I was referring less to model sizes or families and more to the choice of two math-specialized Qwen2.5 models, rather than more general instruction-tuned variants (e.g., Qwen2.5-Instruct) or newer models (e.g., Qwen3). Since math-specialized models may exhibit different likelihood distributions and reasoning characteristics, I would appreciate some discussion of the rationale behind this choice and its potential implications for the generalizability of SFPS.

Apologies for the delayed response. We would like to clarify three points:

1. Method and efficiency. SFPS reuses token-level evidence through a shared-future matrix, rather than repeatedly generating and discarding full trajectories. Polar sampling further improves estimation efficiency. On MATH500, SFPS achieves 77.6% with 3,381 sampled tokens, versus SPS's 76.2% with 34,582 tokens.

2. Correctness across tasks. SFPS exploits the frozen base model's reasoning ability through future-informed position shifts. In our controlled test, changing only the opening improves correctness from 60.67% to 63.67%. Improvements also extend beyond mathematics: SFPS outperforms MCMC Power on GPQA (science: 42.42% vs. 35.35%) and HumanEval (code: 60.98% vs. 55.49%). This shows that the gains are not limited to mathematical reasoning.

3. Model selection and generalization. We chose Qwen2.5-Math-1.5B and Qwen2.5-Math-7B to evaluate different model scales and enable comparisons with publicly available GRPO checkpoints. We did not identify comparable public GRPO checkpoints for non-Math Qwen models, so we additionally evaluated Llama-3.2-3B-Instruct as a general-purpose model from a different family. SFPS outperforms MCMC Power on all ten model–task pairs and the tested GRPO checkpoints on eight of ten, showing improvements across both math-specialized and general-purpose models.

Thank you for the clarifications. However, I feel that the response still does not directly address the remaining concerns I raised in my follow-up discussion. Therefore, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.