acceptodds
Under review as a conference paper at ICLR 2027

Shared-Prefix Self-Consistency

Abstract

Self-consistency improves reasoning by sampling complete reasoning traces in- dependently and aggregating their answers by majority vote, but each addi- tional trace requires another full generation. We introduce Shared-Prefix Self- Consistency (SP-SC), a simple variant that generates complete reasoning trunks, retains a prefix from each, and samples additional continuations. Original trunks and continuations contribute equally to the final vote. Using Qwen3-14B on AIME 2025 and HMMT 2025, we examine how prefix length, trunk count, and continuation budget affect accuracy and generated-token cost. Offline evaluation over fixed generation pools shows that selected configurations improve the ob- served trade-off relative to standard, adaptive, and early-stopping self-consistency. For example, on AIME 2025, an eight-candidate configuration achieves 81.72% accuracy using 24.5% fewer generated tokens than eight-sample self-consistency, which achieves 79.44%. Longer prefixes can reduce generation cost while low- ering accuracy; multiple trunks mitigate this sensitivity in several evaluated con- figurations. A separate rescue analysis shows that erroneous traces can contain prefixes from which correct continuations emerge.

Primary area:
generative models
Keywords:
Efficient inference, reasoning language models, Test-time scaling, Self-consistency, Chain-of-thought reasoning, Shared-prefix sampling, Large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tDTzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Shared-Prefix Self-Consistency (SP-SC). The method first generates one or more complete solutions and retains a prefix from each solution. It then generates new continuations from these prefixes. Finally, all candidate solutions are combined through majority voting, with original solutions and generated continuations receiving equal voting weight. The authors use Qwen3-14B to investigate the effects of prefix length, the number of independent reasoning trunks, and the number of candidate solutions on AIME 2025 and HMMT 2025. Under certain configurations, SP-SC achieves a better trade-off between accuracy and token consumption than standard Self-Consistency, Adaptive Self-Consistency, and Early-Stopping Self-Consistency. In addition, the authors conduct a separate experiment showing that even when a complete solution produces an incorrect final answer, its reasoning prefix may still lead to correct answers through alternative continuations.

Critical strengths
  1. The method is simple and easy to implement. It can be directly applied to existing language models without additional training or a separate verification model.

  2. The method demonstrates effectiveness and practical value. On AIME 2025, the proposed method with eight candidates achieves an accuracy of 81.72%, with an average generation cost of 98.7K tokens per problem. In comparison, standard Self-Consistency with eight independent complete solutions achieves 79.44% accuracy while consuming 130.6K tokens per problem. These results demonstrate that the proposed method can reduce token consumption while maintaining or slightly improving accuracy.

  3. The paper includes comparative experiments and ablation studies. The authors compare SP-SC with several existing Self-Consistency methods and investigate the effects of important design choices, including prefix length and the number of reasoning trunks.

Critical weaknesses
  1. The novelty of the proposed method is somewhat limited. Previous work, such as Path-Consistency, has already explored using reasoning prefixes to guide subsequent generations. The authors should further clarify what SP-SC contributes beyond these existing approaches and identify its main technical novelty.

  2. The experimental results may depend heavily on a small number of selected reasoning trunks. In Sections 5.3–5.4, the baseline methods sample from a pool of 316 complete solutions per problem, while SP-SC uses only four designated reasoning trunks. The single-prefix setting further relies on one fixed trunk.The 500 trials are conducted by repeatedly resampling from a fixed pool of previously generated results, rather than generating new independent reasoning trunks.Therefore, the experiments do not sufficiently demonstrate whether the reported performance would remain stable with different sets of reasoning trunks. It is difficult to determine how much of the improvement comes from prefix reuse itself and how much comes from the quality of these selected trunks. It is possible that the chosen trunks happen to perform particularly well.

  3. The prefix rescue experiment does not fully establish the benefit of prefix reuse. The experiment shows that incorrect solutions may contain useful reasoning prefixes. However, generating 20 continuations for each prefix already provides multiple additional opportunities to find the correct answer. Moreover, it remains unclear whether reusing these prefixes is more efficient than generating entirely new solutions from scratch.I suggest adding a baseline that generates independent complete solutions under the same token budget. This comparison would help determine how much improvement comes specifically from prefix reuse. Additionally, reporting the final majority-voting accuracy in the rescue experiment would help demonstrate whether the correct continuations actually improve the final prediction.

Questions for authors
  1. The novelty of the proposed method is somewhat limited. Previous work, such as Path-Consistency, has already explored using reasoning prefixes to guide subsequent generations. The authors should further clarify what SP-SC contributes beyond these existing approaches and identify its main technical novelty.

  2. The experimental results may depend heavily on a small number of selected reasoning trunks. In Sections 5.3–5.4, the baseline methods sample from a pool of 316 complete solutions per problem, while SP-SC uses only four designated reasoning trunks. The single-prefix setting further relies on one fixed trunk.The 500 trials are conducted by repeatedly resampling from a fixed pool of previously generated results, rather than generating new independent reasoning trunks.Therefore, the experiments do not sufficiently demonstrate whether the reported performance would remain stable with different sets of reasoning trunks. It is difficult to determine how much of the improvement comes from prefix reuse itself and how much comes from the quality of these selected trunks. It is possible that the chosen trunks happen to perform particularly well.

  3. The prefix rescue experiment does not fully establish the benefit of prefix reuse. The experiment shows that incorrect solutions may contain useful reasoning prefixes. However, generating 20 continuations for each prefix already provides multiple additional opportunities to find the correct answer. Moreover, it remains unclear whether reusing these prefixes is more efficient than generating entirely new solutions from scratch.I suggest adding a baseline that generates independent complete solutions under the same token budget. This comparison would help determine how much improvement comes specifically from prefix reuse. Additionally, reporting the final majority-voting accuracy in the rescue experiment would help demonstrate whether the correct continuations actually improve the final prediction.

1 comment
AuthorsOpenReview ↗

Clarifications and expanded experiments explained.

We thank the reviewer for the careful reading and concrete suggestions. We expanded the evaluation and clarify below the distinction from Path-Consistency, the role of trunk selection, and the comparison with independent sampling.

1. Novelty relative to Path-Consistency

Our novelty is the use of prefixes from unfiltered reasoning paths to obtain additional votes, supported by the observation that a trunk with an incorrect final answer can still provide a productive prefix. SP-SC therefore requires no confidence threshold or answer-based ranking to decide which trunks to reuse.

Path-Consistency selects prefixes from paths supporting the leading answer, with its thresholded version waiting until an agreement-based confidence criterion is met. SP-SC samples independent trunks, retains fixed fractions of them, and generates alternative continuations from multiple prefixes. Original trunks and continuations contribute equally to the final vote. This directly studies whether prefix reuse can improve the accuracy–token trade-off without first identifying the most promising paths.

Section 3 provides the motivating evidence. Prefixes from incorrect trunks can produce correct continuations, including groups with at least ten successes out of twenty. The new Gemma HMMT analysis gives another example, where a trunk answering "2/27" incorrectly produces 12/12 correct continuations having the answer "4/9" from its 20% (tau = 0.2) prefix. These observations motivate exploiting alternative outcomes from a prefix even when its original completed solution fails.

2. Trunk variability and expanded evaluation

We agree that four designated trunks, particularly one fixed single-prefix trunk, limited the original evidence. We added Gemma 4 12B IT, from a different model family, on all 30 problems in both AIME 2025 and HMMT 2025. The updated setting generates 100 independent full responses per problem, selects eight without correctness-based filtering, and generates 12 continuations per trunk at each of four prefix fractions. Sections 5.3–5.4 describe the protocol, and Section 6 and the appendix report the results.

For each of 500 trials, the Gemma replay shuffles the eight trunks and selects S without replacement. Single-prefix and smaller multi-prefix settings therefore vary their starting trunks, while S=8 uses all eight. Continuations are also resampled. Averaging over all questions and these trials reduces dependence on any one selected path, although the trials still quantify variability within saved pools. Favorable operating points occur in both Qwen benchmarks and Gemma AIME, covering three of the four model–dataset pairs.

We also report Gemma HMMT’s weaker result and examine problems 20, 29, and 30 in Section 6. Problems 20 and 29 have no correct selected trunk and five missing-answer trunks each. Problem 30 has one correct trunk, three missing-answer trunks, and two matching wrong votes. As a diagnostic, excluding these same three problems for both methods gives SP-SC (.2,8,32) 99.80% accuracy at 812.8K tokens per problem, versus SC N=32 at 98.89% and 846.0K. This illustrates the concentration of failures. The main benchmark results retain all 30 problems.

3. Comparison with independent generation

The blue SC points in Section 6 implement the requested independent-generation baseline. Every SC candidate is a complete solution generated independently from the question. SP-SC instead spends its budget on complete trunks and their continuations. Both methods charge all consumed generation, including incorrect and missing-answer outputs.

A vertical comparison through the graph evaluates their final-vote accuracies at the same mean generated-token budget on the horizontal axis. Since the evaluated SC budgets are discrete, we compare against SC settings at equal or lower mean cost. This shows whether spending tokens on reused prefixes is beneficial compared with spending those tokens on independent complete solutions. Selected SP-SC settings improve this trade-off in three of the four model–dataset pairs, with accuracy and cost averaged over all problems and 500 trials.

The rescue experiment shows that prefixes from incorrect trunks can still produce correct continuations, including groups with at least ten successes out of twenty. The main experiments provide the comparison against independent generation at measured token costs. Rescue supplied the initial motivation for SP-SC by showing why reuse can be useful even without identifying and grouping high-confidence paths beforehand. Its recoverability metrics and the main experiments’ final-vote accuracy answer complementary questions.

Reviewer_QpKyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Shared Prefix Self-Consistency (SP-SC), a training-free variant of self-consistency (SC) for LLM reasoning. Instead of generating every vote from scratch, SP-SC generates one or more full reasoning trunks, retains a prefix from each trunk, and samples several continuations from the retained prefixes. All completed trunks and continuations contribute to a common majority vote. The authors evaluate SP-SC on Qwen3-14B across AIME 2025 and HMMT 2025, showing that it improves the accuracy–token Pareto frontier relative to standard SC, early-stopping SC, and adaptive SC, especially under low token budgets. They also present an oracle “rescue” experiment showing that prefixes from incorrect trunks can still produce correct continuations, motivating prefix reuse even when the original trunk fails.

Critical strengths
  1. Sharing early reasoning prefixes is a natural way to reduce the cost of obtaining multiple votes. The method is easy to understand, training-free, and compatible with existing sampling infrastructure.

  2. The paper sweeps prefix fraction, number of trunks, and candidate budget, and reports Pareto frontiers on two competition mathematics benchmarks. The empirical finding that prefix reuse helps most under tight token budgets is plausible and well illustrated.

  3. The paper correctly identifies that aggressive prefix sharing reduces diversity, and shows that using multiple independent trunks recovers much of the lost accuracy. The single- vs. multi-prefix comparison is useful.

  4. Table 1 shows that prefixes from incorrect traces can still admit correct continuations. This challenges the assumption that incorrect rollouts are necessarily wasted and motivates prefix reuse as complementary to pruning-based efficiency methods.

  5. The use of saved traces, 500 resampling trials, and explicit token accounting is reasonably rigorous for a controlled comparison. The paper is transparent that the trials measure sampling variability over fixed pools rather than new generation runs.

Critical weaknesses
  1. The main experiments use a single model (Qwen3-14B) and only two benchmarks (AIME 2025 and HMMT 2025), each with 30 problems. The results are based on pre-generated saved traces and continuations, not live inference. It is therefore unclear how SP-SC behaves in an actual serving system with prefix caching, batching, and varying generation lengths. The paper acknowledges this, but the empirical evidence is still limited.

  2. The paper compares against SC, ASC, and ESC, but not against recent methods such as DeepConf, Slim-SC, or other trace-pruning/branching approaches. The authors mention DeepConf in related work and note that selected SP-SC configurations use fewer tokens than DeepConf’s warmup, but this is not a direct empirical comparison. Without such baselines, the claimed Pareto improvement is only relative to a subset of efficient SC methods.

  3. Prefix sharing is a straightforward idea, and the paper is primarily an empirical study of a simple modification. The rescue analysis is insightful, but it is not used to design a selective prefix-reuse policy; SP-SC simply reuses arbitrary trunks. The method’s contribution would be stronger if it included an adaptive rule for choosing trunks, prefix lengths, or continuation budgets.

Questions for authors
  1. Online implementation: Were the saved continuations generated by conditioning on the retained prefix, or were they sampled independently and then paired with prefixes? Please clarify the generation protocol. How would SP-SC be implemented online, and what is the expected latency/KV-cache benefit?

  2. Comparison to DeepConf/Slim-SC: Can the authors provide a direct comparison to at least one recent efficient SC method (e.g., DeepConf or Slim-SC) under the same token budget and model? If not, can they explain why the comparison is infeasible?

  3. How were the incorrect trunks selected for Table 1? Were they randomly sampled among incorrect trunks, or selected based on length/confidence? Does the rescue rate depend on trunk length, and could this bias the comparison across milestones?

1 comment
AuthorsOpenReview ↗

Comparisons with recent efficient methods and generation details.

We thank the reviewer for recognizing the value of prefix reuse and suggesting stronger evaluations. We added Gemma results and address online execution, recent baselines, and rescue selection below.

1. Expanded evaluation and online implementation

We added Gemma 4 12B IT, as a different model family, on AIME 2025 and HMMT 2025 for SP-SC. The updated setting uses eight designated trunks per problem, compared with four originally, and 12 continuations per trunk at each of four prefix fractions. Across 500 trials, single-prefix and smaller multi-prefix configurations vary their selected trunks. Sections 5.3–5.4 describe the protocol, and Section 6 and the appendix report the results.

The continuations were generated conditionally on the exact retained prefix token IDs appended to the original prompt. They were never generated independently and then paired with prefixes. The same procedure can run online. With fractional cuts, each full trunk must finish before its branching point is known. Shared prefixes offer opportunities to reuse resident KV-cache states and batch sibling continuations, while suffixes still require generation and cache space. Measuring end-to-end latency, throughput, and memory in a serving system is a valuable next step. Our current efficiency measurements count generated tokens.

2. DeepConf, Slim-SC, and implementation requirements

Section 5.2 and the figures now include a DeepConf warmup reference. Its default online version generates 16 complete traces per problem to set its pruning threshold, and its main efficiency table compares against SC with 512 traces. We estimate the default warmup cost using SC N=16 under our generation settings. The vertical lines correspond to approximately 261.4K and 287.7K tokens for Qwen on AIME and HMMT, and 456.0K and 495.0K for Gemma. Selected SP-SC settings finish below these estimates. This is an initialization-cost comparison. DeepConf accuracy and total online cost remain unmeasured in our setting. Our archived outputs lack the per-token confidence information needed to reconstruct DeepConf’s online pruning. A direct comparison therefore requires additional inference with confidence collection and pruning enabled.

Our direct ESC comparison already shows favorable operating points. On Qwen AIME, SP-SC (.5,4,8) achieves 81.72% at 98.7K tokens, versus 80.62% at 643.1K for ESC (8,64), giving 1.10 percentage points higher accuracy with 84.7% fewer generated tokens. Section 6 and the appendix give the broader curves and variability.

Slim-SC's Table 1 provides additional context. On QwQ-32B/AIME 2024, Slim-SC (DP) reports 84.4% accuracy at 104K tokens, versus ESC's 80.0% at 47K. ESC uses fewer tokens in their reported settings, while Slim-SC reports lower latency. The comparison through ESC suggests a possible SP-SC token-cost advantage over Slim-SC, but differences in models, datasets, generation settings, and variability prevent a direct ranking.

SP-SC uses ordinary conditional generation and voting. It requires no auxiliary embedding model, active-thought similarity index, or confidence threshold for prefix selection. Slim-SC embeds intermediate thoughts and searches a FAISS index to prune similar chains. Path-Consistency iteratively refines prefixes using answer agreement. These are differences in implementation requirements, with serving costs requiring measurement. A direct comparison under the same model and generation conditions remains open.

3. Rescue selection and adaptive prefix reuse

Rescue trunks were selected without confidence ranking or a preference for shorter or longer traces. They just had to be long enough to supply the prefix at the relevant milestone.

The selected incorrect trunks cover a broad range of lengths, from 10,489 to 29,902 tokens on AIME and 6,938 to 29,939 on HMMT. Successful recovery occurs across this range. For example, on AIME problem 24, the original incorrect solution contains 10,489 tokens, and branching from its first 100 tokens produces 15/20 correct continuations. On AIME problem 25, the original incorrect solution contains 26,070 tokens, and its first 100 tokens produce 20/20 correct continuations. On HMMT, the same 100-token cut produces 19/20 correct continuations from problem 28’s 29,568-token solution, but none from problem 19’s 29,939-token solution. These examples illustrate variation in recovery across trunks of different lengths.

We deliberately designed SP-SC around a fixed sampling procedure, making its simplicity part of the contribution. It achieves useful accuracy–token trade-offs without an embedding model, similarity search, or confidence threshold for prefix selection. We agree that adaptive selection of trunks and prefix lengths is a promising extension and identify it as future work alongside online serving measurements.

Reviewer_tbu7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Shared-Prefix Self-Consistency (SP-SC), which reduces generation cost by sharing the beginning of reasoning traces. It generates complete traces as trunks, retains a prefix from each, and samples additional continuations. The original trunks and continuations then vote equally on the final answer. Experiments investigate how prefix length, trunk count, and continuation count affect accuracy and generated-token consumption.

Critical strengths
  1. Simple and potentially useful approach. SP-SC offers an intuitive way to reduce repeated generation while retaining multiple candidate answers. Using several independent trunks provides a natural trade-off between prefix reuse and diversity.
  2. Useful configuration analysis. The paper examines prefix length, trunk count, and continuation budget, with comparisons against standard, adaptive, and early-stopping self-consistency. It also transparently explains the scope of its token accounting and offline resampling.
Critical weaknesses
  1. The results depend on a small fixed set of trunks. Each problem has only four designated SP-SC trunks, and single-prefix evaluation always uses the designated first trunk. In contrast, the baselines sample from 316 independent complete traces. Although the trunks are not selected by correctness, this asymmetry leaves uncertainty about whether the gains reflect the method or the particular trunk sample. Repeating continuation resampling 500 times does not establish robustness to fresh trunks.
  2. The reported improvements lack independent configuration validation. The paper sweeps 80 SP-SC configurations per dataset and highlights favorable settings on the same benchmarks. Evaluation covers one model and two datasets of 30 problems each. The reported variability measures sampling from fixed pools, rather than uncertainty over new generations or problems. Consequently, the results support an exploratory finding more strongly than a reliable, general improvement over self-consistency.
Questions for authors
  1. Do the gains persist across independently generated trunk sets, and how do the designated trunks perform without additional continuations?
  2. Can prefix length and sampling configuration be selected on development data and evaluated unchanged on held-out problems?
  3. Paper mentioned and analyzed DeepConf, but didn't used it as a baseline. Would it be possible to show, even illustratively, the difference compared with DeepConf?
1 comment
AuthorsOpenReview ↗

Expanded evaluation and stronger controls

We thank the reviewer for clearly identifying the trunk-selection and configuration-validation controls. We expanded the evaluation and address each question below.

1. Trunk selection and trunk-only performance

We agree that four designated trunks, with one fixed trunk for single-prefix evaluation, limited the original evidence. We added Gemma 4 12B IT as a different model family on both 30-problem benchmarks and doubled the available trunk pool from four to eight for all Gemma SP-SC configurations. Each configuration uses its specified number of trunks from this expanded pool. The updated setting also uses a 100K context and 12 continuations per trunk at each of four prefix fractions. Sections 5.3–5.4 describe the protocol, and Section 6 and the appendix report the results.

In each of 500 trials, single-prefix and smaller multi-prefix configurations select from the eight trunks without replacement and resample their continuations. The designated first-trunk restriction is therefore removed for Gemma. Trunks are selected without correctness-based filtering, and benchmark accuracy averages equally over all 30 questions. The observed gains span both original Qwen benchmarks and Gemma AIME. This broadens the evidence beyond individual favorable paths.

The trunk-only controls and individual continuations provide evidence beyond favorable starting answers. On Qwen HMMT, voting over the four designated trunks alone achieves 54.10%, while SP-SC (.4,4,28) reaches 62.45%, with additional generation cost. On Qwen AIME problem 18, a trunk incorrectly answering "50" produces 12/12 correct continuations answering "82" from its 30% prefix. On Gemma AIME problem 23, a trunk incorrectly answering "600" produces 11/12 correct continuations answering "610" from its 30% prefix. On Gemma HMMT problem 19, an incorrect "2/27" trunk produces 12/12 correct continuations answering "4/9" from its 20% prefix. These examples show how alternative continuations can add correct votes even when the original trunk ends incorrectly.

2. Development selection and held-out evaluation

We agree that selecting a configuration on development problems and evaluating it unchanged on held-out problems would strengthen the evidence. Given the rebuttal timeline and our compute budget, we prioritized completing the second-model evaluation and expanded trunk sampling. We have not completed a separate held-out configuration evaluation, and this remains future work. Adaptive selection of prefixes, trunks, and continuation budgets is also a promising future direction.

3. DeepConf and efficient SC comparisons

We added an illustrative comparison with DeepConf by plotting its estimated default warmup cost as a vertical reference in the graphs, as described in Section 5.2. Its default online algorithm generates 16 complete traces per problem to set its pruning threshold, and its main efficiency table compares against SC with 512 traces. We estimate this initial cost using SC N=16 under our generation settings. The references are approximately 261.4K/287.7K tokens for Qwen AIME/HMMT and 456.0K/495.0K for Gemma. Selected SP-SC operating points lie to their left. This compares against the estimated default initialization cost. DeepConf accuracy and total online cost remain unmeasured.

Our direct ESC comparison provides a measured reference. On Qwen AIME, SP-SC (.5,4,8) achieves 81.72% at 98.7K tokens, versus ESC (8,64) at 80.62% and 643.1K, giving 1.10 percentage points higher accuracy with 84.7% fewer generated tokens. Section 6 and the appendix provide the curves and variability. Slim-SC, another recent efficient SC method, embeds intermediate thoughts and uses similarity search to prune redundant reasoning chains. Its Table 1 reports QwQ-32B/AIME 2024 accuracy of 84.4% at 104K tokens, versus ESC's 80.0% at 47K. ESC uses fewer generated tokens in those settings, while Slim-SC reports lower latency. The ESC comparisons suggest a possible SP-SC token-cost advantage over Slim-SC, although differences in models, datasets, generation settings, and variability prevent a reliable direct ranking.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.