acceptodds
Under review as a conference paper at ICLR 2027

Path-consistency with Prefix Enhancement for Efficient Inference in LLMs

Abstract

Self-consistency improves reasoning by aggregating multiple sampled solutions, but generating each trajectory from scratch incurs substantial inference cost. We propose path-consistency, an efficient test-time scaling method that reduces this cost through prefix enhancement, a controlled process that progressively extends a shared reasoning prefix across sampling windows and validates each extension with subsequent samples. Answer agreement guides candidate extensions, while trial and recovery determine whether to commit them or roll back to an earlier prefix. All valid answers from completed windows contribute to the final vote. We derive an exact finite-horizon decomposition of the gains and losses in voting accuracy induced by prefix reuse. We also establish a voting error bound for adaptive sampling with finitely many competing answers under explicit conditional-margin assumptions. Our evaluation targets three models of different scales, Llama-3.1-8B-Instruct, Qwen3-30B-A3B, and DeepSeek-V4-Flash, across five reasoning benchmarks. Across diverse models and reasoning benchmarks, path-consistency reduces generated tokens by 25.8% to 42.0%, averaging 34.4% across the long-chain settings, while largely maintaining accuracy. Combining path-consistency with early-stopping methods yields an additional reduction of approximately 20%. These results position path-consistency as a competitive reasoning-sharing method that also integrates flexibly with early-stopping methods for additional savings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language models, Efficient inference, Self-consistency, Prefix reuse, Reasoning, Test-time scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zwLHOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Path-Consistency, a training-free test-time scaling framework that reduces the token generation cost of Self-Consistency by progressively sharing and extending a validated reasoning prefix across sequential sampling windows. The key observation is that independently sampling full trajectories repeatedly generates similar early reasoning steps, which can be amortized through controlled prefix reuse. Specifically, Path-Consistency combines an incomplete-Beta answer agreement gate to trigger prefix extension, a local confidence selection rule that ranks candidate prefixes by their minimum sliding-window token log-probability, and a trial-and-recovery mechanism that rolls back prefixes upon consecutive agreement failures. The authors also provide a theoretical decomposition of voting accuracy and an error bound under conditional-margin assumptions. Experiments across three models and five reasoning benchmarks show that Path-Consistency reduces generated tokens by 25.8% to 42.0% while largely preserving voting accuracy, and achieves an additional 20% token reduction when combined with early-stopping methods.

Critical strengths
  1. The paper identifies substantial early-step redundancy across independently sampled reasoning trajectories and proposes a training-free progressive prefix reuse mechanism with rollback protection. By combining an answer agreement gate, local sliding-window confidence selection, and a trial-and-recovery controller, the method amortizes early reasoning computation while preventing the irreversible lock-in of misleading prefixes.

  2. The proposed approach achieves consistent token reductions on long-chain reasoning models and composes effectively with existing early-stopping techniques. Across three model scales and five benchmarks, Path-Consistency reduces generated tokens by 25.8% to 42.0% with minimal impact on majority-voting accuracy, and provides an additional 20% token saving when integrated with early-stopping methods.

Critical weaknesses

1.The core propose-trial-rollback mechanism closely resembles existing Tree-of-Thoughts and Monte Carlo Tree Search paradigms, yet the paper lacks both conceptual discussion and empirical comparisons against tree-search baselines. Extending a shared high-confidence prefix, evaluating it via multi-path rollouts, and backtracking upon poor outcomes is structurally equivalent to a single-branch depth-first tree search, especially in the sentence-level variant presented in the appendix. However, the authors neither cite foundational Tree-of-Thoughts literature nor compare their method against step-level beam search or MCTS under matched token budgets.

  1. Partitioning a parallel sampling budget into sequential windows reduces generation concurrency and may degrade end-to-end latency in realistic serving environments. While standard Self-Consistency can generate all 32 trajectories in a single parallel batch, Path-Consistency enforces 8 sequential rounds of 4 trajectories where each window must wait for the previous one to finish before updating the prefix. The evaluation only reports single-GPU timing under small batch constraints and does not analyze how this serialization bottleneck affects wall-clock latency and hardware utilization on high-capacity GPUs where 32 trajectories can be decoded concurrently without throughput saturation.
Questions for authors

How does Path-Consistency conceptually and empirically compare against Tree-of-Thoughts, step-level beam search, or rollout-based MCTS methods under an equal token or compute budget? [1][2]

[1] Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models.

[2] Tree of Thoughts: Deliberate Problem Solving with Large Language Models.

2 comments
AuthorsOpenReview ↗

Response to Reviewer zwLH

W1. Relationship to tree search

Thank you for highlighting the connection to Tree of Thoughts and rollout-based search. PC shares proposal, evaluation, and recovery ideas with search and can be interpreted as a restricted prefix-search process. The ToT and Inference Scaling Laws works identified in the review provide the relevant search context.

PC organizes these operations around multi-trajectory voting. Each window uses one committed shared prefix; continuation agreement determines whether an extension is retained or rolled back. All valid answers, including those from unsuccessful trial windows, enter the final majority vote. PC does not maintain a competing frontier or use visit-count-based selection. The search adapters evaluated below maintain explicit candidate states and select a solution using path likelihood or self-evaluation rewards. This locates the contribution in controlled prefix reuse for voting, rather than in proposal or rollback alone.

W2. Sequential windows and serving concurrency

We measured Qwen3-30B-A3B, thinking on, BF16, on two A40s with TP=2 using MATH-500. SC keeps all 32 trajectories of a question resident without OOM. PC uses the same 32-sequence capacity by interleaving up to eight independent questions, four trajectories each; its within-question window dependencies are preserved.

MetricSCPC serving variant
Accuracy (%)97.5597.35
Tokens/question161,05194,565
Timed workload duration (h)54.1233.01
Questions/hour9.0514.84
Tokens/second405.0389.9

The 41.3% generation reduction yields 1.64x question throughput with an accuracy difference of 0.20 percentage points. This serving run uses logprobs=0 and length/source-ID fallback selection; it is distinct from the main-table BLC-ranking configuration. Both arms enable prefix caching, and PC generates its own roots.

SC timing sums decoding-loop wall times (194,837.5 s), excluding loading, prompt construction, and inter-question overhead; PC uses engine-ready-to-completion makespan (118,838.7 s), including scheduling. These scopes are explicitly reported. Mean per-question timed duration is 397.6 s for SC and 1,925 s for interleaved PC. In a separate 32-question isolated schedule, PC takes 1.79x SC's wall time despite 40.9% token savings. The measured benefit is workload throughput when independent requests fill decoding capacity; it does not imply lower isolated-request latency.

Q1. Empirical comparison with ToT, beam search, and MCTS

We completed a comparison on all 1,319 GSM8K test questions with Llama-3.1-8B-Instruct. For each question, the search methods receive a generation-token cap equal to PC's measured usage. Proposed and discarded candidates, evaluator outputs, rollouts, and final completions are all charged.

MethodCorrect / 1,319Accuracy (%)Actual tokens/question
PC, C=0.8121091.745038.7
ToT-BFS, self-evaluation adapter113886.282497.9
Step-level beam116788.482291.9
Rollout MCTS, self-evaluation adapter116388.174526.8

PC has the highest observed accuracy among these configurations, while the search adapters use fewer actual tokens. These are accuracy-cost operating points under equal per-question generation caps; early termination means actual consumption differs. We report both quantities rather than interpret the comparison as equal-realized-token or equal-FLOP dominance.

ToT uses breadth-first expansion with same-model evaluation; beam ranks paths by average cumulative token log-probability; MCTS uses UCT, rollouts, and same-model rewards. The MCTS adapter is not a reproduction of the cited PRM-based system. Sampling uses seed 0, T=0.6, top-p=0.9, and a 2,048-token path limit; reference answers enter alignment and final grading, not search rewards. ToT/beam's 7/6 answer-parsing failures remain incorrect; ToT/MCTS's 49/6 invalid evaluations receive zero reward. All questions are retained. This comparison complements W1 by measuring the outcomes of allocating a bounded generation allowance to prefix-sharing voting or explicit search.

Thank you for the response and the additional experiments.

However, my two main concerns remain largely unresolved. First, regarding W1 and Q1, the added tree-search comparison is not well matched in actual compute. Both ToT-BFS and step-level beam search use less than half of PC's actual tokens, and the self-evaluation adapters with an 8B model are much weaker than standard PRM-guided search methods such as REBASE. Thus, the conceptual novelty over restricted prefix tree search and the empirical advantage under equal realized compute remain unconvincing.

Second, regarding W2, the new A40 serving results directly confirm my concern about serialization latency. Even with 40.9% fewer tokens, isolated-request latency becomes 1.79× slower than SC, and the interleaved serving schedule increases average per-question latency from 397.6 s to 1,925 s while disabling the proposed local confidence selection. This shows that the method trades substantial per-request latency for batch throughput rather than providing general inference acceleration.

Given these remaining limitations, I will maintain my rating of 2.

Reviewer_uwQROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed Path-Consistency (PC); the idea is to try to reuse prefixes on different path to have more efficient self consistency parallel thinking. PC progressively builds a shared prefix that later samples only generate new continuations. They evaluate 3 models across GSM8k, math500, AIME, and GPQA-D, and show comparable accuracy with self-consistency while getting token savings upto 40%.

Critical strengths

I think the general research question is very timely and important, how to be token efficient while retaining capabilities of reasoning models.

Critical weaknesses
  1. Limited contribution and novelty. The main results are in Table 3. On reported benchmarks, SC accuracy is not much different from pass@1. This raises the question of why SC is needed in the first place in these settings. Also, the token reductions can be misleading, since Table 3 compares against an inefficient fixed-budget SC setup. The authors compare with similar methods such as PoLR [1] in Table 4, but only on Llama-3.1-8B and GSM8K. These comparisons should be extended to the main experimental settings. Also general idea of PC looks very close to related works and generic.

  2. Diversity. Sharing prefixes may hurt diversity. However, because pass@1 is already close to SC accuracy in several evaluated settings, it is unclear whether PC actually retains useful diversity or whether the sampled solutions simply offer little useful diversity to begin with. More direct analysis is needed to distinguish these possibilities.

[1] Jindal, Ishan, et al. "The Path of Least Resistance: Guiding LLM Reasoning Trajectories with Prefix Consensus." ICLR 2026.

Questions for authors

Can the authors evaluate another model or benchmark where SC accuracy is significantly higher than pass@1? If PC still retains SC accuracy in that setting, it would provide stronger evidence that the method does not sacrifice too much useful diversity.

1 comment
AuthorsOpenReview ↗

Response to Reviewer uwQR

W1. Contribution, motivation for SC, and baseline budgets

Thank you for asking us to connect the contribution to useful aggregation and stronger comparisons. PC integrates controlled continuous-prefix reuse into multi-trajectory voting. Answer agreement triggers an extension, available confidence ranks candidate prefixes, and a trial-and-recovery controller commits, rolls back, or stops reuse. Valid answers from unsuccessful trial windows still contribute to the final vote. The objective is to reduce repeated generation while retaining sampling opportunities and their voting benefit.

A reduced SC budget provides a direct reference beyond fixed-budget SC-32. The results on Llama-3.1-8B-Instruct/GSM8K are shown below.

MethodAccuracy (%)Generated tokens/question
SC-3292.128,316
SC-2091.285,214
PoLR, Lp=12892.195,863
CPT, Llama adapter, 32 workers92.808,411
PC91.745,039

At these operating points, PC improves on SC-20 by 0.45 percentage points (pp) while using 3.36% fewer tokens. Relative to SC-32, it saves 39.41% with a -0.38 pp accuracy difference. This supports reuse as an alternative to reducing the number of sampling opportunities.

New Qwen/AIME evidence examines offline DeepConf aggregation on the completed SC/PC pools. Across bottom-10, tail, and lowest-group confidence with 10%/90% retention, the SC and PC pools have matching accuracy in 11 of 12 filter-year comparisons; the remaining cell is 24/30 for SC versus 23/30 for PC. This extends the comparison to confidence-based aggregation on long chains. The PoLR/CPT numbers above remain short-chain comparisons; their long-chain evaluation is a separate outstanding comparison.

W2. Useful diversity under prefix sharing

We agree that diversity should be assessed where voting produces a measurable gain. Qwen AIME 2024 has a mean of 2.37 distinct final-answer strings per question for SC and 2.17 for PC; AIME 2025 has 3.73 and 3.07. Both methods retain multiple answer alternatives, while PC preserves the final voting accuracy shown below. Across three DeepSeek AIME calls, the corresponding SC/PC means are approximately 1.08/1.07 in 2024 and 1.16/1.09 in 2025. Final vote strings match on every question in these AIME comparisons.

The evidence combines answer-level diversity, retained aggregation gains, and paired decisions. It supports preservation of useful voting behavior in these settings; distinct-answer counts measure answer diversity rather than the full diversity of reasoning paths.

Q1. Settings where SC improves substantially over Pass@1

Several settings provide this comparison across two model families.

Model / datasetPass@1 (%)SC (%)PC (%)PC token savings
DeepSeek / AIME 2025, original run86.67100.00100.0025.8%
Qwen / AIME 202473.3383.3383.3341.2%
Qwen / AIME 202573.3383.3383.3335.5%
DeepSeek / GPQA-D83.3389.9089.9028.4%

PC retains observed voting gains of 6.57-13.33 pp over Pass@1 while reducing generation. The Qwen gains correspond to 22/30 correct for Pass@1 versus 25/30 for both SC and PC; GPQA-D covers 198 questions. Three independent DeepSeek calls on each AIME benchmark also preserve SC accuracy in every paired run, with token savings averaging 28.75% and 36.04% for 2024 and 2025. These results show that prefix reuse lowers the generation cost of retaining multi-trajectory aggregation gains.

Reviewer_rS6yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes path-consistency (PC), a training-free method that cuts the cost of self-consistency (SC). PC samples in windows of four trajectories that share one reasoning prefix, which grows in steps of 20% across windows.

An answer agreement gate, based on the regularized incomplete beta function, decides when to propose an extension. Local confidence selection (LCS) picks the candidate with the highest minimum sliding-window mean log-probability over its new tokens. A trial window then either commits the extension or returns to the last committed prefix. Two consecutive failures stop reuse for that question. All valid answers, including those from failed windows, enter the final majority vote.

With 32 trajectories on two models and five benchmarks, PC cuts generated tokens by 25.8–42.0% (average 34.4%). Accuracy drops by at most 0.4 points.

Critical strengths
  1. The evaluation covers three model scales, both short and long chain regimes, five benchmarks, and the composition with two early stopping methods. More importantly, the token accounting charges the failed and unprefixed windows in full (Appendix C.3), and it clearly refuses to treat the nominal prefix fractions as the measured savings. This makes the reported token reductions credible as the real token reductions.
  2. Instead of claiming that answer agreement means correctness, the paper proves an exact decomposition but cannot guarantee its sign, and it also constructs the explicit finite-budget cases where the prefix reuse actually harms the accuracy, even the per-trajectory correctness is above 0.5. I think this honesty is valuable.
Critical weaknesses
  1. The baseline (SC) was proposed in 2023. This paper does not compare with other more recent methods that are mentioned in the related work section, so it is hard to know the real position of PC among current works.
  2. The central claim is efficient inference, but it is only supported by the token count. The more decisive metric, which is the wall-clock latency / throughput, is not measured in the regime that really matters, and it is possibly neutral or even worse. The reason is that SC is parallel across all the 32 trajectories, but PC has to serialize the sampling windows, because each window depends on the previous window's outcome and the prefix decision.
  3. The accuracy preservation is only concluded from single runs, and the test sets are too small to support such claim. PC never improves over SC and even degrades a little in several settings. Many entries in Table 3 are numerically identical to SC, and AIME 2024/2025 only have 30 questions each (one question is about 3.3pp), so the "matching accuracy" here is actually within the noise.
Questions for authors
  1. For the long-chain models, what is the wall-clock latency and throughput of PC compared with SC, under the batched serving that can fully use SC's parallelism (not the serial per-question timing)? Does the ~34% token saving really become some real latency / throughput improvement?
  2. Are the accuracies in Table 3 from single runs? What is the variance across different seeds, especially for the 30-question AIME benchmarks? If the authors can report the mean ± standard deviation over several seeds, it will affect a lot how I read the "maintained accuracy" claim.
  3. How does PC compare with the other reasoning-sharing methods (PoLR, CPT) and also DeepConf? Currently these baselines only appear on the short-chain Llama/GSM8K, and DeepConf is never compared in the experiments.
  4. In a paired way, what is the fraction of questions that PC changes compared with SC, and can it be split into “correct->wrong”and “wrong->correct” flips? The aggregate accuracy may hide the offsetting flips that are caused by the appending prefix errors.
1 comment
AuthorsOpenReview ↗

Response to Reviewer rS6y

W1. Comparisons with recent methods

Thank you for asking us to clarify PC's position. The Llama/GSM8K results below compare reuse with both a reduced SC budget and other sharing methods. New Qwen/AIME analyses also evaluate offline DeepConf aggregation (Q3).

MethodAccuracy (%)Tokens/question
SC-3292.128,316
SC-2091.285,214
PoLR, Lp=12892.195,863
CPT, Llama adapter, 32 workers92.808,411
PC91.745,039

PC uses fewer tokens than the other configurations in this comparison. Relative to SC-20, it increases observed accuracy by 0.45 percentage points (pp) with 3.36% fewer tokens. Long-chain PoLR/CPT results are not included in the completed comparison.

W2. Serving efficiency beyond token counts

We evaluated MATH-500 using Qwen3-30B-A3B, thinking on, BF16, two A40s, TP=2. SC decodes 32 trajectories together without OOM; PC fills the same 32-sequence capacity with up to eight independent questions, four trajectories each. PC generates its own roots.

MetricSCPC serving variant
Accuracy (%)97.5597.35
Mean generated tokens161,05194,565
Timed workload duration (h)54.1233.01
Questions/hour9.0514.84
Generated tokens/second405.0389.9

Thus, 41.3% fewer tokens translate into 1.64x question throughput, with an accuracy difference of 0.20 percentage points. The serving run uses logprobs=0 and length/source-ID fallback prefix selection, distinct from the main-table BLC-ranking configuration.

W3. Accuracy preservation and variability

We retain every original run and add two independent DeepSeek repetitions per AIME benchmark at fixed C=0.8, B=32, m=4. PC matches SC's voting accuracy in every repetition, and all six comparisons show token savings. The API repetitions are independent calls, each using its own paired SC roots. This measures run-to-run variability on the same 30 questions per year; repeated calls do not enlarge the benchmark.

Q1. Wall-clock latency and throughput under batching

The result in W2 uses fully resident SC-32. PC maintains about 96% of SC's token throughput while completing more questions per hour. Timing excludes engine loading. SC uses the sum of decoding-loop wall times (194,837.5 s); PC uses engine-ready-to-completion makespan (118,838.7 s), including scheduling. SC's whole-process duration is 194,977 s. We disclose these timer boundaries rather than treat them as identical end-to-end scopes.

Mean per-question timed duration is 397.6 s for SC and 1,925 s for interleaved PC. On a separate 32-question slice with one PC question occupying the device at a time, PC takes 1.79x SC's wall time despite saving 40.9% of tokens. The measured advantage is workload throughput when independent requests fill decoding capacity; individual-request latency remains a separate metric.

Q2. Single runs and repeated-run statistics

The original main table contains single runs. Across three independent DeepSeek calls, SC and PC accuracies are 96.67/100/100% on AIME 2024 and 100/100/100% on AIME 2025.

DatasetSC mean +/- SD (%)PC mean +/- SD (%)Token savings mean +/- SD (%)
AIME 202498.89 +/- 1.9298.89 +/- 1.9228.75 +/- 4.14
AIME 2025100.00 +/- 0.00100.00 +/- 0.0036.04 +/- 8.85

SD is the sample SD; savings are calculated from integer token totals per run, then averaged across runs. The paired accuracy gap is 0.00 +/- 0.00 pp for both datasets, and every run saves at least 25.8% of tokens.

Q3. PoLR, CPT, and DeepConf

W1 reports PoLR/CPT on Llama/GSM8K. On Qwen/AIME, offline DeepConf uses bottom-10, tail, and LGC at 10%/90% retention, six configurations per year.

DatasetEqual SC/PC accuracyDiffering configuration, SC / PCPC pool token savings
AIME 20246/6None41.2%
AIME 20255/6LGC-10, 80.00% / 76.67%35.5%

PC preserves filtered accuracy in 11/12 configurations with lower pool-generation cost. Raw majority voting gives 83.33% for both pools in both years. Mean confidence is outside this quantile comparison. Savings refer to generating the pools; offline filtering itself does not stop generation.

Q4. Paired changes

On Qwen/MATH-500, correct-to-wrong changes affect 0.20% of questions, with no wrong-to-correct changes. Vote strings match on 99.39% of questions. Of the three changed strings, two are different wrong answers and one changes from correct to wrong. On both Qwen AIME sets, all three DeepSeek runs of each AIME set, and DeepSeek/GPQA-D, vote strings and correctness states match throughout. These paired results distinguish preserved decisions from cancellation of gains and losses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.