acceptodds
Under review as a conference paper at ICLR 2027

Where to Recompute: Learning Recoverability for Compute-Efficient LLM Reasoning

Abstract

Test-time computation strengthens large language model reasoning only when the additional work changes the answer that the system ultimately returns, so deciding where and how much to recompute is central to compute-efficient inference. Existing methods face two challenges in making this decision. First, they target candidate correctness, continuation progress, or search coverage, yet a frozen selector can reject a repair or let a higher-scoring error replace a correct answer, so better generation does not by itself yield a better returned answer. Second, question-level allocation does not directly estimate the post-selection value of retaining a particular prefix and assigning it a continuation budget. To address the first challenge, we propose Recoverability-Aware Value of Computation(RAVEC), which learns the post-selection repair and damage of each prefix intervention from randomized same-parent interventions with a frozen generator and selector, and we prove a selector-gating identity under which changing only the selector can reverse the optimal intervention. To address the second challenge, we develop finite-horizon control over reopening positions, suffix budgets, and stopping, prove that path information improves on the best coarser policy exactly when its decision value exceeds estimation regret, and sharpen within-parent action ordering through parent-centered fitting without additional rollouts. Across 14 open models and 12 benchmarks, RAVEC improves core macro accuracy over adapted persistent-pool search (PB-SMC) by 1.34 points on average at equal expected operations, and at matched accuracy it reduces computation by 20.8% on MATH-500 and 25.4% on LiveCodeBench.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, inference-time scaling, adaptive computation, test-time compute, reasoning, verifier-guided search, reinforcement learning, sequential decision making, mathematical reasoning, code generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_A3MpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Recoverability-Aware Value of Computation (RAVEC), a learning-based controller for compute-efficient test-time reasoning. Instead of optimizing candidate correctness or search coverage, RAVEC estimates the expected improvement in the final selected answer after recomputing a reasoning suffix. Specifically, it learns the probabilities of repairing an incorrect answer and damaging a correct answer through randomized interventions on the same parent state. The controller then uses finite-horizon value learning to jointly determine where to recompute, how much computation to allocate, and when to stop. Experiments across multiple models and benchmarks demonstrate improvements in accuracy and computational efficiency over existing test-time search methods.

Critical strengths
  1. The paper addresses an important and interesting problem in compute-efficient LLM reasoning: determining where and how much computation should be allocated to improve the final answer.
  2. The proposed solution is intuitively reasonable. The design of the controller training procedure is well-motivated, including the use of randomized same-parent interventions to collect training data, separate prediction of repair and damage probabilities, and finite-horizon value learning to account for future computation benefits.
  3. The empirical evaluation is comprehensive. The paper evaluates the proposed method across 14 models and 12 benchmarks, covering different types of reasoning tasks. Multiple baselines and extensive ablation studies are included.
Critical weaknesses
  1. A concern with the proposed method is that the proposed controller relies on intermediate hidden representations extracted from the generator. This limits its applicability to API-based LLMs.
  2. The paper motivates RAVEC by highlighting the limitations of a fixed selector, which may reject correct candidates or prefer incorrect ones. However, given the substantial offline computation required to train the proposed controller, comparable resources could instead be invested in training a stronger outcome reward model or using a process reward model for final answer selection, which may directly mitigate the selection errors that motivate the method. The paper does not provide a compute-matched comparison with improved selection strategies, leaving it unclear whether the proposed approach is the most effective use of the additional computation.
  3. Although the proposed method demonstrates computational savings in terms of operation counts, the paper does not report actual wall-clock inference latency. Since RAVEC performs sequential reopenings while BoN can be fully parallelized, operation savings may not translate into latency savings in practice.
Questions for authors

For Qwen3.8-27B, R1-Distill-Qwen-14B, and Gemma-3-12B, the “original fit” values in Figure 9 differ from those reported in Table 1 (e.g., 99.0 vs. 99.3 on MATH-500 for Qwen3.8-27B), while the Qwen3-8B results match exactly. Were these results obtained from separate controller fits or different experimental settings?

4 comments
AuthorsOpenReview ↗

Response to Reviewer A3Mp (1/2): Fig. 9 reference fit and measured latency

Thank you for the positive assessment and for checking Fig. 9 so carefully. We understand your direct question to be the Fig. 9 / Table 1 difference; we address it first.

Summary of changes for you:

  1. The Fig. 9 heading and caption and App. J.4 now state where the reference fit comes from.
  2. Measured latency and throughput, including a parallel-2 variant (Sec. 7.1; App. K.6.1, Table 36).
  3. A controller without hidden states (App. L.1, Table 38; part 2/2).
  4. A compute-matched controller/selector allocation (App. L.2, Table 39; part 2/2).

Fig. 9 "original fit" values differ from Table 1 for Qwen3.8-27B, R1-Distill-Qwen-14B and Gemma-3-12B, while Qwen3-8B matches. Separate controller fits or different settings?

Short answer: they come from separate controller fits; the experimental settings are the same. Table 1 reports the primary-matrix fits. The reference column of Fig. 9 is a fit trained within the centered-fitting experiment, with the same fitting procedure and evaluation settings, so the two differ only in the fitted controller instance. Each signed change in Fig. 9 is computed against that experiment's own reference fit.

The headline accuracy and efficiency results (Table 1, Figs. 2–3, Table 27) do not use Fig. 9. The centered-fitting claim rests primarily on within-parent action ranking: centered fitting reduces Qwen3-8B's math/code action regret from 2.2 / 2.7 to 1.4 / 2.2 and raises τb\tau_b from 0.40 / 0.41 to 0.48 / 0.46 (App. J.4, L2239–2241).

We agree that the label "original fit" was ambiguous, since App. J.4 also used "original 2.4C02.4C_0 matrix" for Table 1. In the paper: the Fig. 9 heading now reads "Reference fit (this experiment) → centered fit", and the caption and App. J.4 (L2236–2248) state that it is a fit instance separate from Table 1, trained with the same procedure and settings.

No wall-clock latency; RAVEC reopens sequentially while BoN is fully parallel.

Short answer: you are right that parallel BoN remains fastest for a single request, and Sec. 7.1 and Limitations now say so explicitly. In concurrent serving, where GPU time sets the cost, RAVEC has the highest throughput.

At matched accuracy (General Response, G2), the single-request p50 is 12.8 / 22.6 s for BoN versus 17.6 / 30.1 s for RAVEC. RAVEC is the fastest of the measured sequential methods: full-pool and adapted PB-SMC take 18.5 / 31.8 s and 19.8 / 33.6 s. Serial depth is shallow: at 2.4C02.4C_0, RAVEC stops after the initial answer on 46.8% / 29.5% of questions and averages 1.31 / 1.82 reopenings (Table 12).

At concurrency 64, RAVEC serves 21.90 / 15.04 questions per minute versus 12.17 / 8.82 for BoN (1.8× / 1.7×), because BoN needs 3.80 / 4.00 C0C_0 to reach the same accuracy.

For latency-sensitive use, executing the top-2 actions in parallel (RAVEC-parallel-2) lowers p50 to 15.1 / 26.3 s at the same matched accuracy, using 1.85 / 2.18 C0C_0; its throughput, 20.76 / 14.15 q/min, remains well above BoN's. In the paper: Sec. 7.1 (L431–465), App. K.6.1 (Table 36), Limitations.

AuthorsOpenReview ↗

Response to Reviewer A3Mp (2/2): controller without hidden states; compute-matched selector comparison

Reliance on hidden states limits applicability to API-based LLMs.

Short answer: hidden states are not essential. Replacing them with external text embeddings retains about three-quarters of the gain on math and code.

In a controlled test, we treated Qwen3-8B as a black box and replaced the four hidden-state projections with Qwen3-Embedding-0.6B embeddings of the question, retained prefix, incumbent and pool. The 20 scalar features (scores, positions, history, resources) already need no internals. Encoder cost is charged within 2.4C02.4C_0; we used five fits and 32 paired replicates.

Controller inputNeeds internalsMATH-500LCBGPQA
PB-SMCNo97.354.459.2
RAVEC, hidden statesYes98.856.458.7
RAVEC, text encoder + scalars (new)No98.47 ± 0.1755.91 ± 0.3058.76 ± 0.25
RAVEC, scalars only (new)No97.86 ± 0.2555.16 ± 0.3758.42 ± 0.28

Takeaway: without internal access, RAVEC retains 78.0% (MATH-500) and 75.5% (LCB) of its gain over PB-SMC; even scalars alone stay above PB-SMC on both. We agree that the remaining interface requirement matters: the controller still needs prefix continuation from a chosen checkpoint, and this proxy does not test a commercial API; Limitations now states both. In the paper: App. L.1 (New), Table 38.

Compute-matched comparison with improved selection (stronger ORM or PRM).

Short answer: a stronger selector helps, and it complements RAVEC rather than replacing it. Under the same offline budget, splitting compute between RAVEC and a stronger selector beats giving all of it to the selector on MATH-500 and LCB, and ties on GPQA.

The offline cap is 307.8 GPU-hours, RAVEC's measured method-specific cost. Every row uses the same 2.4C02.4C_0 online budget in the original 0.6B-ORM unit, with all larger-selector and PRM scoring charged online. This shared unit leaves PRM rows less generation budget than the selector-specific C0C_0 in our reply to mXdf, hence 98.09 here versus 98.37 there. "Scaled 4B ORM" is a new 4B ORM trained with the remaining 265.2 GPU-hours, distinct from the Table 7 checkpoint.

Use of the 307.8 GPU-h offline budgetMATH-500LCBGPQA
PB-SMC + 0.6B ORM (42.6 GPU-h used)97.354.459.2
PB-SMC + PRM final selection (existing checkpoint)98.09n/an/a
PB-SMC + scaled 4B ORM (all to selector)98.2457.4361.58
RAVEC + 0.6B ORM (all to controller)98.856.458.7
RAVEC + 4B ORM (184.7 controller / 123.1 selector GPU-h)98.6358.1661.69

Takeaway: as you anticipated, a stronger selector is valuable, especially on code and GPQA, but it does not substitute for RAVEC; the best use of the budget is a split between the two.

  • The joint allocation leads the selector-only allocation by 0.39 / 0.73 points on MATH-500 / LCB and ties on GPQA (+0.11), although its selector receives less than half the compute.
  • On MATH-500, the controller-only allocation is best.

This is consistent with Proposition 1: a better selector changes which reopenings are valuable but does not remove the need to choose them. With a fixed 4B selector, Table 7 likewise shows refitted RAVEC at 98.4 / 58.0 versus 98.0 / 57.0 for PB-SMC. In the paper: App. L.2 (New), Table 39.

We hope these results address your concerns. Please let us know if anything remains unclear; we are happy to clarify further.

Reviewer_A3MpOpenReview ↗

Thank you for your rebuttal

Thank the authors for the rebuttal. The response addresses some of my concerns, and I will maintain my positive assessment of the paper.

AuthorsOpenReview ↗

Thank you; how the revised paper states the remaining limitations

Thank you for the follow-up and for maintaining your positive assessment. Because our response resolves some of your concerns only in part, we briefly note how the revised paper states each remaining limitation, so that its scope is explicit.

  • Single-request latency. Parallel BoN remains faster for a single request: at matched accuracy, its median latency is 12.8 / 22.6 s on MATH-500 / LCB, against 17.6 / 30.1 s for RAVEC. RAVEC's advantage lies in GPU time and concurrent throughput, with 21.90 / 15.04 questions per minute against 12.17 / 8.82 for BoN at concurrency 64. Sec. 7.1 and the Limitations state this.
  • API-only models. With text embeddings in place of hidden states, RAVEC keeps 78.0% / 75.5% of its gain. It still needs prefix continuation from a chosen checkpoint, however, and our black-box proxy does not test a commercial API (App. L.1; Limitations).
  • Stronger selectors. We agree that a stronger selector helps, especially on code and GPQA. With the offline budget fixed at 307.8 GPU-hours, splitting it between controller and selector outperforms spending all of it on the selector on MATH-500 and LCB, and ties on GPQA (App. L.2, Table 39). A better selector therefore complements the controller rather than replacing it.

After our response, the AC asked about baseline fidelity, and we added a PB-SMC control that follows Algorithm 4 of Tran et al. (2026) (App. F.3, Table F3). RAVEC needs 12.8% [5.1, 19.4] / 12.1% [2.3, 21.6] fewer operations than this control on MATH-500 / LCB. The tuned full-pool variant (8.9% / 10.2%, Table 2) remains the most demanding comparison.

Thank you again for the careful review, in particular your check of Fig. 9, which led us to state the provenance of the reference fit explicitly in the paper.

Reviewer_mXdfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RAVEC, a controller that allocates additional computation at inference time to allow llm (frozen) to produce more answer candidates via continuation. After an initial response, the controller chooses a reasoning checkpoint from which to regenerate, a continuation token budget, or STOP. Then the final response is selected by a scorer. The controller training is framed as: given a current reasoning state and restart action (i.e. checkpoint and budget), the controller is expected to predict whether the action can improve the final answer selected by scorer. The task is treated as a Q-function learning task that estimates cost and revision opportunities.

The evaluation is extensive as it covers 14 models and 12 benchmarks. Across the eight core benchmarks, RAVEC improves average accuracy over adapted PB-SMC by 1.34% points across models. With Qwen3-8b, 20.8% and 25.4% fewer computation cost is reported MATH-500 and LiveCodeBench, respectively.

The main contribution is it proposes an approach applies offline Q-learning to budgeted revision of LLM responses, with rewards based on the correctness of the final selected answer.

Critical strengths
  1. Extensive experiments and evaluations - authors did comprehensive evaluations, including BoN, adaptive sampling, process-verifier methods, backtracking methods, and component ablations. All 14 evaluated models show positive gains over the primary PB-SMC baseline.

  2. Well-motivated objective tied to the final answer. The paper distinguishes generating a correct candidate from actually selecting it as the final answer. Learning from accepted repairs and accepted errors therefore targets the system’s final accuracy, which accounts for the bias introduced from scorer. This is also supported by the swapping experiments in Table 18 that shows changing only the scorer can change the preferred restart position for 23.1% of evaluated starting states.

Critical weaknesses
  1. Unclear attribution of the benefit from additional supervision - The authors acknowledge that "a sufficiently expressive direct gain predictor can learn the same conditional mean as Eq. (2); the two parameterizations therefore access the same target information." While ablations in Table 12 supports the justification that the additional supervision in RAVEC may help the model learn better estimates, the experiments remove both probability supervision and consistency supervision, so it measures the combined benefit. It does not isolate which of those components produces the improvement.

  2. Computation savings need validation against the stronger baseline variant. The reported 20.8% and 25.4% cost benefit in computation on MATH-500 and LiveCodeBench are measured against the primary adapted PB-SMC configuration. However, Table 5 shows that a wider PB-SMC variant achieves better accuracy at the same computation budget, reducing RAVEC’s advantage. It remains unclear whether similar computation savings can still hold against this stronger variant. Evaluating it across computation budgets would provide a more convincing assessment of RAVEC’s efficiency.

  3. Dependence on a fixed scorer. RAVEC learns intervention values specific to the scorer used during training. Updating or replacing that scorer can change the reward target and reduce the effectiveness of the existing controller, as demonstrated in Table 6. The paper demonstrates sensitivity to replacing the scorer and shows that refitting helps. However, the analysis covers a limited range of scorer changes, leaving the broader robustness and adaptation requirements unclear.

Questions for authors
  1. Table 12 removes probability supervision and the one-step consistency loss together. Could you report an ablation that retains probability supervision but removes the consistency loss, comparing it with full RAVEC and direct-Q prediction under matched training data and computation? This would clarify which benefits arise from each component. Are the observed differences robust across controller fits?

  2. How do the reported computation savings change when RAVEC is compared with the wider PB-SMC variant in Table 5?

  3. Beyond the 0.6B-to-4B scorer replacement, how well does a fixed controller transfer to scorers with different training or ranking behavior?

4 comments
AuthorsOpenReview ↗

Response to Reviewer mXdf (1/2): separated ablation (both components contribute); full-pool frontier across budgets

Thank you for the positive assessment. We understand your main question to be the separated ablation in Q1; we address it first.

Summary of changes for you:

  1. The separated ablation you requested (App. G.1, Table 16).
  2. The full-pool PB-SMC frontier across nine budgets (Fig. 3; new Table 2 in Sec. 7.1; Table 13).
  3. Four new selector families with bank-only and full refits (App. F.2, Table 14; part 2/2).

Retain probability supervision but remove the consistency loss; compare with full RAVEC and direct-Q under matched data and compute; robust across fits?

Short answer: both components contribute. Starting from direct-Q, probability supervision alone adds 0.34 (MATH-500) / 0.33 (LCB) points, and the consistency loss adds a further 0.36 / 0.47; all four paired 95% CIs lie above zero.

As suggested, all three variants share the same transitions, trunk width, epochs, learning-rate grid, validation budget, five fit seeds and 64 paired replicates. Probability heads are temperature-calibrated on the calibration split, and budget prices are selected on the same validation split for every variant. Accuracies at 2.4C02.4C_0 (mean ± SD over five fits):

VariantProb. sup.ConsistencyMATH-500LCBGPQAAction regret† (math / code)
RAVEC (full)✓✓98.8 ± 0.1856.4 ± 0.2258.7 ± 0.192.2 / 2.7
No consistency (new)✓✗98.44 ± 0.1955.93 ± 0.2858.61 ± 0.242.6 / 3.0
Direct-ΔY Q only (Table 15)✗✗98.1 ± 0.2255.6 ± 0.3158.5 ± 0.233.4 / 3.9

Paired differences [95% CI], nested over questions and fits:

  • full − no-consistency: 0.36 [0.07, 0.69] / 0.47 [0.09, 0.91];
  • no-consistency − direct-Q: 0.34 [0.04, 0.71] / 0.33 [0.01, 0.72].

On GPQA, where RAVEC's overall gain is smallest, both increments (0.09 and 0.11) have intervals that include zero. †Regret and within-parent Kendall τb\tau_b use the independent-bank action diagnostic of App. J.4, not the 2.4C02.4C_0 benchmark; τb\tau_b follows the same order (0.40 / 0.41, 0.36 / 0.38, 0.20 / 0.30), so the accuracy gains come with better action ranking.

Takeaway: the factorization adds no target information, as App. K.1 (L2431–2436) states; it improves finite-data estimation, and the consistency term adds a separable increment on top. In the paper: App. G.1 (New), Table 16.

Savings against the wider PB-SMC variant, across budgets.

Short answer: 8.9% [1.7, 15.9] on MATH-500 and 10.2% [0.6, 19.1] on LCB at the original targets (General Response, G1).

Following your suggestion to evaluate across budgets, we built the full nine-budget frontier (selected accuracy, %; full-pool / RAVEC):

Budget (C0C_0)1.21.51.62.02.43.24.04.8
MATH-50095.13 / 95.2‡95.83 / 96.196.07 / 96.42‡97.12 / 97.798.1 / 98.898.56 / 99.098.77 / 99.198.94 / 99.2
LCB50.29 / 50.48‡51.82 / 52.152.24 / 52.6‡54.13 / 54.655.2 / 56.456.47 / 57.857.26 / 58.457.74 / 58.9

‡Interpolated between measured RAVEC budgets. Both methods share the Single anchor at 1.0C01.0C_0 (94.6 / 49.4).

Takeaway: RAVEC is above full-pool PB-SMC at every budget, by 0.07–0.70 points on MATH-500 and 0.19–1.33 on LCB. The gap is small near 1.0C01.0C_0, where both stay close to the shared anchor, and largest at 2.42.4–3.2C03.2C_0. The frontier-averaged saving is 10.7% / 10.1%; with equal historical-prefix access, savings are 13.2% / 16.8%. In the paper: Fig. 3 (dashed curve), new Table 2 (Sec. 7.1) and App. F.1 (Table 13); the abstract, Sec. 1 and Sec. 8 now report both baselines.

AuthorsOpenReview ↗

Response to Reviewer mXdf (2/2): transfer to four new selector families

Beyond the 0.6B-to-4B replacement, how well does a fixed controller transfer to scorers with different training or ranking behavior?

Short answer: an unchanged controller falls below PB-SMC once the selector's ranking changes, consistent with the selector dependence formalized in Proposition 1. A bank-only refit with no new generation restores the advantage over PB-SMC for every new selector family on MATH-500 and LCB.

We added four selector families with different training or ranking behavior. Evaluation uses 32 paired replicates per question and fit, and five controller fits on the complete task sets. Each selector defines its own C0C_0, as in Table 7 (formerly Table 6). Entries are MATH-500 / LCB:

SelectorReversal (%)PB-SMCUnchanged RAVECBank-only refitFull refitRefit GPU-h (bank / full)
4B ORM (Table 7)23.198.0 / 57.097.7 / 56.598.22 / 57.6898.4 / 58.019.0 / 64.7
Different-family 8B ORM31.798.14 / 56.9197.48 / 55.8498.48 / 57.5298.61 / 58.1726.8 / 83.5
PRM final selection (math)38.498.3797.6298.7398.8344.2 / 101.6
8B LLM judge35.997.86 / 56.2397.08 / 55.1798.01 / 56.5898.42 / 57.3155.6 / 118.4
Self-consistency (math)44.697.696.9397.8497.962.7 / 46.8

Reversal is the share of fixed-bank parents whose preferred position changes, as defined in Table 23 (formerly Table 18). In the 4B row, reversal is from Table 23, and all accuracies except the bank-only refit are from Table 7.

Takeaway:

  • Refitting is cheap. A bank-only refit costs under one-fifth of the 307.8 GPU-hour initial collection. Its point estimates exceed PB-SMC in all six new MATH-500/LCB cells, and it recovers 76.7% of the full-refit gain at 36.9% of its cost.
  • Full refits lead. A full refit leads PB-SMC by 0.36–1.26 points under every selector.
  • Recovery varies. Recovery is highest for the PRM and lowest for the LLM judge, which benefits more from collecting reached states.

We agree that the original text understated this dependence. In the paper: App. F.2 (New), Table 14; Sec. 8 and Limitations now state that a selector change should be accompanied by at least a bank-only refit.

We hope this clarifies the separation of the two training components and the transfer behavior. Please let us know if anything remains unclear; we are happy to provide further results.

I thank the authors for addressing my comments and questions by providing additional results.

  1. positive: the ablation confirmed the contribution of both components.
  2. minor concern (expected): stronger baseline will significantly reduce the magnitude of the gain.
  3. concern: The scorer-dependency results confirms the limited robustness when scorer varies. (mitigation is provided, but add another layer of complexity on scorer)

I will maintain my recommendation.

AuthorsOpenReview ↗

Thank you; on the two remaining points

Thank you for the follow-up and for confirming that the separated ablation resolves the attribution question. We agree with both remaining points and summarize how the revised paper reflects them.

Stronger baseline. We agree. Against tuned full-pool PB-SMC the savings fall by more than half: from 20.8% / 25.4% to 8.9% [1.7, 15.9] / 10.2% [0.6, 19.1] on MATH-500 / LCB, or 7.1% / 9.7% in measured GPU time. We therefore report the full-pool numbers next to the adapted ones in the abstract, Sec. 1, Sec. 7.1 (Table 2) and Sec. 8 (L526–528), so the conservative figure appears wherever savings are stated. Fig. 4 indicates that the remaining gain comes from the selected-answer target rather than search breadth: switching the coverage reward to the selected-correctness reward adds +1.7 / +2.8 points in the incumbent search domain and +1.2 / +2.2 points even with full historical-prefix access.

Scorer dependence. We agree that this is a real deployment constraint, and the paper now says so directly. Limitations (L2952–2953) and Sec. 8 (L529–532) state that an unchanged controller falls below PB-SMC after a selector change and that a refit is required. This dependence is the other side of the selection-conditioned objective: a selector-independent target needs no refit by construction, but in Fig. 4 it gives up the 1.7–2.8 point gain above. On complexity, a bank-only refit adds no new component. It rescores the stored candidates with the new selector and reruns the same fitting routine, with no new generation. In Table 14 this costs 2.7–55.6 GPU-hours, at most 18% of the initial 307.8 GPU-hours.

Thank you again for the careful reading and the suggested ablation, which made the attribution in the paper sharper.

Reviewer_8AFHOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this work, the authors propose RAVEC, a controller that adaptively determines where to revisit a completed LLM reasoning trajectory, how much computation to allocate to its continuation, and when to stop. The core idea is to estimate how each intervention affects the correctness of the final selected answer, considering both the potential to correct an incorrect answer and the risk of disrupting a correct one. RAVEC learns from randomized interventions originating from the same parent state and uses finite-horizon value estimates to guide its decisions. Experiments across 14 models and 12 benchmarks show an average improvement of 1.34 percentage points over an adapted PB-SMC baseline on the eight-task core macro average under matched expected operation counts.

Critical strengths
  1. The proposed learning objectives are well motivated, and the main target definitions and theoretical arguments appear reasonable.

  2. The search-domain–reward factorial experiments and the matched-feature root-only controller provide complementary evidence for the contributions of the learning objective and adaptive reopening-position selection, respectively.

Critical weaknesses
  1. The operation ledger accounts for selection, control, and prefix-processing costs, but the service-time and break-even analyses in Appendix K.6 rely on assumed parameters rather than measured deployment costs. Similarly, the estimated 23 billion continuation tokens across 14 models represent a design-bound estimate rather than actual expenditure. Since changing the generator or selector requires refitting, the lack of measured latency, memory overhead, and incremental offline costs makes it difficult to assess the method's practical efficiency.

  2. The main comparisons evaluate adapted baselines under a shared action space and budget framework. However, Table 5 shows that an expanded PB-SMC configuration reduces the performance gaps to 0.7 and 1.2 percentage points on MATH-500 and LiveCodeBench, respectively, and outperforms RAVEC on AIME24. Although the factorial experiments support the proposed learning objective, the claimed efficiency advantages remain insufficiently validated against the stronger baseline's full accuracy–operation trade-off.

Questions for authors

When the stronger full-pool PB-SMC configuration from Table 5 is used to construct the accuracy–operation frontier, how much of the reported 20.8% and 25.4% savings remains? Do the corresponding paired confidence intervals still exclude zero?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 8AFH (1/2): savings vs. full-pool PB-SMC are 8.9% / 10.2%, with CIs excluding zero

Thank you for the precise questions. We understand your main concern to be whether the efficiency claim survives the stronger full-pool PB-SMC frontier; we address it first.

Summary of changes for you:

  1. A tuned nine-budget full-pool PB-SMC frontier with paired CIs (Fig. 3; new Table 2 in Sec. 7.1; App. F.1, Table 13).
  2. The abstract, Sec. 1 and Sec. 8 now report savings against both PB-SMC variants.
  3. Measured GPU time, latency, memory and offline cost replace the assumed parameters (App. K.6.1, App. I.1).
  4. Targeted presentation edits in Sec. 1, 3, 4 and 6 (details in part 2/2).

"How much of the reported 20.8% and 25.4% savings remains? Do the corresponding paired confidence intervals still exclude zero?"

Short answer: 8.9% on MATH-500 and 10.2% on LiveCodeBench (LCB) remain at the original 97.0% / 55.0% targets, with paired 95% CIs of [1.7, 15.9] and [0.6, 19.1]; both exclude zero (General Response, G1).

You are right that full-pool PB-SMC is the stronger comparator: it can reopen any legal prefix in its history, whereas primary RAVEC reopens only the current trace. As suggested, we turned its single point (Table 6, formerly Table 5) into a frontier at nine budgets (1.0, 1.2, 1.5, 1.6, 2.0, 2.4, 3.2, 4.0, 4.8 C0C_0), covering all seven Fig. 3 budgets. To give the baseline its best case, we tuned it on validation data at every budget over particle slots {4, 8, 16}, power exponent {1, 2, 4}, uniform-mixture weight {0.1, 0.2, 0.4}, and pools of all legal checkpoints or 8 / 16 per candidate, selecting by validation selected accuracy under the charged budget. Every bootstrap draw rebuilds both frontiers; draws that miss a target are counted, never extrapolated.

Four further checks show that the saving is not specific to one operating point:

  • Every budget. RAVEC is above full-pool PB-SMC at each budget from 1.21.2 to 4.8C04.8C_0, by 0.07–0.70 points on MATH-500 and 0.19–1.33 on LCB (Table 13).
  • Other targets. At full-pool's own 2.4C02.4C_0 accuracy (98.1% / 55.2%), RAVEC saves 10.6% [2.8, 17.9] / 11.1% [1.9, 20.4]. Averaged over the common accuracy range, it saves 10.7% [3.2, 18.4] / 10.1% [0.4, 19.3].
  • Equal search access. Refitted with the same historical-prefix pool and candidate cap (RAVEC-persistent), RAVEC saves 13.2% [5.8, 20.5] / 16.8% [7.3, 26.1].
  • AIME. We agree that full-pool PB-SMC is stronger on AIME24: the difference is −1.4 [−5.9, 3.2] points at 2.4C02.4C_0, and full-pool also leads at 1.6C01.6C_0 and 3.2C03.2C_0. On AIME25 RAVEC leads at all three budgets (67.3 vs. 67.0 at 2.4C02.4C_0). With 30 questions per set, neither direction is statistically resolved.

In the paper: we have revised the abstract, Sec. 1 (L93–95), Sec. 7.1 (Table 2, L424–429) and Sec. 8 to report both comparisons. The abstract now reads "... reduces computation by 20.8% on MATH-500 and 25.4% on LiveCodeBench, and by 8.9% and 10.2% relative to a tuned full-pool PB-SMC variant with broader historical-prefix access." Fig. 3 adds the full-pool frontier, App. F.1 (Table 13) lists its accuracy at every budget, and Sec. 7.2 (L480–483) gives the AIME24 interval.

AuthorsOpenReview ↗

Response to Reviewer 8AFH (2/2): measured serving and offline cost; presentation edits

Service-time and break-even analyses rely on assumed parameters; the 23 billion tokens are a design bound; latency, memory and incremental offline costs are not measured.

Short answer: we agree, and we have replaced the assumptions with measurements of serving cost, offline expenditure, refit cost and break-even.

Serving (General Response, G2; App. K.6.1, Table 36). Measured mean single-request service time also tests the condition of Eq. (35) directly. Against adapted PB-SMC it is 21.6 vs. 22.8 s on MATH-500 and 38.5 vs. 40.2 s on LCB. The measured non-arithmetic overheads are hBh_B = 0.047 / 0.055 and hRh_R = 0.066 / 0.078 (assumed: 0.04 / 0.06). The measured service-rate ratio vv = 0.855 / 0.800 exceeds the thresholds these overheads imply (0.807 / 0.764), so operation savings become service-time savings in the measured system; Eq. (35) predicts ratios of 0.947 / 0.958, matching the measured means. The controller pipeline takes 28 / 43 ms per question, 0.13% / 0.11% of mean service time.

Offline ledger (Qwen3-8B, measured; App. I.1, Table 21):

ItemDesign value (paper)Measured
Transitions≤ 1,824,000≈ 1.562M
Suffix tokens≈ 1.64B (900-token mean)1.208B
Method-specific GPU-hoursnot reported307.8
Incremental vs. adapted / full-pool PB-SMC100k–1M C0C_0 (assumed)265.2 / 239.4 GPU-h
Suffix tokens, all 14 generators≈ 23.0B17.84B

The 307.8 GPU-hours comprise suffix generation 264.8, initial parents 16.7, scoring and features 8.9, controller fitting 4.6, and validation 12.8. Code execution uses 236.8 CPU-hours, reported separately. App. C.4 now labels 23.0B as a design upper bound.

Refitting. Replacing the 0.6B with the 4B selector costs 19.0 GPU-hours for a bank-only refit (6.2% of the initial cost) and 64.7 for a full refit (21.0%). A new generator requires a fresh collection, 307.8 GPU-hours at the 8B scale.

Break-even. In GPU time, the incremental cost is recovered after 1.77M / 0.98M questions against adapted PB-SMC and 4.10M / 2.00M against full-pool PB-SMC (MATH-500 / LCB). Sec. 8 now states that RAVEC targets high-volume serving with a fixed generator.

Presentation. Following your comments, we have made targeted edits rather than restructuring the paper:

  1. a one-sentence statement for each contribution in Sec. 1 (L68–80);
  2. a notation-free paragraph opening Sec. 3 that walks through the online loop (L159–163);
  3. the x+2=x\sqrt{x+2}=x example brought from App. K.1 into Sec. 3, with rr, α\alpha, β\beta and gg given as numbers (L184–192);
  4. ORM and "adapted / harmonized" defined at first use (L349, L357–359); C0C_0 is defined at first use (L350);
  5. each Sec. 4 paragraph now opens with what the controller can do, what it sees and predicts, how it is fitted, and how it acts (L205, L210, L225, L243).

To stay within the 10-page limit, we moved the endpoint decomposition from Sec. 7.4 to App. H (L1837–1840).

We hope these results address your main concern, in particular that the savings against full-pool PB-SMC remain positive with intervals excluding zero. If anything remains unclear, we are happy to provide further analysis.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Response: full-pool PB-SMC frontier, measured cost, selector transfer, and list of revisions

We thank the reviewers for their careful and constructive reviews. We have uploaded a revised PDF. All changes are in blue and tagged by reviewer ([8AFH], [mXdf], [A3Mp]); the main text stays within 10 pages, and new appendix sections are marked "(New)". The three shared concerns were (i) efficiency against the stronger full-pool PB-SMC, (ii) measured serving and offline cost, and (iii) dependence on the selector. We ran each requested experiment; the shared results follow, and each reviewer thread answers the remaining points.

G1. Savings against full-pool PB-SMC (8AFH, mXdf). We agree that full-pool PB-SMC, which can reopen any legal prefix in its history, is the strongest search comparator and belongs in the headline. We extended its single point (Table 6, formerly Table 5) into a Qwen3-8B frontier at nine budgets (1.01.0–4.8C04.8C_0), tuned on validation data at every budget. The protocol is unchanged: 64 paired replicates, five controller fits, and a nested question/fit bootstrap that rebuilds both frontiers in every draw.

Matched accuracyFull-pool PB-SMC (C0C_0)RAVEC (C0C_0)Saving [95% CI]
MATH-500 @ 97.0%1.9541.7818.9% [1.7, 15.9]
LCB @ 55.0%2.3252.08910.2% [0.6, 19.1]
MATH-500 @ 98.1% (full-pool at 2.4C02.4C_0)2.4002.14510.6% [2.8, 17.9]
LCB @ 55.2% (full-pool at 2.4C02.4C_0)2.4002.13311.1% [1.9, 20.4]

Takeaway: savings are smaller than against adapted PB-SMC (20.8% / 25.4%) but remain positive, with intervals excluding zero. RAVEC lies above full-pool PB-SMC at every budget from 1.21.2 to 4.8C04.8C_0 on both tasks (Fig. 3; Table 13). With the same historical-prefix access (RAVEC-persistent), savings are 13.2% [5.8, 20.5] / 16.8% [7.3, 26.1].

G2. Measured serving cost (8AFH, A3Mp). One H100-80GB, BF16, vLLM with prefix caching; Qwen3-8B at matched accuracy (97.0% MATH-500 / 55.0% LCB); entries are MATH-500 / LCB. GPU-s is device time per completed question at concurrency 64, so throughput = 60 / GPU-s. Live memory counts weights, active KV blocks and workspaces within the same 72.4 GiB reservation.

MethodGPU-s / questionThroughput (q/min)p50 latency, single request (s)Peak live memory (GiB)
BoN (parallel)4.93 / 6.8012.17 / 8.8212.8 / 22.656.8 / 60.7
PB-SMC (adapted)3.28 / 4.9618.29 / 12.1019.8 / 33.657.6 / 63.5
PB-SMC full-pool2.95 / 4.4220.34 / 13.5718.5 / 31.865.9 / 70.6
RAVEC2.74 / 3.9921.90 / 15.0417.6 / 30.151.4 / 56.3

Takeaway: operation savings carry over to measured GPU time. RAVEC uses 7.1% / 9.7% less GPU time than full-pool PB-SMC, 16.5% / 19.6% less than adapted PB-SMC and 41–44% less than BoN, with the lowest live memory. Parallel BoN keeps the lowest single-request latency. Offline, Qwen3-8B uses 307.8 method-specific GPU-hours, 239.4 more than full-pool PB-SMC.

G3. Selector dependence (mXdf, A3Mp). Across four new selector families, a bank-only refit with no new generation lets RAVEC exceed PB-SMC in point estimate in all six MATH-500/LCB cells, recovering 76.7% of the full-refit gain at 36.9% of its cost.

Revisions (blue in the PDF)

  • Abstract, Sec. 1 (L93–95), Sec. 7.1 (new Table 2), Sec. 8: savings against both PB-SMC variants [8AFH, mXdf]
  • Fig. 3: full-pool frontier added (dashed); App. F.1 (New, Table 13): protocol and per-budget accuracies [8AFH, mXdf]
  • Sec. 7.1; App. K.6.1 (New, Table 36): measured serving cost [8AFH, A3Mp]
  • App. C.4; App. I.1 (New, Table 21): measured offline cost, refit cost, break-even [8AFH]
  • App. F.2 (New, Table 14): selector-family transfer [mXdf, A3Mp]
  • App. G.1 (New, Table 16): probability supervision vs. consistency loss [mXdf]
  • App. L (New, Tables 38–39): controller without hidden states; compute-matched selector allocation [A3Mp]
  • Fig. 9 and App. J.4: reference-fit provenance [A3Mp]
  • Sec. 1, 3, 4, 6: targeted readability edits; endpoint decomposition moved to App. H to stay within 10 pages [8AFH]
  • Limitations: selector refitting, single-request latency, prefix-continuation interface [all]
AuthorsOpenReview ↗

Why SPS and PB-SMC are adapted rather than run in their native setup

We thank the AC for raising this. The AC is right that our controls look quite different from the published algorithms, and the paper should have explained the reason more directly. We have now done so in Appendix E (Table E1). In brief, Tran et al. (2026) and our paper study different problems. Running the native setup at our operating points would also change the generator, scorer, final-answer rule and cost unit. We therefore kept the two mechanisms that define SPS and PB-SMC and replaced only the components shared by every method in our study. Each change is either required by that shared protocol or gives the baseline more search access, not less.

Why the native setup does not transfer

Native (Tran et al.)Our shared protocolReason
TaskStep-level search from scratchRepair after a completed, shared initial answerOur question is where to reinvest compute after an answer exists
BudgetNN expansions per stepOne suffix per iteration; 2.4C02.4C_0 headlineFour particles grown to completion already generate about four full answers, above 2.4C02.4C_0 before any step scoring
ScorerQwen2.5-Math-PRM-7B, every stepOne 0.6B process head shared by PAV, VGB, SPS, PB-SMC; cost chargedLong-CoT traces contain hundreds of loosely delimited steps; the math PRM does not cover code or GPQA
Final answerBest PRM scoreOne frozen ORM for all methodsBy Prop. 1 the selector gates every gain; per-method selectors would compare selectors, not search
CostGenerated tokensOperations incl. scoring and replay, plus GPU-secondsRevisiting history has a real replay cost, which Tran et al. also note (KV recomputation)

We left unchanged the parts Tran et al. present as their contribution:

  • a persistent pool that keeps checkpoints of non-selected candidates;
  • SPS's top-score choice inside a random subpool (≤8\le 8);
  • PB-SMC's powered, clipped scores (γ∈{1,2,4}\gamma \in \{1,2,4\}, tuned on validation), its mixture proposal q=0.8p+0.2⋅Uniformq = 0.8p + 0.2\cdot\mathrm{Uniform}, and the p/qp/q correction.

Because the surrounding setup differs, we label these controls "adapted" and do not carry over the native SMC guarantees.

Does the adaptation weaken the baselines? We tested this directly:

  • Wider search. Both controls may revisit non-incumbent history, which RAVEC's incumbent grid cannot. Full-pool PB-SMC can reopen any legal prefix and was tuned separately at nine budgets (App. F.1, added for Reviewers 8AFH and mXdf). RAVEC's frontier stays above it over 1.21.2–4.8C04.8C_0, with matched-accuracy savings of 8.9% [1.7, 15.9] on MATH-500 and 10.2% [0.6, 19.1] on LCB (Table 2).
  • Native final-answer rule. Under PRM final selection, PB-SMC reaches 98.37 on MATH-500 and refitted RAVEC reaches 98.83 (Table 14). With a larger 4B ORM, the scores are 98.0/57.0 vs. 98.4/58.0 on MATH-500/LCB (Table 7).
  • Same pool for RAVEC. RAVEC-persistent receives the identical historical pool (Table 2), which separates search access from the controller.

As expected, these stronger variants narrow the margin. The MATH-500 saving drops from 20.8% against adapted PB-SMC to 8.9% against full-pool PB-SMC. Full-pool PB-SMC also leads on AIME24 (74.4 vs. 73.0; −1.4 [−5.9, 3.2] with 30 questions), and we report this in the main text. We therefore do not claim that RAVEC beats every search configuration. Our claim is narrower: under a common budget and selector, selection-conditioned allocation adds value on top of historical-prefix search.

We hope this clarifies the design choice, and we are glad to give further detail on any part of the adaptation.

It seems that original PB-SMC in Tran et al did not use a uniform distribution to explore the history, so even the full-pool PB-SMC is still not the same as the original PB-SMC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.