- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Recoverability-Aware Value of Computation (RAVEC), a learning-based controller for compute-efficient test-time reasoning. Instead of optimizing candidate correctness or search coverage, RAVEC estimates the expected improvement in the final selected answer after recomputing a reasoning suffix. Specifically, it learns the probabilities of repairing an incorrect answer and damaging a correct answer through randomized interventions on the same parent state. The controller then uses finite-horizon value learning to jointly determine where to recompute, how much computation to allocate, and when to stop. Experiments across multiple models and benchmarks demonstrate improvements in accuracy and computational efficiency over existing test-time search methods.
Critical strengths
- The paper addresses an important and interesting problem in compute-efficient LLM reasoning: determining where and how much computation should be allocated to improve the final answer.
- The proposed solution is intuitively reasonable. The design of the controller training procedure is well-motivated, including the use of randomized same-parent interventions to collect training data, separate prediction of repair and damage probabilities, and finite-horizon value learning to account for future computation benefits.
- The empirical evaluation is comprehensive. The paper evaluates the proposed method across 14 models and 12 benchmarks, covering different types of reasoning tasks. Multiple baselines and extensive ablation studies are included.
Critical weaknesses
- A concern with the proposed method is that the proposed controller relies on intermediate hidden representations extracted from the generator. This limits its applicability to API-based LLMs.
- The paper motivates RAVEC by highlighting the limitations of a fixed selector, which may reject correct candidates or prefer incorrect ones. However, given the substantial offline computation required to train the proposed controller, comparable resources could instead be invested in training a stronger outcome reward model or using a process reward model for final answer selection, which may directly mitigate the selection errors that motivate the method. The paper does not provide a compute-matched comparison with improved selection strategies, leaving it unclear whether the proposed approach is the most effective use of the additional computation.
- Although the proposed method demonstrates computational savings in terms of operation counts, the paper does not report actual wall-clock inference latency. Since RAVEC performs sequential reopenings while BoN can be fully parallelized, operation savings may not translate into latency savings in practice.
Questions for authors
For Qwen3.8-27B, R1-Distill-Qwen-14B, and Gemma-3-12B, the “original fit” values in Figure 9 differ from those reported in Table 1 (e.g., 99.0 vs. 99.3 on MATH-500 for Qwen3.8-27B), while the Qwen3-8B results match exactly. Were these results obtained from separate controller fits or different experimental settings?
4 comments
Response to Reviewer A3Mp (2/2): controller without hidden states; compute-matched selector comparison
Reliance on hidden states limits applicability to API-based LLMs.
Short answer: hidden states are not essential. Replacing them with external text embeddings retains about three-quarters of the gain on math and code.
In a controlled test, we treated Qwen3-8B as a black box and replaced the four hidden-state projections with Qwen3-Embedding-0.6B embeddings of the question, retained prefix, incumbent and pool. The 20 scalar features (scores, positions, history, resources) already need no internals. Encoder cost is charged within ; we used five fits and 32 paired replicates.
| Controller input | Needs internals | MATH-500 | LCB | GPQA |
|---|---|---|---|---|
| PB-SMC | No | 97.3 | 54.4 | 59.2 |
| RAVEC, hidden states | Yes | 98.8 | 56.4 | 58.7 |
| RAVEC, text encoder + scalars (new) | No | 98.47 ± 0.17 | 55.91 ± 0.30 | 58.76 ± 0.25 |
| RAVEC, scalars only (new) | No | 97.86 ± 0.25 | 55.16 ± 0.37 | 58.42 ± 0.28 |
Takeaway: without internal access, RAVEC retains 78.0% (MATH-500) and 75.5% (LCB) of its gain over PB-SMC; even scalars alone stay above PB-SMC on both. We agree that the remaining interface requirement matters: the controller still needs prefix continuation from a chosen checkpoint, and this proxy does not test a commercial API; Limitations now states both. In the paper: App. L.1 (New), Table 38.
Compute-matched comparison with improved selection (stronger ORM or PRM).
Short answer: a stronger selector helps, and it complements RAVEC rather than replacing it. Under the same offline budget, splitting compute between RAVEC and a stronger selector beats giving all of it to the selector on MATH-500 and LCB, and ties on GPQA.
The offline cap is 307.8 GPU-hours, RAVEC's measured method-specific cost. Every row uses the same online budget in the original 0.6B-ORM unit, with all larger-selector and PRM scoring charged online. This shared unit leaves PRM rows less generation budget than the selector-specific in our reply to mXdf, hence 98.09 here versus 98.37 there. "Scaled 4B ORM" is a new 4B ORM trained with the remaining 265.2 GPU-hours, distinct from the Table 7 checkpoint.
| Use of the 307.8 GPU-h offline budget | MATH-500 | LCB | GPQA |
|---|---|---|---|
| PB-SMC + 0.6B ORM (42.6 GPU-h used) | 97.3 | 54.4 | 59.2 |
| PB-SMC + PRM final selection (existing checkpoint) | 98.09 | n/a | n/a |
| PB-SMC + scaled 4B ORM (all to selector) | 98.24 | 57.43 | 61.58 |
| RAVEC + 0.6B ORM (all to controller) | 98.8 | 56.4 | 58.7 |
| RAVEC + 4B ORM (184.7 controller / 123.1 selector GPU-h) | 98.63 | 58.16 | 61.69 |
Takeaway: as you anticipated, a stronger selector is valuable, especially on code and GPQA, but it does not substitute for RAVEC; the best use of the budget is a split between the two.
- The joint allocation leads the selector-only allocation by 0.39 / 0.73 points on MATH-500 / LCB and ties on GPQA (+0.11), although its selector receives less than half the compute.
- On MATH-500, the controller-only allocation is best.
This is consistent with Proposition 1: a better selector changes which reopenings are valuable but does not remove the need to choose them. With a fixed 4B selector, Table 7 likewise shows refitted RAVEC at 98.4 / 58.0 versus 98.0 / 57.0 for PB-SMC. In the paper: App. L.2 (New), Table 39.
We hope these results address your concerns. Please let us know if anything remains unclear; we are happy to clarify further.
Thank you for your rebuttal
Thank the authors for the rebuttal. The response addresses some of my concerns, and I will maintain my positive assessment of the paper.
Thank you; how the revised paper states the remaining limitations
Thank you for the follow-up and for maintaining your positive assessment. Because our response resolves some of your concerns only in part, we briefly note how the revised paper states each remaining limitation, so that its scope is explicit.
- Single-request latency. Parallel BoN remains faster for a single request: at matched accuracy, its median latency is 12.8 / 22.6 s on MATH-500 / LCB, against 17.6 / 30.1 s for RAVEC. RAVEC's advantage lies in GPU time and concurrent throughput, with 21.90 / 15.04 questions per minute against 12.17 / 8.82 for BoN at concurrency 64. Sec. 7.1 and the Limitations state this.
- API-only models. With text embeddings in place of hidden states, RAVEC keeps 78.0% / 75.5% of its gain. It still needs prefix continuation from a chosen checkpoint, however, and our black-box proxy does not test a commercial API (App. L.1; Limitations).
- Stronger selectors. We agree that a stronger selector helps, especially on code and GPQA. With the offline budget fixed at 307.8 GPU-hours, splitting it between controller and selector outperforms spending all of it on the selector on MATH-500 and LCB, and ties on GPQA (App. L.2, Table 39). A better selector therefore complements the controller rather than replacing it.
After our response, the AC asked about baseline fidelity, and we added a PB-SMC control that follows Algorithm 4 of Tran et al. (2026) (App. F.3, Table F3). RAVEC needs 12.8% [5.1, 19.4] / 12.1% [2.3, 21.6] fewer operations than this control on MATH-500 / LCB. The tuned full-pool variant (8.9% / 10.2%, Table 2) remains the most demanding comparison.
Thank you again for the careful review, in particular your check of Fig. 9, which led us to state the provenance of the reference fit explicitly in the paper.
Response to Reviewer A3Mp (1/2): Fig. 9 reference fit and measured latency
Thank you for the positive assessment and for checking Fig. 9 so carefully. We understand your direct question to be the Fig. 9 / Table 1 difference; we address it first.
Summary of changes for you:
Short answer: they come from separate controller fits; the experimental settings are the same. Table 1 reports the primary-matrix fits. The reference column of Fig. 9 is a fit trained within the centered-fitting experiment, with the same fitting procedure and evaluation settings, so the two differ only in the fitted controller instance. Each signed change in Fig. 9 is computed against that experiment's own reference fit.
The headline accuracy and efficiency results (Table 1, Figs. 2–3, Table 27) do not use Fig. 9. The centered-fitting claim rests primarily on within-parent action ranking: centered fitting reduces Qwen3-8B's math/code action regret from 2.2 / 2.7 to 1.4 / 2.2 and raises τb from 0.40 / 0.41 to 0.48 / 0.46 (App. J.4, L2239–2241).
We agree that the label "original fit" was ambiguous, since App. J.4 also used "original 2.4C0 matrix" for Table 1. In the paper: the Fig. 9 heading now reads "Reference fit (this experiment) → centered fit", and the caption and App. J.4 (L2236–2248) state that it is a fit instance separate from Table 1, trained with the same procedure and settings.
Short answer: you are right that parallel BoN remains fastest for a single request, and Sec. 7.1 and Limitations now say so explicitly. In concurrent serving, where GPU time sets the cost, RAVEC has the highest throughput.
At matched accuracy (General Response, G2), the single-request p50 is 12.8 / 22.6 s for BoN versus 17.6 / 30.1 s for RAVEC. RAVEC is the fastest of the measured sequential methods: full-pool and adapted PB-SMC take 18.5 / 31.8 s and 19.8 / 33.6 s. Serial depth is shallow: at 2.4C0, RAVEC stops after the initial answer on 46.8% / 29.5% of questions and averages 1.31 / 1.82 reopenings (Table 12).
At concurrency 64, RAVEC serves 21.90 / 15.04 questions per minute versus 12.17 / 8.82 for BoN (1.8× / 1.7×), because BoN needs 3.80 / 4.00 C0 to reach the same accuracy.
For latency-sensitive use, executing the top-2 actions in parallel (RAVEC-parallel-2) lowers p50 to 15.1 / 26.3 s at the same matched accuracy, using 1.85 / 2.18 C0; its throughput, 20.76 / 14.15 q/min, remains well above BoN's. In the paper: Sec. 7.1 (L431–465), App. K.6.1 (Table 36), Limitations.