acceptodds
Under review as a conference paper at ICLR 2027

Think Before Spending: Value-Aware Computation Control for LLM Reasoning

Abstract

Test-time computation improves large language model (LLM) reasoning by generating additional evidence and exploring multiple reasoning paths, but incurs substantial generation cost. Existing adaptive computation methods mainly optimize resource allocation or termination after a reasoning strategy has been fixed, leaving the initial strategy choice unresolved. Reasoning strategies such as direct submission, evidence gathering, and consistency sampling offer different quality–cost trade-offs across requests, making the choice of strategy central to reducing unnecessary computation. To address this challenge, we present Compass, a value-aware computation controller for LLM reasoning. Compass is built on two key observations: (1) observable inference states contain signals about the expected quality and cost of candidate reasoning strategies, and (2) additional computation is valuable only when it improves upon the currently available answer. Based on these observations, Compass introduces two coupled decisions: quality-aware computation strategy selection and marginal-value continuation control. Specifically, Compass estimates the final quality and remaining token cost of candidate computation strategies before committing to expensive execution, selecting a low-cost strategy with near-optimal predicted quality. During execution, Compass dynamically reassesses continuation by estimating the probability of correcting the current answer against the risk of introducing an error and the additional computation cost. Across four datasets, three starting models, and two generation seeds, Compass reduces pooled output tokens per correct answer (TPC) by 20.1–88.1% relative to the evaluated sampling baselines, with macro-accuracy gains of 0.9–2.3 percentage points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large language model reasoning, Test-time computation, Value-aware computation control, Adaptive strategy selection, Marginal-value continuation control.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nqqXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Compass, a framework for LLM test-time compute control. The method first predicts the accuracy and computation cost for different reasoning strategies, and then chooses the optimal one without sacrificing quality or cost. Then during the reasoning process, the method continuously predicts the accuracy improvement and additional token generation cost to dynamically decide whether to terminate the reasoning. The experiments cover four different datasets and three models across two different seeds, showing consistent improvements in the quality-efficiency tradeoff.

Critical strengths
  1. Designed joint computation control method. The method integrates strategy selection and adaptive stopping while explicitly accounting for the expected accuracy benefits and costs of continued computation.

  2. Extensive experimental evaluation. The method demonstrates token efficiency across multiple models, datasets, and random seeds, with ablation studies show the contributions of individual components.

Critical weaknesses
  1. Limited novelty. Prior works such as PRISM [1], UniScale [2], and BEST-Route [3] have explored dynamic reasoning strategy selection and joint computation optimization. Compass does not sufficiently distinguish itself from these approaches. Moreover, removing strategy selection increases TPC by only 2.1%-4.1%, suggesting that most efficiency gains come from adaptive stopping.

  2. Reliance on offline calibration and limited generalization analysis. Compass requires ground-truth labels and strategy rollouts to estimate strategy quality and marginal computation value, introducing additional calibration costs and dependence on train-test distribution similarity. Evaluation is largely restricted to the same datasets and model settings, without evaluating cross-setting transfer or out-of-distribution robustness. Furthermore, TPC excludes offline rollout and calibration costs, leaving the end-to-end efficiency benefits unclear.

[1] Qi, Shihao, et al. "Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs." arXiv preprint arXiv:2509.24377 (2025).

[2] Huang, Kaiyu, et al. "UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling." arXiv preprint arXiv:2605.30898 (2026).

[3] Ding, Dujian, et al. "Best-route: Adaptive llm routing with test-time optimal compute." arXiv preprint arXiv:2506.22716 (2025).

Questions for authors
  1. Could the authors discuss what are the key differences between Compass and prior methods? Why does joint strategy selection provide only marginal gains over adaptive stopping alone?

  2. Could the authors explain how many model calls, input/output tokens, and GPU hours are required to construct the strategy outcome tables and continuation predictor?

  3. Can Compass maintain its accuracy and efficiency gains when transferred to different settings without in-distribution calibration?

Reviewer_pf7rOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Compass, a controller that decides how to spend additional computation after an LLM generates its first answer. It selects among immediate submission, bounded evidence gathering using paired models, and Adaptive-Consistency sampling. During evidence gathering, it decides whether to continue by estimating the probability of correcting the current answer, the risk of making it incorrect, and the remaining output-token cost. Predictions use task-balanced nearest-neighbor retrieval over separate training tasks. Evaluation uses offline replay of model-generated answer banks across four datasets, three starting models, and two generation seeds. Compass improves aggregate output-token efficiency while maintaining competitive accuracy.

Critical strengths
  1. The evaluation includes meaningful controls against alternative explanations. Training and evaluation tasks are disjoint, retrieval limits each training task’s contribution, and training-record construction excludes the queried task from downstream estimates. Importantly, Appendix B.2 includes Starting-to-Trusted AC, which accesses the same trusted model as Compass. Compass reduces TPC by 44.0–44.8% against this control while both achieve approximately 89% macro accuracy. This supports a benefit beyond simply accessing a stronger auxiliary model.

  2. The component ablations support the controller’s contribution. Holding initial actions fixed, replacing learned continuation with native AC increases TPC by 14.8–15.0%. Removing strategy selection increases TPC by 2.1–4.1%, with similar aggregate accuracy. Task-clustered confidence intervals for selected component comparisons strengthen the evidence that the improvements are not merely artifacts of treating repeated records as independent.

Critical weaknesses
  1. The headline results combine several sources of improvement, and their uncertainty is incompletely characterized. Main-table baselines sample the starting model, whereas Compass can use a dataset-specific trusted model. The matched-model control substantially addresses this concern, but deserves greater prominence and paired uncertainty estimates. Furthermore, the macro-accuracy improvement is not uniform: Table 2 shows losses relative to AC in several GSM8K and ASDiv cells. The two generation seeds reuse the same task identities, so they establish sampling robustness rather than independent task replication. Task-clustered intervals for the principal baseline comparisons would make the quality–cost trade-off easier to assess.
  2. The operational meaning and reproducibility of the gains remain limited. TPC counts unweighted output tokens; it does not establish equivalent reductions in latency, total inference compute, or monetary cost when switching between models. The paper appropriately acknowledges this limitation. However, exact definitions of the standard-error-based admissibility margin, generation-confidence features, and syntax-tree features, together with complete prompts and decoding settings, are needed to reproduce the controller and answer banks. The observed contribution of initial strategy selection is also considerably smaller than that of continuation control, which should temper the emphasis on strategy selection as the principal advance.
Questions for authors
  1. Can you provide task-clustered confidence intervals for accuracy differences and TPC reductions against Starting-to-Trusted AC and Strategic Scaling, preserving dependence across models and generation seeds?
  2. What are the exact admissibility-margin formula, confidence computation, feature definitions, and answer-bank generation settings?
  3. How frequently does Compass select each initial branch, broken down by dataset and starting model? This would clarify when learned strategy selection adds value beyond a fixed evidence branch with continuation control.
Reviewer_shQSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a nearest-neighbour rule, built from labeled training problems, that predicts when one or two samples are already enough and stops there. For each dataset, the method uses a "trusted model" alongside the starting model: the starting model gives one answer, and test-time scaling then samples the trusted model, whose majority vote is submitted. A nearest-neighbor lookup over hand-designed features, matched against 64 labeled training problems per dataset, decides how many trusted-model samples to draw and when to stop, reducing the output tokens spent on test-time scaling.

Critical strengths
  1. "Across four datasets, three starting models, and two generation seeds, Compass reduces pooled output tokens per correct answer (TPC) by 20.1–88.1% relative to the evaluated sampling baselines"
  2. The paper proposes tos how that nearest neighbour search works well for inference time scaling -- this idea is reasonably good. But somehow this has NOT used a second model which is lightweight, and rath used a full second model which is equivalent to the primary model.
Critical weaknesses
  1. a separate labeled table for each dataset, model and strategy, with hand-picked features. There is no feature ablation and no transfer test, 64 is the largest size tried. Can you please provide cost of building these tables and ablations?
  2. when i read the title + abstract i expected an RL value function, or atleast an economic value framing. But value here is some notion of marginal utility. Maybe this can be clarified better in the title/abstract?
  3. Main Concern: The primary weakness is that baselines are weak. Compass can query a per-dataset trusted model that no main baseline may use. What if the gains are all coming from this model? Furthermore i think the accuracy tracks the second model. For example, in Table 7 the accuracy gain against baselines goes away, for example, when baseline is "Starting-to-Trusted AC".
  4. I understand that prefill is cheaper than output tokens, but one must account for additional input tokens. Especiallly when input to output token ratio is so high.
Questions for authors
  1. What is the curve for input token scaling at test time? Does it track with the output tokens and how do the different methods compare?
  2. Why not use trusted model = start model and frame the entire paper like that? That framing still allows the authors to claim some token efficiency improvements over adaptive-consistency. The issue is that one does not know what is the trusted model at inference time on the user's queries which are random and not from some known dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.