- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Compass, a framework for LLM test-time compute control. The method first predicts the accuracy and computation cost for different reasoning strategies, and then chooses the optimal one without sacrificing quality or cost. Then during the reasoning process, the method continuously predicts the accuracy improvement and additional token generation cost to dynamically decide whether to terminate the reasoning. The experiments cover four different datasets and three models across two different seeds, showing consistent improvements in the quality-efficiency tradeoff.
Critical strengths
-
Designed joint computation control method. The method integrates strategy selection and adaptive stopping while explicitly accounting for the expected accuracy benefits and costs of continued computation.
-
Extensive experimental evaluation. The method demonstrates token efficiency across multiple models, datasets, and random seeds, with ablation studies show the contributions of individual components.
Critical weaknesses
-
Limited novelty. Prior works such as PRISM [1], UniScale [2], and BEST-Route [3] have explored dynamic reasoning strategy selection and joint computation optimization. Compass does not sufficiently distinguish itself from these approaches. Moreover, removing strategy selection increases TPC by only 2.1%-4.1%, suggesting that most efficiency gains come from adaptive stopping.
-
Reliance on offline calibration and limited generalization analysis. Compass requires ground-truth labels and strategy rollouts to estimate strategy quality and marginal computation value, introducing additional calibration costs and dependence on train-test distribution similarity. Evaluation is largely restricted to the same datasets and model settings, without evaluating cross-setting transfer or out-of-distribution robustness. Furthermore, TPC excludes offline rollout and calibration costs, leaving the end-to-end efficiency benefits unclear.
[1] Qi, Shihao, et al. "Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs." arXiv preprint arXiv:2509.24377 (2025).
[2] Huang, Kaiyu, et al. "UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling." arXiv preprint arXiv:2605.30898 (2026).
[3] Ding, Dujian, et al. "Best-route: Adaptive llm routing with test-time optimal compute." arXiv preprint arXiv:2506.22716 (2025).
Questions for authors
-
Could the authors discuss what are the key differences between Compass and prior methods? Why does joint strategy selection provide only marginal gains over adaptive stopping alone?
-
Could the authors explain how many model calls, input/output tokens, and GPU hours are required to construct the strategy outcome tables and continuation predictor?
-
Can Compass maintain its accuracy and efficiency gains when transferred to different settings without in-distribution calibration?