- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes STEPROUTER, a lightweight step router for reasoning LLMs. A backbone first sketches a solution and extracts reusable candidate steps; a scorer trained from offline outcome-induced pairwise preferences ranks these steps; the solver receives the original problem plus the top 20% of candidates. The central claim is that verification effort is a poor proxy for downstream usefulness, while learned utility ranking improves inference-time routing across math, science, and code, and that the same utility signal can transfer to SFT. The paper reports gains across GPT-5, DeepSeek-R1, and Qwen3-8B on AIME 2025, GPQA-Diamond, and LiveCodeBench, plus same-pool controls and SFT transfer.
Critical strengths
- The same-pool controls in Table 2 are strong and directly relevant: random, fixed-order, effort-based, and learned selectors keep the same candidate pool/count, isolating the value of learned ranking rather than just adding context.
- Large empirical gains in several settings: Qwen3-8B +6.16 pp on AIME and +23.23 pp on LiveCodeBench; DeepSeek-R1 +19.61 pp on LiveCodeBench (Table 1, p.5). The GPT-5 exact-count study reports +3.06 pp over random with a bootstrap interval (Table 3, p.6).
- Useful robustness checks: judge ablation, scorer-data scaling, zero-shot MMLU-Pro, latency, architecture ablation, and data-scaling for SFT (Secs. 6–7; Tables 5–7; Fig. 3).
Critical weaknesses
- Scorer is not problem-conditioned. The scorer encodes only candidate step text (Sec. 3.2, Eq. 5, p.3). Since step utility is inherently problem-dependent, this is a conceptual limitation. The paper notes problem information can appear inside the candidate statement, but the original problem is not separately encoded; this may limit generality beyond the tested pools.
- Evaluation scale is small for broad claims. AIME 2025 is small; the GPT-5 exact-count study uses 30 problems × 12 completions (Table 3, p.6). Several main comparisons in Table 1 do not report confidence intervals or clearly state the number of completions/seeds for each backbone. The paper acknowledges small test sets in Sec. 9, but some claims remain stronger than the uncertainty analysis supports.
- Main comparisons are not fully compute-matched. Table 1 shows STEPROUTER can use substantially more generated tokens than Direct, e.g., Qwen3-8B/LiveCodeBench 8,356 vs 6,536 (+27.85%, p.5). The same-pool controls help, but an equal-token-budget comparison against Direct/self-consistency/PRM would better isolate whether gains come from better allocation or more compute.
- Offline supervision cost is not fully analyzed. Utility labels require judge calls and explicit verification runs per candidate (Sec. 3.2; Sec. 7 cost analysis, p.8). These costs are excluded from inference cost and from Table 7d latency. This is a practicality concern for adopting the method, even if not a validity flaw.
Questions for authors
- How much does problem-conditioning the scorer matter? Can you report a variant that encodes the problem alongside the step, or analyze cases where the same step text has different utility across problems?
- For Table 1, what are the number of completions/seeds and confidence intervals for each backbone/task? Can you provide problem-cluster bootstrap intervals for the main gains?
- Can you add a compute-matched comparison at equal generated-token budget versus Direct, self-consistency, and PRM, especially for Qwen3-8B/LiveCodeBench where STEPROUTER uses more tokens?
2 comments
Dear Reviewer 6TJ3,
Thank you again for your helpful review. As the discussion period is drawing to a close, we wanted to gently check whether our response has addressed your questions. In brief, we added a problem-conditioned scorer variant, completions per problem and paired problem-level bootstrap CIs, a token accounting showing that StepRouter uses fewer tokens than Effort in all 9 cells (and outperforms 10-sample self-consistency on Qwen3-8B/AIME at about one-eleventh of the tokens), and the offline labeling cost (Table 13).
We would be grateful for any further comments, and are happy to provide additional clarification if helpful. Thank you for your time.
Best regards,
The Authors
Response to Reviewer 6TJ3
Thank you for the careful review. Revisions in the PDF are marked in blue; new results are in Appendix E.
Q1. How much does problem conditioning matter?
Empirically, little. We trained a variant that feeds [E(x);E(s)] to the same MLP, with the same data, split, and loss. On Qwen3-8B/AIME, with the same frozen pools and solver, it reaches 71.39, only 2.35 pp above Random (69.04), whereas the step-only scorer reaches 74.32 (5.28 pp above Random). It is consistent with two properties of the setting: extracted steps restate the problem's objects, quantities, and constraints, and since training pairs and routing are both within a problem, problem-only terms cancel. Step-only encoding also lets one frozen scorer transfer zero-shot to GPQA-D and MMLU-Pro (+5.15 pp, Table 7c). We therefore keep step-only encoding.
Q2. Completions and confidence intervals.
Accuracy is the per-problem mean over completions, averaged over problems (Eq. 6, App. A). Completions per problem in Table 1, for every backbone: 12 on AIME, 2 on GPQA-D, and 1 on LiveCodeBench. New intervals, all on Qwen3-8B:
The paper also reports GPT-5/AIME StepRouter − Random: +3.06 [+0.7, +5.4] pp. The direction is consistent across cells: StepRouter exceeds Effort in all 9 cells of Table 1.
Q3. Compute-matched comparison.
The gains are not bought with tokens. Throughout, Tok in Table 1 counts all generated tokens of the full pipeline, i.e. sketching, candidate extraction, and final solving (Table 1 caption; per-stage breakdown in Table 11), so StepRouter is charged for its extra stages.
StepRouter uses more tokens than Direct in two cells: GPT-5/AIME (9,026 vs 8,449) and Qwen3-8B/LiveCodeBench (+27.85%). In the latter, Effort uses more tokens than StepRouter (13,786 vs 8,356) yet scores 10.33 pp lower, and token-matched compression of all candidates reaches 65.3 vs 80.79 (Table 2). We have not run a token-matched Direct or self-consistency baseline for this cell.
Offline cost.
Labels are built once, for the 647 candidates of 126 development problems, and reused without retraining across all backbones and tasks. Following the labeling protocol (Sec. 3.2), this amounts to 647 verification runs, 647 with-step solves, 126 shared no-step solves, and 773 judge calls (Table 13). Half of the labels already beat Effort (71.32 vs 70.25, Table 7b). At inference there is no verification, solving, or judging; only the frozen MLP scores candidates.