acceptodds
Under review as a conference paper at ICLR 2027

Planned Test-Time Scaling with Coordinated Reasoning Paths

Abstract

Test-time scaling with parallel branches is widely adopted to improve performance on challenging reasoning tasks. The predominant approach, repeated sampling, draws branches independently from a single policy, which can produce redundant attempts and thereby limit the gains from additional inference compute. To address this limitation, we propose **Planned Test-Time Scaling (PTTS)**, which replaces independent sampling with a coordinated joint policy: a *planner* generates a solution outline for each branch, steering the branches toward distinct reasoning paths, and an *executor* produces a full solution conditioned on each outline. Formally, we show that PTTS strictly generalizes repeated sampling and, in a stylized setting, provably promotes coverage of complementary reasoning modes and yields better pass@k scaling. We instantiate PTTS on top of strong reasoning models, keeping them fixed as executors while replacing repeated sampling with PTTS inference to further enhance test-time scaling. Concretely, we develop two variants: **PTTS-ZS** prompts a model to jointly generate outlines for all branches in a single autoregressive pass, while **PTTS-RL** directly optimizes the planner against the pass@k reward using truncated execution rollouts for efficient training and a sharper reward signal. Across five mathematical reasoning benchmarks with Qwen3-1.7B and 4B, PTTS-ZS improves pass@64 over repeated sampling by up to 6.7 points, while PTTS-RL further increases the gain to up to 13.4 points. Further analysis indicates that broader coverage of distinct reasoning paths contributes to these gains. Overall, PTTS provides a general framework for improving test-time scaling by coordinating reasoning branches, with zero-shot and trainable instantiations that yield substantial performance gains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
reasoning, test-time scaling, diversity, efficiency

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BSCNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Planned Test-Time Scaling (PTTS), a framework that improves inference-time reasoning by coordinating multiple solution attempts through a planner-executor architecture. Instead of independently sampling solutions, PTTS jointly generates diverse solution outlines and uses a fixed reasoning model to execute each outline. The authors introduce two variants: PTTS-ZS, which relies on zero-shot planning, and PTTS-RL, which optimizes the planner using GRPO with a pass@k reward. The paper also provides theoretical analyses of reasoning-mode coverage and evaluates the proposed approach on five mathematical reasoning benchmarks using Qwen3-1.7B and Qwen3-4B models.

Critical strengths
  1. The proposed planner-executor decomposition provides a straightforward way to introduce structured diversity into test-time scaling without modifying the underlying reasoning model. The ability to use a relatively small planner with a stronger executor is particularly appealing from a practical perspective.

  2. The experiments demonstrate improvements over repeated sampling across multiple mathematical reasoning benchmarks. The ablations on branching factors, truncated execution, and planner-executor configurations provide useful insights into the behavior of the proposed framework.

Critical weaknesses
  1. the method uses a branching factor of only four and independently repeats the planning procedure for larger budgets. Consequently, the experiments do not demonstrate joint coordination across all 64 reasoning attempts, despite the broader motivation of coordinated test-time scaling. The computational efficiency claims are also insufficiently substantiated, as the additional planning overhead is not clearly accounted for in the comparison against repeated sampling.

  2. The central idea of generating diverse solution plans to guide multiple reasoning attempts is closely related to existing methods such as Guided Sampling and other plan-based exploration approaches. The main distinction appears to be the joint generation of outlines and the use of GRPO to optimize the planner. However, the current experiments do not convincingly isolate the benefits of these individual components. In particular, the paper lacks a comparison against a planner trained with conventional per-branch or average-accuracy rewards under the same training budget. Without such a baseline, it remains unclear whether the improvements of PTTS-RL primarily originate from the proposed joint pass@k objective or simply from additional reinforcement learning.

  3. the analysis of useful diversity is not entirely convincing. The proposed metric normalizes the number of distinct successful reasoning clusters by the total number of correct responses, making it sensitive to the number of successful samples. Therefore, a higher value does not necessarily demonstrate better coverage of complementary successful reasoning strategies. The reliance on LLM-based clustering without further validation also makes the interpretation of these results less conclusive.

Questions for authors

Could the authors compare PTTS-RL against a planner trained using conventional GRPO with per-branch accuracy or average correctness as the reward, while keeping the planner architecture, executor, training data, and computational budget identical? This comparison would help establish whether the proposed joint pass@k objective provides a substantial benefit beyond simply training the planner.

2 comments

[W1a] The experiments do not demonstrate joint coordination across all 64 reasoning attempts.

The main results (Figure 4 and 5) are measured with kk=4, demonstrating coordination among four branches rather than all 64. The pass@64 results show that coordination within small groups of reasoning attempts still yields substantial gains at larger inference budgets.

Our ablations (Figure 7) evaluate larger kk for PTTS-ZS. We further extend the experiments to kk=24, 32 and 64, directly coordinating up to 64 branches, with pass@64 results averaged on AIME2024~2026 datasets as follows:

kQwen3-1.7BQwen3-4B
k=1 (Repeated sampling)53.363.3
k=457.866.7
k=856.773.3
k=1258.970.0
k=1655.673.3
k=2458.270.7
k=3254.462.2
k=6445.652.2

We observe that small branching factors (kk=4) already yield substantial gains, with further improvements saturating around kk=10~20 and declining at $k\ge$32, as zero-shot planners struggle to generate long lists of outlines. This shows that PTTS is a general algorithm that can scale to larger kk to coordinate a larger number of attempts. Due to limited computational resources, we leave extending PTTS-RL to larger kk for future work.

[W1b] The computational efficiency comparison does not account for the additional planning overhead.

PTTS incurs negligible inference overhead when accounting for the planning stage, as shown in the table below. Planning accounts for only 1.3~2.2% of the total cost, as outlines are designed to be short, high-level reasoning strategies.

Token cost per branch, averaged across datasets:

Qwen3-1.7BQwen3-4B
Repeated sampling8.3k8.1k
PTTS-ZS8.2k7.1k
  planning stage0.1k (1.6%)0.1k (1.7%)
PTTS-RL8.0k8.1k
  planning stage0.1k (1.3%)0.2k (2.2%)

We replot the main results (Figure 4 and 5) with inference cost measured by total generated tokens (planner + executor) at here and here. The overall trend remains unchanged: with matched inference budget, PTTS-RL consistently performs the best, followed by PTTS-ZS.

[W2] [Q1] The paper lacks a comparison against a planner trained with conventional per-branch or average-accuracy rewards under the same training budget.

We add a baseline where the planner generates a single outline for the executor and is trained with a pass@1 reward. We train the planner from Qwen3-1.7B-Base and use Qwen3-1.7B as the executor, keeping all other training configurations unchanged.

The pass@1-trained planner achieves an average pass@64 of 60.2 across five benchmarks, outperforming PTTS-ZS (58.8) but falling short of PTTS-RL (62.2). Full results are shown at here. These results demonstrate that PTTS-RL's gains stem not only from planner training but also from optimizing the joint pass@k objective.

[W3] The analysis of useful diversity is sensitive to the number of successful samples and relies on LLM-based clustering.

Useful diversity measures whether increased diversity contributes to successful reasoning rather than merely producing more distinct but incorrect solutions. We thus count distinct reasoning clusters only among correct responses. However, as the reviewer notes, this raw count is sensitive to the number of correct responses. To address this sensitivity, we normalize by the total number of correct responses, measuring the proportion of distinct successful reasoning paths rather than their absolute count.

We further validate the two stages of the LLM-based diversity analysis pipeline, concept extraction and concept clustering, against human annotations on AIME2024~2026 outputs. Concept extraction achieves 93.3% agreement with human annotations across 300 responses, while concept clustering achieves 81.1% agreement across 60 groups of 64 responses, supporting the reliability of the pipeline.

Dear Reviewer BSCN, thank you again for your time and feedback. We would appreciate hearing whether our responses and additional results have addressed your concerns. Please let us know if you have any remaining questions.

Reviewer_R7kxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Planned Test-Time Scaling (PTTS) replaces independent sampling with coordinated reasoning branches. A planner generates multiple solution outlines in one pass, and a frozen executor solves each branch. PTTS-ZS uses a zero-shot planner, while PTTS-RL trains the planner with GRPO using a pass@k reward. Experiments with Qwen3-1.7B/4B on math benchmarks report up to 6.7 gain in pass@64 for PTTS-ZS and 13.4 gain in pass@64 for PTTS-RL. PTTS-RL at pass@32 also outperforms repeated sampling at pass@64. A stylized analysis explains how joint planning can avoid mode collapse and improve coverage.

Critical strengths
  • PTTS-ZS improves pass@k with minimal changes to existing reasoning models, which makes it easy to integrate into existing systems. PTTS-RL provides a trainable extension while keeping the executor frozen. The 4k-token training truncation is supported by ablations, and transfer across planner/executor sizes and to the diversity-trained e3 executor strengthens the main contribution(s).

  • Analysis shows why pass@1 optimization can collapse onto one mode while pass@k optimization favors complementary strategies. Empirically, PTTS-RL increases diversity among correct solutions, which supports the proposed explanation for its gains.

Critical weaknesses
  • The evaluation as presently done relies almost entirely on pass@k, which assumes access to a perfect verifier. However, in real-world scenarios, test-time scaling must select one answer. Increased diversity may improve coverage while hurting majority voting, so the practical benefit remains unclear.

    The claimed 2x compute efficiency counts executor samples but ignores planner tokens and RL training cost (though I suppose the latter is significant yet ignorable in the context of the overall evaluation). Without token-matched comparisons and realistic answer selection, neither the accuracy nor efficiency claim is properly established.

  • AIME and HMMT have roughly 30 problems each, so one problem changes accuracy by about 3.3. Some configurations also underperform, for example, Guided Sampling beats PTTS on HMMT-2026 with the 4B model, and certain planner/executor pairings reduce pass@1.

    The paper omits cheap diversity baselines such as higher sampling temperature, varied strategy prompts, and pass@k-trained executors. Without these comparisons, it is unclear how much benefit comes from coordinated planning rather than simply increasing diversity. The diversity analysis also relies maibly on unvalidated LLM-generated labels.

Questions for authors
  1. How does PTTS perform under majority voting or reward-model-based best-of-k selection? Does increased diversity improve or hurt final answer accuracy?

  2. Does claimed 2x efficiency hold when planner tokens are included? Maybe also mention amortized training cost of PTTS-RL?

  3. How does PTTS compare with higher-temperature sampling, varied strategy prompts, and pass@k-trained executors under matched compute budgets?

  4. What distinguishes PTTS from prior multi-strategy planning methods (Kang & Zhang, 2025; Li et al., 2026), beyond single-pass joint generation and stylzed analysis?

  5. Do (or would) the gains persist on larger executors and non-math tasks?

References

  • Kang, Shijia, and Muhan Zhang. "The Road Less Traveled: Enhancing Exploration in LLMs via Sequential Sampling." arXiv preprint arXiv:2510.15502 (2025).
  • Li, Yilong, Suman Banerjee, and Tong Che. "Cast a Wider Net: Coordinated Pass@ K Policy Optimization for Code Reasoning." arXiv preprint arXiv:2605.27000 (2026).
4 comments

[W1] [Q1] The evaluation relies on pass@k, but increased diversity may improve coverage while hurting majority voting.

We report majority voting performance with different kk in here and here. The trend is consistent with the pass@k results, with PTTS methods outperforming the baselines, suggesting that the increased diversity does not come at the expense of majority-voting accuracy.

[W2] [Q2] The compute efficiency analysis ignores planner tokens and RL training cost.

When accounting for the planner tokens, PTTS incurs negligible inference overhead, as shown in the table below. Planning accounts for only 1.3~2.2% of the total cost, as outlines are designed to be short, high-level reasoning strategies.

Token cost per branch, averaged across datasets:

Qwen3-1.7BQwen3-4B
Repeated sampling8.3k8.1k
PTTS-ZS8.2k7.1k
  planning stage0.1k (1.6%)0.1k (1.7%)
PTTS-RL8.0k8.1k
  planning stage0.1k (1.3%)0.2k (2.2%)

We replot the main results (Figure 4 and 5) with inference cost measured by total generated tokens (planner + executor) at here and here. The efficiency gain persists when accounting for total generated tokens: PTTS-RL achieves higher performance than RS (56.8 vs. 55.8) using 192.8k tokens per problem, compared to 528.5k for RS.

For RL training, the cost is approximately 29 hours for the 1.7B model and 42 hours for the 4B model, both on 8xA100 GPUs. However, this is a one-time cost incurred before deployment and does not add to inference cost, which is the focus of our efficiency claims.

[W3] AIME and HMMT have roughly 30 problems each and may lead to variability, and some configurations underperform.

We agree that results on individual datasets can exhibit high variance. To provide a more robust comparison, we evaluate across five benchmarks and report average performance. As shown in Figures 4 and 5, PTTS outperforms the baselines on average, with PTTS-RL improving pass@64 over repeated sampling by 6.4 points for Qwen3-1.7B and 3.6 points for Qwen3-4B.

[W4a] [Q3] Comparison against baselines such as higher sampling temperature, varied strategy prompts, and pass@k-trained executors.

For sampling temperature, we experiment with increased temperature of the executor and compare PTTS-ZS against repeated sampling at each setting. As shown in the table below, PTTS-ZS consistently maintains a gain over repeated sampling. We show the full curve at here.

RSPTTS-ZS
t=0.7 (main setting)46.850.2 (+3.3)
t=1.050.250.9 (+0.7)
t=1.248.553.5 (+5.0)
t=1.446.149.2 (+3.2)

For pass@k-trained executors, we view them as complementary to PTTS rather than competing baselines, since executor-level diversity improvements can be combined with cross-branch coordination. In §4.3 (L462-471), we demonstrate this by adopting e3, a model trained via RL to improve exploration, as the executor. PTTS-ZS improves its pass@64 from 57.8 to 62.2 (+4.4) on AIME2024~2026, showing that PTTS provides additional gains on top of diversity-aware executor training.

[W4b] The diversity analysis relies mainly on LLM-generated labels.

To verify the reliability of LLM-generated labels for the diversity analysis, we validate the two stages of the LLM-based diversity analysis pipeline, concept extraction and concept clustering, against human annotations on AIME2024~2026 outputs. Concept extraction achieves 93.3% agreement with human annotations across 300 responses, while concept clustering achieves 81.1% agreement across 60 groups of 64 responses, supporting the reliability of the pipeline.

[Q5] What distinguishes PTTS from prior multi-strategy planning methods beyond single-pass joint generation and stylzed analysis?

The main distinctions are: (1) PTTS provides a unified joint-policy formulation of coordinated reasoning, enabling formal analysis of the benefits of coordination beyond empirical observations. (2) The planner's single-pass joint generation design enables direct RL optimization of the joint pass@k objective, which is difficult with multi-stage planning pipelines. This allows PTTS-RL to learn effective coordination and achieve further gains over PTTS-ZS.

[Q6] Would the gains persist on larger executors and non-math tasks?

In §4.3 (L449-461), we study the effect of executor sizes and find that smaller planners can effectively guide larger executors, where the 1.7B planner paired with the 4B executor improves pass@64 over repeated sampling by 4.4 points on AIME2024~2026 (70.0 vs. 65.6), matching the performance of the 4B planner (70.0). For non-math tasks, we agree that the evaluation is important to assess the generalizability of PTTS, but leave it for future work due to computational constraints.

Thanks for the clarifications and additional experiments. The majority-voting results, token-cost analysis, and diversity validation address some of my concerns. Limitations do remain regarding baselines and generalization, so I'll keep my score.

Thank you again for your time and effort in reviewing our work and for your thoughtful consideration. We truly appreciate it!

Reviewer_Vj8tOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a simple approach to improving diversity in sampling, but having a planner model first suggest a plan for each roll out, which is then used by the executor to generate the sample. The planner is optimized to improve pass@K during RL, with a frozen executor. Using different plans for rollouts allows increased diversity amongst samples, boosting pass@k.

Critical strengths

The paper is well motivated and easy to follow, and the proposed approach is straightforward and intuitive. Results show significant improvement in pass@k, and there is some nice analysis

Critical weaknesses

It would be good to show how well this method works for other model families and scales - particulary to understand if the impact of the approach increases or decreases with scale It would be interesting to add a baseline using independently sampled plans, to verify how much of the improvement can be explained simply by using plans

Questions for authors

n/a

2 comments

[W1] It would be good to show how well this method works for other model families and scales, particularly to understand if the impact of the approach increases or decreases with scale.

In the main experiments, we evaluate PTTS at two model scales (1.7B and 4B) and observe consistent gains at both. In §4.3 (L449-461), we further study the effect of planner and executor sizes, finding that smaller planners can effectively guide larger executors, where the 1.7B planner paired with the 4B executor improves pass@64 over repeated sampling by 4.4 points on AIME2024~2026 (70.0 vs. 65.6), matching the performance of the 4B planner (70.0). Due to computational constraints, we leave evaluation across additional model families and scales for future work.

[W2] It would be interesting to add a baseline using independently sampled plans, to verify how much of the improvement can be explained simply by using plans.

We add a baseline where the planner independently generates an outline for each reasoning branch, which then guides the executor to generate a full solution. We use Qwen3-1.7B as the executor and train the planner from Qwen3-1.7B-Base with a pass@1 reward.

The pass@1-trained planner achieves an average pass@64 of 60.2 across five benchmarks, outperforming repeated sampling (55.8) but falling short of PTTS-RL (62.2). Full results are shown at here. These results show that independent planning already provides benefits, while joint planning with pass@k optimization yields further gains through branch coordination.

Dear Reviewer Vj8t, thank you again for your time and feedback. We would appreciate hearing whether our responses and additional results have addressed your concerns. Please let us know if you have any remaining questions.

Comments on the paper

1 comment

Hi Reviewers,

The authors have provided responses. Please check if they have addressed your concerns.

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.