acceptodds
Under review as a conference paper at ICLR 2027

Global Selection, Local Credit: Decoupled Topology Search for Adaptive Test-Time Scaling

Abstract

Feedback-guided inference-time tree search is widely used to improve the reasoning of large language models on complex tasks. Existing adaptive methods use feedback to allocate a limited inference budget between generating new solutions and revising existing ones. However, they still traverse the tree level by level to choose a solution to revise, even though a complete solution can serve directly as the starting point. Consequently, whether a candidate receives further computation depends on both its estimated value and its position in the search tree, which can lead to inefficient allocation and limit search performance. To allocate inference compute more efficiently, we propose Decoupled Topology Search (DTS), an inference-time framework that searches over complete solutions. Before each model call, DTS uses the available feedback to decide globally whether to generate a new solution or revise an existing one. If it chooses revision, it selects directly from all valid candidate solutions based on their current quality estimates and uncertainty, without traversing the tree level by level. DTS retains the links through which candidates were generated to propagate feedback and continually update their value estimates. We evaluate DTS on code generation, mathematical reasoning, and molecular optimization. At the same candidate-call budget, DTS outperforms repeated sampling and existing tree-search baselines. On LiveCodeBench v6, DTS improves the Pass@1 of Qwen3-14B by up to 7.2 percentage points over repeated sampling.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, test-time scaling, inference-time search, iterative refinement, code generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_neGqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies test-time scaling for large language models under a fixed inference budget, with a particular focus on how computation should be allocated between generating new solutions and refining existing ones. The authors argue that existing methods such as MCTS and AB-MCTS still rely on ancestor-by-ancestor traversal when selecting a refinement parent, causing access to a complete candidate to depend jointly on its estimated quality and its position in the search tree. To address this issue, the paper proposes Decoupled Topology Search (DTS), which separates global refinement-parent selection from lineage-based information sharing. DTS uses Thompson sampling to adaptively choose between fresh generation and refinement. When refinement is selected, all valid complete candidates compete globally for the next model call, using either Expected Improvement or a Pareto–Thompson acquisition rule. Parent-child relations are retained only for forward inheritance and backward credit propagation of statistical evidence. The method is evaluated on code generation, mathematical reasoning, and molecular optimization against repeated sampling, sequential refinement, MCTS, and AB-MCTS baselines. On the main code-generation benchmarks, DTS consistently improves final-answer accuracy across multiple Qwen3 and AReaL settings, with a maximum gain of 7.2 percentage points over repeated sampling on LiveCodeBench v6 with Qwen3-14B. Additional topology controls, ablations, and low-budget diagnostics indicate that global parent selection, adaptive computation allocation, and lineage-based evidence sharing all contribute to the observed gains, while also revealing settings in which feedback-guided search is less effective.

Critical strengths
  1. The paper identifies a clear and meaningful structural limitation of complete-solution tree search. The distinction between using lineage for statistical information sharing and using lineage as an access path for refinement is conceptually clean. Decoupling these two roles gives DTS a clear methodological identity relative to standard MCTS and AB-MCTS.

  2. The paper includes a well-targeted control for its central claim. The coupled-DTS comparison keeps the controller, acquisition mechanism, and lineage updates fixed while changing the refinement-parent topology. This substantially strengthens the causal argument that globally selecting complete candidates is useful beyond the other components of DTS.

Critical weaknesses
  1. Matching the number of candidate calls does not fully establish compute efficiency. A refinement call can involve a substantially longer prompt because it includes the parent solution and feedback, while repeated sampling uses only the task prompt. In addition, DTS-EI performs nontrivial numerical acquisition work over the candidate pool. The paper explicitly notes that candidate-call matching does not equalize token usage or wall-clock cost.

  2. The lineage-transfer weight K is important to the method but remains largely heuristic. The surprisal-weighted token-probability formula does not have an obvious probabilistic interpretation as the degree to which a child's observation should update its parent or ancestors. The ablations show that K-aware transfer is useful in one major setting, but the evidence is concentrated on LiveCodeBench v6 with Qwen3-14B. The paper would benefit from a stronger justification of this weighting rule and from sensitivity or alternative-weight experiments across multiple models/tasks.

Questions for authors
  1. A closely related prior work (which the authors cited in the 3rd paragraph, page 3, context: "... Bayesian methods guide program refinement (Tang et al.)..."), formulates LLM program refinement as an arm-acquiring bandit problem and uses Thompson sampling to allocate refinement calls among candidate programs under an exploration–exploitation tradeoff. This appears closely related to DTS's global competition among complete candidates for refinement budget. Could the authors clarify the precise algorithmic and conceptual differences between DTS and Tang et al. (2024), beyond the addition of lineage-based credit propagation? In particular, does the prior bandit formulation already permit direct selection of any existing complete candidate independent of its generation lineage? If so, which part of “decoupled topology search” is genuinely new relative to that formulation?

  2. Could the authors further evaluate the robustness of DTS across a broader range of non-code tasks and feedback conditions? The strongest gains are concentrated on code-generation benchmarks. On AIME-Base, DTS-EI improves over repeated sampling by only 0.6 percentage points, while DTS-Pareto performs 2.7 points worse. On PMO, DTS-EI outperforms repeated sampling, whereas DTS-Pareto again underperforms it. It would therefore be useful to include 1-2 additional mathematical reasoning, natural-language reasoning, or other tasks with different levels of feedback noise, and to analyze why the relative performance of the two acquisition rules changes across domains. Such experiments would help clarify whether the benefits of DTS generalize broadly or depend strongly on the task and feedback structure.

p.s.: There seems to be some formatting issues (in page 5, Alg. 1: Eq. equation 7 -> Eq. 7; page 20, Figure 7 title: 8B_model [which seems to be a script name in code editor rather than a formal name to be illustrated in the paper], etc.). However, these issues merely require the author's attention and are not the primary basis for my rating.

8 comments

Thank you for recognizing the distinction between selection and evidence sharing, and the value of the coupled-DTS control. We address actual cost, the relationship to REx, and robustness below.

1. Token cost and acquisition overhead

DTS achieves higher accuracy than AB-MCTS-A/M with lower average token usage per saved log on Qwen3-14B/LiveCodeBench v6. Final accuracies are three-run means at 128 candidate calls. Token counts are averages per saved message body, excluding chat templates.

MethodInput tokens/logOutput tokens/logTotal tokens/logFinal accuracy (%)
Repeated sampling59413,31113,90564.4
Sequential refinement9907,3948,38467.8
Standard MCTS9527,9728,92465.7
AB-MCTS-A8169,41710,23365.1
AB-MCTS-M8689,64810,51764.2
DTS-EI8578,9899,84670.6
DTS-Pareto8489,0989,94571.6

DTS-EI/Pareto use 3.8%/2.8% fewer tokens per saved log than AB-MCTS-A and 6.4%/5.4% fewer than AB-MCTS-M. Relative to repeated sampling, the reductions are 29.2%/28.5%. Compared with sequential refinement, DTS uses 17.4%/18.6% more tokens per saved log, with accuracy gains of 2.8/3.8 percentage points.

Token usage also depends on the model. On Qwen2.5-7B-Instruct, the corresponding totals are 851 tokens/log for repeated sampling and 1,166/1,149 for DTS-EI/Pareto.

EI acquisition is computed numerically from candidate statistics and requires no additional LLM calls (Appendix B.5).

2. What DTS adds beyond REx

DTS advances AB-MCTS-style adaptive search through global action allocation and bidirectional evidence sharing. REx already permits global parent selection, but its single fresh-generation arm competes with a growing pool of candidate arms. DTS learns GEN/CONT separately from parent acquisition, so pool growth does not directly add competitors to GEN. REx also keeps each program's search statistics local; DTS passes accumulated evidence to new children and returns descendant feedback to ancestors. These differences address how calls are allocated and how much information each call can contribute. EI or Pareto acquisition then selects a parent without a preset branching limit, allocation ratio, or tuned exploration coefficient.

The direct REx comparison uses the same 175 LCBv6 problems, Qwen3-14B, 128 candidate calls, and final selection rule, with early stopping disabled. REx uses public pass-rate feedback during search. Final accuracy is 63.43 ± 0.00% for REx over two runs, versus 70.60 ± 0.20%/71.60 ± 0.60% for DTS-EI/Pareto over three runs. Values are means ± SD.

3. Evidence for K and transfer across models

The completed fixed-weight controls support using a weight that varies with each refinement. On Qwen3-14B/LCBv6, Pareto obtains 71.60 ± 0.60%, compared with 63.14 ± 2.02% for fixed K = 0.883626, 63.43 ± 0.81% for K = 0.5, and 66.50 ± 1.20% for K = 1. The first two controls have two runs; the others have three.

Across three models and both acquisition rules, K correlates with lexical parent–child code overlap (Spearman 0.495–0.648). This supports using K to vary how much evidence related candidates share. Forward inheritance also raises Qwen3-8B accuracy from 58.67 ± 2.10% to 61.14 ± 0.00% for EI and from 58.29 ± 2.03% to 63.05 ± 0.27% for Pareto, over three runs. Qwen2.5 results are mixed.

4. Non-code tasks and differences between EI and Pareto

The existing non-code results cover both mathematical reasoning and molecular optimization, with different feedback sources. The following are three-run means ± SD:

Setting and metricRepeated samplingDTS-EIDTS-Pareto
AIME-Base accuracy (%)78.3 ± 1.478.9 ± 0.875.6 ± 2.1
AIME-RM accuracy (%)60.0 ± 3.663.9 ± 3.460.6 ± 1.6
PMO final objective3.7775 ± 0.02873.8051 ± 0.03843.7182 ± 0.0352

AIME uses the same 60 problems and Qwen3-8B at 64 calls, with oracle feedback in Base and ReasonFlux-PRM feedback in RM. PMO uses GPT-4o-mini at 128 calls and five molecular objectives. On AIME-RM, EI achieves higher final accuracy despite lower coverage than repeated sampling (74.4% versus 78.3%), extending the coverage–selection observation beyond code.

EI scores expected improvement above the current best mean; Pareto–Thompson samples among candidates that trade off estimated reward and uncertainty. These rules allocate calls differently, so their relative value can depend on the task's refinement opportunities and feedback. Pareto also trails repeated sampling with oracle AIME feedback and on PMO, so feedback noise alone cannot explain the reversal. The non-code results show that the choice of acquisition rule matters across tasks.

For the typographical points, the intended references in Algorithm 1 are Eqs. (7) and (2), and the model in Figure 7 is Qwen3-8B.

Dear Reviewer neGq,

Thank you for your careful review and constructive suggestions.

With the author–reviewer discussion period approaching its end, we would sincerely appreciate it if you could review our rebuttal and the newly added experimental comparisons, particularly our clarification of the differences between DTS and REx.

We would be grateful for any further feedback on whether the additional evidence addresses your concerns.

Thank you very much for your time and consideration.

Best regards, The Authors

Reviewer_p8h6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Decoupled Topology Search (DTS), an adaptive test-time search framework that separates refinement-parent selection from lineage-based feedback sharing. Its central observation is that complete solutions can be refined directly, without the ancestral traversal imposed by conventional tree-search methods. DTS combines global candidate acquisition, adaptive generation/refinement allocation, and lineage-aware statistical updates. Experiments show improvements over repeated sampling and tree-search baselines, with additional analyses of search topology, candidate coverage, and final-answer selection.

Critical strengths

S1. An interesting research perspective and a clearly motivated design.

I appreciate the paper's starting point, particularly its distinction between the selection and evidence-sharing roles of a search tree. Decoupling these two roles is an interesting and intuitive perspective on inference-time search. Although I still have concerns about the novelty and broader significance of this formulation (see W1 and W2), I believe the underlying research question is worth investigating.

S2. Reasonably extensive experiments and informative analyses.

The experimental coverage is reasonably broad, spanning multiple models, benchmarks, and task domains, together with component ablations and controlled comparisons. Some improvements are relatively modest, but I personally place greater weight on the insight and significance of a method than on isolated benchmark gains.

I also appreciate the analyses beyond headline accuracy. In particular, the observation that DTS can achieve higher Pass@1 despite lower oracle coverage (Section 5.2, Table 2) is interesting. It suggests that the advantage may lie not simply in discovering more correct solutions, but in how the search process organizes candidates and converts them into successful final outputs. I would welcome further analysis of this phenomenon.

Critical weaknesses

W1. Novelty with respect to existing global refinement approaches.

The authors motivate their work through the observation in the Introduction that "Sharing feedback therefore does not require selecting refinement targets along ancestral paths." I think this is a good starting point. However, I am not fully convinced that the novelty of the resulting framework has been sufficiently established.

In particular, REx (Tang et al., NeurIPS 2024) and DTS share several important elements, including Beta distributions, Thompson Sampling, complete-program refinement, and dynamically growing candidate pools. More importantly, REx already allows refinement targets to be selected directly from a global program pool, rather than requiring ancestral traversal.

The Related Work section describes REx rather briefly as an instance of "Bayesian methods [that] guide program refinement." In my view, this characterization substantially understates its methodological proximity to DTS.

The distinction between DTS and existing global parent-selection approaches is therefore central to assessing the actual contribution. I would like to see a more careful analysis of what is fundamentally different, supported by direct experimental comparisons rather than only comparisons with conventional tree-search baselines. [Q1]

W2. Whether the broader research significance is sufficiently established.

I understand that the authors deliberately focus on inference-time search over complete solutions, as stated in the Introduction: "We study the latter setting." It is reasonable to define this scope. Nevertheless, I remain concerned about whether optimizing this particular family of tree-search strategies is sufficiently meaningful from a broader inference-time scaling and practical deployment perspective.

Existing research has demonstrated that tree-based search is not always the most computation-efficient way to improve reasoning, and that alternative sequential or controlled reasoning strategies can achieve competitive performance without extensive tree expansion. Relevant examples include s1: Simple Test-Time Scaling (Muennighoff et al., 2025), Chain-in-Tree (Li, 2026), and On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference (Yu et al., 2026). State-conditioned reasoning approaches, such as State of Thought (Gong et al., 2026), also offer a different perspective on allocating inference computation.

These studies do not establish that such strategies necessarily outperform DTS under identical conditions. However, they raise an important question: even if DTS improves upon conventional MCTS-style baselines, is it competitive with other practical ways of spending the same inference budget?

This matters because an improvement within the tree-search family may have limited deployment value if a simpler sequential reasoning strategy or a lightweight adaptive controller achieves comparable or better accuracy at lower actual cost.

The existing repeated-sampling and sequential-refinement baselines are useful, but they do not fully address this concern. I would therefore appreciate comparisons with at least one representative direct reasoning strategy and one adaptive reasoning/control strategy, using the same backbone and tasks where feasible, with reasonably matched token or computational budgets. [Q2]

A related concern is the paper's reliance on the concept of generation lineage. I understand its structural definition as the parent-child relationships through which candidate solutions are produced, and I see why these relationships can be useful for evidence sharing. However, I am less convinced that the paper provides sufficient empirical evidence for the claimed effect of lineage-constrained access on actual search behavior. Figure 1 gives an illustrative argument, and Table 3 provides useful topology controls, but the underlying mechanism could be analyzed more directly. I would especially like to understand how often ancestral traversal actually prevents or delays the selection of promising refinement targets, and how much of the observed improvement can be attributed to this effect. [Q2.0]

Presentation notes (not separate critical weaknesses, with my personal view). I genuinely enjoyed the writing style and logical organization of the paper. The exposition is clear and comfortable to follow. I have a few suggestions that may further improve the presentation:

  • Figure 1 and the motivation. I understand the illustrative example, and the caption does clarify that the probabilities and feedback are illustrative. However, since this figure motivates much of the paper, I would particularly appreciate a deeper empirical investigation of this phenomenon, preferably with real search trajectories and statistical summaries. A dedicated analysis would make the motivation considerably more convincing.
  • Terminology and notation. Please clarify why refinement is denoted as CONT (presumably continuation). Also, some figures use NN for the generation budget while the main text uses BB; consistent notation would improve readability.
  • Pareto-based selection. If the Pareto acquisition mechanism is an important part of the method, I would welcome a more visible analysis in the main text, ideally with an empirical visualization of its selection behavior. The existing schematic is helpful, but a deeper analysis would improve the presentation.
  • Code availability. I generally value papers that provide reproducible implementations.
Questions for authors

The following questions correspond to my concerns above. I would appreciate clarification and, where feasible, additional empirical evidence.

[Q1] Novelty and comparison with global refinement methods.

Could the authors clarify the essential differences between DTS and REx, especially regarding global parent selection, adaptive generation/refinement allocation, and Beta-based updates? Please provide a direct comparison or a closely matched global-selection baseline to establish the additional contribution of DTS.

[Q2] Broader significance and comparison with alternative reasoning strategies.

Could the authors compare DTS with at least one representative direct/sequential reasoning approach and one adaptive reasoning/control approach, using comparable backbone models, tasks, and actual inference budgets? I am particularly interested in whether the benefits persist when accounting for token consumption and computational cost.

[Q2.0] The empirical role of generation lineage.

Could the authors clarify the precise role of generation lineage in the claimed limitation of existing search methods? Beyond the illustrative argument in Figure 1, is there empirical evidence quantifying how frequently ancestral traversal delays access to promising candidates, and how this relates to the improvements observed in Table 3?

[Q3] Could the information associated with different search depths be exploited rather than bypassed?

I understand the motivation for abandoning level-by-level traversal, and I find this an insightful direction. However, a candidate's depth and its relationship to other nodes may still carry useful information about refinement potential.

Have the authors considered a depth-aware or probability-aware adaptive global-selection strategy that retains this information without requiring ancestral traversal? It would be interesting to investigate whether this could improve upon the current DTS acquisition rules.

[Q4] Technical questions about the statistical design.

[Q4.1] GEN/CONT controller and its optimization signal.

My understanding is that the controller in Section 3.2 learns from the absolute observed reward of each action, rather than its marginal improvement or expected contribution to final-answer correctness.

For example, suppose a program already passes all public tests. If repeated refinements continue to pass these tests, the CONT state repeatedly receives positive feedback even when these refinements do not improve hidden-test correctness.

Could the authors compare this design with a reward-improvement-based or recovery-based controller? I would also appreciate an explanation of why absolute reward is a suitable signal for allocating further computation between GEN and CONT.

[Q4.2] Interpretation of candidate-level Beta states.

The candidate-level Beta distributions in Section 3.4 are interesting, but their probabilistic interpretation is not entirely clear to me. These states are updated through inherited parent statistics and backward propagation of descendant feedback.

However, candidate correctness and candidate refinability are not necessarily the same quantity. An incorrect candidate may be close to a correct solution and easy to repair, while a candidate with high observed reward may offer little potential for further improvement.

What exactly is each Beta state intended to estimate: the candidate's current quality, its probability of successful refinement, or a heuristic combination of these quantities? Could the authors provide empirical diagnostics, such as the relationship between estimated candidate values and observed refinement success rates, to support this interpretation?

[Q4.3] Justification of the transfer weight KK.

The surprisal-weighted transfer coefficient in Eq. (5) appears to be one of the most important heuristics in DTS:

\frac{\sum_{j=1}^{m} P_j(-\log P_j)} {\sum_{j=1}^{m}(-\log P_j)}.$$ I understand that lower-probability tokens receive greater surprisal weighting, while $K_c$ remains a weighted average of token probabilities. However, **why should token predictability reliably measure the amount of statistical evidence that can be transferred between a parent and its refined child?** These quantities do not appear to have an obvious or necessarily stable relationship. Could the authors empirically examine whether $K_c$ correlates with parent-child similarity, refinement outcomes, or the usefulness of shared evidence? Comparisons with simpler fixed or alternative transfer weights would also help establish why this particular formulation is appropriate. --- I would be grateful if the authors could address these concerns, particularly those related to **novelty and the empirical evidence supporting the method's underlying motivation**. These points are central to my assessment of the paper's significance and contribution.
7 comments

Thank you for your questions. We address the contribution, search behavior, and statistical design below.

1. Contribution relative to REx (Q1)

DTS extends AB-MCTS with global action allocation, global parent selection, and bidirectional evidence sharing. REx already offers global program selection, but fresh generation competes as one arm against the growing candidate pool, and accumulated search statistics are not shared across the lineage. DTS separates GEN/CONT allocation from parent selection and lets related candidates reuse evidence through forward inheritance and weighted ancestor updates. EI or Pareto acquisition then selects parents. This design adapts allocation and evidence sharing without a preset branching limit or hand-tuned allocation and transfer weights.

The completed direct comparison uses Qwen3-14B, the same 175 LCBv6 problems, 128 candidate calls, no early stopping, and the same final selector. REx uses public pass rates during search. Final accuracy is 63.43 ± 0.00% for REx (two runs), versus 70.60 ± 0.20%/71.60 ± 0.60% for DTS-EI/Pareto (three runs). Values are means ± SD. DTS thus improves accuracy over a baseline that already uses global access.

2. How lineage affects selection (Q2.0 and Q3)

Ancestral routing makes access depend on first selecting the candidate's ancestor branches. In Table 3, changing only this access rule raises Qwen3-14B accuracy from 63.81 ± 1.35% to 70.60 ± 0.20% for EI and from 65.33 ± 0.97% to 71.60 ± 0.60% for Pareto.

In one complete 175-problem run per rule, 42.36%/22.51% of EI/Pareto refinements revisit candidates that already have children. Replaying global and ancestral selection on identical recorded states with shared random draws changes the selected parent in 71.86%/81.61% of comparisons. This quantifies how routing affects parent selection.

A real Pareto trajectory revisits candidate 7 on the global frontier. Offline evaluation confirms that its earlier child fails hidden tests, while a later child repairs loop boundaries and passes. This illustrates how global acquisition can return to an earlier candidate after an unsuccessful refinement.

Depth and probability information (Q3). DTS uses parent–child relationships for inheritance and ancestor updates, and token probabilities determine the transfer weights. These signals guide evidence sharing while all valid candidates remain accessible. Explicit depth-based acquisition is a possible extension.

3. Controller, Beta states, and transfer weights (Q4)

Q4.1. Feedback for GEN/CONT

Absolute reward puts GEN and CONT on the same scale. If a parent and its refinement both pass public tests, reward improvement is zero even when the refinement repairs a hidden error. Absolute reward still credits the action for producing a publicly passing candidate. The fixed 50/50 control reaches 64.60 ± 2.20%, versus 71.60 ± 0.60% for adaptive Pareto (three runs); this supports learning the generation/refinement balance from feedback.

Q4.2. Meaning of the Beta state

The Beta state is a heuristic summary of reward evidence used to select the next candidate to refine. It combines the candidate's own reward, inherited parent evidence, and descendant feedback. Its mean summarizes the reward signal, and its concentration represents the amount of accumulated evidence. EI and Pareto use these statistics to balance reward and uncertainty.

The sharing ablations support this use. On Qwen3-14B, Pareto accuracy is 71.60 ± 0.60% with full sharing, 65.50 ± 2.30% without forward inheritance (three runs each), and 62.00 ± 0.40% when backward updates reach only the direct parent (two runs).

Q4.3. Transfer weight K

K controls how much evidence related candidates share. It is computed from token probabilities conditioned on the parent and its feedback. The motivation is that more predictable revisions tend to preserve more of the parent, making evidence sharing more useful. Across three models and two acquisition rules, K correlates with lexical code overlap (Spearman 0.495–0.648).

Qwen3-14B/LCBv6 Pareto accuracy is 71.60 ± 0.60% with dynamic K, versus 63.14 ± 2.02%, 63.43 ± 0.81%, and 66.50 ± 1.20% with fixed K = 0.883626, 0.5, and 1. The first two controls have two runs; the others have three. These controls support adapting the transfer weight to each parent–child pair.

4. Broader significance and presentation (Q2)

Complete-candidate search supports both test-time inference and RL trajectory collection. MARTI-MARS² uses multi-agent refinement trajectories for policy learning, and its inference method builds on Multi-Agent AB-MCTS. DTS retains complete candidates, evaluation feedback, and parent–child lineage while changing search allocation. This preserves the candidate and lineage information needed by such rollouts, making DTS a compatible search component at this interface. The connection gives a concrete application for better allocation beyond selecting a final answer: collecting trajectories for learning.

On Qwen3-14B, DTS-Pareto reaches 71.6%, compared with 67.8% for sequential refinement and 64.4% for repeated sampling. Saved-log averages are 9,945, 8,384, and 13,905 tokens respectively: DTS has higher accuracy, with per-log token usage between the two baselines. The token figures are message-body averages per saved log under the same candidate-call budget.

The suggested methods and DTS address the following allocation decisions:

MethodMain decision
s1How long to continue a reasoning trace through budget forcing
Chain-in-TreeWhen to branch during stepwise tree search
Sequential Reward FilteringWhich high-reward generations to retain as context for later generations
State of ThoughtWhich historical reasoning support to activate given the model's reasoning state
DTSWhether to generate or refine a complete candidate, which parent to select globally, and how to share evidence along its lineage

CONT denotes continuing search by refining an existing complete candidate; B is the candidate-call budget. Appendix B contains the full algorithm and update rules, and the Reproducibility Statement describes code availability.

Thank you for the additional comparisons and ablations. They are helpful, but a few points remain important to my assessment.

[Q1] REx adaptation and final selection. With early stopping disabled, how are REx's statistics updated after a refinement passes all public tests? Please specify the update rule and prior-strength setting. Could you also report REx's coverage and final accuracy under uniform tie-breaking, extending the comparison already provided for MCT?

[Q2] The controller's reward signal. The fixed-50/50 ablation tests adaptive allocation, not whether absolute reward is the appropriate signal. Could you provide an improvement-based controller comparison, or analyze hidden-test recovery versus regression among refinements whose parents already pass public tests? Hidden-test labels should remain restricted to offline diagnostics.

[Q3] What makes dynamic evidence sharing effective? The fixed-KK controls are useful, but do not isolate whether each weight needs to match its particular parent–child pair. Could you test a randomized-KK control with a comparable weight distribution? Please also provide the complete Qwen2.5 inheritance results currently described as “mixed.”

[Q4] Accuracy at comparable actual cost. Could you provide accuracy–cumulative-token curves, particularly against sequential refinement, rather than only average tokens per saved log? Please clarify the logging coverage and report acquisition/evaluation overhead separately. This would help establish whether the gains persist at comparable inference cost.

Presentation and revised PDF. I would appreciate an updated PDF incorporating these results and clarifications. The presentation still needs work, particularly the positioning against prior global-selection methods and the interpretation of the statistical states. Please make the central contribution and its supporting evidence clearer in the main text, rather than leaving important explanations only in the rebuttal.

Reviewer_B4tmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Decoupled Topology Search (DTS), an inference-time search framework over complete candidate solutions. A global Thompson-sampling controller allocates calls between generating fresh solutions and refining existing ones. Refinement parents are selected directly from the eligible candidate pool using expected improvement or Pareto-filtered Thompson sampling. Generation lineage is retained for forward inheritance of Beta statistics and backward propagation of feedback, weighted by a heuristic derived from generated-token probabilities. Validation primarily covers code generation, with additional mathematical reasoning and molecular optimization experiments, under matched candidate-call budgets. Comparisons with sampling and tree-search baselines, coupled-selection controls, component ablations, and final-selection analyses support improvements in several settings, while also identifying cases where feedback-guided search is ineffective.

Critical strengths
  1. Meaningful empirical improvements supported by a useful controlled comparison. On LiveCodeBench v6 with Qwen3-14B at 128 calls, DTS-Pareto achieves 71.6% final accuracy, compared with 64.4% for repeated sampling and 67.8% for sequential refinement. The coupled-DTS control preserves the major algorithmic components while changing parent selection, making it more informative than a comparison between unrelated systems. These results provide credible evidence that the proposed combination can improve budget allocation in the evaluated settings.

  2. Informative examination of the limits of feedback-guided search. The paper separates oracle coverage from final-answer accuracy, evaluates alternative final tie-breaking rules, and reports settings where repeated sampling outperforms all evaluated feedback-guided methods. This matters because improved search-visible scores need not translate into correct final answers. The analyses help qualify the empirical gains rather than presenting refinement as universally beneficial.

Critical weaknesses
  1. The central novelty is insufficiently distinguished from closely related prior work. REx [1] already treats complete programs as an expanding set of bandit arms, selects programs globally with Thompson sampling, and represents generation from scratch through an empty-program arm. More directly, Self-Refine [2], describes selecting among eligible answer nodes without starting from the root while retaining value propagation to parents and ancestors. DTS differs in its eligibility rules, acquisition functions, controller, and statistical updates, but the manuscript does not sufficiently explain why these differences constitute a substantial advance. Both works are cited but neither is directly evaluated. The coupled-DTS comparison establishes an advantage over ancestral routing within the authors’ framework; it does not establish an advantage over existing methods that already avoid such routing. The “Global TS” ablation also retains DTS’s controller and lineage updates, so it is not a substitute for REx. This is consequential because the strongest novelty claim concerns precisely the separation of selection from lineage-based feedback sharing.

  2. The distinctive evidence-sharing mechanism lacks sufficiently targeted validation. The token-probability weight K is explicitly heuristic, but generation confidence does not by itself establish how transferable a child’s score is to its parent or ancestors. Moreover, the Beta states combine own-candidate scores, inherited statistics, and descendant feedback, leaving their interpretation as uncertainty about refinement value unclear. The principal ablations are concentrated in one model–benchmark setting and do not distinguish informative edge-specific weighting from generic discounting or regularization. In particular, “No K-aware transfer” still propagates feedback to ancestors with unit weight; it is not a fully independent-candidate baseline. Without a clearer interpretation and stronger controls, the experiments do not establish the independent value of the component that appears most distinctive relative to prior work.

[1] Code Repair with LLMs gives an Exploration-Exploitation Tradeoff. NeurIPS 2026

[2] Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B: A Technical Report.

Questions for authors
  1. Clarification and comparative evidence: What is the precise conceptual distinction between DTS’s global selection plus lineage updates and the mechanisms in REx and MCT Self-Refine? Can you provide comparisons under aligned model, feedback, budget, stopping, and final-selection protocols? This would materially affect my assessment of the contribution beyond existing global candidate-selection methods.

  2. Clarification: What quantity is each candidate’s Beta distribution intended to estimate: its own quality, the expected quality of its immediate refinements, or another quantity? How does combining inherited, own, and descendant scores support this interpretation, and what evidence connects K to the reliability of transferring feedback?

3 comments

Thank you for recognizing the empirical gains and the value of the controlled topology comparison. We address the distinction from prior work and the evidence for lineage sharing below.

1. Contribution and direct comparisons with REx and MCT Self-Refine

DTS advances the adaptive search developed in AB-MCTS by making both action allocation and parent selection global, while retaining lineage for sharing evidence. The search has no preset branching limit, fixed generation/refinement ratio, or exploration coefficient to tune. Every valid complete candidate remains available for refinement, including candidates that already have children.

REx already selects programs globally. Two further differences explain the motivation for DTS's design:

  • Action allocation. REx represents fresh generation as one empty-program arm competing with all candidate arms. Holding existing arm beliefs fixed, adding candidates cannot increase this arm's chance of winning. Thus, pool growth itself affects the generation/refinement balance. DTS learns a separate GEN/CONT decision from action outcomes, then selects a parent by EI or Pareto–Thompson acquisition, separating the two decisions.
  • Evidence reuse. REx tracks each program's heuristic quality and its own refinement attempts. A child does not inherit its parent's accumulated statistics, and deeper descendant outcomes do not update earlier ancestors. DTS supports both: a child inherits evidence, and its feedback updates ancestors with refinement-dependent weights. This lets related candidates share useful search history within a limited budget. The inheritance and ancestor-update controls in Section 3 below test the value of this reuse.

MCT Self-Refine also selects globally among eligible nodes and propagates Q values to ancestors. DTS uses the full valid candidate pool, an explicit global GEN/CONT controller, and forward inheritance together with weighted backward updates of Beta statistics. These differences concern how computation and evidence are allocated throughout the search.

We completed both direct comparisons on the same 175 LiveCodeBench v6 problems with Qwen3-14B and 128 candidate calls per problem. Invalid and duplicate outputs count toward the budget, early stopping is disabled, and all methods return the latest candidate among those with the highest binary public-test score. Hidden tests are used only for evaluation.

MethodFinal accuracy (%)Runs
REx, public pass-rate feedback63.43 ± 0.002
MCT Self-Refine, public-feedback adaptation64.86 ± 0.402
DTS-EI70.60 ± 0.203
DTS-Pareto71.60 ± 0.603

REx uses public pass rates during search. The MCT adaptation retains global UCT selection, expansion eligibility, and recursive Q updates, while replacing its mathematical self-evaluation with binary public-test feedback and the shared generation interface; each candidate uses one model call. Values are means ± SD.

DTS-EI/Pareto exceed REx by 7.17/8.17 percentage points and the adapted MCT by 5.74/6.74 points. Under uniform tie-breaking, they also exceed MCT by 4.48/4.08 points. These comparisons establish the overall gain over existing global selection methods; the coupled-DTS control separately tests the effect of ancestral routing.

2. Meaning of the Beta states and the transfer weight K

Each candidate's Beta state is a search statistic used to choose refinement parents. It summarizes the candidate's own feedback, inherited parent statistics, and weighted descendant feedback. Its mean provides a score estimate and its concentration controls exploration. DTS uses these statistics to balance promising candidates with further exploration.

K sets how much evidence a parent and child share. A larger K strengthens inheritance from the parent and feedback to ancestors; a smaller K weakens both. It is computed from token probabilities conditioned on the parent and feedback. The rationale is that more similar revisions can share more evidence.

Across three models and two acquisition rules (one complete run per setting):

CheckObservation
K versus parent–child code similarityPositive Spearman correlations of 0.495–0.648 across all six runs
Parent–child hidden-test outcome agreementHigher for the top 25% of K values than for the bottom 25% in all six runs

These observations link K to parent–child similarity; the fixed-K controls below test its value for search.

3. Dynamic weighting, inheritance, and multi-generation updates

The completed controls separate dynamic weighting from fixed discounting, and forward inheritance from longer-range backward updates. Under the same Qwen3-14B/LCBv6 protocol:

Pareto configurationFinal accuracy (%)Runs
Full DTS71.60 ± 0.603
Fixed K = 0.883626, the empirical mean63.14 ± 2.022
Fixed K = 0.563.43 ± 0.812
Static K = 166.50 ± 1.203
Backward updates to the direct parent only62.00 ± 0.402
No forward inheritance65.50 ± 2.303

The parent-only control preserves forward inheritance and direct-parent updates. Its EI counterpart also falls from 70.60 ± 0.20% to 64.86 ± 1.21% over two runs. Thus, longer-range updates contribute under both acquisition rules.

Inheritance also helps on Qwen3-8B: over three runs, EI improves from 58.67 ± 2.10% without inheritance to 61.14 ± 0.00%, and Pareto from 58.29 ± 2.03% to 63.05 ± 0.27%.

These controls support dynamic weighting, forward inheritance, and multi-generation updates. The “No K-aware transfer” control uses unit-weight ancestor updates.

Dear Reviewer B4tm,

Thank you for your thoughtful review and constructive feedback.

As the author–reviewer discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our rebuttal and additional experimental evidence, particularly the comparisons with prior global-selection methods and the lineage-credit ablations.

Your feedback would be very helpful in determining whether our clarifications and new results adequately address your main concerns.

Thank you again for your time and consideration.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary and Manuscript Revisions

Dear Area Chair and Reviewers,

Thank you for the discussion. A complete solution can be revised directly. DTS chooses which solution to revise from the full pool of eligible candidates and uses the generation tree to share feedback between related solutions. A separate controller adapts how many calls go to new solutions and how many go to revisions.

In the original submission, we held call allocation and feedback updates fixed and changed only how the refinement parent was selected. On LiveCodeBench v6 with Qwen3-14B, direct selection improved Pass@1 by 6.3 percentage points for DTS-Pareto (Table 3). REx and MCT Self-Refine also select parents directly, so we added comparisons with both. In the same setting, with 128 calls, the same public tests, and a common final selection rule, DTS reached 70.6–71.6% Pass@1, compared with 63.4% for REx and 64.9% for MCT Self-Refine. Both DTS variants remained ahead under uniform tie-breaking (Table 8).

The revision shows what changes during search. It includes a recorded trajectory and replays both parent selectors from the same states (§5.3, Figure 9). When we shuffled the transfer weights within each problem, the weight distribution stayed similar and DTS-Pareto accuracy fell from 71.6% to 64.6%. This supports assigning weights to individual parent–child pairs (Table 10). We also explain how the Beta states combine feedback from a candidate and its lineage, and show both repairs and regressions when refining solutions that already pass the public tests (§3.3, Appendix C.5).

We compared costs by replaying the saved runs at the mean token use of sequential refinement. DTS reached 69.4–70.8% accuracy versus 67.8% for sequential refinement in this setting (Figure 7, Table 9). On AIME-RM, DTS-EI reached 63.9% versus 60.0% for repeated sampling with the same learned verifier. Repeated sampling led on the two low-recovery Qwen2.5 settings, and gains were small or mixed on AIME-Base and PMO. The revised Section 5.4 and Appendix E.3 use these cases to explain when refinement is worth trying and how to assess a new setting.

We responded to all three reviews, including p8h6's four follow-up questions, and incorporated the main results and clarifications in the revised manuscript. Reviewer neGq confirmed that our explanation addressed their remaining concern about generalization.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.