- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies test-time scaling for large language models under a fixed inference budget, with a particular focus on how computation should be allocated between generating new solutions and refining existing ones. The authors argue that existing methods such as MCTS and AB-MCTS still rely on ancestor-by-ancestor traversal when selecting a refinement parent, causing access to a complete candidate to depend jointly on its estimated quality and its position in the search tree. To address this issue, the paper proposes Decoupled Topology Search (DTS), which separates global refinement-parent selection from lineage-based information sharing. DTS uses Thompson sampling to adaptively choose between fresh generation and refinement. When refinement is selected, all valid complete candidates compete globally for the next model call, using either Expected Improvement or a Pareto–Thompson acquisition rule. Parent-child relations are retained only for forward inheritance and backward credit propagation of statistical evidence. The method is evaluated on code generation, mathematical reasoning, and molecular optimization against repeated sampling, sequential refinement, MCTS, and AB-MCTS baselines. On the main code-generation benchmarks, DTS consistently improves final-answer accuracy across multiple Qwen3 and AReaL settings, with a maximum gain of 7.2 percentage points over repeated sampling on LiveCodeBench v6 with Qwen3-14B. Additional topology controls, ablations, and low-budget diagnostics indicate that global parent selection, adaptive computation allocation, and lineage-based evidence sharing all contribute to the observed gains, while also revealing settings in which feedback-guided search is less effective.
Critical strengths
-
The paper identifies a clear and meaningful structural limitation of complete-solution tree search. The distinction between using lineage for statistical information sharing and using lineage as an access path for refinement is conceptually clean. Decoupling these two roles gives DTS a clear methodological identity relative to standard MCTS and AB-MCTS.
-
The paper includes a well-targeted control for its central claim. The coupled-DTS comparison keeps the controller, acquisition mechanism, and lineage updates fixed while changing the refinement-parent topology. This substantially strengthens the causal argument that globally selecting complete candidates is useful beyond the other components of DTS.
Critical weaknesses
-
Matching the number of candidate calls does not fully establish compute efficiency. A refinement call can involve a substantially longer prompt because it includes the parent solution and feedback, while repeated sampling uses only the task prompt. In addition, DTS-EI performs nontrivial numerical acquisition work over the candidate pool. The paper explicitly notes that candidate-call matching does not equalize token usage or wall-clock cost.
-
The lineage-transfer weight K is important to the method but remains largely heuristic. The surprisal-weighted token-probability formula does not have an obvious probabilistic interpretation as the degree to which a child's observation should update its parent or ancestors. The ablations show that K-aware transfer is useful in one major setting, but the evidence is concentrated on LiveCodeBench v6 with Qwen3-14B. The paper would benefit from a stronger justification of this weighting rule and from sensitivity or alternative-weight experiments across multiple models/tasks.
Questions for authors
-
A closely related prior work (which the authors cited in the 3rd paragraph, page 3, context: "... Bayesian methods guide program refinement (Tang et al.)..."), formulates LLM program refinement as an arm-acquiring bandit problem and uses Thompson sampling to allocate refinement calls among candidate programs under an exploration–exploitation tradeoff. This appears closely related to DTS's global competition among complete candidates for refinement budget. Could the authors clarify the precise algorithmic and conceptual differences between DTS and Tang et al. (2024), beyond the addition of lineage-based credit propagation? In particular, does the prior bandit formulation already permit direct selection of any existing complete candidate independent of its generation lineage? If so, which part of “decoupled topology search” is genuinely new relative to that formulation?
-
Could the authors further evaluate the robustness of DTS across a broader range of non-code tasks and feedback conditions? The strongest gains are concentrated on code-generation benchmarks. On AIME-Base, DTS-EI improves over repeated sampling by only 0.6 percentage points, while DTS-Pareto performs 2.7 points worse. On PMO, DTS-EI outperforms repeated sampling, whereas DTS-Pareto again underperforms it. It would therefore be useful to include 1-2 additional mathematical reasoning, natural-language reasoning, or other tasks with different levels of feedback noise, and to analyze why the relative performance of the two acquisition rules changes across domains. Such experiments would help clarify whether the benefits of DTS generalize broadly or depend strongly on the task and feedback structure.
p.s.: There seems to be some formatting issues (in page 5, Alg. 1: Eq. equation 7 -> Eq. 7; page 20, Figure 7 title: 8B_model [which seems to be a script name in code editor rather than a formal name to be illustrated in the paper], etc.). However, these issues merely require the author's attention and are not the primary basis for my rating.
8 comments
3. Evidence for K and transfer across models
The completed fixed-weight controls support using a weight that varies with each refinement. On Qwen3-14B/LCBv6, Pareto obtains 71.60 ± 0.60%, compared with 63.14 ± 2.02% for fixed K = 0.883626, 63.43 ± 0.81% for K = 0.5, and 66.50 ± 1.20% for K = 1. The first two controls have two runs; the others have three.
Across three models and both acquisition rules, K correlates with lexical parent–child code overlap (Spearman 0.495–0.648). This supports using K to vary how much evidence related candidates share. Forward inheritance also raises Qwen3-8B accuracy from 58.67 ± 2.10% to 61.14 ± 0.00% for EI and from 58.29 ± 2.03% to 63.05 ± 0.27% for Pareto, over three runs. Qwen2.5 results are mixed.
4. Non-code tasks and differences between EI and Pareto
The existing non-code results cover both mathematical reasoning and molecular optimization, with different feedback sources. The following are three-run means ± SD:
| Setting and metric | Repeated sampling | DTS-EI | DTS-Pareto |
|---|---|---|---|
| AIME-Base accuracy (%) | 78.3 ± 1.4 | 78.9 ± 0.8 | 75.6 ± 2.1 |
| AIME-RM accuracy (%) | 60.0 ± 3.6 | 63.9 ± 3.4 | 60.6 ± 1.6 |
| PMO final objective | 3.7775 ± 0.0287 | 3.8051 ± 0.0384 | 3.7182 ± 0.0352 |
AIME uses the same 60 problems and Qwen3-8B at 64 calls, with oracle feedback in Base and ReasonFlux-PRM feedback in RM. PMO uses GPT-4o-mini at 128 calls and five molecular objectives. On AIME-RM, EI achieves higher final accuracy despite lower coverage than repeated sampling (74.4% versus 78.3%), extending the coverage–selection observation beyond code.
EI scores expected improvement above the current best mean; Pareto–Thompson samples among candidates that trade off estimated reward and uncertainty. These rules allocate calls differently, so their relative value can depend on the task's refinement opportunities and feedback. Pareto also trails repeated sampling with oracle AIME feedback and on PMO, so feedback noise alone cannot explain the reversal. The non-code results show that the choice of acquisition rule matters across tasks.
For the typographical points, the intended references in Algorithm 1 are Eqs. (7) and (2), and the model in Figure 7 is Qwen3-8B.
Dear Reviewer neGq,
Thank you for your careful review and constructive suggestions.
With the author–reviewer discussion period approaching its end, we would sincerely appreciate it if you could review our rebuttal and the newly added experimental comparisons, particularly our clarification of the differences between DTS and REx.
We would be grateful for any further feedback on whether the additional evidence addresses your concerns.
Thank you very much for your time and consideration.
Best regards, The Authors
Thank you for recognizing the distinction between selection and evidence sharing, and the value of the coupled-DTS control. We address actual cost, the relationship to REx, and robustness below.
1. Token cost and acquisition overhead
DTS achieves higher accuracy than AB-MCTS-A/M with lower average token usage per saved log on Qwen3-14B/LiveCodeBench v6. Final accuracies are three-run means at 128 candidate calls. Token counts are averages per saved message body, excluding chat templates.
DTS-EI/Pareto use 3.8%/2.8% fewer tokens per saved log than AB-MCTS-A and 6.4%/5.4% fewer than AB-MCTS-M. Relative to repeated sampling, the reductions are 29.2%/28.5%. Compared with sequential refinement, DTS uses 17.4%/18.6% more tokens per saved log, with accuracy gains of 2.8/3.8 percentage points.
Token usage also depends on the model. On Qwen2.5-7B-Instruct, the corresponding totals are 851 tokens/log for repeated sampling and 1,166/1,149 for DTS-EI/Pareto.
EI acquisition is computed numerically from candidate statistics and requires no additional LLM calls (Appendix B.5).
2. What DTS adds beyond REx
DTS advances AB-MCTS-style adaptive search through global action allocation and bidirectional evidence sharing. REx already permits global parent selection, but its single fresh-generation arm competes with a growing pool of candidate arms. DTS learns GEN/CONT separately from parent acquisition, so pool growth does not directly add competitors to GEN. REx also keeps each program's search statistics local; DTS passes accumulated evidence to new children and returns descendant feedback to ancestors. These differences address how calls are allocated and how much information each call can contribute. EI or Pareto acquisition then selects a parent without a preset branching limit, allocation ratio, or tuned exploration coefficient.
The direct REx comparison uses the same 175 LCBv6 problems, Qwen3-14B, 128 candidate calls, and final selection rule, with early stopping disabled. REx uses public pass-rate feedback during search. Final accuracy is 63.43 ± 0.00% for REx over two runs, versus 70.60 ± 0.20%/71.60 ± 0.60% for DTS-EI/Pareto over three runs. Values are means ± SD.