- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies when one continuous latent state cannot preserve the useful futures of an explicit beam, and how to allocate independent reasoning paths. It gives a conditional obstruction to lossless beam emulation and derives a collision-based marginal-coverage rule with an explicit width cap. The deployed protocol uses a deterministic first path, followed by additional independent paths; ProsQA uses answer-candidate probabilities, while open-answer tasks use a vocabulary proxy. Experiments support separate continuations and adaptive allocation at matched mean path count. Resource measurements additionally compare adaptive execution with always using the full width cap. The evaluation concerns pass@K coverage, rather than an implemented selector's final-answer accuracy.
Critical strengths
- The theory distinguishes the need for separate futures from the allocation of additional paths. The surrogate objective, stopping condition, and width guarantee are explicit.
- The matched shared-state control and fixed-width frontier make the separation and allocation results informative. Main allocation comparisons include paired intervals.
- Proxy ablations, architectural transfer, and measured inference overhead provide useful evidence about practical scope, including the mixed CoLaR allocation results.
Critical weaknesses
- W1 — The introduction moves to technical contributions before sufficiently motivating the problem. After a brief description of latent reasoning, the introduction moves directly to the method and contribution list (p.1, lines 35–53; p.2, lines 68–77). It does not sufficiently explain the distinction between implicitly representing multiple futures in one state, maintaining independent latent continuations, and allocating their width across problems before introducing unresolved futures and collision-based marginal gains. Figure 1 provides a policy comparison, and §§3–4 later formalize the concepts, but the reader must assemble much of the motivation from these later sections. For readers not already working on COCONUT-style methods, this makes the problem framing and the connection between the separation and allocation contributions harder to follow.
- W2 — The efficiency and allocation comparisons use different budget notions. Table 2 matches mean path count, while §5.3 and Table 4 report resource savings against full width without a joint cost–coverage frontier. Path length, batching, and the deterministic first path can affect the relationship between path count and actual cost. Tables 6–7 additionally compare stopping rules and proxies at their own mean widths. Their uplifts over the respective fixed frontiers are informative, but are not direct coverage differences at a common budget. Thus, the practical-budget advantage and the incremental value of particular allocation choices are less directly established than the individual reported comparisons (pp.7–9, 14–15, 18–19).
- W3 — The downloaded artifact lacks the promised low-cost verification records. Appendix A.5 states that stored path, proxy, and compute records permit regenerating the analyses without model inference (pp.13–14). The supplied archive's README instead excludes stored experiment results, and its inventory contains no such records. Recreating outputs is therefore required before checking the reported frontier, bootstrap, and resource analyses. This is a fixable artifact/documentation discrepancy; it does not establish that the reported results are wrong.
Questions for authors
- Q1 — Clarifying the problem and motivation (W1). Could the introduction explain, before the technical contribution list, the concrete limitation of a single implicit beam, why a fixed number of independent paths is insufficient, and how the separation and per-problem allocation questions are connected? A short running example distinguishing these settings would help readers unfamiliar with COCONUT-style work understand what the method addresses.
- Q2 — Common-budget quality and efficiency (W2). At matched budgets, how does the proposed policy compare in coverage with fixed-width and simpler adaptive policies? Please connect coverage with measured latency or estimated FLOPs for the practical-cost comparison, and distinguish that result from comparisons at matched mean path count. When comparing stopping rules, keep the proxy fixed so their incremental contribution can be assessed.
- Q3 — Availability of verification records (W3). Are the path, proxy, and compute records described in Appendix A.5 available in an omitted location? Please identify the matching records and analysis entry point, or clarify the artifact's actual reproduction scope.
3 comments
Thank the authors for rebuttal and explaining. I think the blue lines indeed help me understand how you want to use multiple latent cots conceptually, but I doubt if current version has too much examples in the opening two paragraphs as a rigid academic paper. I think there are better ways for the opening.
Thank you for the follow-up. We revised the first two paragraphs of the introduction to reduce the number of examples while preserving the intuition behind separate latent trajectories. The revised opening explains why a shared latent state may lose distinctions that matter for later reasoning and how separate continuations let each trajectory proceed from its own latent state. We replaced the task-specific examples with one brief illustration of why different problems may benefit from different numbers of trajectories. This leads directly to the question of how many latent trajectories are worth exploring for each problem. We believe these revisions make the introduction easier to follow and clarify the motivation for adaptive branching, and we hope they address your concern.
Thank you for the thoughtful review, comments and suggestions!
We are glad you see the benefit of the theory for establishing surrogate objectives, stopping conditions and width guarantees, and that you found find the separation and allocation results informative, and see useful evidence about practical scope!
We hope we address any of your remaining concerns below.
We revised the introduction to motivate separation and adaptive width before presenting the collision-based rule and listing our contributions. The new examples, marked in blue, include a running scheduling example that connects separate continuations to the choice of how many paths to explore per problem.
We have now added Fig 16, which shows the coverage comparison from Fig 4 at matched mean estimated FLOPs per example. At the eight-path cap, COCONUT TREE improves coverage over the fixed-width frontier by 0.35–3.02 percentage points across all ten settings, with paired 95% confidence intervals above zero in each setting. We have also added Fig 14(b), comparing adaptive rules at matched mean estimated FLOPs, alongside the matched mean path-count comparison in Fig 14(a). All adaptive rules use the same probability proxy. Under both budget measures, COCONUT TREE outperforms all four calibrated baselines on GSM8K, GSM-Hard, and SVAMP on both backbones at every tested cap above one. On ProsQA and MultiArith, coverage differences are smaller and vary by baseline. The low mean path counts on these tasks leave fewer extra paths to reallocate.
Sorry, this was an oversight during submission, when the system limited submissions to 50MB. We have now updated the supplementary material to include the compressed records for reproducibility, as well as refined the reproducibility statement and appendix details on this, and updated the README.md with instructions. Thank you for being so thorough with the checks!
Thank you for the comments and questions, we hope we have addressed any remaining concerns.