acceptodds
Withdrawn from ICLR 2027

When Is Test-Time Allocation Actionable? Exchangeability and Breadth Dominance in Hierarchical Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Retrospective oracle gains and accurate value prediction do not by themselves establish that adaptive test-time allocation can improve generation. We audit when information that distinguishes actions becomes available. Under joint conditional exchangeability of future child observations and costs, unseen child labels reduce to one expand action per parent. With hidden fresh randomness and no candidate observation before spawn, policies cannot select a particular unseen parent realization. These conditions restrict action identity but do not establish breadth optimality. In a frozen two-stage TRELLIS generator, value models achieve pooled Spearman correlations of 0.935–0.970, while only 0.108–0.534% of retrospective target variation lies within decision sets. For the primary input-silhouette proxy, the empirical within-parent sum-of-squares share is 4.038% and the ANOVA one-child ICC is 0.959. Three development attempts fail to improve on the selected heuristic comparator. A one-shot 80-unit test of a common spawn-first controller with learned branch selection yields a normalized-utility gain of 0.004281 (95% CI [0.000903, 0.008504]) over uniform branching. Raw silhouette-Chamfer improves by 0.409%, with an interval including zero. A zero-parameter rule attains 92.2% of the learned gain's point estimate; the incremental benefit of the 76-feature Q model remains unresolved. These are finite-table replay results under an estimated cost contract, not end-to-end latency or perceptual quality improvements. The study provides an actionability audit and evidence of small tested-policy gains in one breadth-dominant hierarchy.

Primary area:
generative models
Keywords:
test-time compute, adaptive compute allocation, hierarchical generation, conditional exchangeability, breadth dominance, best-of-N sampling, verifier evaluation, 3D generation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_CotpOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates when adaptive test-time compute allocation can actually help in a hierarchical (two-stage) generator. In this setting, a "spawn" action creates a new first-stage parent, and an "expand" action draws another second-stage child from an existing parent. The authors propose an "actionability" audit with three parts. The empirical study uses frozen TRELLIS on 200 conditioning units (80 dev / 40 calibration / 80 test), with 4 parents × 4 children per unit. Three adaptive methods fail during development. On the test set, a narrowed spawn-first controller with learned branch selection gains 0.0043 in normalized utility over uniform branching.

Critical strengths
  1. The central message is useful to the test-time scaling community. High pooled verifier accuracy and large oracle gaps do not by themselves justify adaptive allocation, because decisions depend on ranking within choice sets. Many papers motivate adaptive methods with exactly these diagnostics.

  2. The empirical protocol is unusually careful: physically separated splits, explicit cost and failure conventions, unit-level bootstrap intervals, and the full chronological sequence of failed attempts. Reporting negative results honestly is commendable.

Critical weaknesses
  1. Conceptual novelty is limited. The formal results are close to definitional. If future children are conditionally exchangeable, unlabeled slots are interchangeable. If no information is observed before spawn, unseen parents cannot be ranked. The value–decision dissociation is essentially a between- vs. within-group variance decomposition (ANOVA/ICC), a standard statistical observation. The related-work section itself acknowledges that the discovery-vs-repeated-sampling trade-off and the strength of greedy breadth are already known (many-armed bandits, progressive widening, knowledge gradient). The remaining contribution, auditing when a parent becomes an identifiable arm, is thin for a full conference paper.

  2. If the main contribution is a diagnostic workflow, its value should be shown by applying it across several systems and showing that it separates regimes where adaptation helps from regimes where it does not. The paper instead studies one generator, one 4×4 hierarchy, two budgets, and 80 test units. The authors also state that the variance share, ICC, and effect sizes do not transfer. As a result, the reader learns neither whether the workflow generalizes nor whether the TRELLIS findings do.

  3. The normalized-utility gain is small (0.0043, CI [0.0009, 0.0085]). The raw silhouette-Chamfer improvement of 0.409% has an interval including zero. A zero-parameter rule recovers 92.2% of the point estimate, and the incremental value of the 76-feature Q model is unresolved. The practical takeaway is therefore unclear.

  4. The paper is very hard to read. Nearly every claim comes with multiple hedges ("does not establish", "remains unresolved", "is not a consequence of"), and many ad hoc terms and labels (B0/B1, candidate payload, information contract, action-space quotient) are introduced quickly. The main conclusion is difficult to extract. Substantial rewriting and condensation would help.

Questions for authors
  1. Can the audit be applied to at least one other hierarchical system, for example LLM reasoning with shared prefixes and sampled continuations, or diffusion models with intermediate denoising predictions? Ideally one of these would be a regime where the audit predicts that depth/adaptivity should help, and that prediction would be confirmed.

  2. The discussion notes that partial denoising observations could reveal realization-specific information, but the "peek" protocol was not executed. Since this appears to be the most natural way to make spawn actionable, why was it left out?

  3. How sensitive are the conclusions to the hierarchy size (4×4) and to the budget levels? Would larger child pools per parent change the within-parent variance share or the observed branch gains?

Reviewer_AX7YOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

As far as I understand it, the paper’s main point is that adaptive test-time compute helps only when a policy can use information available to it before it acts. A retrospective oracle may appear powerful because it sees outcomes that a real policy could not know in advance (i.e. it gives an upper bound on what could be achieved with hindsight).

In the paper’s TRELLIS case study, the learned policy improves a little bit over uniform branching on a proxy metric; however, a simple rule gets nearly the same gain. The authors take those results to conclude that this is evidence of a small benefit for branch selection in their tested setup, instead of proof that adaptive allocation is generally unnecessary or that the result transfers to other generators.

Critical strengths

The good thing about the paper is that it is quite careful about the scope of its claims. It does says explicitely that exchangeability and a high ICC (intraclass correlation coefficient) do not necessarily prove that breadth is optimal, or neither that adaptive allocation is unnecessary.

Additionally, I do find the empirical protocol to be transparent about the development sequence, in addition to the replay rules, failures, and limitations. Also rather than presenting the learned policy’s result in isolation, the authors also report the simple-rule comparison.

Another idea also seems useful, which is to check what information is available before each action, and then evaluate rankings within actual decision sets.

Critical weaknesses

I find that while the theory presented in the paper has some interesting ideas, its contribution seems rather limited. As far as I understand it (and I had a bit of a difficult time understanding the paper), the exchangeable-slot result follows directly from the stated exchangeability and label-invariance assumptions. Similary, the no-ranking result follows from an assumption about the pre-spawn payload being constant. I am therefore not yet convinced that these contributions add substantial theoretical insight. The paper would benefit from a more sharper account of what is technically / theoretically new, especially compared to related work.

Additionally, I believe the experiment seems to test a narrower question than the paper’s framing might suggest. The held-out primary comparison seems to test, which observed parent to expand; however, not whether a policy should spawn a new parent or expand an existing one. It thus provides some evidence for branch selection within this controller, but is likely not a general demonstration yet that adaptive test-time allocation is actionable. If I'm wrong here and misunderstood somethign, please correct me on this.

Also importantly, the practical gain over a simple heuristic seems unclear at the current state of the paper. The difference between Q and the current-utility rule appears small, even though the learned Q policy improves on uniform branching on the metric. I agree with the authors that this does not establish equivalence; however, it leaves the main practical benefit of the learned model unresolved, which I do believe is an important point to address.

As I mentioned above, I had a rather difficult time digesting the paper overall. One reason for this could be that the central message is spread across many parts of the paper. Also it's not always clear which type the results are, e.g. theoretical, exploratory, or confirmatory; the writing is a bit dense and some acronyms are introduced without explaining them first. A simpler overview early in the paper, followed by a clearer separation of the main result from the audit details, would make the contribution easier to assess for the reader.

In conclusion, I find the paper’s central distinction sensible and potentially useful (i.e. a policy can exploit only information it has when it acts). However, unfortunately I found the paper too hard to follow, and I am not yet convinced that the current combination of observations and experiment s establishes a sufficiently strong contribution for ICLR.

Questions for authors
  • Could the authors clarify the main novelty relative to existing work on hierarchical sampling and adaptive allocation? What is the most important contribution of the paper?

  • Why is the held-out confirmation restricted to a spawn-first controller? Can the authors provide evidence (if available) that directly evaluates the spawn-versus-expand decision?

  • What evidence would be needed to establish that the learned Q model provides a useful increment over the current-utility rule?

Reviewer_uaSpOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper provides test-time compute methods for analyzing which parts of the sampling space deserve more compute allocation.

Critical strengths

The paper investigates a relevant topic and provides insights to a variety of existing methods

Critical weaknesses

The paper lacks significant empirical evidence to support the claims made in there. There are a few correlation analysis and synthetic examples but it is unclear whether these methods provide real-world impact.

Questions for authors

Why were these methods for tested on more realistic datasets and experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.