acceptodds
Under review as a conference paper at ICLR 2027

Dual-Resolution Recursive Energy for Selective Deliberation in Sequential Decision Making

Abstract

Fast action selection and slow deliberation are often modeled as separate computational modes. We ask whether they can instead be realized as two evaluation resolutions of a single structured decision model. We introduce a dual-resolution recursive energy model in which contracted macro-energies and explicitly composed AND/OR subtrees evaluate the same action-support structure on a common energy scale. Local expansion controls the extent of explicit deliberation within a decision. Wake–sleep training distills expanded subtree energies into reusable macro-energies and learns estimates of their contraction errors. At inference, these estimates guide which frontier nodes to expand and when the action-energy margin permits stopping. The fully expanded winner is preserved when the predicted errors cover the relevant contraction gaps, or when any remaining undercoverage fits within the action margin. Across BabyAI, MiniHack, and Overcooked-AI, the model reduces explicit recursive expansion by –% while retaining near-Full macro success, matching Full on Overcooked-AI. It achieves higher macro success than the matched-budget controls in the benchmark-level summaries. On identical held-out decision states, Ours reduces task-averaged median action-selection latency by –% relative to Full while retaining at least % task-averaged action agreement.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
energy-based models, adaptive deliberation, structured reasoning, sequential decision making

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WF3AOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper represents each action by an AND/OR energy tree and permits any internal subtree to be evaluated either explicitly or through a learned macro-energy on the same scale. Wake-sleep training distills expanded energies and learns contraction-error estimates. At inference, propagated estimates and action margins determine which frontier node to expand and when to stop. The winner-preservation result is conditional on error coverage or sufficiently small remaining undercoverage. Across 17 tasks/layouts in BabyAI, MiniHack, and Overcooked-AI, mixed-resolution inference uses roughly 51-78% fewer explicit expansions than the same-checkpoint Full evaluator while retaining similar macro success. Fixed expert-state timing shows lower action-selection latency; separate closed-loop audits quantify early-stop disagreements.

Critical strengths
  1. The common-energy interface provides a coherent mechanism for local adaptive computation. A macro-energy approximates a particular expanded subtree, so mixed-resolution evaluation has a precise target rather than switching between unrelated policies. The propagation argument and margin condition make the approximation's role in action selection explicit. Importantly, the paper correctly describes learned radii as estimates and the preservation result as conditional.
  2. The matched-checkpoint Full comparison and the allocation controls provide useful evidence for the mechanism. Across six training seeds, success remains close to Full while expansions decrease substantially. Appendix D's RF-order + Stop control further separates where to expand from when to stop: comparable success requires more expansion under random ordering. Measured fixed-state latency and the closed-loop reliability audit complement expansion counts. These controls support a weak-acceptance recommendation for the local computation-allocation contribution within the stated structured setting.
Critical weaknesses
  1. Deployment reliability is less settled than the high agreement reported on held-out expert states might suggest. Among closed-loop early stops, Full-winner disagreement is 6.09% on BabyAI and 2.81% on MiniHack (Table 1). The conditional proposition does not provide an operational error-rate guarantee because its residual conditions are post-hoc quantities. The retained BossLevel test is encouraging, but largely falls back to expansion of 32.52/34 nodes and excludes unseen skills. Thus the evidence for simultaneously retaining substantial speedups and reliable stopping under shift remains limited.
  2. Applicability depends on supplied task-specific AND/OR scaffolds and, for BabyAI/MiniHack, upstream subgoal and consequence interfaces. The controls fairly share these inputs, so this does not invalidate the within-scaffold comparison. However, independent scaffold construction is evaluated on only one task, and fixed-state timing excludes high-level planning and consequence generation. The current evidence therefore establishes savings inside a structured action evaluator more strongly than end-to-end efficiency or robustness across scaffold constructions. This is the principal limitation on the broader significance of the result.
Questions for authors
  1. On the existing closed-loop evaluation states, what wall-clock savings accompany the early-stop disagreements, and are the disagreements concentrated in particular tasks or failure-critical decisions? Relating these quantities would clarify the actual reliability-efficiency trade-off.
  2. Can a small sweep of a conservative radius multiplier, using held-out validation choices, show how stopping disagreement, success, and expansion change? This would indicate whether the deployment trade-off is controllable without relying on unobservable residuals.
  3. For the independently constructed scaffolds in Appendix L, how variable are expansion savings and stopping reliability in addition to success? Please also state what fraction of end-to-end decision time is represented by the evaluator timing, if those measurements are available.
6 comments
AuthorsOpenReview ↗

Response to Reviewer WF3A (1/3)

W1. Deployment reliability and OOD robustness

To address your concern about deployment reliability and OOD robustness, we emphasize that our learned radii provide conditional rather than unconditional guarantees. We further examine closed-loop disagreements and the reliability–efficiency trade-off in Q1 and Q2.

We agree that computational savings are limited under zero-shot BossLevel transfer, where Ours expands 32.52 of 34 nodes. However, this is consistent with our adaptive mechanism: when estimated error bounds cannot reliably separate competing actions, the controller expands more subtrees rather than stopping early.

To examine whether task-specific learning can improve efficiency, we trained the model on BossLevel using the same inference mechanism. Expansion reduction increased from 4.34% to 38.63%, while success remained close to Full (74.00% vs. 76.50%). This shows that task-specific training can recover substantial computational savings on complex tasks. We have added these results to Section 4.2 and Appendix E of the revised manuscript.

Q1. What wall-clock savings accompany early-stop disagreements, and how do these disagreements affect task outcomes?

To answer your questions about runtime savings and the consequences of early-stop disagreements, we conducted three complementary analyses across six BabyAI tasks: complete closed-loop episode timing, paired disagreement–outcome analysis, and first-disagreement action interventions.

Q1(a). What wall-clock savings are observed during closed-loop execution?

To examine whether selective evaluation reduces runtime during complete task execution, we measured episode wall-clock time for Full and Ours on scenarios where both methods succeeded. Both controllers independently executed their low-level policies in closed loop, using saved high-level plans and replans without new LLM API calls. This allows us to compare episode execution time while keeping the high-level planning outputs consistent.

BabyAI taskFull (ms)Ours (ms)Speedup
GoToObj94.48376.1831.240×
OpenDoor236.566189.2361.250×
PickupLoc135.071121.3561.113×
UnblockPickup1418.4011273.0081.114×
UnlockPickup1174.527654.6261.794×
BlockedUnlockPickup2155.4561535.6491.404×

Ours achieves 1.11–1.79× whole-episode speedups across all six tasks. These results show that selective evaluation provides measurable runtime savings during closed-loop execution, beyond the fixed-state decision-latency measurements. We have added the complete episode timing results to Appendix G of the revised manuscript.

Q1(b). Where do early-stop disagreements occur, and how do they affect task outcomes?

To examine whether early-stop disagreements lead to task failures, we analyzed 3,600 paired initial scenarios (six seeds × 100 episodes per task). We identified 808 Ours rollouts with at least one early-certified Full-winner disagreement. Among the 647 corresponding scenarios where Full succeeded, Ours also succeeded in 511 (78.98%).

Disagreements were concentrated in UnlockPickup and BlockedUnlockPickup, accounting for 680 of the 808 affected rollouts (84.2%). The conditional success rates of Ours when Full succeeded were 13.33% (2/15) in UnblockPickup, 52.11% (37/71) in OpenDoor, 86.79% (184/212) in UnlockPickup, and 82.90% (286/345) in BlockedUnlockPickup.

To further investigate whether individual disagreeing actions affect the final outcome, we conducted a first-disagreement intervention. For each affected rollout, we replaced only the first disagreeing action with Full's action at the same state, then resumed Ours for the remainder of the episode. Across the 808 affected rollouts, 73 failures became successes, 25 successes became failures, and 710 outcomes remained unchanged.

Takeaway. These results show that early-stop disagreements do not necessarily lead to task failure, although some disagreements can change the final outcome. The intervention experiment further distinguishes disagreements that affect task completion from those that leave the outcome unchanged. Together with the whole-episode timing results, these analyses provide additional evidence on the reliability–efficiency trade-off during closed-loop execution. We have retained the decision-level reliability analysis in Section 4.4 and added the closed-loop disagreement–outcome and first-action intervention analyses to Appendix F.3 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer WF3A (2/3)

Q2. Can a conservative radius multiplier control the reliability–efficiency trade-off?

To examine whether stopping reliability can be adjusted without changing the inference mechanism, we conducted a radius-scaling experiment by multiplying the predicted radii by a factor λ\lambda, giving r^n′=λr^n\hat{r}'_n = \lambda \hat{r}_n. Increasing λ\lambda makes the stopping condition more conservative, requiring additional expansion before a decision can be certified. This allows us to examine how early-stop disagreement, expansion, and closed-loop success change as the stopping criterion becomes more conservative. At λ=1\lambda=1, all metrics reproduce the main results.

Metricλ=1Intermediateλ=5
Early stopping (%)78.0863.2847.66
Early-stop disagreement (%)6.093.242.03
Expansion (% Full)49.1268.1188.19
Closed-loop success (%)87.4288.6789.03

Six-task macro averages; disagreement is conditional on early certification. The intermediate setting uses λ=2 for four tasks and λ=1.5 for the two UnlockPickup tasks.

Takeaway. Increasing the radius multiplier reduces early-stop disagreement from 6.09% to 2.03%, while expansion increases from 49.12% to 88.19% of Full. Closed-loop success also increases from 87.42% to 89.03%. These results show that the reliability–efficiency trade-off can be adjusted through radius scaling: more conservative stopping reduces disagreement at the cost of additional computation. We have added the radius-scaling experiments and their reliability–efficiency analysis to Section 4.4 and Appendix F of the revised manuscript.

W2 / Q3. Independent scaffolds and evaluator fraction

Q3(a). How variable are expansion savings and stopping reliability across independently constructed scaffolds?

To address your concern about whether our method depends on particular predefined scaffolds, we extended the independent construction experiment in Appendix L. For the same BabyAI UnblockPickup task, we independently generated three AND/OR scaffolds with 29, 32, and 33 internal nodes. We then evaluated each scaffold using six independent training and evaluation, following the main benchmark protocol, to examine whether the expansion savings and stopping behavior remain consistent across different structures and training runs. Within each run, Full and Ours share the same scaffold and trained checkpoint, ensuring a matched comparison.

NodesFull success (%)Ours success (%)Expansion (% Full)Early stopping (%)Disagreement (%)Residual-valid (%)Realized-valid (%)
2983.00±2.9780.67±1.8642.86±4.3598.61±0.678.52±4.1180.24±6.6187.80±4.91
3285.17±2.2384.50±3.7850.37±2.2299.05±0.504.61±2.7391.43±2.9094.46±3.00
3388.83±2.5687.00±3.1039.70±5.5399.41±0.708.90±5.4381.88±7.1287.65±7.15

Mean ± SD across six seeds. Early stopping uses all audited decisions, while disagreement and validity rates are conditional on early certification.

Takeaway. Across the three independently generated scaffolds, Ours reduces explicit expansion by 49.63–60.30%, while maintaining success within 0.67–2.33 percentage points of Full. Early-stopping rates remain above 98%, while disagreement rates vary from 4.61% to 8.90%. These results show that selective evaluation maintains substantial expansion savings across different scaffold constructions for the same task, while also revealing variation in stopping reliability across structures. We have extended Appendix L of the revised manuscript with the six-seed scaffold experiments and corresponding stopping-reliability analyses.

AuthorsOpenReview ↗

Response to Reviewer WF3A (3/3)

Q3(b). What fraction of complete episode time is spent in the evaluator?

To answer your question about the evaluator's contribution to overall execution time, we examined both the sources of computation cost and the fraction of complete episode time spent on model evaluation.

First, we conducted an independent stage-level profiling experiment across the six BabyAI tasks to identify where computation time is spent. We measured five components: candidate-future generation, input preparation, model decision, environment execution, and other planning/state overhead. The model-decision component includes neural encoding, energy scoring, recursive aggregation, and adaptive control. This breakdown allows us to examine whether the computational savings from selective evaluation primarily arise within the evaluator or from other stages of task execution.

The table reports mean latency per executed step (Full / Ours, ms) from the independent profiling runs.

TaskFuture generationInput prepModel decisionEnvironmentOther
GoToObj2.739 / 2.7590.523 / 0.53013.617 / 10.8880.292 / 0.2520.945 / 0.944
OpenDoor8.756 / 8.2220.672 / 0.64715.561 / 8.3100.441 / 0.4041.761 / 1.661
PickupLoc3.527 / 3.6770.612 / 0.57816.527 / 14.2640.280 / 0.2450.847 / 0.821
UnblockPickup18.423 / 18.2020.790 / 0.79423.978 / 15.3340.465 / 0.4372.682 / 2.561
UnlockPickup6.576 / 6.6790.961 / 0.95452.325 / 22.5560.237 / 0.2251.357 / 1.307
BlockedUnlockPickup6.762 / 6.7300.971 / 0.96961.297 / 40.8740.235 / 0.2121.530 / 1.522

The results show that the main computational savings occur during model decision, while future generation, input preparation, and environment execution remain broadly comparable. For example, on UnlockPickup, model-decision latency decreases from 52.325 to 22.556 ms per step.

To quantify the evaluator's contribution to complete task execution, we separately used evaluator and episode wall times recorded in the same paired-successful closed-loop runs described in Q1. We calculate the evaluator fraction as

Evaluator Fraction=∑iTeval,i∑iTepisode,i×100%\mathrm{Evaluator\ Fraction} = \frac{\sum_i T_{\mathrm{eval},i}}{\sum_i T_{\mathrm{episode},i}} \times 100\%

The evaluator accounts for 52.02–86.59% of Full's complete episode time, compared with 41.10–81.37% for Ours, with lower fractions across all six tasks. These fractions are calculated directly from the paired episode measurements, while the independent stage-level profiling results explain the sources of computation cost rather than being used to reconstruct the fractions.

Takeaway. The measurements show that model evaluation represents a substantial portion of complete episode execution time, and selective evaluation reduces both its computational cost and its share of overall runtime. Together with the whole-episode speedups reported in Q1, these results clarify how improvements in evaluator efficiency translate into savings during closed-loop task execution. We have added the complete episode timing, evaluator-fraction measurements, and detailed execution-time breakdown to Appendix G of the revised manuscript.

Reviewer_m3JzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a dual-resolution recursive energy model that allows test-time mixing of the macro(compressed) energy and explicitly expanded subtree energies. These two energy resolutions are incorporated into an AND/OR tree decision model that could selectively expand based on the trained error-radius estimation and action-energy margins. The model alternates itself between wake and sleep phases for further action-learning, self-distillation of the expanded energies, and error-radius estimation. A conditional mathematical bound is given to further understand whether the winning action preserves when fully expanded. In the controlled experiments (that extract the low-level decision making from high-level planning in BabyAI and MiniHack), the authors' method shown great reduction in expansion rate with lower latency w.r.t the fully expanded counterpart, while retaining almost intact success rate.

Critical strengths
  1. The paper is presented with fluent English writing that flows naturally from problem setup, intuition to solution. Clear mathematical definitions and symbol usage are accompanied with delicate figures and tables, shown great overall presentation quality.
  2. The expandable AND/OR energy tree design insightfully split the monolithic decision problem into local sub-problems, providing an interpretable decision when necessary (to expand) and efficiency (when compression is desired). The combination of energy is explained by the mathematical design (additive AND, soft-min OR, and the 1-Lipschitz property for error propagation).
  3. Experiment across three benchmark families shown great reduction in expansion rate and latency while retaining almost full success rate. Ablation studies from various settings are provided for checking the expansion mechanism, the stopping rule, expansion order, structure and leaf supervision.
Critical weaknesses
  1. The AND/OR tree structures are generated (LLM agents + manual auditing) rather than derived from training. This dependence on predefined structure might harm expandability and constrain further model improvement, and the robustness evaluation of the generated structure remains limited.
  2. The authors acknowledge high expansion rate on BossLevel transfer tasks, showing potential limitations of the computational saving in more challenging settings. This raises concerns whether its efficiency advantage could preserve on more complex tasks.
Questions for authors
  1. Will this dependence on predefined AND/OR tree harm scalability or constrain further model improvement, especially to unseen structures or complex tasks?
  2. Would a more accurate macro-energy estimation or a tighter but reliable error-radius might be favorable on more challenging tasks? Are additional conditions or mechanisms needed to maintain this computational efficiency while maintaining reliability?
  3. How might this structure eventually be applied in real-world problems? What is the potential cost?
5 comments
AuthorsOpenReview ↗

Response to Reviewer m3Jz (1/2)

We thank the reviewer for the constructive questions on scaffold robustness, scalability, and practical applicability.

W1. Robustness of independently generated scaffolds

To address your concern about whether our method depends on a particular predefined AND/OR structure, we extended the independent scaffold experiment originally reported in Appendix L. For the same BabyAI UnblockPickup task, we independently ran the LLM-based scaffold generation process three times, producing three different trees with 29, 32, and 33 internal nodes.

To examine whether selective inference remains effective across these structures, we fixed each scaffold and extended its evaluation from a single training run to six independent training and evaluation, following the main benchmark protocol. Within each run, Full and Ours share the same scaffold and trained checkpoint, allowing us to compare their inference behavior under matched conditions while assessing robustness across different scaffold constructions. The results are shown below:

NodesFull success (%)Ours success (%)Expansion (% Full)Early cert. (%)Disagree. (%) ↓Residual-valid (%)Realized-valid (%)
2983.00±2.9780.67±1.8642.86±4.3598.61±0.678.52±4.1180.24±6.6187.80±4.91
3285.17±2.2384.50±3.7850.37±2.2299.05±0.504.61±2.7391.43±2.9094.46±3.00
3388.83±2.5687.00±3.1039.70±5.5399.41±0.708.90±5.4381.88±7.1287.65±7.15

Mean ± SD over six seeds. Early certification uses all audited decisions; disagreement and margin-validity rates are conditional on early certification.

Across all three scaffolds, Ours reduces explicit expansion by 49.63–60.30%, while maintaining success within 0.67–2.33 percentage points of Full. Early-certification rates remain above 98%, while disagreement rates range from 4.61% to 8.90% across the three structures.

Takeaway. These results show that selective evaluation remains effective across independently generated scaffolds for the same task, rather than relying on a particular LLM-generated tree. Despite differences in scaffold structure and training seeds, Ours consistently reduces expansion while maintaining success close to Full. We have extended Section 4.2 and Appendix L of the revised manuscript with the six-seed scaffold experiments and corresponding stopping-reliability analyses.

Q1. Will dependence on predefined AND/OR trees limit scalability to unseen structures or more complex tasks?

Our framework supports new tasks through progressive scaffold construction and reuse of existing semantic factors, rather than requiring every task structure to be generated entirely from scratch.

For example, we begin with a simple BabyAI task, GoToObj, where an LLM constructs an initial scaffold containing navigation-related predicates. After manual review, this scaffold and its predicate library are provided to the LLM when constructing the next task, such as OpenDoor. The LLM can reuse existing navigation predicates while adding new conditions for door-related interactions. As tasks become more complex, such as BlockedUnlockPickup, the scaffold can be further extended with additional task-specific conditions. Related MiniHack tasks also allow partial reuse of existing semantic factors.

Once constructed, each scaffold remains fixed during training and inference. The contract–expand mechanism then uses predicted contraction errors and action-energy margins to determine which subtrees require explicit evaluation. For new or modified structures, the corresponding energy and error-radius predictors can be trained or adapted, while the same adaptive inference mechanism continues to apply.

Our experiments provide evidence for both scaffold reuse and robustness to different constructions. In Overcooked-AI, asymmetric_advantages reuses the same 54-node scaffold as cramped_room, with model parameters retrained for the new layout. The independent scaffold experiments in W1 further show that selective evaluation maintains its expansion savings across three different constructions of the same UnblockPickup task.

In summary, these results illustrate two complementary aspects of scalability: existing scaffolds and predicates can be reused or extended across related tasks, while the adaptive inference mechanism can operate on different task-specific structures. We have clarified the progressive construction process and cross-task scaffold reuse in Appendix B.6 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer m3Jz (2/2)

W2 / Q2. Can computational savings be maintained on more challenging tasks, and are additional inference mechanisms needed?

To examine whether our method can maintain computational savings on more challenging tasks, we conducted an additional BossLevel-specific training experiment alongside our original zero-shot transfer evaluation. Both settings use the same 34-node scaffold and contract–expand inference mechanism, allowing us to investigate whether task-specific learning can improve computational efficiency without changing the inference rule.

Under zero-shot transfer from UnblockPickup, without BossLevel-specific training or recalibration, Ours expands 32.52 of 34 nodes, indicating more conservative evaluation under distribution shift. We then trained the model specifically for BossLevel and compared Full and Ours using the same inference mechanism. The results are shown below:

SettingFull successOurs successExpanded nodesExpansion reduction
Zero-shot transfer68.67±8.62%68.33±8.24%32.52±0.614.34±1.80%
BossLevel training76.50±5.13%74.00±4.90%20.86±1.4338.63±4.19%

Takeaway. With task-specific training, expansion reduction increases from 4.34% to 38.63%, while Ours maintains success close to Full (74.00% vs. 76.50%). These results show that the existing contract–expand mechanism can recover substantial computational savings on more complex tasks through task-specific learning, without requiring additional inference mechanisms in this setting. We have added the BossLevel-specific training comparison to Section 4.2 and Appendix E of the revised manuscript.

Q3. How could the framework be applied to real-world problems, and what would be its potential cost?

Our framework can be applied to structured sequential decision-making problems by combining task-specific AND/OR decision structures with the same adaptive inference mechanism. Each application can define its own semantic factors and action-conditioned conditions, while the contract–expand mechanism determines which parts of the decision structure require explicit evaluation.

Potential applications include robotic manipulation, autonomous navigation, industrial control, and multi-step agent planning. For example, a robotic manipulation task may involve grasp feasibility, collision avoidance, and placement constraints, while navigation tasks may involve accessibility, prerequisites, and alternative paths. These conditions can be organized into AND/OR structures through domain knowledge or LLM-assisted construction.

The cost of applying our framework can be divided into two parts:

  • Offline setup costs: scaffold and predicate-library construction, LLM-based predicate annotation, task-specific model training, and manual verification where applicable. These costs can be amortized through repeated use of the same scaffold, while compatible structures and predicates can be reused across related tasks.
  • Online inference costs: neural scoring, recursive energy aggregation, and adaptive expansion. Once trained, the evaluator predicts predicate values directly, without requiring LLM annotation at each decision.

In summary, our framework separates task-specific scaffold construction from the reusable adaptive inference mechanism, allowing the same decision procedure to be applied to different structured tasks. Its practical cost comes from offline scaffold construction and model training, as well as online neural scoring and adaptive evaluation. Our experiments across BabyAI, MiniHack, and Overcooked-AI demonstrate the mechanism in different structured environments, while the independent scaffold experiments provide further evidence of its flexibility. We have described the scaffold construction and cross-task reuse workflow in Appendix B.6 of the revised manuscript.

Thank you for the clarifications and additional experiments. The multi-seed scaffold evaluations and BossLevel-specific training results partially address my concerns about structural robustness and computational savings. The discussion of deployment costs is also helpful. Broader generalization and zero-shot efficiency remain concerns, but my overall assessment is unchanged. I will maintain my rating.

Reviewer_uC57OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Dual-Resolution Recursive Energy for action selection. Each candidate action is scored by an AND/OR tree of learned predicate checks. Any subtree can be evaluated either in full or through a learned “macro-energy” shortcut on the same energy scale.

A wake–sleep scheme distills the full subtree values into the shortcuts and learns error radii for them. At inference, these radii propagate up the tree. The controller expands the most uncertain branches until the winning action is separated from its competitors, and a conditional result guarantees the full-tree winner is preserved when the errors fit within the margin.

Across BabyAI, MiniHack and Overcooked-AI, the method cuts explicit expansion by 51–78% and P50 latency by 27–58% versus full evaluation. It keeps near-full success and at least 99.2% action agreement.

Critical strengths

The idea of selecting actions based on energy and uncertainty sounds nice, especially for problems where decisions vary in difficulty. Overall, the work and evaluation seem reasonable.

  1. Shortcuts and fully expanded subtrees live on the same energy scale, so the two can be mixed inside one decision. It replaces a global fast/slow switch with fine-grained, per-subtree control backed by a correct winner-preservation bound.
  2. The compared models share one trained checkpoint. The RF-order + Stop ablation shows that radius-guided selection reaches similar success with substantially less expansion. This makes the efficiency claim about where to expand more or less independent of the rest of the pipeline.
Critical weaknesses
  1. The performance of the method probably depends heavily on the LLM-based components. Some kind of comparison of the variants, to some baseline without the LLM components would help to understand the methods contribution. A RL baseline without planner and other LLM additions would give more insights in "raw" performance.
  2. Full’s latency scales roughly with node count, which suggests a sequential implementation. Full has no data-dependent control flow, so it can be computed in one batched pass, while Ours requires an inherently sequential loop. Information about implementation would help to understand fairness of evaluation.
  3. Node count is an unreliable cost proxy in the paper’s own tables because "Random fixed" is slower than "Full" on large_room despite fewer nodes.
Questions for authors
  1. How is Full implemented for timing? What is the latency of a batched Full pass, and how much time does the Ours controller loop itself take?
  2. Does Flat-BC receive the same action-conditioned consequence features as Ours? If not, how does it perform with them?
  3. How does it compare to a RL baseline?
6 comments
AuthorsOpenReview ↗

Response to Reviewer uC57 (1/3)

We sincerely thank the reviewer for the constructive feedback. In particular, the questions about batching and expanded-node counts led us to identify an inconsistency in our original Overcooked-AI timing implementation. The batching suggestion also motivated additional experiments examining when selective computation translates into actual latency savings. We appreciate these observations, which have helped us strengthen the evaluation.

W2 / Q1. How is Full implemented for timing, and what is the latency of a batched Full pass?

Q1(a). How is Full implemented for timing?

In our matched recursive/iterative comparison, Full evaluates the complete AND/OR tree, while Ours selectively expands subtrees using the same evaluation procedure. Both methods share the checkpoint, scaffold, and fixed decision states, so the comparison measures the effect of computation allocation under a common execution protocol.

Q1(b). What is the latency of a batched Full pass?

As you pointed out, Full has no data-dependent expansion process and can compute all leaf scores in a single batch. To examine whether static batching offsets the savings from selective expansion, we compared batched Full with an optimized implementation of Ours.

Batched Full computes all leaf scores in one batch, while optimized Ours reuses encoded features, batches only the requested leaves within each expansion round, and caches computed scores. To keep the comparison consistent with our main evaluation, we use the same 6 BabyAI tasks, 8 MiniHack tasks, and 3 Overcooked-AI layouts as in the main experiments. We measure the complete low-level decision latency (ms) on matched fixed decision states. Each benchmark result is the unweighted average of task-level mean latencies across its corresponding tasks/layouts.

BenchmarkBatched Full (ms)Optimized Ours (ms)
BabyAI (6 tasks)9.63710.189
MiniHack (8 tasks)5.6216.353
Overcooked-AI (3 layouts)1.0610.848

The results show that Batched Full is faster in 11 of the 17 tasks, while optimized Ours is faster in the remaining six. This confirms that static batching can be more efficient when individual leaf evaluations are lightweight. We have included this comparison in Section 4.3 and Appendix G.

Your question also raises an important issue:

When does selective evaluation become more efficient than full batching?

To investigate this, we designed a controlled per-leaf cost-scaling experiment across six BabyAI tasks. Full benefits from evaluating all leaves in parallel, whereas Ours avoids unnecessary leaf evaluations but requires sequential expansion rounds. The relative efficiency therefore depends on the computational cost of each leaf. To test this, we added the same computational workload to each evaluated leaf in both methods, increasing its computation cost without changing its output score. We varied the number of processed vectors TT and computation layers DD, using a fixed subset of held-out decision states from our original latency benchmark for each task, shared across all methods and workload settings. This design allows us to examine how increasing per-leaf computation cost affects the trade-off between Full's batching advantage and Ours' selective computation savings. The table reports mean complete-decision latency in milliseconds, averaged equally across the six tasks.

Per-leaf cost (T/D)Batched Full (ms)Optimized Ours (ms)Mean task-wise reduction (%)Ours faster
0/4 (no extra cost)11.02711.675−30.5%2/6
2048/421.24722.349−4.3%3/6
4096/432.49627.044+17.4%4/6
16384/4103.70869.310+32.8%6/6
16384/32771.627458.695+40.5%6/6

Mean task-wise reduction is calculated as 100 × (Full − Ours) / Full for each task and then averaged across the six tasks. Positive values favor Ours.

Takeaway. The results confirm that Full benefits from static batching, particularly when individual leaf evaluations are lightweight. More importantly, the cost-scaling experiment reveals a cost-dependent crossover: as per-leaf computation becomes more expensive, the savings from selective evaluation increasingly outweigh Full's batching advantage. At the highest tested workload, Ours achieves a 40.5% mean task-wise latency reduction and is faster on all six tasks. These findings clarify when selective expansion becomes more efficient than full batching.

In our revised manuscript, we have expanded Section 4.3 to discuss the effects of execution strategies and per-leaf computation costs, and added the batched comparison and complete 12-setting cost-scaling experiments to Appendix G, including implementation details and full results.

AuthorsOpenReview ↗

Response to Reviewer uC57 (2/3)

Q1. How much time does the Ours controller loop itself take?

Across all 17 tasks, the Ours controller spends an average of 0.706 ms per decision on frontier updates, expansion selection, and stopping checks, with a task-averaged control share of 4.0%. To examine where the remaining computation is spent, we separately profiled Ours under the recursive/iterative evaluation protocol, measuring four components:

  • Encoding: processing observations, history, candidate actions, and optional consequence features.
  • Scoring: neural leaf/macro-energy/radius predictions and recursive energy aggregation.
  • Control: frontier updates, expansion selection, and stopping checks.
  • Other: remaining execution and measurement overhead.

The table reports mean per-decision component times in milliseconds (ms), averaged equally across tasks within each benchmark. Percentage shares for control and scoring are calculated within each task before averaging, with all 17 tasks weighted equally in the overall summary.

BenchmarkEncodingScoringControlOtherTotalControl share
BabyAI (6)3.57115.0710.8631.15320.6594.2%
MiniHack (8)3.59918.5990.6630.83323.6933.0%
Overcooked-AI (3)0.3986.0810.5100.9887.9776.5%
All 17 tasks3.02415.1450.7060.97419.8494.0%

Takeaway. The controller contributes only 4.0% of the instrumented decision time, compared with 74.6% for scoring. This shows that neural scoring and recursive energy evaluation dominate the computational cost, while frontier selection and stopping introduce relatively little overhead. We have added the detailed timing breakdown, including per-task results, to Appendix G and expanded the discussion of adaptive control overhead in Section 4.3 of the revised manuscript.

W3. Expanded-node count as a proxy for computational cost

To address your concern that "Random fixed" was slower than "Full" on large_room despite expanding fewer nodes, we examined how neural predictions were scheduled during evaluation.

In the original Overcooked-AI timing, neural predictions for all leaves were computed in a single batch, including those not requested by selective expansion, while tree-energy aggregation remained recursive/iterative. Random fixed and Node-wise also precomputed macro-energy and certificate predictions for all internal nodes, including those not subsequently used.

We have now unified the evaluation protocol across all three benchmark families, using demand-driven recursive/iterative evaluation that computes predictions only when required by each method's expansion procedure. The resulting three-run P50 latencies (mean ± SD, ms) are shown below:

LayoutFullOursRFNode-wiseGlobal
cramped_room13.850±1.8461.031±0.0813.342±0.6232.069±0.3270.826±0.119
asymmetric_advantages13.270±2.6613.449±0.6683.681±0.5562.086±0.2540.818±0.116
large_room23.246±3.66015.554±1.61216.119±1.76517.266±2.40621.926±2.240

Under the unified protocol, Random fixed achieves 16.119±1.765 ms on large_room, compared with 23.246±3.660 ms for Full, directly addressing the comparison you highlighted. Ours also achieves a lower latency of 15.554±1.612 ms in this layout.

Takeaway. These results clarify the relationship between expanded-node counts and measured latency under a common execution protocol. Expanded-node count reflects structural computation allocation, while wall-clock latency measures actual execution cost. Reporting both provides a more complete evaluation of selective computation. In our revised manuscript, we have updated the detailed expansion and decision-latency results in Appendix G.

AuthorsOpenReview ↗

Response to Reviewer uC57 (3/3)

Q2. Does Flat-BC receive the same action-conditioned consequence features as Ours?

Yes. Flat-BC receives the same applicable runtime inputs as Ours, including action-conditioned consequence features when used. The difference lies in how the models are trained and how they compute action scores:

  • Flat-BC: directly predicts actions from the shared inputs, without predicate-level supervision or recursive composition.
  • Flat MLP: uses the same runtime inputs and predicate-level supervision as Ours, but predicts actions without AND/OR composition.
  • Ours: composes predicted predicates through recursive energies and selectively expands subtrees.

These baselines serve different purposes. Flat-BC provides a standard imitation-learning reference, while Flat MLP isolates the contribution of recursive AND/OR composition under matched predicate supervision. Thus, the comparison with Flat-BC does not involve an input advantage for Ours.

In our revised manuscript, we have clarified the shared runtime inputs, predicate supervision, and representation differences among these methods in Appendix B.10.

W1 / Q3. How much does the method depend on LLM components, and how does it compare with an RL baseline?

To clarify the role of LLMs in our framework, LLMs serve two upstream roles: (1) high-level subgoal planning and replanning for long-horizon tasks (e.g., CrossDoor → CrossDoor → Pickup), which is unnecessary for simple tasks such as GoToObj, OpenDoor, and PickupLoc; and (2) offline AND/OR scaffold construction and predicate annotation. The low-level contract–expand inference mechanism operates on the resulting task interface and scaffold, without online LLM calls for recursive evaluation.

To provide a reference for performance without our LLM-based decomposition or other project-specific structured inputs, we additionally evaluated BC and PPO under the official BabyAI configuration. On UnblockPickup, BC achieves 50% success with 2,000 demonstrations and 32% with 400 demonstrations, while PPO achieves 0% after 2M environment steps.

Separately, PPO using our shared task interface achieves 82.2% macro success across six BabyAI environments after 1M online steps per environment, including 66% on UnblockPickup.

Takeaway. These experiments provide complementary BC/PPO references under different task interfaces. The official BabyAI experiments examine performance without our project-specific structured interface, while the shared-interface PPO experiment provides an RL reference under the same task interface used by our framework. Our matched-input comparisons and same-checkpoint inference ablations more directly isolate the contributions of recursive representation and adaptive evaluation within this shared interface.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.