acceptodds
Under review as a conference paper at ICLR 2027

What Makes a Good Symbol for Hybrid Discrete–Continuous Planning?

18 views

Abstract

The physical world has continuous state and action spaces, yet robot planning often benefits from hybrid discrete-continuous formulations that explicitly expose combinatorial structure, such as contact modes or discrete primitive actions. Existing methods typically rely on human-defined symbol sets or symbols derived from language supervision, without asking what makes a symbol computationally useful. In this paper, we take an inference-algorithm–centric view to define usefulness: a good symbol should expose the discrete commitments that simplify solving the original planning problem. We first introduce the concept of solver-sufficient symbolization by studying finite-horizon piecewise-affine systems, for which solver sufficiency can be precisely characterized, and develop a counterexample-guided algorithm for symbol invention. We then extend this principle to invent symbols for optimization objectives parameterized by general neural networks as energy functions, showing that solver-sufficient symbols can nevertheless be induced from their continuous optimization landscapes. Experiments on simulated and real-robot tasks show that the resulting symbols reduce solving cost while preserving solution quality, while also improving interpretability and steerability. Furthermore, our theoretical and algorithmic insights help explain why certain human-defined symbols are computationally useful.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neuro-Symbolic Learning, Optimization, Robot Planning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_9u1xOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work presented solver-sufficient symbolization, where commitments reduce the optimization problem to one whose solution is simplified by discrete commitments and relative affine-TU certificates, which allow LP relaxation in exact manner after conditioning the problem on symbolic assignments. In the case of neural energy objectives, τ-descent faithfulness, which is an accessibility requirement relying on energy increase bounds, is introduced.

Critical strengths

Relative affine-TU constructions, in the context of PWA, provide a way to verify sufficient conditions for LP relaxation to be exact after symbolic assignment. This matters since it provides a concrete target for symbol discovery.

Critical weaknesses

Please check Questions For Authors

Questions for authors

Definition 3.2 defines v^tau{s,R}(x) via an infimum taken over all possible admissible continuous paths. Thus, Proposition B.4 ensures that the correct endpoint is reachable according to the path framework. However, it does not guarantee that one single execution of a gradient descent or noisy descent algorithm achieves this target and how many steps are necessary for this.

Appendix D assigns the maximum value of the objective function at three points (endpoints and middle point) for each edge and suggests that in this way the barrier estimation for the segment is not underestimated. This statement is false, since the maximum value at three sampled points might be lower than the maximal value of the function on the whole segment. The graph will miss all barriers.

What are the neural energy functions and how do they convert the baselines? The authors need to clarify hs(x), describe how it works, discuss the management of the diffusion time and the modification of each baseline.

Experiments do not provide the evidence of the optimality preservation and even lead to the degradation of the quality. In the case of neural networks, Table 2 shows Aligning success deteriorating from 0.94 to 0.84 if symbols are added.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 9u1x (Part I: Q1)

We thank the reviewer for the precise technical comments, which help us correct and sharpen the paper. We address each point below.

Q1 (Prop. B.4 shows that a near-optimal endpoint is reachable, but not that a single descent reaches it, nor how many steps this takes.)

Thanks for bringing this up. We respectfully note that, in our setting, the solver and its capability are given: the paper studies how to partition a problem, not how to solve each part, and the solver is chosen by the user (Eq. 2 is defined with respect to a given solver). We agree that Prop. B.4 shows only that a good path exists, not that any given optimizer would follow it. Here, we support the practicality of Prop. B.4 empirically and with existing theory, as detailed below.

(1) Empirically. Our implemented solver, projected noisy gradient descent (a Langevin-type dynamics), reaches the τ\tau-neighborhood in most cases: with one confined descent per region, its best endpoint is within τ\tau of the best value found by a 200-start multi-start on 29/32 states (Avoiding) and 30/30 states (Pushing); see Q4.

(2) Theoretically. There are established methods that provably ensure that an optimizer that can climb by τ\tau is not trapped in basins shallower than τ\tau, given enough steps; for such an optimizer, a τ\tau-descent faithful region has no basin that keeps it away from the τ\tau-neighborhood of the optimum. We give two examples. For simulated annealing, Hajek [1] defines the depth of a local minimum as the smallest climb needed to reach a lower state, which is the quantity that τ\tau bounds, and proves (for finite state spaces) that annealing leaves every local minimum whose depth is at most its cooling constant. Projected Langevin algorithms [2], which have the same form of update as our solver, provably reach near-optimal points of non-convex objectives on compact convex sets. These methods act on the optimizer side, i.e., on how to solve each part, which our method takes as given and does not modify.

Number of steps. The number of steps is likewise a property of the given solver, so the paper gives no bound on it. Determining it is an inherent challenge of diffusion- and flow-based energy methods; our method inherits it and does not modify it. The simulated tasks (Table 2) use 600 steps per descent.

In the revised version, we have stated the solver assumption (see Remark 3.2, to which Prop. B.4 now refers), added the two points above, and rewritten the three sentences that read as a guarantee for the implemented descent (Sec. 3.1 after Eq. (5), App. A.1, and App. B.5). In the PWA setting, no such assumption is needed, since both the certificate and the LP solver are exact. Thank you very much again for raising this concern.

[1] Cooling Schedules for Optimal Annealing, 1988.

[2] Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning, 2021.

AuthorsOpenReview ↗

Response to Reviewer 9u1x (Part II: Q2, Q3)

Q2 (The maximum over three points can underestimate the barrier along a segment, so the statement in Appendix D is false.)

The reviewer is right; we thank the reviewer for catching this writing error and have corrected it. For a neural energy, the exact maximum along a segment has no closed form, so sampling gives a provable lower bound on it. In our implementation for all results in the paper, we evaluate 50–200 points per segment depending on the task, not three (cf. Fig. 4).

Q3 (What is the neural energy hsh_s, how is the diffusion time handled, and how is each baseline converted?)

We clarify below how hsh_s is constructed for each baseline and how the diffusion time is handled; the construction for each baseline is listed in App. F.2 of the revised version. In short, hsh_s is −log⁡π(x∣s)-\log \pi(x \mid s) up to a constant, or a surrogate of it when the likelihood has no closed form; it is evaluated on a fixed grid of diffusion times with fixed noise; and every baseline is frozen.

Energy. One rule covers all policies: viewing a trained policy as a conditional distribution π(x∣s)\pi(x \mid s) over the decision xx, we set hs(x)=−log⁡π(x∣s)+c(s)h_s(x) = -\log \pi(x \mid s) + c(s), where c(s)c(s) is independent of xx and affects neither minimizers, gradients, nor barrier heights. When the likelihood has no closed form (e.g., diffusion policies), we use a surrogate of −log⁡π-\log \pi. For DDPM, this is the denoising training loss at xx, with nn fixed pairs of diffusion time and noise (ti,ϵi)(t_i,\epsilon_i) and xtix_{t_i} the correspondingly noised xx:

hs(x)=1n∑i=1n∥ϵθ(xti,ti,s)−ϵi∥2.h_s(x)=\frac{1}{n}\sum_{i=1}^{n}\big\Vert\epsilon_\theta(x_{t_i},t_i,s)-\epsilon_i\big\Vert^2 .

This is the DDPM variational bound (ELBO) without its time weights on a fixed set of diffusion times: a surrogate of −log⁡π(x∣s)-\log \pi(x \mid s), not a bound on it. BESO, also a diffusion policy, uses the same construction with its own denoising loss. Explicit energy models (IBC) give hs=Eθh_s = E_\theta, and explicit densities (e.g., GMM) give hs=−log⁡πh_s = -\log \pi directly. For most baselines (DDPM, BESO, IBC), hsh_s is thus read out from the frozen network with no additional training; for ACT and BeT, it is an energy network fitted to action samples from the frozen policy. Values come from forward passes and gradients from automatic differentiation with respect to xx.

Diffusion time. Each tt sets a smoothing scale of the energy: at small tt the energy is sharp and has many spurious basins (not corresponding to distinct action modes); at large tt it is over-smoothed and distinct modes merge. We therefore use a fixed grid of intermediate tt with fixed noise (four diffusion times combined with four fixed noise draws, n=16n = 16 pairs, uniform weights), which makes hsh_s deterministic and its basin structure stable.

Baselines. No baseline network is modified: each is frozen and its energy is obtained as above, with the same solver and budget for all. The energy of each baseline is listed in App. F.2.

AuthorsOpenReview ↗

Response to Reviewer 9u1x (Part III: Q4)

Q4 (The experiments give no evidence that optimality is preserved, and quality even degrades on Aligning (0.94 to 0.84).)

Thanks. Here, we add direct evidence that optimality is preserved, and we explain the drop on Aligning. (i) In the PWA setting, the conditioned LP attains the MILP optimum on all tested states; in the neural setting, the symbol-guided solver ends within τ\tau of the best value found by multi-start on 91% (Avoiding) and 100% (Pushing) of the states. (ii) The drop on Aligning comes from the descent step limit and from under-sampling; once both are addressed, SR returns to 0.94. We also respectfully note that the paper already reports optimality evidence through DCδ\mathrm{DC}_\delta (Table 2).

Guarantee and evidence. Reaching the optimum requires (a) that the optimum of each conditioned problem is reachable and (b) that the solver reaches it. Our properties guarantee (a); (b) depends on the given solver (Q1).

  1. PWA setting. Total unimodularity makes the LP relaxation exact (a) and LP solvers are exact (b), so each conditioned problem is solved to optimality without further assumption. Empirically, the conditioned LP at ψ⋆=Wz⋆\psi^\star = W z^\star, with z⋆z^\star the MILP optimum, attains the MILP optimal value on all tested states.
  2. Neural setting. If a region has the τ\tau-descent faithfulness property (verified on samples), a path to a near-optimal point exists from every start in it (a); whether the solver follows it to the end (b) depends on the solver, e.g., on its step limit. DCδ\mathrm{DC}_\delta measures this reachability: the minimal number of descents needed to reach the τ\tau-neighborhood of the optimum with probability at least 1−δ1-\delta (Table 2). In addition, the symbol-guided solver reaches this neighborhood, relative to the multi-start optimum, on most states.
States where the optimum is reached (%)MPushTHookShelfBlockCuboidPenAvoidingPushing
exact (PWA) / within τ\tau (neural)10010010010010010091100

The drop on Aligning. Its cause is consistent with the decomposition above: the step limit violates (b), and under-sampling leaves a few uncertified regions that violate (a). A descent that occasionally falls into such a basin needs more steps to leave it, so we allow more descent steps; denser sampling (1.2× the samples) then removes these regions. Together they restore SR to that of the multi-start solver, while the DCδ\mathrm{DC}_\delta advantage remains.

AligningDDPM (multi-start, 24 starts)DDPM (Symbols, Table 2; ≈2.5 starts)+ more descent steps+ more descent steps + denser sampling (1.2×)
SR0.940.840.880.94
DCδ\mathrm{DC}_\delta10.312.412.122.36

Exact certification is out of reach for modern networks; our framework instead makes its premises explicit and gives testable diagnoses when they fail.

We have incorporated these corrections, clarifications and results into the revised version. We hope they address your concerns; if so, we would be grateful if you would consider raising your score.

Reviewer_rgpYOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a solver-centric definition of a useful symbol for hybrid discrete-continuous planning. A symbol is a discrete summary of the decision variables and is called solver-sufficient if fixing its value preserves global optimality while reducing the complexity of the resulting subproblem. The authors instantiate this for MILP-based piecewise-affine (PWA) systems, via a relative affine-TU certificate that makes the conditioned problem solvable by LP relaxation, and for neural energy-based policies, via a τ-descent-faithfulness condition on basins, and construct the symbols with a shared counterexample-guided algorithm (CPR).

Experiments on six PWA tasks and five neural tasks show that the invented symbols preserve success rate while speeding up solving over the full MILP, and reduce the number of descents needed to reach a near-optimal solution in the neural setting. The theory is further used to explain why some hand-designed binary variables matter more than others, and to let a user choose between a policy's alternative strategies, such as passing an obstacle on the left or right, by fixing a symbol value.

Critical strengths
  1. The central research question, what makes a symbol useful for hybrid discrete-continuous planning, is well posed, and the answer is principled: usefulness is defined by what fixing the symbol does to the remaining solve, not by descriptiveness or downstream success rate. The paper is also very well written and carefully designed, and remains readable for non-experts.
  2. The experimental evaluation is extensive across problem types (contact-rich 2D tasks, dexterous in-hand reorientation, D3IL benchmarks, two real-robot tasks) and against both solver-oriented and symbol-oriented baselines. The Q2 analysis, which shows that the hand-designed binary variables the solver cannot relax away are also the ones whose removal hurts task success most, is a convincing demonstration that the theory has explanatory value beyond the proposed method.
Critical weaknesses
  1. The theoretical guarantee holds only for PWA systems. For neural policies, τ-descent faithfulness is verified only on a finite set of sampled descents, so nothing is established about the full continuous action space. Given the complexity of neural networks this limitation is understandable, but the main text presents the two settings as parallel and mentions the gap only briefly in Sec. 3.2 and Appendix A.
  2. The reported speedups exclude the cost of symbol discovery. Table 6 (p. 20, l. 1026-1034) reports up to 962 s at a converged setting and 2350 s at the finest resolution for a single task and horizon, and Table 5 shows symbols must be rediscovered for each horizon.
Questions for authors
  1. Is the symbol matrix W computed once per task from the constraint matrices, independent of start state and goal? If so, please state it; if not, please report the amortized symbol-discovery time for the PWA tasks in Table 1.
  2. How many evaluation episodes are behind each entry in Tables 1, 2 and 4, and are they averaged over seeds? Please provide standard deviations, since several reported differences are small.
  3. Do the authors have an intuition for why SR drops for Block in Table 1 (0.88 vs 0.94 for GBD-MPC) and for Aligning in Table 2 (0.94 to 0.84 after adding symbols)?
4 comments
AuthorsOpenReview ↗

Response to Reviewer rgpY (Part I: W1, W2)

We thank the reviewer for the careful reading and the constructive comments. We address each point below.

W1 (The guarantee is exact only for PWA; for neural policies it is verified on finite samples, yet the main text presents the two settings as parallel.)

Thanks! We agree, and we will state this difference more explicitly in the revised version. The two settings are not totally symmetric in this sense. The PWA certificate is exact. On neural networks, our method is empirical: τ\tau-descent faithfulness is verified on a finite set of sampled descents and holds only relative to this evidence. Verifying even local properties of a neural network exactly is NP-hard, so sample-based verification is a standard compromise.

W2 (The reported speedups exclude the cost of symbol discovery, and the symbols must be rediscovered for each horizon.)

This is correct: the reported speedups are online costs. Note that our symbol discovery is a one-time offline cost, of the same kind as training. It is performed once per horizon, offline, and the same symbols are reused across initial states. No symbol is invented at deployment. Reporting online and offline costs separately is common practice and applies to every method we compare: the classifier training of CoCo, the offline learning of the symbol-invention baselines, and the training of the policies are likewise excluded from the reported online metrics. The discovery times are reported separately, in Q1 for PWA and in Tables 7–8 of the revised paper (Tables 6–7 of the original submission) for the neural setting. For comparison, training CoCo's classifier takes about 6.14 times as long as symbol discovery.

On the horizon. In the neural setting, the decision variable is the action chunk a1:Ha_{1:H}, so changing HH changes the policy and hence hsh_s; each horizon therefore has its "own symbols", discovered once offline. A deployed policy has a fixed HH, so no further discovery takes place at deployment.

AuthorsOpenReview ↗

Response to Reviewer rgpY (Part II: Q1, Q2, Q3)

Q1 (Is WW computed once per task, independent of the start state and the goal?)

Yes. WW depends only on the coefficient matrices (Ed,Pd,Bd)(E^d, P^d, B^d). The start state and the goal do not change these matrices; they enter only the right-hand sides (and the cost). Total unimodularity guarantees integrality for every integral right-hand side (Hoffman–Kruskal, Sec. 2.1), so the same WW is valid for all states and goals and is computed once per task. This computation takes on the order of seconds.

MPushTHookShelfBlockCuboidPen
CPR time (s)0.431.660.480.110.110.18

Q2 (How many evaluation episodes and seeds are behind each entry of Tables 1, 2 and 4, and what are the standard deviations?)

Each entry in our paper is computed over 64 evaluation episodes (8 seeds); we now state this in the experiment setup (Sec. 4). The standard deviations are given below. SR and PR are binomial proportions, so their standard deviation is p(1−p)/n\sqrt{p(1-p)/n}; IS is reported as geometric mean ± standard deviation, and DCδ\mathrm{DC}_\delta as mean ± standard deviation across states. We list the results of our method (IS for the six PWA tasks, DCδ\mathrm{DC}_\delta for the three simulated neural tasks); the complete tables with all methods will appear in the revised version.

OursMPushTHookShelfBlockCuboidPenAvoidingPushingAligning
SR ± std1.00 ± 0.001.00 ± 0.000.94 ± 0.030.88 ± 0.040.94 ± 0.030.88 ± 0.041.00 ± 0.000.91 ± 0.040.84 ± 0.05
IS / DCδ\mathrm{DC}_\delta ± std4.81 ± 2.913.07 ± 0.662.77 ± 0.431.53 ± 0.121.34 ± 0.171.94 ± 0.182.00 ± 0.001.00 ± 0.002.41 ± 0.30

Our claim is that symbols preserve SR while reducing cost, so SR differences within these standard deviations are consistent with it.

Q3 (Why does SR drop for Block (0.88 vs. 0.94 for GBD-MPC) and for Aligning (0.94 to 0.84 with symbols)?)

We investigated both cases: the drops stem from the receding-horizon gap (Block) and from the step budget and the sampling density (Aligning), not from a flaw of the method itself. As the reference, note that our method is designed to match end-to-end exact solving (E2E), not to outperform other approximate methods.

  • Block (PWA). Our controller is a receding-horizon controller: at every step, symbols are selected over a finite planning horizon, which is shorter than the task. A commitment that is optimal within this horizon can therefore invalidate a later transition (Sec. 4.1); on Block, a symbol switch takes several time steps. This gap is a general issue of receding-horizon control, not specific to our method. A general remedy is a dwell constraint, which holds a symbol for the required number of steps; it returns SR to the E2E level.

  • Aligning (neural). (1) The number of steps allowed for a single descent was too small; allowing more steps raises SR. (2) In addition, we found a few regions that violate τ\tau-descent faithfulness and were missed because of under-sampling (cf. W1); denser sampling removes them. SR then matches that of the multi-start solver, and the DCδ\mathrm{DC}_\delta advantage remains.

BlockE2EGBD-MPCOurs (Table 1)Ours + dwell constraints
SR1.000.940.881.00
AligningDDPM (multi-start)DDPM (Symbols, Table 2)+ more descent steps+ more descent steps + denser sampling
SR0.940.840.880.94
DCδ\mathrm{DC}_\delta10.312.412.122.36

We will add these clarifications and results to the revised paper. We hope they address your concerns; if so, we would be grateful if you would consider raising your score.

I thank the authors for their responses and additional information, which address my main concerns.

AuthorsOpenReview ↗

Follow-up on Rebuttal to Reviewer rgpY

Dear Reviewer rgpY,

Thank you for your thoughtful feedback and for recognizing the value of our work. We sincerely appreciate your time and consideration throughout the review process!

Reviewer_kaYHOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to generate useful symbols that simplify hybrid discrete-continuous robotic planning. For piecewise-affine systems formulated as MILPs, it introduces a relative affine-TU formulation and a counterexample-guided algorithm to find symbolic summaries that make the remaining optimization problems solvable through LP relaxations. For neural objectives, it constructs symbols by partitioning the action space based on optimization basins. Experiments across simulated and real-robot tasks show improvements in solving efficiency while largely preserving solution quality.

Critical strengths
  • Interesting question and a solver-oriented approach to generating symbols that simplify optimization in robotic planning. Rather than relying on human-defined symbols, it proposes deriving them from the optimization problem, providing a more principled way.
  • The invented symbols improve solving efficiency over competing methods in several settings, including manipulation tasks. Automatically discovering these symbols without requiring manual specification is a useful contribution.
Critical weaknesses
  • The neural implementation lacks some important details, particularly how the scalar energy function is obtained from the trained policies. These details are needed to understand and reproduce the proposed method.
  • The computational benefits of symbol discovery are not fully isolated from simpler partitioning strategies, such as clustering action modes from a fixed DDPM. Without such comparisons, it is unclear how much of the improvement comes from the proposed symbol-invention method rather than simply separating different action modes.
Questions for authors
  1. How are nonlinear manipulation dynamics represented in the MILPs? Which aspects of the dynamics and contact models are approximated in these formulations?

  2. What is the scalar objective hs(x)h_s(x) for the neural policies, and how are its values and gradients computed? Is it an explicit neural energy, an approximation of negative log-likelihood, or derived from the DDPM's denoising predictions?

  3. How is descent constrained to a selected symbol region during DDPM inference? Is this handled through projection, backtracking, or another method?

  4. How does the method handle smaller or unexplored basins that may be missed during sampling? How stable are the discovered symbols across different states or previously unseen action modes?

3 comments
AuthorsOpenReview ↗

Response to Reviewer kaYH (Part I: W1, W2)

We thank the reviewer for the positive assessment and the constructive comments. We address each point below.

W1 (The paper does not explain how the scalar energy function is obtained from the trained policies.)

Thanks for pointing this out and we have clarified in the revision. In short, one rule covers all policies: Our algorithm only needs an hsh_s that can be evaluated and differentiated; it does not depend on any policy architecture. We view a trained policy as a conditional distribution π(x∣s)\pi(x \mid s) over the decision xx and set

hs(x)=−log⁡π(x∣s)+c(s),h_s(x) = -\log \pi(x \mid s) + c(s),

where c(s)c(s) does not depend on xx and thus changes neither the minimizers, the gradients, nor the barrier heights. When the likelihood has no closed form (e.g., for diffusion policies such as DDPM), we use a surrogate of −log⁡π-\log \pi.

The other two classes need no surrogate: for explicit energy models (IBC), π∝exp⁡(−Eθ)\pi \propto \exp(-E_\theta) gives hs=Eθh_s = E_\theta, with the partition function absorbed into c(s)c(s); for explicit densities (e.g., GMM), hs=−log⁡πh_s = -\log \pi directly. Gradients are taken with respect to xx by automatic differentiation. For most evaluated policies, such as DDPM, BESO, and IBC, hsh_s is read out from the frozen network with no additional training; the construction for each policy is listed in App. F.2.

W2 (It is unclear whether the gain comes from the proposed symbol invention or simply from separating action modes, e.g., by clustering.)

This is a great suggestion. We add the requested clustering ablation below. In general, clustering needs more descents than our symbols and does not improve SR. Our hypothesis is that separating action modes is not what makes a region easy to solve. Whether action modes are clearly separated is a property of the task semantics, whereas whether a region is easy to solve is a property of the energy landscape: it depends on whether the solver gets trapped (Def. 3.2). The two are related but not equivalent.

Why clustering methods were not compared. Clustering-based methods, such as ENAP [1] and [2], invent symbols from a given dataset, such as demonstrations. Our problem setting is different: we are given only the system itself (the MILP or the trained network) and invent symbols by querying it. Our method thus bypasses some of their requirements, such as a dataset from which the symbols are extracted. For a controlled comparison, we apply clustering to the same network below.

Empirically. We add a clustering baseline on the two D3IL tasks. HDBSCAN (no preset number of clusters) partitions the decision space using either (a) action modes (action samples from the DDPM policy) or (b) output modes (endpoints of descents on hsh_s); each cluster defines one region. The energy hsh_s, the solver, and the budget are the same as ours. The results show that clustering either over-splits or under-splits: it never attains a lower DCδ\mathrm{DC}_\delta than CPR symbols without losing SR.

MethodAvoiding SR↑Avoiding DCδ\mathrm{DC}_\delta↓Pushing SR↑Pushing DCδ\mathrm{DC}_\delta↓
DDPM (multi-start, no partition)1.007.030.901.00
DDPM (Clustering on action modes)1.0016.250.884.00
DDPM (Clustering on output modes)1.0021.250.843.62
DDPM (Symbols, ours)1.002.000.911.00

Theoretically. (i) Clustering may fail outright, since tasks such as PushT or dexterous in-hand manipulation often have no such clearly identifiable modes. (ii) Over-splitting adds regions and reduces the speedup; under-splitting leaves a trapping basin inside a region, so the optimum may be lost. In short, the key question is the minimum number of regions the solver needs, and clustering has no criterion for it.

AuthorsOpenReview ↗

Response to Reviewer kaYH (Part II: Q1, Q2, Q3, Q4)

Q1 (How are nonlinear manipulation dynamics represented in the MILPs, and which aspects are approximated?)

Nonlinear dynamics are approximated by a set of mode-dependent local affine models. Concretely, our MILPs admit linear complementarity systems (LCS) [3] (intuitively, by enumerating possible contact modes) and, more generally, gated formulations in which the binary variables zz select the contact mode.

According to the existing literature [3, 4], this form covers most rigid-body dynamics and contact models used in manipulation. It does not cover strongly coupled nonlinearities, such as bilinear terms or deformable objects. This limitation motivates the neural extension in Section 3.

Q2 (What is the scalar objective for the neural policies, and how are its values and gradients computed?)

For diffusion policies such as DDPM, it is derived from the policy's denoising predictions and approximates the negative log-likelihood (W1); no separate energy network is trained. Concretely, we noise xx with nn fixed pairs of diffusion time and noise (ti,ϵi)(t_i,\epsilon_i) to obtain xtix_{t_i}, let the frozen policy predict each noise, and average the squared errors:

hs(x)=1n∑i=1n∥ϵθ(xti,ti,s)−ϵi∥2.h_s(x)=\frac{1}{n}\sum_{i=1}^{n}\big\Vert\epsilon_\theta(x_{t_i},t_i,s)-\epsilon_i\big\Vert^2 .

This is the DDPM training loss evaluated at xx; a lower value indicates that the policy considers xx more likely. Values are computed by forward passes and gradients by automatic differentiation with respect to xx, with the network frozen. Because the pairs (ti,ϵi)(t_i,\epsilon_i) are fixed, hsh_s is deterministic. We use a scalar rather than the score because verification compares barrier heights, which requires function values, whereas the score only gives a direction.

Q3 (How is the descent kept inside a selected symbol region: by projection, backtracking, or another method?)

By projection; no backtracking is needed. Instead of running the reverse diffusion sampler, we run noisy gradient descent on hsh_s: it starts inside the selected region, and the iterate is projected back onto the region after every step. Each region is an intersection of linear cuts on uj⊤xu_j^\top x (Eq. 5), hence convex, so the projection is unique and has a closed form for each cut.

Q4 (How does the method handle small or unexplored basins missed by sampling, and how stable are the symbols across states?)

In short, basins that sampling misses are not covered by our certificate, which holds relative to the samples; and the symbols transfer to unseen states of the same network (a value gap on the order of τ\tau), but there is no guarantee for new tasks or networks. Details and a new experiment follow.

  1. Small or unexplored basins. Our certificate holds relative to the sampled evidence (Sec. 3.2). (1) Theoretically, if a random start reaches a basin with probability pp, all NN independent starts miss it with probability (1−p)N(1-p)^N, which decreases as NN grows. (2) Empirically, SPARSIFY adds samples at a basin that appears sealed.

  2. Generalization across states. (1) For unseen tasks or new networks, there is no guarantee. (2) For unsampled states of the same network, the symbols generalize: we evaluate on states and test scenarios that were not used to construct the symbols. On held-out states, the value reached by the symbol-guided solver stays close to the multi-start optimum (a gap on the order of τ\tau), and the closed-loop SR on unseen scenarios stays close to that of the multi-start solver.

TaskValue gap to multi-start on held-out states (in units of τ\tau) ↓SR on unseen scenarios ↑ (Symbols)SR on unseen scenarios ↑ (Multi-start)
Avoiding0.171.001.00
Pushing0.000.910.90

We have added these clarifications and the two new experiments to the revised paper (see updated PDF file).

References

[1] Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories, 2026.

[2] From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning, 2018.

[3] Task-Driven Hybrid Model Reduction for Dexterous Manipulation, 2024.

[4] A Direct Method for Trajectory Optimization of Rigid Bodies Through Contact, 2014.

AuthorsOpenReview ↗

Follow-up on Rebuttal to Reviewer kaYH

Dear Reviewer kaYH,

Thank you for your thoughtful feedback and for recognizing the value of our work. We sincerely appreciate your time and consideration throughout the review process!

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

We thank all reviewers for their constructive comments. We have uploaded a revised paper. The method and all reported results are unchanged. The revision adds one comparison and a few clarifications, which further support the main message: a good symbol leaves the given solver a problem it can solve, exactly for PWA systems and, as verified on samples, within τ\tau for neural policies.

New evidence

  • Separating action modes does not explain the gain (Sec. 4.4, Table 6, new). With the same energy, solver, and budget, our symbols need 2.00 (Avoiding) and 1.00 (Pushing) descents, whereas clustering-based partitions need 16.25–21.25 and 3.62–4.00, with no gain in SR.
  • Failures are diagnosable (Sec. 4.1). On Block, the failures come from the receding horizon and are removed by a dwell constraint. On Aligning, the drop comes from the step limit and the sampling density; once both are addressed, SR returns to 0.94 with DCδ=2.36\mathrm{DC}_\delta = 2.36 (multi-start: 10.31).

Clarifications

  • Scope of the guarantee (Remark 3.2, new). We study how to partition a problem, with the solver given. In the neural setting, τ\tau-descent faithfulness, verified on samples, ensures a path to a near-optimal point from every start; that the optimizer follows it is an assumption on the solver and a standard capability (e.g., simulated annealing, projected Langevin algorithms).
  • WW is computed once per task (Sec. 2.2) and reused for every start state and goal.
  • How the energy hsh_s is obtained (Sec. 4, App. F.2, new). Every policy is kept frozen. The energy is the negative log-likelihood of the policy up to a constant, or a surrogate when the likelihood has no closed form (e.g., the denoising loss for diffusion policies); App. F.2 lists the construction for each evaluated policy.
  • Protocol. For instance, each entry uses 64 evaluation episodes (8 seeds) (Sec. 4); the reference optimum comes from a 200-start multi-start (Sec. 4.1); each edge barrier is evaluated at 50–200 points sampled along its segment (Sec. 3.2, App. D).

Tables 6–10 of the original submission are now Tables 7–11, Figs. 7 and 8 have exchanged numbers, and App. F.3 is now App. F.4.

Thanks!

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.