acceptodds
Under review as a conference paper at ICLR 2027

When Interaction Changes Controllability: Physics-informed Goal-Conditioned Reinforcement Learning for Manipulation

Abstract

Learning to reach arbitrary goals requires agents to infer a rich notion of reachability across state-goal pairs, which becomes increasingly difficult over long decision horizons and in systems with complex dynamics. Physics-informed Goal-Conditioned Reinforcement Learning (Pi-GCRL) addresses this problem primarily via Eikonal-based losses, achieving strong results in navigation while remaining less effective in robotic manipulation. In this paper, we show that this gap stems from the state-dependent controllability of manipulation, where object coordinates are not locally controllable throughout the task. We first establish that full-state Eikonal regularization becomes inconsistent whenever goal-relevant coordinates are locally uncontrollable. This insight leads to two complementary approaches: a loss that regularizes value gradients only along feasible directions, and a hierarchical decomposition that restricts physics-informed learning to a controllable low-level space. We evaluate both approaches on offline manipulation benchmarks and a physical robot, showing substantial improvements over naive full-state Eikonal regularization and demonstrating that accounting for local controllability improves the effectiveness of Pi-GCRL in robotic manipulation.

Primary area:
reinforcement learning
Keywords:
Goal-Conditioned Reinforcement Learning, Physics-Informed Neural Networks, Robotic Manipulation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_usgKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper shows why standard Eikonal-based physics-informed GCRL fails in robotic manipulation and establishes that physics-informed value learning must account for which state dimensions the robot can actually control at each interaction stage, rather than imposing uniform geometric constraints across the entire state space.

Critical strengths
  1. This paper provides a formal analysis explaining why full-state Eikonal regularization can conflict with locally feasible dynamics.
  2. They demonstrate encouraging improvements over baselines and has real world validation on physical UR5e robot.
Critical weaknesses
  1. The algorithmic novelty is limited as the transition-based HJB residual was previously proposed, while the hierarchical approach combines existing flow-based subgoal generation and value-learning techniques.
  2. Figure 4 shows that excluding object poses from the low-level state improves even non-physics-informed hierarchical methods. Then how much of the performance improvement comes specifically from the proposed formulation?
  3. Physical experiments involve only one object and a relatively simple pick-and-place task, with 20 evaluation episodes.
Questions for authors
  1. Since Equation 4 uses observed transitions rather than optimizing over feasible actions, under what conditions is the resulting regularizer consistent with the optimal HJB equation? How sensitive is performance to suboptimal offline trajectories?
  2. Could the authors provide uncertainty estimates for the physical evaluation and discuss whether the improvements remain consistent across different object configurations?
2 comments

We thank the reviewer for the thorough review and valuable feedback, which provided an opportunity to improve our work. We have carefully addressed your concerns and revised the manuscript accordingly, with changes highlighted in blue. We have also conducted additional experiments to further investigate the robustness of our proposed regularizer. We hope that our responses and revisions satisfactorily address your concerns. Thank you!

W1: Algorithmic novelties

The algorithmic novelty is limited as the transition-based HJB residual was previously proposed, while the hierarchical approach combines existing flow-based subgoal generation and value-learning techniques.

We acknowledge that our formulations build on existing components, as explicitly discussed in the manuscript (lines 285-290 and 304-307, 322–323). However, our main contribution is to explain why existing physics-informed regularization methods struggle in manipulation due to state-dependent controllability, and to derive principled solutions that overcome these limitations.

Two important distinctions from prior work are:

  1. Giammarino et al. (2026): This work primarily focused on navigation and introduced the HJB-based regularizer as an underperforming method compared to the Eikonal regularization. In contrast, our work shows that this relationship can be reversed in robotic manipulation and explains why this occurs. Specifically, Theorems 4.1 and 4.2 establish when and why Eikonal regularization becomes inconsistent with the local dynamics and when it remains a reasonable approximation. This provides new insights into the design and application of physics-informed GCRL algorithms beyond navigation.

  2. SHARSA (Park et al., 2026): While we adopt flow-based subgoal generation and critic-based selection, our specific hierarchical decomposition is motivated by our theoretical analysis. This leads to separating object-centric high-level reasoning from robot-centric low-level control, allowing physics-informed regularization to operate on a representation more closely aligned with the robot's controllable dynamics.

These distinctions, together with the components shared with prior work, are highlighted in Section 5 (Lines 281-284 and 293-309).

Q1 : HJB regularizer consistency and sensitivity to suboptimal trajectories

Since Equation 4 uses observed transitions rather than optimizing over feasible actions, under what conditions is the resulting regularizer consistent with the optimal HJB equation? How sensitive is performance to suboptimal offline trajectories?

We thank the reviewer for raising this point. The empirical HJB residual approximates the optimal HJB condition when observed transitions are sufficiently close to optimal goal-reaching directions. Indeed, the HJB equality holds exactly along optimal directions, while suboptimal transitions may introduce additional approximation errors.

As highlighted in lines 273-277, the HJB regularizer is computed using future goals sampled along the same trajectory as the observed transition. Furthermore, the OGBench Play datasets, used for the experiments in Fig. 3, are collected using expert policies, providing goal-directed trajectories that make the approximation reasonable for many local transitions [1].

[1] Park S, Frans K, Eysenbach B, Levine S. Ogbench: Benchmarking offline goal-conditioned rl. In International Conference on Learning Representations 2025 May 1 (Vol. 2025, pp. 94937-94982).

How sensitive is performance to suboptimal offline trajectories?

To answer this question, we conducted additional experiments comparing GCIVL and HJB-GCIVL on the OGBench Noisy manipulation datasets, which contain substantially more action noise and suboptimal trajectories than the Play datasets. These new experiments are summarized in Fig. 5 and lines 461–480 (both highlighted in blue in the revised manuscript).

The results show that the benefits of HJB regularization can persist under noisy trajectories, although its effectiveness varies across environments. In Triple, HJB-GCIVL loses its advantage over GCIVL, with both methods achieving comparable performance. On the other hand, in the more challenging Quadruple environment, HJB-GCIVL outperforms GCIVL (approximately 45% versus 20% final success rate). These findings suggest that while HJB regularization is sensitive to data quality, suboptimal trajectories do not undermine its benefits, highlighting opportunities for further improvements.

W2: hierarchical methods

Figure 4 shows that excluding object poses from the low-level state improves even non-physics-informed hierarchical methods. Then how much of the performance improvement comes specifically from the proposed formulation?

We thank the reviewer for highlighting this observation. We agree and already acknowledge (lines 451-454) that the controllability-aware decomposition improves both physics-informed and unregularized algorithms. This is itself an interesting finding, demonstrating that our theoretically motivated decomposition benefits GCRL beyond physics-informed regularization.

We also want to acknowledge that, in Figure 4, H-Flow-GCIVL and Pi-H-Flow-HJB-GCIVL achieve comparable performance, with a marginal difference in mean peak success rate (+2% percentage points) and very similar learning-curve performance. Thus, the additional benefit of HJB regularization is limited in this setting.

However, two other results demonstrate its effectiveness:

  1. Figure 3: Without hierarchy, HJB-GCIVL outperforms GCIVL, particularly in the challenging Triple and Quadruple environments.

  2. Real-world experiments (Section 6): Pi-H-Flow-HJB-GCIVL achieves higher success rates than H-Flow-GCIVL across all three evaluation thresholds.

These findings suggest that controllability-aware decomposition and HJB regularization offer complementary, but not necessarily additive, benefits. We have clarified this distinction in the revised manuscript (lines 455-461, highlighted in blue).

W3, Q2: real-world experiments

Physical experiments involve only one object and a relatively simple pick-and-place task, with 20 evaluation episodes.

Could the authors provide uncertainty estimates for the physical evaluation and discuss whether the improvements remain consistent across different object configurations?

We thank the reviewer for this question. We would like to emphasize that our real-world experiments already involve randomized initial object poses (lines 484-485), and we did not observe systematic performance differences across these configurations. Therefore, the reported success rates (lines 502-509) reflect evaluations across different initial configurations rather than a single fixed pick-and-place scenario.

Regarding statistical uncertainty, we acknowledge that 20 evaluation episodes per method provide limited ability to establish statistically significant performance differences at conventional thresholds (e.g., with a p-value less than 0.05). Nevertheless, we find it encouraging that Pi-H-Flow-HJB-GCIVL consistently achieves higher observed success rates across all three evaluation thresholds in real-world experiments, suggesting that the benefits observed in simulation can also translate to physical manipulation.

Finally, while we were unable to extend the physical experiments to multi-object rearrangement tasks, our simulated multi-object experiments (Fig. 3) suggest that similar benefits may extend to these settings.

We have revised the discussion of our real-world results accordingly (lines 508-512).

Reviewer_Y3TUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies why Eikonal-based regularization, which has been useful in physics-informed goal-conditioned reinforcement learning for navigation, can fail in manipulation. The paper shows that object coordinates can affect the long-term cost-to-go even when they cannot change locally, creating a mismatch between full-state Eikonal constraints and feasible dynamics. Based on this analysis, the authors intriduce HJB-GCIVL, which regularizes value gradients along observed transitions, and Pi-H-Flow, which separates high-level object-centric planning from low-level learning in a robot-centric representation. Experiments on a hybrid toy problem, OGBench manipulation tasks, and physical UR5e pick-and-place show improvements over full-state Eikonal regularization and unregularized baselines. The representation study also shows that excluding object poses from the low level benefits several hierarchical methods.

Critical strengths
  • The paper provide a clear theoretical explanation for why full-sate Eikonal regularization can fail in manipulation. Theorem 4.1 identifies a mismatch when object coordinates affect the long-term cost-to-go but cannot change locally before contact. The toy example illustrates how this mismatch distorts the learned value function. This provides a useful principle for designing physics-informed regularizers that respect feasible motion.

  • The empircial results are promising. Fig.3 shows substantial improvements over GCIVL on the harder Triple and Quadruple tasks, while Fig.4 shows that excluding object poses from the low-level representation helps several hierarchical methods. This is an useful finding beyond the proposed algorithm. The physical-robot experiment also provides encouraging evidence that the approach extends beyond simulation.

Critical weaknesses
  • The HJB residual may needs further justification for the training setup. Eq.4 encourages a unit decrease in cost-to-go along sampled transitions, but the offline trajectories may include pauses or detours. Therefore, sampling a future goal could not guarantee the optimal progress at every step. Training also uses discounted values, while the residual follows an undiscounted derivation. A clearer explanation of these differences would strengthen the connection between the theory and implemented loss.

  • The additional benefit of physics-informed regularization remains unclear. Fig.4 shows that removing object poses also helps with unregularized methods, and H-Flow-GCIVL performs similarly to its HJB-regularized version with “Joints + Contacts". This suggests that the simpler representation may explain much of the hierarchical improvement. A direct comparison with the hierarchy and representation held fixed, including uncertainty estimates, would help establish the additional benefit of the HJB term.

Questions for authors
  • Eq. 4 enforces an equality that holds only along optimal transitions. How is Eq.4 justified for feasible but suboptimal offline transitions, and how is it reconciled with the discounted value objective? Could you clarify which goal samples receive this regularizer and test its sensitivity to pauses or detours in the data?
  • What variables are included in "Joints+Contacts" , contact flags, gripper–object relative quantities, gripper width? If it includes contact flags or object-relative quantities, how do these fit the controllability assumptions? With this representation fixed, does physics-informed regularization consistently improve over the same hierarchical policy without regularization at final checkpoints?
4 comments

We thank the reviewer for the thorough review and valuable feedback, which provided an opportunity to improve our work. We have carefully addressed your concerns and revised the manuscript accordingly, with changes highlighted in blue. We have also conducted additional experiments to further investigate the robustness of our proposed regularizer.

We hope that our responses and revisions satisfactorily address your concerns and, if so, we would kindly ask you to consider raising your rating.

W1 and Q1: Justification of the HJB regularizer

How is Eq. 4 justified for feasible but suboptimal offline transitions? Could you clarify which goal samples receive this regularizer and test its sensitivity to pauses or detours in the data?

We thank the reviewer for pointing this out. We agree that the HJB equality is exact only along optimal transitions, and suboptimal trajectories may introduce additional approximation errors.

As highlighted in lines 273-277, the HJB regularizer is computed using future goals sampled along the same trajectory as the observed transition. Furthermore, the OGBench play datasets, used for the experiments in Fig.3, are collected using expert policies, providing goal-directed trajectories that make the approximation reasonable for many local transitions.

However, an important question is whether the approximation introduced by HJB regularization adversely affects performance when the offline data become more suboptimal. To investigate this, we have conducted additional experiments comparing GCIVL and HJB-GCIVL on the OGBench Noisy manipulation datasets, which contain substantially more action noise and suboptimal trajectories than the Play datasets. The results for these new experiments are summarized in Fig. 5 and lines 461-480 (both highlighted in blue in the new version of the manuscript).

The results show that HJB regularization remains beneficial under noisy trajectories, although its effectiveness varies across environments. In Triple both methods achieve comparable performance; while in the more challenging Quadruple, HJB-GCIVL outperforms GCIVL (approximately 45% versus 20% final success rate). We believe that these findings demonstrate that suboptimal trajectories do not necessarily undermine the benefits of the HJB regularizer, while highlighting opportunities for further improvements.

How is Eq. 4 reconciled with the discounted value objective?

We would like to clarify that Eq. (4) is an auxiliary inductive bias, not a replacement for the discounted Bellman objective. As in previous physics-informed GCRL formulations [1, 2], the regularizer encourages a physically meaningful value landscape, while the original GCIVL objective remains responsible for discounted value learning.

Furthermore, the first-order directional approximation employed in Eq. (4) does not generally coincide exactly with the discrete-time Bellman residual at finite sampling intervals, even without discounting:

d(s’,g)−d(s,g)=∇sd(s,g)⊤(s’−s)+O(∣∣s’−s∣∣2)d(s’,g) - d(s, g) = \nabla_s d(s, g)^{\top}(s’-s) + O(||s’ - s||^2)

As a result, Eq. (4) is not intended to enforce exact agreement between the two formulations, but rather to provide additional structure to the learned value function, while the original discounted GCIVL objective remains responsible for value learning.

We have clarified this distinction and the approximate nature of the regularizer in the revised manuscript (Lines 289-291)

[1] Giammarino V, Ni R, Qureshi A. Physics-informed value learner for offline goal-conditioned reinforcement learning. Advances in Neural Information Processing Systems. 2026a Apr 23;38:137769-96.

[2] Giammarino V, Qureshi AH. Goal reaching with eikonal-constrained hierarchical quasimetric reinforcement learning. InInternational Conference on Learning Representations 2026 Apr 20 (Vol. 2026b, pp. 59145-59182).

W2: Contribution of physics-informed regularization

The contribution of physics-informed regularization remains unclear.

We respectfully disagree with this comment. Fig. 3 clearly demonstrates the benefits of physics-informed regularization, with HJB-GCIVL substantially outperforming GCIVL, particularly in the challenging Triple and Quadruple environments. Furthermore, our real-world experiments provide additional evidence of its effectiveness.

More importantly, prior literature has already established the benefits of physics-informed regularization for goal-conditioned value learning. The main contribution of our paper is to explain why these approaches struggle in robotic manipulation and derive principled solutions that enable their successful application to these tasks. We believe our theoretical analysis and experimental results clearly support this contribution.

Q2: Controllable representation and final checkpoint performance

What variables are included in "Joints+Contacts", contact flags, gripper–object relative quantities, gripper width? If it includes contact flags or object-relative quantities, how do these fit the controllability assumptions?

The "Joints+Contacts" representation includes robot proprioception, joint states, end-effector pose, gripper opening, and continuous-valued gripper-contact feedback. No object-state coordinates, object-relative quantities, or discrete contact flags are included.

Importantly, the gripper can close on itself independently of the presence of an object. Therefore, its contact feedback can be influenced directly through robot actuation, without requiring interaction with an external object.

With this representation fixed, does physics-informed regularization consistently improve over the same hierarchical policy without regularization at final checkpoints?

We thank the reviewer for highlighting this observation. We agree that the controllability-aware decomposition improves both physics-informed and unregularized hierarchical algorithms. This is itself an interesting finding, demonstrating that our theoretically motivated decomposition benefits GCRL beyond physics-informed regularization.

In Figure 4, H-Flow-GCIVL and Pi-H-Flow-HJB-GCIVL achieve comparable performance, with a marginal difference in mean peak success rate (+2 percentage points) and similar learning-curve performance. Thus, the additional benefit of HJB regularization is limited in this setting.

These results, together with Figure 3 and our real-world experiments, suggest that controllability-aware decomposition and HJB regularization offer complementary, but not necessarily additive, benefits. We have clarified this distinction in the revised manuscript (lines 454-460, highlighted in blue).

We hope these clarifications and additional analyses address your concerns and would appreciate your reconsideration of the overall assessment.

W2: Contribution of physics-informed regularization.

Thank you for clarifying the scope of this contribution. Your explanation helps address my concern. The flat policy results supports the value of HJB regularization, while the representation study supports the controllability-aware decomposition. I agree that these are meaningful contribution even if the benefits are not consistently additive. I no longer consider that as a major concern for the overall contribution.

Q2: Thank you for the additional explanation. It addresses my concern more directly and clarifies the respective contributions of the representation and the regularizer.

I would like to raise my overall rating. Thanks for addressing the concerns.

We thank the reviewer for the constructive and insightful discussion. We sincerely appreciate the time and effort you dedicated to reviewing our work and considering our responses and revisions. We are grateful for your positive reassessment and for recognizing the improvements made to the manuscript. Thank you!

Reviewer_pJFPOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies an interesting topic about why does physics-informed goal-conditioned RL with Eikonal-based value regularization work well in navigation but poorly in robotic manipulation? The authors argue that this gap stems from state-dependent controllability. The paper presents theoretical results showing that the full-state Eikonal constraint cannot be simultaneously satisfied with the local HJB equation when goal-relevant coordinates are locally uncontrollable, while explaining why the Eikonal constraint remains a reasonable approximation when the dynamics are merely anisotropic. Based on this analysis, the paper proposes two solutions: (i) HJB-GCIVL, which replaces the Eikonal residual with a transition-based HJB residual, and (ii) Pi-H-Flow, a hierarchical method where a flow-based high-level policy generates subgoals and the physics-informed loss is applied only to the controllable low-level policy. Experimental results in simulation and the real world demonstrate the effectiveness of the proposed methods.

Critical strengths
  1. The research question is interesting and practical. Analyzing why Eikonal-based Pi-GCRL succeeds in navigation but struggles in robotic manipulation provides valuable insights for the robotics community.
  2. The theoretical analysis provides a convincing explanation for why Eikonal regularization fails in manipulation: it inappropriately penalizes value gradients along object coordinates that are not locally controllable.
  3. The proposed methods, HJB-GCIVL and Pi-H-Flow, are well-motivated and clearly outperform full-state Eikonal regularization and standard GCIVL on the OGBench manipulation tasks and real-robot experiments.
Critical weaknesses
  1. The main concern is that the proposed methods are largely combinations of existing components. As the paper itself notes in Section 5, the transition-based HJB residual in Eq. (4) was already introduced in prior work (Giammarino et al., 2026). Similarly, the hierarchical design using flow-based subgoal generation and critic-based subgoal selection closely follows SHARSA (Park et al., 2026). While I appreciate the focus on adapting Pi-GCRL to manipulation, the authors should more clearly highlight what is algorithmically unique in their formulation.
  2. In Figure 4, the state-representation ablation shows that restricting the low-level state to controllable robot coordinates 'Joints+Contacts' substantially improves non-physics-informed hierarchical baselines as well, meaning this gain is not specific to the physics-informed loss. This suggests that the primary performance boost in the hierarchical setting comes from removing object coordinates from the low-level state space and using a flow-based hierarchy, rather than from the physics-informed HJB regularizer itself.
Questions for authors
  1. In Figure 4, unregularized baseline methods using the 'Joints+Contacts' representation perform very similarly to 'Pi-H-Flow-HJB-GCIVL'. Could you clarify how much marginal improvement the HJB loss actually provides once the low-level state is restricted to controllable robot coordinates?
  2. What is the main algorithmic difference between HJB-GCIVL and the transition-based HJB regularizer in Giammarino et al. (2026), beyond the motivation from state-dependent controllability?
3 comments

We thank the reviewer for the positive feedback and constructive comments. We address your concerns below and have updated the manuscript accordingly, with changes highlighted in blue. We hope that our responses and revisions satisfactorily address your concerns.

W1 and Q2: Algorithmic novelty and relationship to prior work

The main concern is that the proposed methods are largely combinations of existing components.

The authors should more clearly highlight what is algorithmically unique in their formulation.

What is the main difference between HJB-GCIVL and the HJB regularizer in Giammarino et al. (2026), beyond the motivation from state-dependent controllability?

We acknowledge that our formulations build on existing components, as explicitly discussed in the manuscript (lines 285-290 and 304-307, 322-323). However, our main contribution is to explain why existing physics-informed regularization methods struggle in manipulation due to state-dependent controllability, and to derive principled solutions that overcome these limitations.

Two important distinctions from prior work are:

  1. Giammarino et al. (2026): This work primarily focused on navigation and introduced the HJB-based regularizer as an underperforming method compared to the Eikonal regularization. In contrast, our work shows that this relationship can be reversed in robotic manipulation and explains why this occurs. Specifically, Theorems 4.1 and 4.2 establish when and why Eikonal regularization becomes inconsistent with the local dynamics and when it remains a reasonable approximation. This provides new insights into the design and application of physics-informed GCRL algorithms beyond navigation.

  2. SHARSA (Park et al., 2026): While we adopt flow-based subgoal generation and critic-based selection, our hierarchical decomposition is motivated by our theoretical analysis. This leads to separating object-centric high-level reasoning from robot-centric low-level control, allowing physics-informed regularization to operate on a representation more closely aligned with the robot's controllable dynamics.

These distinctions, together with the components shared with prior work, are highlighted in Section 5 (Lines 281-284 and 293-309).

W2 and Q1: Contribution of the HJB regularizer

In Figure 4, the state-representation ablation shows that restricting the low-level state to controllable robot coordinates 'Joints+Contacts' substantially improves non-physics-informed hierarchical baselines as well, meaning this gain is not specific to the physics-informed loss.

Could you clarify how much marginal improvement the HJB loss actually provides once the low-level state is restricted to controllable robot coordinates?

We thank the reviewer for highlighting this observation. As already acknowledged in the manuscript (lines 450-455), the controllability-aware decomposition improves both physics-informed and unregularized algorithms. This is itself an interesting finding, demonstrating that our theoretically motivated decomposition benefits GCRL beyond physics-informed regularization.

We also want to acknowledge that, in Figure 4, H-Flow-GCIVL and Pi-H-Flow-HJB-GCIVL achieve comparable performance, with a marginal difference in mean peak success rate (+2% percentage points) and very similar learning-curve performance. Thus, the additional benefit of HJB regularization is limited in this setting.

However, two other results demonstrate its effectiveness:

  1. Figure 3: Without hierarchy, HJB-GCIVL outperforms GCIVL, particularly in the challenging Triple and Quadruple environments.

  2. Real-world experiments (Section 6): Pi-H-Flow-HJB-GCIVL achieves higher success rates than H-Flow-GCIVL across all three evaluation thresholds.

These findings suggest that controllability-aware decomposition and HJB regularization offer complementary, but not necessarily additive, benefits. We have clarified this distinction in the revised manuscript (lines 454-460, highlighted in blue).

I thank the authors for their detailed responset. The authors have addressed most of my questions. I maintain a positive evaluation and support acceptance of the paper (Rating: 3 / Weak acceptance).

We thank the reviewer for the constructive discussion and for taking the time to carefully consider our responses and revisions. We sincerely appreciate your valuable feedback and your continued support of our work.

Comments on the paper

1 comment

Dear AC and reviewers,

We would like to thank you all for your time and consideration. As the discussion period is coming to an end, we would like to briefly summarize how we have addressed the reviewers' feedback in the revised manuscript. All changes are highlighted in blue.

  1. Algorithmic novelty (reviewers pJFP and usgK): We have acknowledged that our formulations build on existing components and further clarified their differences from prior work, emphasizing the contributions motivated by our theoretical analysis (Section 5).

  2. Effect of the HJB regularizer on performance (reviewers pJFP, Y3TU, and usgK): We have further clarified the contribution of physics-informed regularization, referring to the results in Fig. 3 and our real-world experiments (Section 6).

  3. Sensitivity of the HJB regularizer to suboptimal trajectories (reviewers Y3TU and usgK): To address these concerns, we conducted additional experiments using noisy offline datasets. The results are presented in Fig. 5 and discussed in the Noisy trajectory ablation paragraph of Section 6.

  4. Consistency of the HJB regularizer (reviewers Y3TU and usgK): We clarified that the empirical HJB regularizer is evaluated along observed transitions toward future goals sampled from the same trajectory. We also emphasized that it serves as an auxiliary inductive bias rather than an exact optimality constraint, complementing the original discounted Bellman objective (lines 274–291).

  5. HJB regularization and hierarchical decomposition (reviewers Y3TU and usgK): We have further discussed the complementary, but not necessarily additive, benefits of these two components (lines 456-462).

  6. Controllable state representation (reviewer Y3TU): We have further clarified the variables included in the Joints+Contacts representation and their relationship to our controllability assumptions (Section 6, lines 447-451).

  7. Real-world experiments (reviewer usgK): We clarified that our evaluation considers randomized initial object configurations and explicitly discussed the statistical limitations associated with the limited number of evaluation episodes (lines 509–512).

We sincerely appreciate the constructive feedback and discussions, which have helped us improve the manuscript.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.