acceptodds
Under review as a conference paper at ICLR 2027

Geometry-Informed Planning

Abstract

We introduce Geometry-Informed Planning (GIP), a novel planning framework for offline goal-conditioned reinforcement learning (GCRL). GIP views the state space as a smooth manifold, where temporal distance is defined by optimal goal-reaching time. Optimal trajectories then correspond to geodesics, which can be represented by initial tangent vectors through the exponential map. Guided by this geometric structure, GIP represents an entire path with a single latent path vector and learns to sample such vectors for arbitrary state-goal pairs to infer geodesics. Our geometric formulation enables theoretical recovery of optimal goal-reaching paths under suitable assumptions, while joint learning of temporal distances with generated paths discourages invalid shortcuts that leave the state manifold. GIP enables efficient planning with a fixed-dimensional latent vector independent of the planning horizon, while training without temporal difference learning supports its scalability. We empirically validate these properties on OGBench, where GIP outperforms model-free and model-based baselines, particularly on long-horizon tasks, while requiring substantially lower planning cost.

Primary area:
reinforcement learning
Keywords:
offline goal-conditioned reinforcement learning, planning, geometry, geodesic

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xqMfOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Geometry-Informed Planning (GIP) for offline goal-conditioned reinforcement learning. The central idea is to view the state space as a smooth manifold equipped with a temporal quasimetric, such that optimal goal-reaching trajectories correspond to temporal geodesics. Motivated by the exponential-map representation of geodesics, GIP represents an entire goal-reaching path using a single fixed-dimensional latent path vector. The method jointly learns a temporal distance function, a path map, a latent path-vector selector, and a local controller. The distance-learning objective additionally penalizes generated paths through unsupported regions, with the aim of discouraging invalid shortcuts.

The paper provides theoretical results characterizing zero geodesic cost, asymptotic path recovery under idealized assumptions, and an out-of-support penalization mechanism. Empirically, GIP is evaluated on OGBench across locomotion and manipulation tasks, where it achieves strong average performance, particularly on long-horizon humanoidmaze tasks. The paper also reports substantially lower inference cost than test-time search methods at comparable performance, together with scalability experiments, qualitative path visualizations, and several ablations.

Critical strengths
  1. Novel and practically meaningful formulation of long-horizon planning. The strongest contribution is the idea of amortizing an entire goal-reaching plan into a single fixed-dimensional latent path vector. Compared with explicit graph search or horizon-length trajectory generation, this provides a conceptually different planning interface whose inference cost need not scale directly with the planning horizon. This is both technically interesting and potentially important for real-time long-horizon control.
  2. Strong empirical results, particularly on challenging long-horizon tasks. GIP achieves strong performance on OGBench and is particularly competitive on the high-dimensional humanoidmaze tasks. The reported inference-efficiency gains are also substantial: on the humanoidmaze-giant settings, the paper reports 25.4× lower inference FLOPs than HIQL+TTGS at comparable performance. Together with the larger-scale colossal experiments, these results provide meaningful empirical support for the proposed amortized-planning formulation.
Critical weaknesses
  1. The connection between the strongest theoretical guarantees and the implemented algorithm could be made more precise. The path-recovery and distance-recovery results rely on idealized assumptions, including access to the true temporal distance, on-manifold paths, realizability, and global optimization, while the practical method does not directly guarantee these conditions. In addition, the learned encoder may not preserve the identity property required by some quasimetric arguments.
  2. The experimental study does not yet fully isolate the contribution of the core geometric components. While the paper includes ablations on latent dimension and state representation, it lacks targeted ablations for key ingredients such as the constant-speed geodesic objective, out-of-support penalization, joint distance learning, and the selector regularization. This makes it difficult to determine which parts of the proposed geometry-informed formulation are responsible for the observed gains.
Questions for authors
  1. The theoretical analysis uses the identity property d(s,g)=0⇒s=gd(s,g)=0\Rightarrow s=g, whereas the practical distance is obtained by applying an MRN quasimetric to learned representations ϕ(s)\phi(s) and ϕ(g)\phi(g). Is there any mechanism that guarantees or approximately enforces injectivity of ϕ\phi? If not, how should the theoretical results be interpreted when distinct states may share the same learned representation?
  2. The out-of-support analysis assumes a strictly increasing path-length objective, while the implementation uses a clipped surrogate that stops applying additional spreading pressure after reaching a threshold proportional to d(s,g)d(s,g). Does an analogous OOS exclusion argument hold for the actual surrogate? Also, since the practical method uses only K=4K=4 intermediate points, do you observe paths crossing unsupported regions between sampled anchors, and can you quantify this effect?
  3. Could the authors provide matched ablations removing, individually, (i) the constant-speed geodesic loss, (ii) the generated-path/OOS term, (iii) joint distance learning, and (iv) the proximity/multimodal selector regularization? These results would materially strengthen my assessment by showing which components are responsible for the empirical gains.
  4. Can the authors provide a more controlled quantitative comparison with the strongest planning/trajectory-modeling baselines, such as CompDiffuser?
5 comments
AuthorsOpenReview ↗

Response to the review (1/3)

We thank the reviewer xqMf for the detailed feedback on the practical implementations and experiments, which helped us clarify our implementation details. Below, we address the reviewer's concerns and questions.

  • Implementation of the endpoint boundary condition without MRN distance.

As you pointed out, the MRN architecture does not satisfy the identity property. Since we did not include the MRN distance formulation in the main text, we expressed the endpoint boundary term in Eq. (7) (line 238) as $ \rho(d^{*}(x_1,g)) $ for simplicity of presentation. In our implementation, however, we adopt a practical endpoint residual, $ \lVert \varphi_\phi(x_1) - \varphi_\phi(g) \rVert_2 + \lambda_\mathrm{state} \lVert x_1-g \rVert_2 $, which combines the norm between the endpoint and goal MRN representations with the norm between their raw states. We do not impose any injectivity on the encoder.

A strictly positive λstate\lambda_{\mathrm{state}} ensures that a zero endpoint residual implies equality of the endpoint and goal states. We also conducted an ablation study on λstate\lambda_{\mathrm{state}}.

Table 1. Ablation of λstate\lambda_{\mathrm{state}} in the endpoint residual (4 seeds).

Environmentλstate=0\lambda_{\mathrm{state}} = 0λstate=0.2\lambda_{\mathrm{state}} = 0.2
humanoidmaze-large-navigate77 ± 176 ± 2
humanoidmaze-giant-navigate81 ± 381 ± 2

Interestingly, adding the raw state term did not yield a consistent improvement in performance. We therefore use only the representation norm in our final implementation. This is an empirical design choice: the final endpoint residual guarantees equality of representations, but not necessarily equality of the original states.

We had omitted these implementation details from the original manuscript, and we thank the reviewer for pointing out this omission. We now added a detailed description of the endpoint residual implementation, along with the ablation results, in Appendices D.4 and F.4 (lines 1332 and 1814).

  • Does an analogous OOS exclusion argument hold for the actual surrogate?

An analogous argument holds under the assumptions of the original OOS analysis. It is important to note that τmin>1\tau_\mathrm{min} > 1. Zero expected temporal geodesic cost requires sampled path length ℓϕ=∑k=0K−1dϕ(xk,xk+1)\ell_\phi=\sum_{k=0}^{K-1}d_\phi(x_k, x_{k+1}) to be exactly dϕ(s,g)d_\phi(s,g), whereas clipping occurs at the larger value τmin∗dϕ(s,g)\tau_\mathrm{min} * d_\phi(s,g).

With generated points and targets fixed, the distance modification in the original proof increases the length of a path containing sampled OOS points while preserving dϕ(s,g)d_\phi(s,g). Starting from a zero-cost path, this both reduces the clipped distance loss and makes the path length greater than dϕ(s,g)d_\phi(s,g), ruling out zero geodesic cost under the modified distance. The expected loss decreases strictly when such OOS points are sampled with positive probability. Clipping stops further spreading only after the path already violates the zero-cost condition, so the clipped loss still eliminates possibility of zero-cost OOS paths.

  • K intermediate points.

First, we would like to clarify a detail that we missed in the original manuscript. In Eq. (11), the interior times t1,t2,…,tK−1t_1, t_2, \ldots, t_{K-1} are sampled from Unif[0,1]\mathrm{Unif}[0,1] at each training iteration. We noticed that this was not explicitly stated and have clarified it in the revised manuscript (line 283):

... 0=t0<⋯<tK=10=t_0<\cdots<t_K=1 are the discretization times, with interior times resampled from Unif[0,1]\mathrm{Unif}[0,1] at each iteration, ...

If the time points were sampled only once with K=4K=4, a path could cross an unsupported region between the sampled points. We observe such crossings, particularly early in training. However, since the interior times are independently resampled at every training iteration, repeated sampling allows the entire interval [0,1][0,1] to be probabilistically covered over the course of training.

Since the time points are resampled at each iteration, it is difficult to directly quantify violations as a function of KK. Instead, we conducted an ablation study on KK.

Table 2. Ablation of KK (4 seeds).

EnvironmentK=2K = 2K=4K = 4K=8K = 8
humanoidmaze-large-navigate68 ± 276 ± 277 ± 4
humanoidmaze-giant-navigate75 ± 381 ± 183 ± 3

The results show that increasing KK to 8 yields a modest improvement in performance, but naturally increases training time as well. Considering this tradeoff between efficiency and performance, we chose K=4K=4 as a reasonable balance.

AuthorsOpenReview ↗

Response to the review (2/3)

  • Ablations of removing (i) the constant-speed geodesic loss, (ii) the generated-path/OOS term, (iii) joint distance learning, and (iv) the proximity/multimodal selector regularization.

We first provide ablation results obtained by individually removing the constant speed term, path length maximization term for OOS penalization, and proximity regularization term from GIP training.

Table 3. Ablation of geometric objectives (4 seeds).

EnvironmentFullw/o constant speedw/o path-length maximizationw/o proximity regularization
humanoidmaze-large-navigate76.6 ± 0.511.1 ± 5.267.7 ± 6.076.7 ± 2.2
humanoidmaze-giant-navigate81.5 ± 2.81.8 ± 1.536.6 ± 10.981.7 ± 1.3

The constant speed condition is essential to temporal geodesics, as established in the definition of geodesics in Eq. (3) and Proposition 4.2. Without this term, the path model fails to learn temporal geodesics properly, resulting in a severe performance drop. Without path length maximization, distances along paths passing through regions outside the state manifold can be underestimated, causing such paths to be incorrectly regarded as valid and leading to infeasible plans (e.g., paths crossing walls in maze environments).

Although proximity regularization has little effect on OGBench performance, we observed in the cylinder example that removing it causes premature mode collapse, limiting early exploration and preventing the overall geodesic cost landscape from being learned properly. We have added these results in Figure 8 of Appendix F.1. The ablation results are presented in Figure 10 of the revised manuscript, with further discussion in Appendix F.4 (line 1875).

To examine the effect of joint distance learning, we first jointly trained the distance model and temporal geodesic for 200k steps, which is sufficient for distance convergence, and then froze it while continuing temporal geodesic learning. The results are shown below.

Table 4. Ablation of joint distance learning (4 seeds).

EnvironmentJoint distanceFrozen distance
humanoidmaze-large-navigate77.1 ± 1.565.2 ± 8.7
humanoidmaze-giant-navigate81.1 ± 2.665.2 ± 6.0

Without joint distance learning, distance regression and OOS penalization using the intermediate points generated by temporal geodesics cannot be performed effectively, resulting in reduced performance. We have added these results as Figure 11 and included the corresponding discussion in Appendix F.4 (line 1886).

AuthorsOpenReview ↗

Response to the review (3/3)

  • Comparison to CompDiffuser.

We note that the main AntMaze and HumanoidMaze experiments of CompDiffuser use xy-coordinate planning, whereas GIP generates full-state paths. Moreover, for HumanoidMaze medium, large, and giant, CompDiffuser uses maximum evaluation episode lengths of 5,000, 5,000, and 8,000 steps, respectively, compared with 2,000, 2,000, and 4,000 under the standard OGBench protocol followed by GIP. These differences prevent a fully controlled comparison of the reported scores. Furthermore, training and evaluating CompDiffuser in high-dimensional settings such as HumanoidMaze would require approximately one month with our available computational resources, which was not feasible during the rebuttal period.

Nevertheless, we compared GIP with the published results of CompDiffuser.

Table 5. Comparison of GIP (full-state planning) with CompDiffuser (xy-coordinate planning). CompDiffuser results are taken from Table 2 of the original paper (5 seeds); GIP results are taken from our full OGBench results (8 seeds).

EnvironmentCompDiffuserGIP
antmaze-medium-stitch96 ± 286 ± 6
antmaze-large-stitch86 ± 278 ± 3
antmaze-giant-stitch65 ± 377 ± 6
antmaze-medium-explore81 ± 287 ± 4
antmaze-large-explore27 ± 171 ± 5
humanoidmaze-medium-stitch91 ± 191 ± 2
humanoidmaze-large-stitch72 ± 367 ± 2
humanoidmaze-giant-stitch67 ± 481 ± 5
Average73.179.8

The results in Table 5 suggest that GIP achieves comparable goal-reaching success while generating full-state paths and operating under the shorter standard evaluation limits, which are more restrictive than those used by CompDiffuser.

CompDiffuser also reports results on antmaze-stitch tasks using full 29-dimensional state planning. Table 6 compares these published full-state results with GIP.

Table 6. Comparison of GIP (full-state planning) with CompDiffuser (full-state planning). CompDiffuser results are taken from Table 4 of the original paper (5 seeds); GIP results are taken from our full OGBench results (8 seeds).

EnvironmentCompDiffuser (29D)GIP (29D)
antmaze-medium-stitch97 ± 286 ± 6
antmaze-large-stitch66 ± 578 ± 3
antmaze-giant-stitch28 ± 477 ± 6
Average63.780.3

Although GIP achieves a lower success rate than CompDiffuser on the medium environment, CompDiffuser exhibits a severe performance drop as the task horizon increases. In contrast, GIP shows better scalability with respect to the horizon, as its latent path vector has a fixed dimension independent of the planning horizon. Consequently, the performance gap becomes larger as the environment size increases.

Another notable difference is the per-step inference FLOPs. Averaging the inference FLOPs per environment step across the available OGBench tasks in Table 1 of our manuscript, CompDiffuser requires approximately 5,000 times more computation than GIP. This demonstrates that GIP's implicit planning achieves remarkable computational efficiency while retaining the benefits of planning for long-horizon tasks.

We did not include CompDiffuser in the main result tables (Tables 1 and 5 of the manuscript) to ensure a fair comparison with other baselines, since CompDiffuser primarily plans in xy coordinates and its reported results were evaluated under more relaxed episode limits. However, we have added CompDiffuser to Figure 1 to allow an overall comparison of efficiency and performance.

Reviewer_d31rOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper views the goal-conditioned RL problem as a planning problem of finding the shortest path using the number of actions required to reach from one state to another as quasi-distance. It proposes representing the plans as vectors of the fixed dimension. The authors jointly train a distance quasimetric that predicts the temporal distance between 2 states, a latent path vector selector that proposes a path vector from the state to a goal, and a decoder that given a start state, a path vector, and of a particular length decodes a state that is length away from the start on the path. After that they extract a policy conditioned on a current state and a path vector using difference between predicted distances to the goal from the current and next state minus one as advantage. At inference the selector proposes a path vector, then a subgoal is selected and decoded along this path, selector is used to determine the path vector to the subgoal and the policy produces an action given current state and subgoal path vector. The main contributions are the algorithm of training and using the latent path vector for planning as well as its theoretical study and empirical validation, showing that the algorithm has a competitive performance at lower inference costs compared to prior works. The validation uses standard tasks in OGBench as well as a bigger task designed to probe the scalability.

Critical strengths
  • Viewing optimal trajectories as geodesics and jointly training the path selector with quasi-distance to push apart points on unsupported paths proposed by the selector is a neat and well-motivated idea. It uses the geometric structure to perform planning efficiently without diffusion, autoregression, or test-time search. This makes inference cost independent of the planning horizon, which is an important property in some practical applications.
  • The experiments are showing good empirical performance of the proposed method and visualizations of the mazes and toy cylinder experiments are informative for understanding the paper and validating the mechanism. The long horizon and larger dataset size experiments are a good addition to the standard environments. The strong performance on humanoidmaze stitch and colossal variants shows that the method is capable of composing the dataset trajectories and producing long-horizon plans.
Critical weaknesses
  • The paper is missing modern baselines with no inference time planning. Eik-HiQRL (Giammarino & Qureshi, 2026) is another hierarchical quasimetric RL method and should be included in comparisons, so is MQE (Zheng et al., 2026). It also raises the question of gains attribution: Eik-HiQRL reports 83% success rate on humanoidmaze-giant-navigate and GIP reports 46.6% without encoder and the gap is not likely to be explained by the mismatch in evaluation settings. Unless GIP outperforms baselines under equal configuration (raw observations), the addition of the encoder to the inputs is a major component of the method and should be clearly described as one. Adding a matched encoder representation as input to the components of MQE would help to disentangle if the encoder gains are specific to GIP. With more baselines the claim "10.8×10.8\times higher success than the closest-compute baseline" does not hold.
  • Experimental scope: the majority of experiments is performed on state-based mazes with manipulation results limited to two environments, on one of which GIP underperforms the baselines. The long-horizon manipulation environments (cube-double and triple, puzzle) are not covered. The paper should either include more tasks or be upfront about the scope of applicability, the statement "where GIP outperforms model-free and model-based baselines, particularly on long-horizon tasks" is too broad and is only established for maze navigation.
Questions for authors
  • Can you add the baselines to your paper, and update the tables, the abstract claim, and Figure 1?
  • Can you publish the performance with raw observations for other tasks? Can you test if including them in only some of your networks (path map, selector, policy) explains the majority of the gain? This would help to distinguish if the encoder is something that helps your method specifically or is a general augmentation.
  • Are the results available for other manipulation tasks? If no, will you update the text?
4 comments
AuthorsOpenReview ↗

Response to the review (1/2)

We thank Reviewer d31r for their thoughtful feedback on our experiments and presentation, which has helped us improve the overall completeness of our work. Below, we address the reviewer's concerns and questions.

  • Updated baselines and corresponding statements.

We have added Eik-HiQRL and MQE as new baselines in Figure 1 and Tables 1 and 5. In our effort to highlight the computational efficiency of GIP's implicit planning, we focused primarily on planning baselines and, in doing so, inadvertently overlooked recent quasimetric learning baselines. We appreciate the reviewer for pointing out this oversight and suggesting these important baselines. Accordingly, we have updated Figure 1, its caption, the Introduction (line 77), baseline descriptions (line 420), and discussion of experimental results (line 450).

  • Additional manipulation experiments and revised statements.

We retained the benchmark task sets in Tables 1 and 5 to align with those reported for Eik-HiQRL, but also conducted additional experiments on long-horizon manipulation tasks. GIP partly inherits distance learning principle of QRL and, like QRL and its variants, exhibits severe failures on these tasks. We report these limitations in Section 7 (line 507) and Appendix F.3. We also agree that the original abstract was overly broad and have revised its claims accordingly as below.

... We empirically validate these properties on 13 OGBench environments, where GIP achieves the highest average success rate with substantially lower inference cost than competitive planning baselines.

AuthorsOpenReview ↗

Response to the review (2/2)

  • Role of the state augmentation in GIP.

We agree that state augmentation plays an important role in GIP, and we would like to clarify that its benefit is specific to GIP's direction-based control rather than a general improvement applicable to other methods. We first compared GIP with a variant that removes φϕ\varphi_\phi augmentation from all modules across the OGBench tasks.

Table 1. Ablation of φϕ\varphi_\phi state augmentation across OGBench tasks (4 seeds).

EnvironmentGIPw/o φϕ\varphi_\phi
antmaze-large-navigate91 ± 389 ± 2
antmaze-giant-navigate74 ± 427 ± 8
antmaze-large-stitch78 ± 374 ± 4
antmaze-giant-stitch77 ± 622 ± 7
antmaze-large-explore71 ± 561 ± 8
humanoidmaze-medium-navigate92 ± 187 ± 2
humanoidmaze-large-navigate76 ± 262 ± 2
humanoidmaze-giant-navigate81 ± 147 ± 6
humanoidmaze-medium-stitch91 ± 280 ± 6
humanoidmaze-large-stitch67 ± 248 ± 5
humanoidmaze-giant-stitch81 ± 535 ± 9
scene-play65 ± 1270 ± 4
cube-single-play32 ± 518 ± 3
Average75.155.4

Removing augmentation reduces the average success rate from 75.1 to 55.4. We further examined which GIP modules benefit from augmentation by applying it to different combinations of the path map, selector, and policy.

Table 2. Ablation of φϕ\varphi_\phi augmentation across GIP modules (4 seeds). Columns indicate which modules receive augmentation.

EnvironmentEξ,σψ,πθE_\xi, \sigma_\psi, \pi_\thetaσψ,πθ\sigma_\psi, \pi_\thetaEξ,πθE_\xi, \pi_\thetaEξ,σψE_\xi, \sigma_\psiπθ\pi_\thetaσψ\sigma_\psiEξE_\xi∅\varnothing
humanoidmaze-large-navigate76.6 ± 0.576.7 ± 2.477.1 ± 2.225.4 ± 3.876.1 ± 2.262.1 ± 2.143.2 ± 3.662.0 ± 1.7
humanoidmaze-giant-navigate81.5 ± 2.881.1 ± 2.678.1 ± 2.110.9 ± 2.572.1 ± 12.950.5 ± 13.136.4 ± 5.046.6 ± 6.2

Examining these combinations, we find that state augmentation plays a particularly important role in the policy, and removing it can lead to a substantial performance drop. By contrast, removing it from the path map or selector alone has little effect. These results suggest that GIP can learn temporal geodesics without augmentation, but that augmentation is particularly valuable for learning and executing direction-based local control.

To test whether this is a generally useful augmentation, we applied the same φϕ\varphi_\phi features to the policies of QRL, Eik-HiQRL, and MQE, which use quasimetric representations such as MRN or IQE for value learning.

Table 3. Ablations of φϕ\varphi_\phi augmentation on other baselines (4 seeds).

EnvironmentQRLQRL w/ φϕ\varphi_\phiEik-HiQRLEik-HiQRL w/ φϕ\varphi_\phiMQEMQE w/ φϕ\varphi_\phi
humanoidmaze-large-navigate5 ± 13 ± 174 ± 576 ± 334 ± 828 ± 1
humanoidmaze-giant-navigate1 ± 00 ± 083 ± 682 ± 347 ± 348 ± 7

As shown in Table 3, this augmentation does not consistently improve the other methods. Its role in GIP arises from the geometry of direction-based control. Just as tangent vectors at different points belong to different tangent spaces, the same latent path direction can represent different motions depending on the reference state. Even nearby raw states may therefore require different actions for similar state–direction inputs.

In fact, this state augmentation was inspired by dual representations [1]. Ideally, such a representation uniquely distinguishes states based on their temporal distances to all other states, while capturing information about temporal distance relationships. Our learned φϕ(s)\varphi_\phi(s) naturally serves as a dual representation (analogous to the "symmetric" representation in the original paper). This temporal distance-aware feature helps the policy disambiguate latent directions by distinguishing states more precisely, while the raw state retains the information needed for low-level control.

We agree that state augmentation plays an important role not as a general augmentation method, but as a component that enables GIP's distinctive direction-based control with geometric properties, and we have explicitly stated this in Section 4.3 (line 334) of the revised manuscript. We have also included an analysis in Appendix F.4 (line 1857, Table 4, Figure 14), along with the experimental results presented in Tables 1 and 2.

[1] S. Park et. al., "Dual Goal Representations", ICLR, 2026.

Reviewer_d31rOpenReview ↗

Reviewer's response

Thank you for updating the paper and running the experiments I requested. I've raised my rating to 3. There are a few issues I would appreciate you fixing for camera-ready:

  • Please include the new results (Table 3. Ablations of ... augmentation on other baselines) in the paper.
  • Please discuss the empirical importance of the augmentation in the main text.
  • Tables 2 and 3 in the rebuttal would benefit from additional environments to draw conclusions; currently they only cover humanoidmaze.
  • Figure 1 caption says "15% higher success rate than Eik-HiQRL" but the actual difference is about 10pp (I understand that it is ×1.15\times 1.15, but it reads as an absolute claim).
  • "13 OGBench environments" claim is technically correct, but it would be nice to specify that main gains are on locomotion.

I appreciate the effort you put into the rebuttal process.

Thank you for your detailed response to our rebuttal. We have the following plans to address the points you raised.

  • Regarding Tables 2 and 3, we plan to conduct experiments on additional environments. We will include the updated Table 2 in a new Section 6.4 (Ablations) to discuss the empirical importance of state augmentation in detail in the main text, and will add the updated Table 3 to the state representation part of Appendix F.4.

  • To eliminate ambiguity in our previous statements, we have revised the wording in the Figure 1 caption (line 38), Introduction (line 78), and Experiments (line 452) to use "10 pp".

  • We have also revised the Abstract (line 22) and Introduction (line 77), which previously referred only to "13 OGBench environments," to explicitly clarify that the main gains are observed in long-horizon locomotion tasks as below.

    ... with particularly strong performance on high-dimensional, long-horizon locomotion tasks ...

Once again, we sincerely thank you for recognizing the value of our approach, suggesting several directions for further ablation studies, and providing advice on the presentation. We recognize how valuable such a thoughtful review is and how much effort it takes to provide one. Our discussion with you has been truly helpful in improving the quality of our research and presentation.

Reviewer_WwN7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a offline RL method for learning a planner, called Geometry-Informed Planning (GIP). The method relies on learning to map environment states onto an exponential map where a tangent vector can represent the shortest path between 2 points. To do this, their method learns two functions: a distance network which takes 2 states and estimates how many steps are between the 2 states (critically, this is an asymmetric function); a path network which takes a start state, a “launch vector” (the main object used for planning), and a fraction between 0 and 1, and gives where you’d be along that route. GIP training proceeds as follows. A launch vector is generated for a state-goal pair. The distance network is frozen and scores the path networks output, penalizing it if misses the goal or if the progress to the goal it generates isn’t evenly distributed across time (i.e. the shortest path). The path network penalizes the distance network from leaving the sum of distances along the generated path. Because a local constraint caps the distance of every one-step dataset transition at 1, this pressure can only lengthen segments that leave the data support (e.g., passing through walls), which pushes the path network away from infeasible shortcuts. At test time, a subgoal a fixed number of steps along the planned path is decoded, and a separate low-level policy, trained with advantage-weighted regression and conditioned on the path’s direction, outputs the actions. This ends up being a mostly geometric approach for learning to take actions towards a goal.

Critical strengths

This is a very different approach to control compared to typical RL, this may inspire new ideas in the literature.

Their method seems to scale well with dataset size and is very FLOP-efficient. They avoid doing step-by-step planning which is nice.

Critical weaknesses

My understanding is that this has relatively strong assumptions on the state-space and on the data/generalization goal goal setting.

  1. it assumes the state space can be identified with a smooth manifold.
  2. it seems to assume a deterministic environment.
  3. It seems to really rely on the data having coverage of paths you would want to take (even if "stitched together" from multiple paths) to goals from states of interest.
Questions for authors
  1. A state space can be identified with a smooth manifold. This is controversial. Can you give examples of real-world domains where this be plausibly true?
  2. It seems that this method can't take routes outside of the states in the data? Can you motivate why this is a reasonable assumption.
  3. Does this method assume deterministic dynamics?
  4. Can you provide experiments showing how your method scales as the length from state to goal increases?
2 comments
AuthorsOpenReview ↗

Response to the review (1/2)

We thank the reviewer WwN7 for the constructive feedback on the assumptions, which helped us clarify the required conditions and limitations. Below, we address the reviewer's concerns and questions.

  • Can you give examples of real-world domains where a state space can be identified with a smooth manifold?

Smooth manifold structures naturally arise in a broad class of robotic planning and control problems. By the regular level-set theorem [1], a feasible state space defined by smooth equality constraints h(s)=0h(s)=0 with a full-rank Jacobian forms a smooth manifold. In robotic systems, feasible configuration spaces defined by such constraints can form smooth manifolds, and their corresponding state spaces, including velocities, can also be modeled as smooth manifolds.

Concrete examples include (i) robotic manipulation, where object and end-effector poses lie on SE(3) [2], and (ii) constrained robotic planning, where feasible configurations form smooth constraint manifolds [3, 4]. While these assumptions may not hold in environments with discontinuous state transitions (e.g., teleportation) or unreachable states, we believe they provide a reasonable geometric abstraction for continuous robotic control problems with smooth constraints and sufficiently regular goal-reaching dynamics.

[1] J. M. Lee, Introduction to Smooth Manifolds, 2nd ed., Springer, 2013.
[2] C. Tie et al., “ET-SEED: Efficient Trajectory-Level SE(3) Equivariant Diffusion Policy,” ICLR, 2025.
[3] Y. Zhang et al., “Constraint Manifolds for Robotic Inference and Planning,” ICRA, 2023.
[4] Y. Wang and Y. Chen, “Quasi-static Path Planning for Continuum Robots by Sampling on Implicit Manifold,” ICRA, 2024.

  • Can you motivate avoiding routes through states outside the data as a reasonable design choice?

We clarify that this is not a hard assumption restricting feasible routes to the dataset. Rather, GIP's out-of-support (OOS) penalization includes a conservative regularization mechanism commonly adopted in offline RL. Since offline RL learns solely from a fixed dataset without additional environment interactions, evaluating out-of-distribution (OOD) state-action pairs can lead to inaccurate value estimates, particularly overestimation. To mitigate this issue, offline RL methods commonly adopt pessimistic value estimation or regularize policies toward the data distribution, relying on sufficient data coverage.

GIP's OOS penalization includes the same principle. It discourages generated paths from passing through unsupported regions, including off-manifold regions and regions that lie on the state manifold but fall outside the data support. This helps prevent spurious shortcuts caused by underestimated temporal distances. As in offline RL more broadly, GIP aims to find the best achievable paths by combining transitions from the available data.

  • Deterministic dynamics assumption and dependence on transition coverage.

We agree that the deterministic dynamics assumption is used to justify the local transition constraint in Eq. (12) during temporal distance learning. Specifically, under deterministic dynamics, an observed one-step transition s,s′s,s' guarantees d∗(s,s′)≤1d^*(s,s') \leq 1. This follows the formulation of QRL, upon which our temporal distance learning objective is built, and is consistent with the assumptions adopted in several prior offline GCRL studies [5, 6]. Similarly, the dependence on transition coverage is inherited from the QRL principle, where observed local transitions must sufficiently constrain the global temporal distance structure. This limitation is therefore shared by the broader family of QRL-based distance learning methods, rather than being intrinsic to temporal geodesic learning itself.

Importantly, our central theoretical result, Theorem 5.1 (Asymptotic Temporal Geodesic Recovery), establishes geodesic recovery given the true temporal distance, without explicitly relying on deterministic dynamics or transition coverage conditions in its proof. This separation suggests a promising direction for extending GIP by incorporating alternative quasimetric temporal distance learning approaches that can handle stochastic dynamics and limited transition coverage while preserving our out-of-support penalization mechanism, which we leave for future work. We have incorporated this discussion into Section 7 of the revised manuscript.

[5] S. Park et al., “Foundation Policies with Hilbert Representations,” ICML, 2024.
[6] V. Giammarino and A. H. Qureshi, “Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learning,” ICLR, 2026.

AuthorsOpenReview ↗

Response to the review (2/2)

  • Additional experiment: Does GIP scale as start-to-goal distance increases?

The OGBench maze benchmarks show that the start-to-goal distance increases as the maze size grows from medium to large to giant. Additionally, to more directly investigate how GIP scales with increasing start-to-goal distance, we conducted a new experiment on humanoidmaze-giant-navigate-v0.

Specifically, we fixed a single start state and divided goals into four distance bins based on the number of maze-grid cells along feasible paths from the start: 0–7, 8–14, 15–21, and 22 or more cells. We selected three distinct goals from each bin, yielding a total of 12 goals, and evaluated GIP's average success rate within each bin.

Table 1. Average success rate of GIP across start-to-goal distance bins (4 seeds, 50 evaluation per each goals).

Start-to-goal distanceBin 1 (0–7)Bin 2 (8–14)Bin 3 (15–21)Bin 4 (≥22)
Success rate93±193 \pm 193±293 \pm 285±285 \pm 285±285 \pm 2

As expected, GIP achieves higher success rates for nearby goals. Nevertheless, its performance remains strong even for distant goals, maintaining an 85% success rate in the two farthest distance bins. These results provide further empirical evidence that GIP is robust to increasing planning horizons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.