acceptodds
Under review as a conference paper at ICLR 2027

Masked Generative Motion Planning with Geometry-Guided Token Search

11 views

Abstract

Generative motion planners typically use learned trajectory priors for initial generation, while leaving test-time repair to local continuous refinement. We introduce Masked Generative Motion Planning (MGMP), which extends the learned prior from efficient parallel generation to structural repair. A masked generative transformer generates discrete trajectory candidates in parallel, and Geometry-Guided Token Search (GGTS) uses scene geometry to target where to edit and which prior-supported alternatives to evaluate. This turns refinement into an efficient search over discrete motion alternatives, enabling route-level restructuring beyond local trajectory deformation. MGMP achieves 96% success on Ring Maze and 82% repair success on Controlled Route Invalidation on Kuka, exceeding the strongest external baselines by 23 and 25 percentage points, respectively. It further generalizes to unseen layouts, additional obstacles, unseen geometries, single- and dual-arm planning, and real-world Baxter tasks. Further results and videos are available at: https://anonymous.4open.science/r/MGMP_GGTS-5FC6/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Motion planning, Generative model, Robotics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8AT9OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper's idea of reusing the learned masked-generation prior for test-time structural repair is interesting. The Ring Maze and Controlled Route Invalidation benchmarks are informative diagnostics, and MGMP shows clear gains in success rate, valid-sample fraction, and planning time across a broad range of simulated and real-world settings. However, the central claim that discrete, prior-guided token search is what enables route-level repair is not yet convincing. Key comparisons are missing: sampling-based planners such as BIT on the diagnostic benchmarks, and a continuous generative model on the same backbone.

Critical strengths

None.

Critical weaknesses
  1. The main claim isn't really isolated. The paper says the gains come from prior-guided discrete search, but BIT* is missing from both basin-escape benchmarks, and there's no continuous-generative baseline on the same backbone.

  2. Some evaluations don't quite test what they claim. Obstacles are given to the model only as center points, so on Ring Maze and the unseen meshes the prior can't see the actual geometry, and most of the work is likely done by the DeepSDF objective.

Questions for authors
  1. How does BIT* (or another sampling-based planner) do on Ring Maze and Controlled Route Invalidation under a similar time budget?

  2. What happens if GGTS uses random codebook candidates instead of MGT proposals, or remasks only the colliding tokens?

  3. In Ring Maze, how is the opening direction of each C-shaped obstacle given to the MGT, if only obstacle centers are encoded? The same question applies to the unseen meshes: do the baselines get the same DeepSDF cost?

  4. In Controlled Route Invalidation, how do you check that a local fix near Route A is actually infeasible, rather than just assuming it because a distant Route B exists?

  5. How many tokens are remasked, verified, and committed per round in practice?

11 comments
AuthorsOpenReview ↗

Reply to reviewer 8AT9 (1/4)

We are grateful to the reviewer 8AT9 for the thorough and insightful comments. These suggestions have helped us improve the clarity of the work, and we address each point in detail below.

The main claim isn't really isolated. The paper says the gains come from prior-guided discrete search, but BIT* is missing from both basin-escape benchmarks, and there's no continuous-generative baseline on the same backbone. How does BIT* (or another sampling-based planner) do on Ring Maze and Controlled Route Invalidation under a similar time budget?

We thank the reviewer for this suggestion. Our original Ring Maze evaluation was primarily designed to investigate whether classical trajectory optimization and generative planners with test-time guidance or correction become trapped in local minima, and whether MGMP can overcome these limitations by exploring alternative route topologies. In contrast, sampling-based planners such as BIT* explore the configuration space through sampling and tree expansion, rather than relying on local trajectory optimization, and are therefore not subject to the same optimization-induced local minima.

However, we recognize that this original focus should not have limited our comparison with other planning methods. We appreciate the reviewer for pointing out so we therefore evaluated BIT* on the Ring Maze environment under constrained planning-time budgets. With a 0.1 s budget, BIT* achieves a 0.0% success rate. Under such a tight time budget, BIT* cannot perform sufficient sampling and collision-checked tree expansion to reliably identify feasible paths. Increasing the budget to 1, 2, and 3 s improves its success rate to 61.95%, 88.33%, and 91.35%, respectively. In comparison, MGMP achieves 96% success with approximately 0.08 s running time. These results highlight the computational limitations of sampling-based planning, which requires substantial online exploration to identify feasible paths in complex environments. In contrast, MGMP leverages learned trajectory priors and geometry-guided refinement to efficiently generate feasible solutions, achieving substantially higher success rates under tight planning-time budgets. Moreover, for Controlled Route Invalidation, it is designed to isolate trajectory repair from initial generation, requiring all methods to refine the same invalidated trajectory. Standard BIT* performs sampling-based planning rather than trajectory repair, and replanning from scratch would not satisfy this evaluation protocol.

To further isolate the contribution of discrete token search from the choice of generative architecture, we additionally introduce a same-backbone Diffusion Transformer (DiT) baseline. It retains MGMP's Transformer architecture, obstacle encoder, and start–goal conditioning, but replaces masked discrete token prediction with continuous trajectory denoising. DiT is trained on the same demonstrations using a standard noise-prediction objective. Following MPD's cost-guided denoising strategy (Carvalho et al., 2023), we apply geometric guidance using the same planning objective JJ as MGMP. DiT uses 15 DDIM steps, with 30 gradient updates across the final five denoising steps. We evaluate both methods on identical Ring Maze and Kuka unseen-layout problems, using 20 candidates per problem. Despite using the same Transformer backbone architecture and conditioning, guided DiT achieves 0.81/0.92 SR and 0.42/0.75 VSF on Ring Maze and 7-DoF Kuka, respectively, compared with 0.96/0.97 SR and 0.67/0.87 VSF for MGMP. On Ring Maze, guided DiT has an estimated RT of 0.35 s, compared with 0.08 s for MGMP. The larger performance gap on Ring Maze suggests that geometry-guided discrete token search is particularly beneficial when feasibility requires substantial route changes. Together with the MGT+EmbOpt-30 comparison (0.75/0.90 SR and 0.29/0.74 VSF on Ring Maze/Kuka; Tables 1 and 4), these results suggest that MGMP's advantages cannot be explained solely by differences in network architecture, providing further evidence for the effectiveness of prior-guided discrete search in structural trajectory repair. We will include the additional BIT* results on Ring Maze and comparisons with a continuous generative baseline using the same backbone in the revised paper.

AuthorsOpenReview ↗

Reply to reviewer 8AT9 (2/4)

What happens if GGTS uses random codebook candidates instead of MGT proposals, or remasks only the colliding tokens?

We thank the reviewer for this suggestion. To further investigate the contributions of MGT-guided proposals and geometry-guided adaptive remasking, we introduce two additional ablations. Random Codebook Proposals replaces the MGT-predicted top-M candidates at each remasked position with M uniformly sampled codebook indices and uses uniform weights in the screening of Eq. 2, while the same exact verification procedure is retained. Collision-only Remasking restricts token updates to positions associated with detected collisions, excluding all additional positions selected by Eq. 1. All other components and refinement budgets remain unchanged.

We evaluate both variants on Ring Maze and 7-DoF Kuka under unseen obstacle layouts, following the settings of our existing ablation study (Table 4). We additionally evaluate Controlled Route Invalidation on Kuka, where all variants start from the same invalidated trajectory, to directly assess structural repair.

MethodRing Maze SRRing Maze VSFRing Maze RTKuka Unseen Layouts SRKuka Unseen Layouts VSFKuka Unseen Layouts RTControlled Kuka RS
Random Codebook Proposals0.770.340.080.890.680.430.53
Collision-only Remasking0.870.520.070.930.790.420.68
Full MGMP0.960.670.080.970.870.440.82

Replacing MGT proposals with random codebook candidates reduces SR from 0.96 to 0.77 on Ring Maze and from 0.97 to 0.89 on Kuka under unseen layouts, indicating that the learned trajectory prior provides more effective replacement candidates than random codebook sampling. Similarly, restricting remasking to collision-associated tokens reduces SR from 0.96 to 0.87 on Ring Maze and repair success from 0.82 to 0.68 on Controlled Route Invalidation. This suggests that modifying only collision-associated tokens can be insufficient when successful repair requires broader changes to the trajectory structure. Together, these results support the complementary roles of the learned trajectory prior in proposing meaningful motion alternatives and geometry-guided adaptive remasking in enabling structural repair beyond local collision correction. We will include these ablations in the revised paper.

In Ring Maze, how is the opening direction of each C-shaped obstacle given to the MGT, if only obstacle centers are encoded? The same question applies to the unseen meshes: do the baselines get the same DeepSDF cost?

We thank the reviewer for this question. Each C-shaped Ring Maze obstacle is constructed from multiple rectangular blocks, and the center of each block is provided individually to the obstacle encoder. The opening direction is not given as a separate condition; it is implicitly encoded by the spatial arrangement of the blocks, from which the MGT learns to generate feasible trajectories. Thus, the model must infer the opening direction from the obstacle configuration rather than receiving an explicit orientation label. We will clarify this representation in Sec. 3.2 and App. B.2 in the revised paper.

In the unseen-geometry experiments of Table 3, all methods are evaluated on the same ShapeNet meshes and start–goal queries. MGMP, CHOMP, STOMP, MPD, PBD and SafeFlowMatcher share the same DeepSDF-based signed-distance fields. These fields provide collision distances and gradients for trajectory optimization, sampling guidance, or safety-constrained correction, according to each method's planning mechanism. In contrast, TrajOpt uses mesh-derived collision distances and contact normals, RMP uses obstacle-distance information for reactive avoidance, and RRT*, P-RRT*, and BIT* rely on direct mesh collision checking. These methods can therefore use obstacle geometry without requiring DeepSDF. Importantly, all returned trajectories are evaluated against the same ground-truth meshes using identical PyBullet collision-checking and endpoint-validity criteria. We will clarify the method-specific geometry interfaces in App. B.10 in the revised paper.

AuthorsOpenReview ↗

Reply to reviewer 8AT9 (3/4)

In Controlled Route Invalidation, how do you check that a local fix near Route A is actually infeasible, rather than just assuming it because a distant Route B exists?

We thank the reviewer for raising this question. Controlled Route Invalidation (CRI) is designed to evaluate trajectory repair under identical initialization, independently of initial trajectory generation. For each problem, we relocate an obstacle to invalidate a previously verified feasible Route A while preserving a verified alternative Route B that is separated from Route A in joint space. This construction ensures that a feasible alternative exists after invalidation, although it does not exclude the possibility of local repairs near Route A. Importantly, all methods receive the same invalidated trajectory and modified obstacle configuration, without generating new trajectories from scratch. Repair success is determined solely by the feasibility of the resulting trajectory, regardless of whether the repair is local or involves substantial route changes. Thus, the evaluation does not inherently favor large-scale modifications.

Our results demonstrate GGTS's ability to perform large-scale trajectory repair under identical initialization. MGT+EmbOpt-30 shares MGMP's backbone, initial token representation, decoder, and objective, differing primarily in its use of continuous embedding optimization instead of discrete token search. As shown in Table 1, GGTS without polishing achieves 76% repair success, compared with 60% for MGT+EmbOpt-30. Moreover, successful GGTS repairs exhibit a substantially longer changed span (73.4% vs. 45.2%), indicating its ability to modify extended portions of the trajectory rather than only the immediate collision region. Continuous polishing further improves repair success to 82%.

To further investigate this difference, we conduct a paired analysis on the same 500 CRI problems, comparing GGTS without polishing and MGT+EmbOpt-30 under identical initial trajectories and obstacle configurations. We classify the outcomes into four groups: both methods succeed (285), only GGTS succeeds (95), only EmbOpt-30 succeeds (15), and both fail (105). GGTS successfully repairs 95 problems where EmbOpt-30 fails, whereas EmbOpt-30 succeeds in only 15 problems where GGTS fails. For problems repaired exclusively by GGTS, the changed span reaches 80.0%, compared with 71.2% for problems successfully repaired by both methods. These results demonstrate that GGTS recovers feasible solutions on additional problems that continuous embedding optimization fails to resolve from the same initialization. Together with the larger changed span, this provides further evidence that discrete token search enables effective, extended trajectory modifications beyond those typically achieved by continuous embedding optimization. We will clarify the CRI construction and evaluation protocol and include the additional paired analysis in the revised paper.

Reviewer_8a1HOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MGMP encodes robot trajectories as discrete tokens with a VQ-VAE and generates them in parallel with a masked generative transformer. At test time, Geometry-Guided Token Search (GGTS) uses a geometric objective J to choose which tokens to re-mask and which candidate tokens to try. This lets repair change the route, not only deform it locally. Experiments cover trajectory repair benchmarks, Maze2D, 7-DoF Kuka, 14-DoF DualKuka, and real Baxter tasks.

Critical strengths
  1. Method: the design is simple with good motivation. One discrete token space is used for both parallel generation and test-time repair, and GGTS splits repair into two clear decisions: where to edit (geometric attribution) and what to fill (prior-supported proposals, first-order screening, then exact verification). This keeps edits within the learned trajectory prior while still allowing route-level changes, at low cost.

  2. Results: the experiments support the main claim. In the authors designed Controlled Route Invalidation experiments (where all methods start from the same invalidated route), the proposed method MGMP repairs 82% of cases vs. 60% for continuous embedding optimization (Table 1). It also holds up on harder settings, reaching 0.96 success on 14-DoF DualKuka vs. 0.85 for BIT* (Table 2), and it works on real Baxter tasks.

Critical weaknesses
  1. The method depends heavily on J. Its gradient drives both edit selection (Eq. 1) and candidate screening (Eq. 2), and J is evaluated exactly for every shortlisted candidate. App. A.2 defines J as an SDF cost plus start/goal cost, but does not explain how the SDF is computed for the robot’s links. The unseen-geometry setting also needs a known DeepSDF code for each object (App. B.10). It is unclear whether J stays tractable in complex scenes, which limits how general the method is.

  2. Several core design choices are unclear. I am a bit concerned about how generalizable will Eq. 1 be. Could the authors provide more insights on the mechanism behind it? The masking scheme and the handling of failed candidates are also under-specified. This makes generality and reproducibility hard to judge.

Questions for authors
  1. How is J computed for each robot and scene type? How J is designed and how can it scale to more complex obstacles?
  2. What motivates the form of Eq. 1? Does it directly transfer across tasks without re-tuning?
  3. How is masking implemented in training and inference (Sec. 3.2)? What masking ratio and positional embedding do you use? In training, do you randomly mask out some of the intermediate tokens or simply mask out all intermediate tokens between the csc_s and cgc_g?
  4. In Sec. 3.3.2, it is not clear to me what happens when no shortlisted candidate improves J? Does the search just stop or the algorithm will simply use the original token?
  5. Figure 1 suggests that token-level replacement can jump to better local minima (H2, H3) more easily, while continuous-space refinement cannot. Could you explain or illustrate why token level replacement can escape local minima more esaily, and do you have experiments that show this directly?
  6. What is the difference between planning time (PT) and running time (RT)? It is said that "RT measures the wall-clock cost of generating and refining the full candidate set", but I do not understand why we need to refine the full candidate set.
  7. At around line 38, could the author explain what is continuous guidance? Is it basically classifier-free guidance? It might be better to clarify what is the difference between continuous guidance and trajectory optimization for board audience can understand it.

I am willing to increase my rating if the authors address the concerns above.

9 comments
AuthorsOpenReview ↗

Reply to reviewer 8a1H (1/5)

Thanks for the positive reception of the paper, and the detailed, constructive feedback. We have carefully considered the comments and addressed the raised points below.

How is J computed for each robot and scene type? How J is designed and how can it scale to more complex obstacles?

We thank the reviewer for this question. We clarify that JJ combines a hinge-based collision penalty (JcollJ_{\mathrm{coll}}) and an L1 start–goal penalty (JsgJ_{\mathrm{sg}}). The collision penalty is zero when the required clearance is satisfied and increases with clearance violations, while the start–goal penalty measures the L1 deviation of the first and last decoded configurations from the specified start and goal, respectively. We compute the collision penalty for each setting as follows (App. A.2):

Maze2D, Concave Maze2D, and Ring Maze. We model the robot as a point and compute analytic signed distances to the rectangular blocks that form the obstacles. We sum the hinge penalties over blocks and decoded positions, using a margin of 0.02 in workspace.

7-DoF Kuka. We approximate the arm with 20 collision spheres whose centers are computed through differentiable forward kinematics. At each decoded configuration, we apply a hinge penalty when a sphere's signed distance to the nearest obstacle falls below its radius plus a 0.05 m clearance margin. The penalties are summed over spheres and decoded configurations, considering all obstacles, including those introduced in the additional-obstacle setting.

14-DoF DualKuka. We apply the same environmental collision penalty to both arms and add an inter-arm collision term. At each decoded configuration, we compute the center-to-center distances for all (20×2020 \times 20) inter-arm sphere pairs and penalize separations below the sum of their radii plus a 0.005 m clearance margin. The resulting inter-arm penalties are summed and weighted by 3.0 in the GGTS objective.

Scalability to more complex obstacles. The formulation of JJ is not tied to a particular geometry representation. Box obstacles use analytic signed distances, while complex object meshes use DeepSDF-based distance queries at the collision-sphere centers (App. B.10). To further validate this flexibility, we conduct two additional experiments on Kuka and DualKuka using geometry reconstructed from noisy depth observations, without retraining MGMP. First, replacing DeepSDF with a depth-reconstructed voxel ESDF, MGMP achieves 85%/80% SR and 59%/48% VSF on Kuka/DualKuka, with planning times of 0.24/0.79 s. Despite using observation-derived geometry rather than ground-truth object shapes, it maintains performance close to the original DeepSDF results (90%/87% SR) and still outperforms PBD (76%/75%) and CHOMP (65%/57%) in success rate. Second, we evaluate a gradient-free GGTS variant using binary collision checks directly on the reconstructed occupancy grid, replacing gradient-based token selection and candidate screening while retaining the original sequential verification and commitment procedure. This variant achieves 76%/70% SR and 40%/30% VSF, with planning times of 0.23/0.80 s. Together, these experiments demonstrate that MGMP can use observation-derived geometry without precomputed object-specific SDFs, while its discrete search can also operate without differentiable geometry. Further implementation details are provided in our response to Reviewer mwxt (Question 1). We will clarify the task-specific collision objectives and geometry interfaces in the revised paper, and include the additional computational scalability analysis.

AuthorsOpenReview ↗

Reply to reviewer 8a1H (2/5)

What motivates the form of Eq. 1? Does it directly transfer across tasks without re-tuning?

The core of Eq. 1 is to prioritize tokens with high geometric attribution to constraint violations, while favoring lower-confidence tokens among those with similar geometric influence. The gradient gi=∂J∂ei=∑t(∂xt∂ei)⊤∂J∂xtg_i = \frac{\partial J}{\partial e_i}= \sum_t \left(\frac{\partial x_t}{\partial e_i}\right)^\top \frac{\partial J}{\partial x_t}, where xtx_t denotes the decoded trajectory at timestep tt, is computed by backpropagating the geometric objective through the trajectory decoder. It measures the local sensitivity of JJ to changes in token embedding eie_i, capturing both where the trajectory violates constraints and how strongly token ii influences the affected trajectory regions. Therefore, Eq. 1 contains two terms: si=∥gi∥2+λconf∥gi∥2(1−ci)s_i = \lVert g_i \rVert_2 + \lambda_{\mathrm{conf}}\lVert g_i \rVert_2(1-c_i).

(a) Geometric term (Gauss–Southwell rule). ∥gi∥2\|g_i\|_2 is the maximum rate at which JJ can change under an infinitesimal perturbation of eie_i. Ranking positions by ∥gi∥2\|g_i\|_2 is the Gauss–Southwell (GS) rule in block coordinate descent. Generally, under coordinate-wise Lipschitz smoothness and strong convexity, GS selection converges faster than random selection [Nutini et al., ICML 2015; Nutini et al., JMLR 2022]. Moreover, in our setting the decoder upsamples to recover temporal resolution from the compressed token sequence, and the receptive fields of adjacent tokens overlap, so a single colliding timestep is influenced by several tokens simultaneously. Backpropagating through the decoder attributes each violation to individual tokens according to their actual contribution, yielding a graded ranking.

(b) Confidence term. This term indicates which token position should be prioritized for modification when the geometric influences are similar. cic_i is the MGT's predicted probability of the current token. A low cic_i means the learned prior provides other plausible candidates for this position, so re-predicting the token is likely to yield a different but plausible candidate. Replacing a high-confidence token tends to produce off-manifold candidates. The multiplication ensures that confidence can raise a score by at most a factor of (1+λconf)(1+\lambda_{\mathrm{conf}}), so high-confidence tokens with strong geometric influence remain eligible.

Regarding hyperparameter transferability, we use the same λconf=0.5\lambda_{\mathrm{conf}}=0.5 and ρ=0.8\rho=0.8 across Maze2D, Ring Maze, Kuka, and DualKuka, without task-specific re-tuning. Both the token ranking based on sis_i and the cumulative-fraction selection controlled by ρ\rho are invariant to a global positive rescaling of JJ. Since cic_i is a softmax probability, the confidence is normalized, which gives λconf\lambda_{\mathrm{conf}} the same meaning in every task. We will clarify the motivation and theoretical intuition behind Eq. 1 in Sec. 3.3.1 and specify the shared hyperparameter settings in the revised paper.

How is masking implemented in training and inference (Sec. 3.2)? What masking ratio and positional embedding do you use? In training, do you randomly mask out some of the intermediate tokens or simply mask out all intermediate tokens between csc_s the and cgc_g ?

We thank the reviewer for this question. During training, we randomly sample a masking ratio r∼U(0.5,1.0)r \sim \mathcal{U}(0.5, 1.0) for each trajectory and mask the corresponding proportion of randomly selected trajectory tokens. Among the remaining tokens, each has a 50% probability of being replaced with a randomly sampled codebook index. The start and goal configurations (csc_s and cgc_g) are provided as separate condition tokens and are never masked.

During inference, we initially mask all trajectory tokens and generate their code indices in parallel through a single MGT forward pass, using Gumbel-Max sampling (τ=0.5\tau = 0.5). During subsequent GGTS refinement, token positions are adaptively re-masked based on geometric attribution and token confidence, with a maximum re-masking ratio of 0.4 (Table 8).

For mask representation, we use a dedicated learnable [MASK] embedding rather than zeroing token embeddings. The MGT uses learnable absolute positional embeddings initialized with sinusoidal encodings and updated during training. The same implementations are used across Maze2D, Kuka, and DualKuka. We will clarify these details in Sec. 3.2 and App. B.2 in the revised paper.

AuthorsOpenReview ↗

Reply to reviewer 8a1H (3/5)

In Sec. 3.3.2, it is not clear to me what happens when no shortlisted candidate improves J? Does the search just stop or the algorithm will simply use the original token?

We thank the reviewer for this question. When none of the shortlisted candidates improves JJ at a given token position, GGTS retains the original token rather than terminating the search. The unsuccessful position–token pairs are recorded in a tabu list to prevent repeated evaluation, and GGTS continues verifying candidates at the remaining selected positions. The tabu list is cleared after a successful commit, allowing previously rejected replacements to be reconsidered for the updated trajectory (Alg. 1).

Even if no replacement improves JJ across all selected positions, GGTS retains the current trajectory and proceeds to the next refinement step if the search budget remains. It then recomputes geometric attribution and generates new candidate proposals. Gumbel noise introduces stochasticity into candidate screening, allowing GGTS to explore different token replacements across refinement steps. The search is bounded by two refinement rounds with two token updates per round (Table 8), terminating when a feasible trajectory is found or the budget is exhausted. The benefit of continuing refinement is also supported by our ablation study: increasing the refinement schedule from 1×11 \times 1 to 2×22 \times 2 improves Kuka SR from 0.93 to 0.97 (Table 11b). We will further clarify the handling of unsuccessful candidate replacements and the search continuation criteria in Sec. 3.3.2 and App. A.1 of the revised paper.

Reviewer_mwxtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MGMP, a masked generative motion planning framework that combines a learned motion prior with gradient-guided token refinement for collision avoidance. The method identifies trajectory regions that require modification and selectively resamples them instead of regenerating the entire trajectory. The authors evaluate the approach on robotic manipulation and navigation benchmarks, demonstrating improvements in planning success rate and computational efficiency.

Critical strengths
  • The paper addresses an important problem of efficient motion planning in complex environments.
  • The proposed masked trajectory refinement is interesting and provides a reasonable way to preserve valid trajectory segments while modifying problematic regions.
  • The evaluation covers multiple environments and demonstrates promising improvements over existing approaches.
Critical weaknesses
  • The method uses box obstacles during training and shows generalization to unseen objects during test time using DeepSDF. DeepSDF however trains on precomputed SDF values for already known objects. This doesn't seem to be very practical since it assumes knowing exact object geometries and additional training of the DeepSDF network. It is unclear how the method would handle previously unseen objects without access to their geometry.
  • The method is primarily evaluated on single-arm manipulation setups. It is unclear how well it would scale to bimanual or humanoid robots with higher-dimensional configuration spaces. Extending the method to these settings would make it more broadly applicable.
Questions for authors
  1. Can the method handle arbitrary unseen obstacles without requiring precomputed SDFs or additional DeepSDF optimization? Could it use SDFs reconstructed directly from depth observations?
  2. How does the method scale to bimanual or humanoid robots? What are the expected limitations in terms of computational cost and planning success rate?
4 comments
AuthorsOpenReview ↗

Reply to reviewer mwxt (1/2)

We thank the reviewer for the positive assessment and thoughtful comments. Our responses follow.

Can the method handle arbitrary unseen obstacles without requiring precomputed SDFs or additional DeepSDF optimization? Could it use SDFs reconstructed directly from depth observations?

Our complex-obstacle experiments use DeepSDF computed from ground-truth shapes. However, MGMP is not tied to DeepSDF or any particular geometry representation. GGTS only requires collision costs for exact candidate verification and, for gradient-guided search, corresponding gradients to identify and screen promising token replacements. These quantities can be obtained from alternative geometry backends, without relying on DeepSDF. To investigate these capabilities, we conduct two additional experiments on the Kuka and DualKuka benchmarks with complex obstacle geometries, without retraining MGMP:

1. Depth-Reconstructed ESDF. To simulate imperfect sensing, we render depth images of unseen obstacles from multiple camera viewpoints, remove the robot and table using segmentation masks, and add Gaussian noise to the depth observations. We back-project and fuse the noisy depth images into a point cloud, then construct a voxel occupancy grid using ray casting to distinguish observed free space from occupied regions, treating unobserved regions conservatively as occupied. We then compute a signed Euclidean distance field (ESDF) from the occupancy grid to provide collision costs and gradients for GGTS. GGTS uses this ESDF in place of DeepSDF, retaining the same robot collision spheres and geometric objective JJ. Obstacle centers for MGT conditioning are estimated from the reconstructed point cloud rather than ground-truth geometry.

2. Gradient-Free GGTS with Binary Collision Checking. We further evaluate whether GGTS can perform trajectory repair using only binary collision feedback. We adapt the existing GGTS framework to use binary collision checks from the same noisy depth-reconstructed occupancy grid, without access to ground-truth obstacle geometry. Specifically, for token selection (Eq. 1), we identify collision timesteps along the decoded trajectory and remask their associated token positions, using MGT confidence to select additional positions within the remaining budget. For candidate screening (Eq. 2), we retain the MGT-proposed replacement tokens but shortlist the highest-probability candidates instead of using gradient-based scores. The sequential verification and commitment procedure (Eq. 3) remains unchanged, except that the objective combines the number of colliding sphere–timestep pairs computed from the occupancy grid with the original start–goal error. A replacement is committed only if it strictly reduces this objective; otherwise, the original token is retained. No SDF distances, geometric gradients, or continuous polishing are used. This evaluates whether prior-guided discrete repair remains effective using only binary collision feedback.

We summarize the results below, alongside the baselines reported in Table 3 of the original paper. SR and VSF are reported in %, and PT in seconds. Both depth-based variants use reconstructed geometry for planning, while final feasibility is evaluated against ground-truth meshes.

MethodKuka SRKuka VSFKuka PTDualKuka SRDualKuka VSFDualKuka PT
BIT*91–0.9983–1.88
CHOMP65332.2057342.68
MPD34140.6830111.98
PBD76470.2375360.96
SafeFlowMatcher31212.4733172.92
MGMP (DeepSDF)90660.2187570.72
MGMP + Noisy Depth ESDF85590.2480480.79
MGMP + Noisy Depth Binary Collision76400.2370300.80

With noisy depth-reconstructed ESDFs, MGMP achieves 85%/80% SR on Kuka/DualKuka, compared with 90%/87% using precomputed DeepSDF, while still outperforming PBD (76%/75%) and CHOMP (65%/57%) on both tasks. This demonstrates that MGMP can maintain strong planning performance using geometry reconstructed from noisy depth-camera observations and estimated obstacle positions, without precomputed object-specific SDFs or model retraining. Furthermore, using only binary collision checks on the noisy depth-reconstructed occupancy grid, GGTS achieves 76%/70% SR with planning times of 0.23/0.80 s, despite having no access to SDF distances, geometric gradients, or continuous polishing. Together, these results show that MGMP can perform motion planning using obstacle geometry reconstructed from simulated depth-camera observations, approximating practical robotic perception settings, while its discrete search mechanism can also operate with non-differentiable collision feedback. We will clarify the geometry interfaces and incorporate these additional experiments and implementation details into the revised paper.

AuthorsOpenReview ↗

Reply to reviewer mwxt (2/2)

The method is primarily evaluated on single-arm manipulation setups. It is unclear how well it would scale to bimanual or humanoid robots with higher-dimensional configuration spaces. Extending the method to these settings would make it more broadly applicable. How does the method scale to bimanual or humanoid robots? What are the expected limitations in terms of computational cost and planning success rate?

We thank the reviewer for this question. We would like to clarify that MGMP already supports coordinated 14-DoF dual-arm motion planning, accounting for both environmental and inter-arm collisions. We extensively evaluate DualKuka across three settings in our paper (Tables 2–3):

  • Unseen layouts: MGMP achieves 96% SR, 70% VSF, and 0.39 s PT, outperforming BIT* (85% SR, 2.01 s PT) and PBD (35% VSF).

  • Additional obstacles: MGMP achieves 84% SR, 54% VSF, and 0.90 s PT, outperforming CHOMP (68% SR, 26% VSF, 2.95 s PT).

  • Unseen geometries: MGMP achieves 87% SR, 57% VSF, and 0.72 s PT, outperforming BIT* (83% SR, 1.88 s PT) and PBD (36% VSF).

Importantly, MGMP demonstrates strong scalability from 7 to 14 DoF. On unseen layouts, MGMP maintains nearly identical success rates (97% → 96%), whereas BIT* drops from 97% to 85% and PBD from 93% to 79% (Table 2). This extension retains the same codebook size (8192), MGT architecture, refinement schedule, and per-position candidate budget, while using finer temporal tokenization (12 → 24 tokens). Despite the longer token sequence, MGMP achieves the lowest planning time among all evaluated methods across all three 14-DoF settings (0.39 s, 0.90 s, and 0.72 s; Tables 2–3). These results suggest promising scalability to higher-dimensional planning problems.

For full-body humanoids, we anticipate two main challenges. First, higher-dimensional motions may require richer codebooks and finer tokenization to maintain reconstruction fidelity, increasing learning and inference costs. Second, additional self-collision, balance, and contact constraints may increase computational costs and affect planning success. These challenges could be addressed by extending the motion tokenizer and geometric objective, while retaining the core GGTS search framework. Although full-body humanoid planning remains future work, our results provide encouraging empirical evidence for extending MGMP to higher-dimensional robotic systems.

We believe this addresses all the points raised – please let us know if you have any further questions.

AuthorsOpenReview ↗

Summary of Paper Revisions for Reviewer mwxt

We thank the reviewer for the constructive feedback. We have revised Sec. 4.2 and App. B.10 to address the dependence on precomputed DeepSDF representations. Specifically, we added two experiments using geometry reconstructed from noisy simulated depth observations: Depth-Reconstructed ESDF and Gradient-Free GGTS with Binary Collision Checking, without retraining MGMP. We report their performance on Kuka and DualKuka in Sec. 4.2 and provide implementation details in App. B.10, including depth reconstruction, ESDF construction, gradient computation, and binary collision-based refinement.

AuthorsOpenReview ↗

Reply to reviewer mwxt

Thank you again for your constructive feedback. We have carefully addressed your concerns through additional experiments and clarifications, and we hope these updates have strengthened the paper. As the rebuttal period draws to a close, we would greatly appreciate it if you could revisit our responses and reconsider your score in light of the new evidence. If any concerns remain, we would be grateful for your further feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Response and Updates for Revision

We sincerely thank all three reviewers for their thoughtful and constructive feedback. We hope our detailed responses have addressed the concerns raised. As a brief reminder, we summarize below the key updates made to strengthen the experimental evidence for structural repair, better isolate the contributions of GGTS, and clarify the method and evaluation protocols. The paper revisions are highlighted in red:

  1. [8a1H, 8AT9] Basin-escape and structural repair analysis. We added a direct cost-landscape comparison between GGTS and continuous embedding optimization (Fig. 4, Sec. 4.1.1), four additional visualizations (Fig. 9, App. C.1), and token-replacement statistics and paired repair analysis on Controlled Route Invalidation (Sec. 4.1.2).

  2. [mwxt, 8a1H] Planning with depth-reconstructed geometry. We added Depth-Reconstructed ESDF and Gradient-Free GGTS with Binary Collision Checking experiments on Kuka and DualKuka, demonstrating planning without precomputed object-specific SDFs or retraining, including with non-differentiable collision feedback (Sec. 4.2, App. B.10, Fig. 8).

  3. [8AT9] Additional baseline comparisons. We added BIT* comparisons under different planning-time budgets on Ring Maze (Sec. 4.1.1, App. C.1) and a guided Diffusion Transformer (DiT) baseline with the same backbone architecture and conditioning as MGMP (Sec. 4.3, App. C.5).

  4. [8AT9] New ablations and practical search statistics. We introduced Random Codebook Proposals and Collision-only Remasking ablations to isolate the contributions of learned proposals and adaptive remasking (Sec. 4.3, Table 3, App. C.5). We also report practical statistics on remasked positions, position visits, exact candidate evaluations, and committed replacements (App. C.5).

  5. [8a1H] Methodological and implementation clarifications. We detailed the task-specific geometric objective (App. A.2), motivated Eq. 1 and the role of confidence modulation (Sec. 3.3.1), clarified training and inference masking (App. B.2), and explained how GGTS handles unsuccessful candidate replacements and continues refinement (Secs. 3.3.2–3.3.3). We also clarified the meaning of continuous guidance (Introduction).

  6. [8AT9] Geometry representations and evaluation protocols. We clarified how Ring Maze openings are encoded (App. B.2), the geometry interfaces used by different baselines, and the common feasibility criteria (App. B.10). We also clarified that the existence of Route B does not rule out local repairs near Route A (Sec. 4.1.2).

We hope these additional experiments, analyses, and clarifications address the reviewers' concerns and further strengthen the paper. Detailed responses and revision summaries are provided in the corresponding reviewer discussions. If any concerns remain, we would be happy to provide further clarification. If these updates have resolved your concerns, we would sincerely appreciate your consideration of revising your scores accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.