acceptodds
Under review as a conference paper at ICLR 2027

RoboSTG: Training-Free Spatial-Temporal Guidance for Generative Robot Policies

Abstract

Pretrained generative robot policies provide expressive behavioral priors, yet their deployment-time behavior remains limited by the objectives encoded in training. The central challenge is not simply how to apply guidance at inference time, but how to convert manipulation-specific requirements into differentiable objectives that are compatible with generative action synthesis. We introduce RoboSTG, a training-free framework for reshaping frozen generative robot policies with manipulation-oriented spatial-temporal guidance. RoboSTG expresses task requirements through differentiable spatial fields and relational constraints in physical space, and lifts them to action-chunk objectives via substage-aware spatial-temporal modulation. These objectives produce gradients that directly steer iterative action generation, without policy fine-tuning, learned value models, or candidate ranking. Experiments on LIBERO and RoboTwin and four real-world manipulation settings show strong task success and robust task-aligned execution. Together, the results demonstrate that structured, temporally aligned manipulation objectives provide an effective interface for adapting pretrained generative policies at inference time.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generative Robot Policies, Inference-Time Guidance, Robotic Manipulation, Spatial-Temporal Guidance

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bPujOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RoboSTG, an inference-time guidance framework that does not update policy parameters. The method uses a differentiable mapping to convert generated action sequences into predicted physical states, then scores the trajectories using spatial score fields and geometric relations between entities. The authors further introduce temporal weights and manipulation-stage-dependent objective weights, and inject gradients of the objective into action generation to guide the frozen policy toward satisfying task requirements. Using a frozen X-VLA policy, the paper describes paired evaluation settings on LIBERO-Object and RoboTwin, but provides insufficient quantitative results. It also reports experiments on two real-world UR5 manipulation tasks. Each method is evaluated over 30 trials per task, with the total number of successful trials increasing from 41/60 to 48/60.

Critical strengths

The method provides a unified interface for incorporating task geometry into a frozen generative policy. It integrates spatial proximity, relations between entities, and execution stages into a trajectory objective, and propagates the objective gradients back to the action sequence through a differentiable mapping. This allows task requirements to be expressed explicitly while accounting for temporal dependencies between actions, without retraining the base policy.

Critical weaknesses
  1. The experimental evidence is insufficient. The simulation section describes the setup but reports no quantitative experimental results. The real-robot evaluation covers only two tasks, with 30 trials per method per task, which limits the scope of validation. The baseline method used in the real-robot experiments is also not specified.
  2. The symbol ‖Δa‖_2 on the vertical axis of Figure 4 is not defined. Moreover, Figure 4 only shows how the guidance signals change over time. Without the key control comparisons, it cannot establish that this design improves performance.
Questions for authors
  1. Could you clarify what experiments were conducted on LIBERO-Object and RoboTwin and provide the quantitative results? How did RoboSTG compare with the pretrained policy in terms of task success rate and mean executed actions?
  2. Which pretrained policy checkpoint is used in the real-robot experiments, and how are the object states and task geometry required for guidance obtained?
  3. What does Δa in Figure 4 represent, and how is its norm computed? Could you provide ablations using only spatial guidance, only relational guidance, and fixed objective weights to establish the contribution of each design?
1 comment

Please take a moment to respond the authors. Thank you!

Reviewer_ZbSPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RoboSTG steers frozen generative robot policies at inference time using differentiable spatial and relational objectives with stage-aware temporal weighting.

Critical strengths

The idea of adapting frozen policies through task-specific spatial and temporal guidance is interesting and practically relevant.

Critical weaknesses
  1. The paper leaves essential implementation details unspecified. Although the policy prior is identified as frozen X-VLA, the exact checkpoints and task-adaptation procedures are unclear. More importantly, the source of the task geometry, target relations, and stage information required by the guidance is insufficiently explained.

  2. The construction of the spatial fields is described mainly through generic formulations. The heatmaps do not establish how these fields are obtained in practice, whether they require manual specification or privileged simulator information, or how much task-specific engineering is needed. Keeping the policy frozen does not by itself establish that adaptation requires little additional effort.

  3. The experimental evidence is limited and incompletely reported. Section 4.2 describes the simulation metrics without providing the corresponding numerical results, while the conclusion claims improved success and shorter executions. RoboTwin task coverage is also unclear, and the real-world evaluation covers only two placement tasks.

  4. The evaluation does not isolate the contributions of the proposed components. The stage-transition plots illustrate guidance behavior but do not demonstrate that spatial objectives, relational objectives, and temporal modulation are each necessary. Comparisons against simpler guidance objectives are also missing.

Questions for authors

please address the concerns above

1 comment

Please take a moment to respond the authors. Thank you!

Reviewer_5wQLOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies whether a frontier multimodal model can act directly as a robot manipulation policy and improve from its own experience. GPT-6-Astra controls a simulated two-arm mobile robot through end-effector waypoints chosen from camera images and robot state, with one annotated demonstration per task, and is compared with seven post-trained vision-language-action and world-action policies on the 26 test tasks of EBench. It ranks second overall (46.7% success) and first on short-horizon tasks, but falls behind on long-horizon and precise tabletop tasks, which the authors trace, from recorded episodes, to unreliable physical execution. Two ways to narrow this gap, calling π0.5 as a tool and a self-improvement loop in which a critic from the same model revises the agent’s playbook and harness, give no significant gain, including in a smaller replication of the loop with Claude Opus 5.5.

Critical strengths
  • A clear and general idea. Formulating task requirements as spatial fields and relational residuals, spreading them over the action chunk with stage-dependent weights, and steering a frozen policy without retraining, a value model or candidate ranking is simple and could apply to many generative policies.
Critical weaknesses
  • The simulation results are missing. The abstract says the LIBERO and RoboTwin experiments “show strong task success”, and the conclusion reports “a modest aggregate success increase and shorter executions on LIBERO-Object” and a small reduction in episode length on RoboTwin (l. 456-458). However, Section 4.2 (l. 372-378) only describes the metrics, and no table or figure gives a success rate or an executed-action count for either benchmark. The only quantitative result in the paper is the real-robot Table 1, so the main simulation claims cannot be checked.
  • The experiments do not test the paper’s premise. The motivation is adapting a frozen policy to requirements that “emerge after policy training” (l. 43-46), such as keeping a relative pose or satisfying a geometric constraint. Yet every experiment runs standard benchmark tasks that the policy was trained for, on which the authors say the baseline is near saturation (l. 369, 377). No experiment adds a new requirement, such as a region to avoid or a different target pose, and measures whether guidance enforces it.
  • There are no baselines or ablations. The only comparison is guidance on versus off. Existing steering methods that the paper cites, such as value guidance (Nakamoto et al., 2024), steering through human interaction (Wang et al., 2025), constrained inpainting (Hao et al., 2025) or universal guidance, are not compared, and there is no ablation of the spatial term, the relational term, the temporal kernels, the stage modulation or the guidance scale. The authors note that Fig. 4 is “not a causal ablation” (l. 417-418), so the claimed contributions are untested.
  • It is not described where the objectives come from, and they may use privileged information. The method “assumes task geometry” (l. 430-431), but the paper never says how the fields, keypoints, desired relations, weights, kernels and stage boundaries are obtained for each task: whether they are hand-designed per task, read from simulator object poses, or estimated from the camera on the real robot. If the guided policy receives true object geometry that the unguided policy does not, part of the gain may come from extra information rather than from the method.
  • The positioning misses the closest prior work. Composable dense 3D value maps for manipulation (VoxPoser, Huang et al., CoRL 2023) and stage-wise relational keypoint constraints optimized over time (ReKep, Huang et al., CoRL 2024) are very close to the spatial fields, relational constraints and stage structure here, and neither is cited. The short related-work section does not explain what is new relative to universal or posterior-sampling guidance applied to action chunks.
Questions for authors
  • Please report the LIBERO-Object and RoboTwin results: success and executed actions per task, with and without guidance, with uncertainty, and the list of RoboTwin tasks.
  • How are the fields, relations, weights, kernels and stage boundaries specified for each task, and are simulator ground-truth object poses used? On the UR5, how is the task geometry obtained?
1 comment

Please take a moment to respond the authors. Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.