acceptodds
Withdrawn from ICLR 2027

OneDemo-α: From Spatial Reasoning to One-Shot Robot Skill Transfer

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Imitation learning and pretrained robot models pursue generalization through large-scale training, yet rapidly transferring a newly demonstrated skill across objects and goals remains challenging. Vision-language models (VLMs) with strong spatial reasoning offer a complementary route to adaptation, but repeated inference and action generation during execution can incur substantial latency. Our key insight is to combine reusable motion experience from a demonstration with a spatially proficient VLM’s ability to determine how that motion should change. We introduce OneDemo-α, a framework for one-shot skill acquisition and transfer without task-specific training. From one RGB-D demonstration with synchronized robot poses, the VLM predicts an editable skill-adaptation vector α for the target scene and goal. Its functional bindings, motion assignments, and sparse edits guide a geometric executor, preserving reusable motion while concentrating inference on transfer decisions. We further introduce OneDemo-Bench, which systematically varies object position, appearance, size, morphology, and task requirements to evaluate transfer from a fixed demonstration. Experiments show that OneDemo-α achieves higher transfer success than trajectory-transfer baselines and substantially shorter execution times than direct VLM control, enabling efficient one-shot skill reuse without task-specific training.

Primary area:
applications to robotics, autonomy, planning
Keywords:
One-shot skill acquisition, robot manipulation, object-centric transfer, geometric re-anchoring, OneDemoBench

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kxP1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OneDemo-α transfers a demonstrated robot skill by reusing recorded motion and using a frozen VLM to predict functional bindings, motion assignments, and sparse edits. The paper introduces OneDemo-Bench and evaluates transfer success and inference cost in simulation and on a physical robot.

Critical strengths
  • Reusing demonstrated motion while adapting it to new objects and goals is a useful and well-motivated direction.
  • Combining spatial reasoning with a geometric executor is intuitive, and the simulation and physical results provide encouraging evidence.
Critical weaknesses
  • I find it difficult to assess how representative OneDemo-Bench is. Appendices B–E describe its construction, but more detail on task selection, perturbation design, exclusions, and development exposure would help establish that the evaluation is not overly tailored to the proposed transfer interface.

  • The simulation results span multiple platforms, but largely follow the same custom transfer protocol. This provides coverage, though I remain unsure whether the gains extend to an independently defined one-shot transfer evaluation.

  • The component contributions are not sufficiently clear. Re-anchor and Rigid-only controls are described, but I could not find their aggregate success rates. Comparisons across VLMs do not isolate the contributions of functional bindings, motion assignments, and sparse edits.

  • Some comparisons are difficult to interpret. Native Astra receives different source information, and its success rates and costs in Table 1 come from single-response and multiround settings, respectively. These differences are disclosed, but limit conclusions about the success–cost tradeoff and the benefit of the adaptation mechanism itself.

Questions for authors
  • Could the authors provide more detail supporting the task and perturbation choices in OneDemo-Bench, including exclusions and the separation between development and evaluation conditions?
  • Is there evidence that the gains hold under an independent one-shot transfer protocol or on tasks outside the benchmark construction process?
  • What are the aggregate results for the full method, Re-anchor, and Rigid-only on identical targets?
  • Could Native Astra’s success rate and cost be reported under the same control setting, with the effect of differing source information clarified?
Reviewer_coR9OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents OneDemo-α, a framework for transferring a robot skill from one demonstration without task-specific training. A frozen VLM predicts functional correspondences, phase assignments, and trajectory edits for a target scene. A geometric executor applies these changes to the recorded motion. The paper also introduces a transfer benchmark and reports simulation and real-world experiments.

Critical strengths
  1. Practical motivation. Reusing demonstrated motion while asking a VLM to predict necessary changes is a reasonable approach to reducing repeated action generation.

  2. Explicit interface. Functional bindings and indexed edits make the adaptation inspectable. The paper provides concrete geometric definitions and operation semantics.

  3. Encouraging physical results. The experiments include three real-world skills. The method achieves 60.0% transfer success versus 36.7% for DITTO, despite similar replay performance, suggesting benefits beyond accurate replay.

Critical weaknesses
  1. Geometric adaptation is not sufficiently demonstrated. Matching functional parts does not make a demonstrated trajectory valid for a new object. Even with the same grasp and articulation type, changing a hinged object's hinge-to-handle distance requires a different motion arc. Likewise, changes in handle dimensions can invalidate the demonstrated grasp or approach. The authors acknowledge compatibility limits, and α can modify trajectory shape through edits. However, the experiments do not clearly establish whether those edits resolve geometry changes that functional rebinding alone cannot handle. This is important to the claimed transfer capability.

  2. Limited evidence for the central methodological contribution. The main contribution appears to be the structured editing interface around a pretrained VLM. I am not yet convinced that the experiments establish what this interface adds beyond Astra's existing reasoning and access to recorded motion. Section 5.3 describes Re-anchor and Rigid-only controls, but their success rates are not reported. The empty-edit check verifies compiler consistency rather than editing effectiveness. Edit counts and magnitudes are also missing, leaving unclear how much motion is reused versus regenerated.

  3. The direct-control comparison does not isolate α. Appendix D explicitly states that Native Astra receives source RGB without source numerical motion or depth, making it a system reference rather than an input-matched ablation. This comparison supports the complete system, but cannot establish the advantage of the editing representation itself. Table 1 also combines single-response Native Astra success rates with multiround costs. Although disclosed, these do not describe one accuracy–cost operating point.

  4. The generalization results need more physical detail. The task inventory and transfer categories are useful, but the supplied manuscript does not systematically describe source–target object pairs and their geometric differences. Category labels overlap, and Tables 1–3 use different evaluation populations. Aggregate Geometry and Substitution scores therefore leave unclear which changes require new motion geometry and which preserve a compatible interaction. Per-task results and paired uncertainty estimates would also help assess the smaller margins, including the real-world comparison with Native Astra.

Questions for authors
  1. Editing ablations. What are the Full, Re-anchor, and Rigid-only success rates on identical targets? How frequently are waypoints offset, replaced, inserted, or deleted, and what fraction remain unchanged?

  2. Controlled geometry transfer. Can the authors test a small set of dimension changes within an existing task, such as changing the opening radius of a hinged object? Comparing Full and Re-anchor with oracle functional frames would help separate correspondence errors from failures to adapt the motion.

  3. Input-matched baseline. Does α outperform direct trajectory prediction when the same Astra model receives the same source poses, depth, images, and instruction? Can success and cost be reported together for each Native Astra operating mode?

  4. Execution scope. The pouring criterion includes grasping, while Appendix A.4 allows an execution entry that excludes home motion. Can the authors clarify initial grasp states and execution entries across tasks, and explain how connections from the current state and between rebound phases are generated?

  5. Reproducibility and interpretation. Can the authors provide representative source–target pairs, per-task outcomes, complete prompts, and worked predictions showing landmarks, phase assignments, and numerical edits?

The system is useful and the physical results are encouraging. My main concern is that the paper does not sufficiently isolate the contribution of its editing interface or demonstrate how it handles geometry changes that invalidate the demonstrated motion. The missing component results are particularly important. I lean toward rejection, but convincing editing ablations and targeted geometry-transfer evidence could change my assessment.

Reviewer_yt4sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ONEDEMO-alpha transfers a recorded robot demonstration to new objects and goals without updating model weights. A frozen VLM predicts functional bindings, phase assignments and sparse trajectory edits; a geometric executor applies them while retaining the remaining demonstrated motion. The method requires RGB-D observations and synchronized robot poses, with a shared source and target embodiment. Evaluation includes a transfer benchmark built from existing simulators, comparisons across VLMs and three physical robot skills.

Critical strengths

Letting the VLM edit an existing motion is a sensible way to reduce repeated action generation. I like that the bindings and indexed edits make its decisions inspectable, while the executor preserves motion that does not need to change.

One fixed demonstration per task is tested across changes in geometry, object identity and goals, with a 20.8-27.0 percentage-point gain over DITTO in Table 1. The physical result, 60.0% versus 36.7% transfer success over 90 trials per method (Table 5), supports practical value beyond accurate replay.

Critical weaknesses

The paper does not establish the added value of sparse motion edits. Re-anchor and Rigid-only are described in Sec. 5.3 and App. D but have no reported success rates; the single empty-edit drawer check only verifies that the compiler preserves a trajectory.

Table 1 combines Native Astra's single-response success rates with its multiround inference and token costs, so the row does not describe one achieved success-cost tradeoff. Other tables include multiround comparisons, but the headline comparison still needs success and cost from the same executions.

Questions for authors

What are the success rates of full adaptation, Re-anchor and Rigid-only on identical targets, especially Geometry, Goal and Compound? The full versus Re-anchor comparison with fixed responses and grounded frames would directly test the editing contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.