acceptodds
Under review as a conference paper at ICLR 2027

Contact as Interface: Composing Object-Centric Interactions with Executor-Aware Fields for Long-Horizon Pushing

Abstract

Long-horizon planar pushing in clutter faces two coupled challenges: collision-free object paths may require contacts that a robot cannot realize, and uncertain interaction dynamics can cause deviations from planned motions. We present Contact as Interface (CaI), a framework for long-horizon pushing that connects object-level planning and embodiment-specific execution through a reusable contact interface. An inverse interaction policy implements this interface by mapping normalized object geometry and local motion intents to contact actions. We learn this local mapping using only isolated pushes generated automatically in simulation, without complete task demonstrations, and keep the resulting policy fixed across downstream tasks. Deployment requires no object-specific dynamics identification. Executor-aware cost-to-go fields combine object-space reachability with direction-dependent execution costs to guide both nominal path construction and corrective intent search. An embodiment-specific executor selects and executes feasible contact actions. When progress stalls or repeated attempts fail to find feasible contact actions for the current intent, feedback-driven recovery searches for alternative motion intents. On two benchmarks we construct, CaI achieves 82.29% success in coupled local pose control on held-out geometries and 89.29% overall success in long-horizon pushing, exceeding the success rates of the evaluated baselines. Further experiments demonstrate robustness to nonuniform mass distributions, cross-embodiment reuse, and direct sim-to-real transfer without policy fine-tuning. These results show that a fixed local interaction policy can support long-horizon pushing in clutter when object-level planning accounts for robot feasibility and motion intents are adapted in response to observed deviations in object motion.

Primary area:
applications to robotics, autonomy, planning
Keywords:
non-prehensile manipulation, object-centric policy learning, long-horizon plannning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hmPNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tackles the problem of pushing an object in clutter. separates three layers. A small fixed policy learns the inverse mapping from a local motion inten represented as a translation and rotation, plus normalized object geometry to K candidate (contact point, push direction) pairs. It is trained only on isolated simulated pushes, with no task demonstrations. They also present a planner that builds executor-aware cost-to-go fields. It runs Dijkstra over a grid with direction-dependent costs derived from coarse IK and collision probes, so routes favor motions the robot can realize. They also introduce a robot-specific executor that picks a feasible proposal and executes it.

Critical strengths

Separating a reusable local interaction policy from task-level planning and from robot-specific execution is a clean decomposition. It's nice that the same fixed policy trained on isolated pushes is used across all benchmark tasks, a second embodiment in simulation, and a real robot platform without fine-tuning. I also like the fact that inverse supervision from automatically generated simulated pushes needs no teleoperation, no task demonstrations, and no per-object dynamics identification.

Critical weaknesses
  1. The baselines are all run inside the authors’ own benchmark and framework, with adapters (eg, Push Anything’s reach region is replaced with the benchmark’s pusher workspace). The paper doesn’t evaluate on any benchmark the RHP or Push Anything authors defined.
  2. The cost fields come from a planar grid with eight directions and finite probes. This gives no feasibility guarantee, and that online validation and feedback are essential. The method also needs known object and scene geometry in advance, and it has many hand-crafted components, so the tuning burden and sensitivity are unclear.
  3. The setting in the real world is not practical and the scene is relatively simple (no hard clutter). The perception challenge is sidestepped as geometry is reconstructed offline and tag-based pose tracking from three cameras.
Questions for authors
  1. How reliable are the baselines? RHP’s long-horizon success is very low, and its failures are dominated by virtual-action validation. How would RHP and CaI compare on a benchmark and protocol defined by the RHP authors? And how much do the results depend on baseline hyperparameters that were not tuned for these maps?
  2. How does it scale? The policy was trained on 1,520 pushes from 36 shapes, and the held-out sets are small. How does performance change with more or fewer training pushes and shapes, and how does it fail on concave, thin, or highly elongated objects? Does a policy trained this way still work for materials or surfaces with very different friction?
Reviewer_6YreOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for non-prehensile pushing of an object to a desired goal pose while avoiding obstacles on a 2D surface. The method can adapt to different object geometries.

First, pre-execution planning produces an executor-aware path. Then, an inverse policy maps object geometry and local motion intent to contact action proposals. This is trained on data collected by automatically sampling and executing local pushes in simulation, without any teleoperation or complete task demonstrations. Then the executor selects and executes a feasible proposal. Finally, there's a feedback-driven recovery when progress stagnates.

Critical strengths
  • Positive results in sim and in real for the task at hand
Critical weaknesses
  • A specific approach designed for a specific and relatively simple task
  • With generalist end-to-end policies (VLAs, WAMs, and even Agentic VLMs such as Astra) performing various tasks with high success rates, the specialized approach in the paper is not appealing
  • Limited no. of baselines
Questions for authors

None

1 comment
Reviewer_6YreOpenReview ↗

Stick to the original score

Since there is no response from the authors, I will stick to my original score.

Reviewer_nHdDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies long-horizon planar pushing in clutter. The central challenges are that collision-free object paths may require contacts the robot cannot realize, and unknown interaction dynamics can cause execution to deviate from the plan. Contact as Interface (CaI) connects object-level planning to embodiment-specific execution through a reusable contact interface. An inverse interaction policy learned from isolated simulated pushes maps normalized geometry and local motion intents to contact-point and pushing-direction proposals, with its weights fixed across downstream tasks and embodiments. Executor-aware cost-to-go fields combine geometric reachability with direction-dependent execution costs to guide nominal paths and corrective-intent search. An executor validates and realizes contact actions, while feedback-driven recovery revises motion intents when execution stalls or contacts become infeasible.

Critical strengths

The controlled comparisons support the value of task-level composition. Because BIT*–CaL shares the policy and executor with the full system, the difference in success rate between them can be attributed to planning and recovery jointly, and disabling recovery lowers success further, which points to the importance of feedback-driven correction. Since this pipeline-level comparison controls for differences in the local policy and executor, it isolates the contribution of the higher-level components. In local control, RHP-Policy (S) and (P) share a checkpoint and differ only in push-length assignment, and the gap between them clearly reflects the influence of that single factor.

Critical weaknesses
  1. The contributions of the new mechanisms and their interactions are not sufficiently isolated.The ablations support the importance of recovery. However, when the geometric field and recovery are retained, executor-aware costs improve average success by approximately 3 percentage points, with gains varying across difficulty tiers. Without a configuration that removes both executor-aware costs and recovery, it is unclear whether recovery compensates for the absence of these costs. Each long-horizon task is evaluated once, without paired analysis or uncertainty estimates from repeated runs. Correlations induced by reusing objects and maps also need to be considered, so the aggregate results do not establish the stability of this gain. Runtimes are not reported for the ablation configurations, leaving the relationship between the benefit and computational cost unclear. The local policy uses normalization, intent conditioning, and inverse supervision similar to RHP. The individual effects of multiple proposals and two-stage calibration have not been isolated through task-level ablations. Stage II reduces conditional directional error on the validation set, but its downstream benefit is not quantified. The effects of passage refinement and contact reuse are also not sufficiently isolated, limiting the assessment of the new mechanisms' contributions. A small factorial ablation study with repeated trials and runtime reporting would help disentangle the individual contributions of each proposed component and their interactions.

  2. The long-horizon comparisons lack sufficient diagnostic and reproducibility support.Training RHP on shared data is a useful control. Its training scale and evaluation settings differ from those in the original paper, so success rates cannot be compared directly across papers. The submission identifies failed virtual-action validation as RHP's dominant termination mode and notes that the hybrid often cannot replan because post-interaction states fail the planner's collision checks. However, it does not provide a quantitative breakdown or sanity checks of the adaptation. The available information does not sufficiently distinguish robot-execution feasibility difficulties from the effects of training and adaptation, limiting the explanation of CaI's relative advantage. CaI itself also lacks a failure breakdown and statistics on recovery activation, successful recovery, and recovery exhaustion, making it difficult to assess when recovery works or fails. The evaluation maps come from the same procedural family, so evidence for generalization to other layouts remains limited. Although the supplement includes policy and training-objective code, I did not find the complete task pipeline, map specifications, or per-task results, making the main comparisons difficult to verify externally. Adding a failure-mode breakdown and the missing reproducibility artifacts (full pipeline, map specs, per-task results) would make these long-horizon comparisons far easier to verify and interpret.

Questions for authors
  1. Mechanism contributions and interactions. Could you provide per-task paired results, or an appropriate uncertainty analysis, for the full system and the variant without executor-aware costs, accounting for correlations from shared objects and maps? Could you add a configuration with both executor-aware costs and recovery removed to clarify their interaction and which layouts benefit most? These results would affect my assessment of the contribution of executor-aware planning.
  2. RHP diagnosis and adaptation. What proportions of virtual-action validation failures arise from IK, collision checks during approach/pushing/retreat, solver timeouts, or other causes, and at what stages of the task do they mainly occur? How do the training and execution adaptations correspond to the original implementation, and what sanity checks were performed? This would help determine whether the low performance mainly reflects the feasibility difficulties targeted by this paper.
  3. Recovery and failure mechanisms. Could you report CaI's main failure types by difficulty, together with recovery activations, cases in which recovery enables successful continued progress, and recovery exhaustion? This would affect my assessment of recovery reliability and applicability. Please also clarify whether the complete task pipeline, map specifications, and per-task results can be made available to verify the main comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.