acceptodds
Withdrawn from ICLR 2027

PIVLA: Physical-Interaction-aware Vision-Language-Action Policy for Force-Sensitive Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language-Action (VLA) models produce semantically and geometrically correct motions, yet the same motion can crush or drop an object depending on grasp force. Recent force-aware VLAs treat force as an input or control target but do not represent what interaction a given context calls for. We present our method, a VLA that learns an intermediate physical interaction representation: a dedicated token fusing vision, language, and contact observations, supervised by per-group safe-force distributions and recurring interaction prototypes, and used to directly guide a flow-matching action expert. We validate on two real-robot tasks transferring i) damage-prone produce and ii) appearance-identical cartons with disjoint safe grasp intervals, where the required grasp can only be resolved through contact. Our method outperforms force-agnostic and equal-input force-aware baselines, and ablations confirm the contribution of each component. Code and data will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, VLA model, physical interaction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_y2JgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PiVLA, a vision-language-action policy for force-sensitive robotic manipulation. The core idea is to fuse vision, language, and tactile observations into an explicit physical-interaction representation. This representation is supervised by safe interaction distributions and recurring physical-interaction prototypes, and conditions a flow-matching action expert.

Critical strengths

This paper targets a key weakness of current VLA policies: geometrically correct actions do not always mean safe physical interaction. Rather than predicting deterministic force targets, it models interaction distributions explicitly, a promising route to physically grounded action generation.

The authors build a dataset of damage-sensitive objects and visually similar cartons with different payloads. They report task completion, object damage, and grasp-force deviation to examine how physical feedback shapes manipulation.

Critical weaknesses

On Task 1, PiVLA hits 84.9% safe success, versus 80.6% for the no-force π0 baseline. The authors note the gap is not statistically significant at this sample size. On Task 2, both systems reach 100% safe success under both carton conditions. PiVLA wins on some force-deviation metrics but loses on standardized deviation for empty cartons. Overall, the results barely support the claim that the learned representation delivers meaningful, consistent gains in task-level safety.

Table 3 shows that dropping the direct physical guidance pathway Gphy does not hurt performance. That variant hits 100% safe success under both carton conditions, matching or beating the full model. This undercuts the argument that conditioning actions directly on the learned physical representation is essential.

The ablations also use a run-until-five-successes protocol, so trial counts differ across variants. The authors concede this inflates success rates. Carton wear and shifts in the safety criterion add more confounders, making causal claims about individual components shaky.

Questions for authors

see weakness

Reviewer_9XrQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PIVLA, a force/tactile-aware VLA policy that learns an intermediate physical interaction representation. A contextualized tactile token fusing vision, language, and contact is supervised by per-group safe-force distributions and K-means interaction prototypes, and is used to condition a flow-matching action expert. The authors collect a real-robot dataset with two task families: damage-prone produce transfer and appearance-identical cartons with empty/filled hidden states. Experiments compare PIVLA with no-force π0 and an adapted ForceVLA baseline, plus ablations on physical awareness, direct guidance, and contact-conditioned tokens. The paper claims that PIVLA outperforms force-agnostic and equal-input force-aware baselines and that ablations confirm each component.

Critical strengths

The paper correctly identifies a real gap: semantically correct VLA motions can still crush or drop objects, and a single force target cannot represent disjoint safe-force regimes. Learning a physical interaction token supervised by distributions and prototypes is a reasonable direction. The “appearance-identical cartons” task family, if properly controlled, would be a useful testbed.

Critical weaknesses

1. The flagship Task 2 does not establish that contact is necessary.
The task is motivated by appearance-identical cartons whose safe grasp intervals are disjoint. However, no-force π0—without any contact input—achieves 100% safe success on both conditions. The ablation also notes that “both prints appear in training, so vision alone partially separates conditions before contact.” Together with carton wear, the aperture proxy, and excluded bulged-carton trials, the contact-token ablation is confounded. The paper therefore does not demonstrate the central claim that contact resolves an ambiguity unavailable to vision.

2. The safety supervision and metrics have validity issues.
The “safe-force distribution” is a Gaussian over 50-D taxel voltages, with σg\sigma_g reflecting demonstration variation rather than physical tolerance (the authors acknowledge this). The evaluation’s calibrated [fmin⁡,fmax⁡][f_{\min}, f_{\max}] intervals are not clearly consistent with the observed outcomes: all policies exceed the demonstrated range, yet no-force π0 remains 100% safe. The dd metric is therefore decoupled from actual safety. Grasp force is reported in taxel voltage (a.u.), not calibrated Newtons, which weakens the “force-sensitive” framing.

Questions for authors
  1. On Task 2, no-force π0 achieves 100% safe success on both empty and filled cartons. How does this support the claim that contact is necessary? Can you design a version where no-force π0 demonstrably fails, control for carton prints, and avoid carton wear confounds?

  2. The ablation “No extra guidance” matches or exceeds the full model. What is the evidence that the direct action guidance pathway is necessary? Please separate safe-distribution and prototype losses, and run fixed-trial ablations rather than “until five successes.”

Reviewer_xZSQOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a physical-interaction-aware VLAf for force-sensitive robotic manipulation. The key idea is to learn an intermediate physical interaction representation that captures appropriate contact behaviors rather than treating force as an additional input or a fixed control target. The physical interaction representation is supervised by two objectives: safe interaction distribution prediction and physical interaction prototype learning. The authors also introduce PiVLA-data and evaluate the proposed method on two real-world robotic manipulation tasks involving fragile objects and visually identical cartons with different physical states. Experimental results show improvements in safe manipulation and grasp-force consistency compared to the force-aware baseline.

Critical strengths
  1. The paper addresses an important limitation of existing VLA models: generating successful actions does not necessarily guarantee safe physical interactions.
  2. The authors conduct experiments on a real robotic platform with force-sensitive objects. In particular, the visually identical carton task provides a useful setting for studying physical-state ambiguity and contact feedback.
  3. Comprehensive Evaluation Metrics. Beyond task success, the paper evaluates safe success, object damage, slipping, and grasp-force deviations. These metrics provide a more detailed understanding of physical interaction quality than task success alone.
Critical weaknesses
  1. Although PiVLA improves grasp-force consistency, its advantage over the force-agnostic π0 baseline in safe task completion is limited and statistically insignificant. In addition, the force-agnostic baseline achieves perfect safe success in the tactile payload sensing task, raising questions about whether the proposed physical interaction representation is necessary for the evaluated scenarios.
  2. The ablation study does not convincingly establish the necessity of the proposed components. Removing direct physical guidance does not degrade performance, while the distribution and prototype losses are only ablated jointly. Consequently, the individual contribution of each proposed mechanism remains unclear.
  3. The proposed safe interaction distribution is estimated from successful demonstration statistics rather than physically calibrated safety limits. As a result, the learned distribution may reflect demonstrator behavior instead of actual physical feasibility. Furthermore, the framework does not explicitly enforce safety constraints during action generation, making the claimed connection between distribution prediction and physical safety insufficiently justified.
  4. Although the proposed physical interaction prototypes are intended to capture reusable interaction patterns, their generalization to unseen object categories or task combinations is not evaluated. Without prototype-specific ablations or transfer experiments, it remains unclear whether these prototypes learn transferable physical interaction patterns or primarily serve as an auxiliary regularizer.
Questions for authors
  1. Why do the authors choose K-Means for physical interaction prototypes instead of learnable methods such as VQ-VAE?

  2. Could the authors provide separate ablations for the safe interaction distribution loss and prototype loss to clarify their individual contributions?

  3. How sensitive is the performance to different values of K? Section 3.3 describes 50-dimensional statistics for K-Means, while Section 4 mentions 5-dimensional descriptors. Could the authors clarify which features are actually used?

  4. Since tactile feedback is unavailable before contact, how does PiVLA infer the appropriate physical interaction representation during the initial grasping stage?

  5. Does the learned physical interaction latent encode physically meaningful properties, such as object stiffness, weight, or slip risk, or does it primarily capture task-object-stage identity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.