acceptodds
Under review as a conference paper at ICLR 2027

ObjectBinder-VLA: Permutation-Consistent Multi-View Grounding and Intent-Conditioned Dynamics

Abstract

Vision-language-action (VLA) policies commonly condition control on global visual-language representations that do not explicitly encode task-relevant object support, visibility, multiplicity, or cross-view correspondence. We introduce ObjectBinder-VLA, a predictive multi-view object-state interface that preserves global VLA context while exposing a compact, measurable state for control. For each task-conditioned slot and view, a shared Object Binder predicts a categorical distribution over 64 image patches and a NO_OBJECT state, together with independent visibility. A fine-tuned SAM3.1 teacher provides training-only pseudo masks; Hungarian set matching avoids arbitrary identities for exchangeable instances, and teacher-aware duplicate regularization discourages slot collapse. An Intent Query and Object Dynamics Model predict this state at on LIBERO and on RoboTwin. Detached future distributions are projected into view-slot tokens for a flow-matching action expert. Stop-gradient boundaries isolate direct updates to the task-specific perception and prediction modules, while the shared Qwen backbone remains jointly optimized. Held-out diagnostics demonstrate spatial grounding and visibility prediction, with no failures across 100 target-order checks of the set objective. The final 50K model achieves 98.10% success (1,962/2,000) on Original LIBERO. These results indicate that explicit object binding and intent-conditioned dynamics can provide auditable intermediate states without replacing global policy context.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, object-centric representation learning, robot manipulation, multi-view visual grounding, object dynamics modeling, imitation learning, embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Zz9tOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the lack of explicit object-state structure in VLA policies. ObjectBinder-VLA predicts object locations, absence, and visibility for each camera view while retaining global visual-language features. Training uses segmentation pseudo-labels and set matching for exchangeable instances. An Object Dynamics Model predicts future object distributions from current inputs. Detached future-state tokens then condition the action expert. The main strength is an inspectable interface that represents several distinct aspects of object state. However, future prediction lacks quantitative validation, and the matched control gains are small. The final 50K model reaches 98.10% on Original LIBERO. Reported success is 74.61% at the episode level on LIBERO-Plus and 81.2% across six RoboTwin tasks. At a matched 30K budget, the full model improves LIBERO success from 97.25% to 97.55%. Thus, full-system performance is promising, but the added modules' benefit to control remains uncertain.

Critical strengths
  1. Clear object-state semantics. Localization, absence, and visibility have separate outputs. Set matching handles exchangeable targets, while retained global features preserve broader visual context.

  2. Useful current-state diagnostics. The Binder is evaluated for localization, visibility, absence, and multiplicity against teacher references. Weak results for small wrist-view objects are also disclosed.

  3. Broad benchmark coverage. Evaluation includes complete Original LIBERO and LIBERO-Plus protocols, plus six RoboTwin tasks. The paper distinguishes aggregation methods and acknowledges limits of its ablations. The evaluation range is broad enough.

Critical weaknesses
  1. Future prediction lacks quantitative validation. Current Binder metrics and qualitative examples do not establish whether future distributions outperform copying the current state. This is central to the proposed action-conditioning interface.

  2. The matched control gain is inconclusive. The 30K improvement amounts to six extra successes out of 2,000. The authors acknowledge this uncertainty, but independent training repeats are still needed to establish module benefit.

  3. RoboTwin averages are not directly comparable. The summary includes published results with different task sets and data budgets. The protocol caveat is appropriate, but combining these averages still invites an unsupported ranking.

Questions for authors

I still have three questions:

  1. Does future prediction beat a current-state persistence baseline on independent trajectories, including moving and static objects separately?

  2. How stable is the gain over the matched 30K baseline across independent training seeds?

  3. Can external RoboTwin results be presented separately, with task sets and training-data budgets clearly stated?

Reviewer_Eyf2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes to extract object-centric and task-relevant information from VLM for action prediction in VLA models. The idea is generally reasonable, but the paper is hard to follow and it is far from being polished. There are many terms used before well defined. The experiments are conducted on several simulation benchmarks without any real robot experiments.

Critical strengths

The papers studies an important question in VLA models. The vision-language tokens are usually redundant for generating actions. Such redundancy might affect the model's robustness.

Critical weaknesses
  1. It needs to introduce many important concepts properly before using them such as what is a slot? What does ODM mean? In line 078, What does the "repeated instances" mean? In line 074-075: The ODM predicts its intent-conditioned evolution at t+ 16 on LIBERO and t+ 49 on RoboTwin. What do they mean? The paper needs huge effort to polish before submission.

  2. Why do you think the tokens from the ViT encoder in Qwen3-VL represents patches in the original image space? After many layers of transformer, the tokens may have mixed all the information from all the positions.

  3. The method relies on a task-specific rule-based parser to identify object phrases. I wonder if it can generalize to unseen objects.

  4. The ablation in Table 4 shows that the method does not change the performance much.

Questions for authors

How can the method work if there is only one camera in the setting?

Reviewer_qCm5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ObjectBinder-VLA, which introduces explicit object-centric guidance into VLA policies. Slot queries built from instruction phrases attend to the per-camera patch features of Qwen3-VL, and the attention weights, interpreted as a distribution over patches and a NO-OBJECT option, are supervised with SAM3.1 pseudo-masks. An Object Dynamics Model (ODM) further predicts the object state h steps ahead, and the predicted states condition a flow-matching DiT action expert. The method is evaluated on LIBERO, LIBERO-Plus, and RoboTwin 2.0.

Critical strengths

The method handles the arbitrary instance ordering of same-category objects by matching within each exchangeability group, per frame and per view. The multiplicity point recall of 89.06% (Table 7) suggests that exchangeable instances are localized reasonably well. This is a sound design choice, but it does not outweigh my concern that the performance gains may not come from the proposed modules.

Critical weaknesses

Major

  1. The contribution of the proposed modules is not clearly isolated. In the 30K matched ablation, ObjectBinder-VLA and its ODM variant improve over the baseline by only 0.10 and 0.30 percentage points, which the authors themselves regard as not statistically conclusive (Table 4 caption), and no variance across runs is reported. Moreover, the baseline sharing the same backbone and action expert appears only in this 30K ablation, while all main results (Tables 1–3) are obtained under the 50K-equivalent budget, which the ablation does not isolate (Section 5.1). It is therefore unclear whether the headline 98.10% on LIBERO, as well as the results on LIBERO-Plus and RoboTwin, stem from the object-centric components or from the backbone and longer training.

  2. Most baseline numbers in Table 3 (e.g., Video2Act at 54.1%) are transcribed from Video2Act, where, according to its Table 1, they are averaged over six tasks: beat block, click bell, hanging mug, place object, place shoe, and turn switch. In contrast, the tasks evaluated in this paper (Table 6), such as blocks ranking, place dual shoes, and place object on scale, do not appear to overlap with these by task name. Although the caption notes that each baseline follows the settings of its original paper, presenting averages over different task sets side by side may lead readers to interpret 81.2% versus 54.1% as a direct comparison, particularly given the large variation in per-task difficulty on RoboTwin. In addition, the criteria for selecting these six tasks are not described in the main text.

  3. Quantitative results are reported only for the current-time Binder (Table 7), while the ODM predictions are supported only by qualitative examples: a single LIBERO episode compared with the future state (Figure 3) and RoboTwin predictions without the future target (Figure 4). Since the future teacher states are already cached and the metrics in Appendix C could be applied directly, reporting them for the ODM, together with a copy-current-state baseline, would help verify that the ODM captures meaningful future information.

Minor

  1. Several symbols are used without definition, e.g.:eve_v, LN, EpE_p, fLf_L, fprojf_{\text{proj}}, and WFW_F appear only in equations without being defined in the main manuscript. The prediction horizon is denoted inconsistently, as hh in the caption of Figure 2 and hDh_D in the main manuscript.

  2. Figure 2 does not fully reflect the overall flow and procedure of the proposed method, and it is difficult to match the elements of the figure with the notation used in the main manuscript.

Questions for authors
  1. Could the authors report the mean and standard deviation over multiple training seeds for the baseline and the ODM variant in the 30K matched ablation (Table 4)? This would clarify whether the gain of 0.30 percentage points exceeds run-to-run variation.

  2. Would it be possible to provide results for the baseline without the Binder and ODM trained under the same 50K budget on LIBERO, LIBERO-Plus, and RoboTwin?

  3. Regarding the RoboTwin 2.0 evaluation, how were the six tasks in Table 6 selected? Results on the task set used by the compared methods would allow a more direct comparison; alternatively, clarifying in Table 3 that the averages are not directly comparable would also address this concern.

  4. Could the authors report the metrics in Appendix C (e.g., point recall and top-k support IoU) for the ODM predictions against the cached future teacher states, along with a baseline that copies the current state?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.