acceptodds
Withdrawn from ICLR 2027

PAIR: Bridging Perception and Action in Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision–language–action (VLA) models map visual observations and language instructions to continuous robot actions. This task requires a transition from representations that describe the scene and instruction to representations that support action generation. Many continuous-action VLAs leave this transition implicit and supervise it mainly through the final action-prediction loss. We introduce PAIR, a framework that learns a shared perception–action representation between these two spaces. During training, a Masked Action Autoencoder encodes expert action chunks into horizon-aligned Action Latent Tokens. A Bridge Module extracts task-relevant features from the current visual–language representations. PAIR aligns these features with the Action Latent Tokens to form Bridge Tokens that preserve task information and capture the structure of expert actions. The Bridge Tokens are then projected into the action-token space and injected into the initial Action Tokens, providing an action-ready starting point for Action Expert refinement. At inference, the autoencoder is removed, and the Bridge Tokens are generated only from the current observation and instruction. Experiments on LIBERO, LIBERO-Plus, and CALVIN ABC→D show gains for the evaluated OpenVLA-OFT and VLA-Adapter models. On LIBERO-Plus, PAIR raises VLA-Adapter's success rate from 59.1% to 64.2%. On CALVIN, it increases VLA-Adapter's average completed sequence length from 4.42 to 4.53. Across seven real-world tasks, PAIR raises OpenVLA-OFT's success rate from 51.4% to 65.0%. Representation analyses show that Bridge Tokens retain task information while making continuous-action information accessible before Action Expert refinement. These results support a shared intermediate representation as a useful interface between perception and action in continuous-action VLAs.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DVVgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PAIR, which learns an intermediate representation (Bridge Tokens) bridging perception and action through a masked action autoencoder and an alignment loss, and injects it at the entrance of the Action Expert. Improvements are reported on LIBERO, LIBERO-Plus, CALVIN, and real-world tasks with OpenVLA-based models.

Critical strengths
  1. Easy to read: the paper is clearly written and well organized.
  2. Clear motivation: the perception–action representation gap is well articulated, and the method design is coherent.
  3. Careful validation: linear probes, layerwise decoding, and RSA/CKA analyses support that the intermediate representation carries both task and action information, and the ablations cover the key design choices.
Critical weaknesses
  1. Limited novelty: the method combines known components (action tokenization and latent action representations, query-based bottleneck tokens, cosine alignment to frozen targets, gated residual injection), differing from many closely related works such as LARA, LAPA, and UniVLA mainly in the aligned object and the injection position; it is an engineering combination rather than a new principle.
  2. Small gains without statistical support: +1.1 on LIBERO (a near-saturated benchmark) and +0.11/0.12 on CALVIN; no variance is reported for the baselines and no significance tests are conducted.
  3. Selective baselines in Table 2: under the same protocol, the official LIBERO-Plus benchmark evaluated OpenVLA-OFT (Total 70.0) and π0-FAST (64.2), yet Table 2 omits both, keeps only weaker baselines, and bolds +PAIR's 64.2 as the best; the actual improvement is thus limited.
  4. Model selection: the two backbones come from the same family (both small OpenVLA-derived models); parameter counts, latency, and the two-stage training overhead are unreported, and there is no comparison with intermediate-representation methods such as LARA under the same backbone.
  5. Incomplete ablations: λalign is tested only over {0, 0.01, 0.1}, where the largest value happens to be the base optimum and no larger values are tested; moreover, the base setting is the best in every ablation table, weakening the evidential value.
Questions for authors
  1. What is the baseline selection criterion for Table 2? Please add the LIBERO-Plus results of OpenVLA-OFT and π0-FAST; is +PAIR still the best under comparable settings?
  2. Please provide multi-seed variance and significance tests for the gains in Tables 1/3 and the ablations in Table 5.
  3. What are the results for larger λalign values (e.g., 0.5, 1.0)?
  4. What gains does +PAIR achieve on different backbones such as π0 or π0-FAST?

Note: the score may be raised if the authors, in the rebuttal, add comparisons against SOTA models and additional backbones, conduct detailed comparisons, and demonstrate statistically significant gains.

1 comment
AuthorsOpenReview ↗

Author Response to Reviewer DVVg

We thank the reviewer for the detailed assessment and constructive suggestions. We further clarify PAIR's methodological contribution and provide additional evidence on stronger VLA backbones, challenging benchmarks, and representation learning.

W1. Methodological Contribution

PAIR establishes an explicitly supervised perception-to-action interface, where Bridge Tokens jointly capture task-relevant perception information and learned action structure before entering the Action Expert.

Unlike approaches centered on latent action tokenization or visual-dynamics modeling, PAIR learns action structure from expert trajectories and uses it to supervise perception-conditioned Bridge Tokens. Through cross-attention and action-latent supervision, these tokens integrate task-relevant visual-language information with action structure, providing a task-conditioned, action-informed initialization that facilitates the transition from perception to action.

The injection-position and alignment-target ablations further validate this design, demonstrating the benefits of early integration and learned action representations for downstream action generation.

W2. Generalization to π0.5 and RoboTwin 2.0

We further extend PAIR to π0.5 on RoboTwin 2.0, covering 50 bimanual manipulation tasks. Following the official co-training protocol, we jointly train on 2,500 Clean demonstrations and evaluate 5,000 episodes each under Clean and Random settings.

MethodClean (%) ↑Random (%) ↑Avg. (%) ↑
π0.5 (Official)70.7046.0058.35
π0.5 + PAIR (3 seeds)75.12 ± 0.7552.08 ± 0.8063.60 ± 0.63
Improvement (pp)+4.42+6.08+5.25

PAIR achieves a +5.25 pp average improvement over the official π0.5 baseline, with gains of +4.42 pp on Clean and +6.08 pp on Random evaluation. Notably, the larger Random improvement is achieved using exclusively Clean demonstrations for training.

Together with the existing LIBERO, LIBERO-Plus, CALVIN, and real-world evaluations, these results further demonstrate PAIR's effectiveness across different manipulation settings and VLA architectures.

W3. Controlled Baseline Comparisons

Table 2 focuses on controlled comparisons under the same backbone and training protocol, directly isolating PAIR's contribution. On LIBERO-Plus, PAIR improves VLA-Adapter from 59.1% to 64.2% (+5.1 pp).

This controlled evaluation directly demonstrates the benefit of PAIR as an enhancement to an existing VLA backbone. The additional π0.5 results further establish its applicability to a stronger model and a distinct benchmark.

W4. Backbone Generality and Efficiency

PAIR is evaluated with OpenVLA-OFT (7B), VLA-Adapter (0.5B), and π0.5, demonstrating compatibility across different backbone scales and architectures.

The Action Autoencoder serves as a training-time teacher and is removed during inference. Only the Bridge and gated injection remain, without introducing additional autoregressive generation steps.

This design provides action-informed guidance at the entrance of the Action Expert while preserving the original action-generation process.

W5. Contribution of Action-Latent Supervision

The alignment-weight ablation in Table 5 demonstrates the contribution of learned action-latent supervision:

Alignment Weight00.010.1
LIBERO Avg. (%)97.297.898.4

Increasing the alignment weight from 0 to 0.1 improves performance by +1.2 pp, demonstrating the effectiveness of explicitly supervising Bridge Tokens with learned Action Latents.

The remaining ablations examine complementary design choices, including injection position, supervision target, injection strength, and Action Autoencoder corruption. Together, these results support the effectiveness of PAIR's explicitly supervised perception-to-action transition.

Reviewer_TXsqOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces PAIR, which aims to mitigate the representational gap between perception and action in Vision-Language-Action (VLA) models. During training, the proposed method utilizes latent representations of expert actions to supervise Bridge Tokens," which are then injected into Action Tokens at the onset of the action decoding process. During inference, these Bridge Tokens are generated solely from the current observation and language instruction, requiring no input from expert actions. Experimental results demonstrate that PAIR achieves performance improvements across the evaluated simulation benchmarks as well as in seven real-world robotic tasks.

Critical strengths
  1. The paper is well-motivated and clearly written, addressing the important perception–action representation gap in VLA models.
  2. The proposed framework is intuitive and can be integrated into existing VLA architectures without major modifications.
  3. The experiments are comprehensive, covering multiple simulation benchmarks and real-world robotic tasks. The results demonstrate improvements over the evaluated baselines.
Critical weaknesses
  1. The core idea of bridging perception and action through a shared latent representation overlaps with VITA [1] (CVPR 2026). Although PAIR introduces different alignment and injection mechanisms, its conceptual novelty over VITA remains unclear. The authors should discuss this closely related work and better clarify PAIR's distinctive contributions.

[1] Xiangkai Ma et al. Unifying Perception and Action: A Hybrid-Modality Pipeline with Implicit Visual Chain-of-Thought for Robotic Action Generation. CVPR, 2026.

  1. Table 5 shows that removing alignment reduces performance from 98.4% to 97.2%, even below the original baseline (97.3%, Table 1), while Table 6 shows that learned Action Latent Tokens outperform raw-action alignment. However, these results do not establish whether the two-stage Masked Action Autoencoder is necessary, as simpler alternatives such as auxiliary action prediction from Bridge Tokens are not evaluated.
Questions for authors
  1. Appendix C.1 specifies 50k training steps for the Action Autoencoder. How do the authors determine whether the model has sufficiently converged?

  2. How is the quality of the learned Action Latent Tokens evaluated? Could the authors report reconstruction errors or other relevant metrics?

  3. How sensitive is PAIR's downstream performance to the quality of the pretrained Action Autoencoder? What happens if the Autoencoder is undertrained?

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer TXsq

We thank the reviewer for the positive assessment of the paper's motivation, clarity, and experimental evaluation, as well as the constructive suggestions. We clarify PAIR's distinction from VITA and provide additional experimental evidence on Action Latent learning and its downstream effects.

W1. Distinction from VITA (CVPR 2026)

VITA and PAIR address the perception–action gap through fundamentally different mechanisms.

VITA unifies visual dynamics and action generation through a shared discrete codebook. Its autoregressively generated latent tokens are jointly decoded into future visual observations and robot actions, integrating both modalities within a generative framework.

PAIR instead focuses on facilitating the transition from perception to action within existing VLA architectures. Specifically, a Masked Action Autoencoder learns continuous, action-derived latent representations, which supervise perception-conditioned Bridge Tokens. Through cross-attention, these tokens capture task-relevant visual-language information while acquiring action structure through latent alignment. They are then injected at the beginning of the Action Expert, providing a task-conditioned, action-informed initialization for downstream action generation.

Thus, PAIR establishes an explicitly supervised perception-to-action interface, rather than unifying visual and action generation in a shared generative space. It requires neither future visual prediction nor additional autoregressive reasoning during inference. We will incorporate VITA into the related-work discussion to clarify this distinction.

W2. Effectiveness of Learned Action Latent Supervision

We further evaluate direct action-space supervision as a simpler alternative to the Masked Action Autoencoder, comparing two objectives:

  • Raw-action cosine supervision (Table 6): Project Bridge Tokens into the action space and maximize their cosine similarity with ground-truth action chunks.
  • Direct action prediction (L1): Predict ground-truth action chunks directly from Bridge Tokens using an auxiliary L1 regression loss, without the Action Autoencoder.

Both alternatives retain the Bridge and its injection mechanism, replacing learned Action Latent supervision with direct action-space objectives.

MethodLIBERO Avg. (%) ↑
Raw-action cosine supervision (Table 6)97.1
Direct action prediction (L1)96.9
Full PAIR (Learned Action Latents)98.4

Full PAIR outperforms direct L1 prediction by +1.5 pp and raw-action cosine supervision by +1.3 pp. These results demonstrate the benefit of learning structured Action Latent Tokens as supervision for the perception-to-action transition, beyond directly supervising Bridge Tokens with raw actions.

Q1. Action Autoencoder Training Convergence

We train the Action Autoencoder for a fixed 50K steps and monitor validation reconstruction L1 throughout training. The reconstruction error decreases rapidly during the first 10K steps, followed by gradual stabilization.

Across the four LIBERO suites, the average validation L1 decreases from 0.0395 (20K–30K) to 0.0374 (40K–50K), an additional 5.3% reduction. Validation fluctuations during the final 10K steps remain small (standard deviations of 0.0013–0.0022), indicating stable reconstruction performance toward the end of training.

Q2. Quality of Learned Action Latent Tokens

We evaluate Action Latent quality through reconstruction of continuous action chunks, including masked action steps. All errors are measured on q99-normalized actions.

SuiteReconstruction L1 ↓Masked-step L1 ↓
Spatial0.0390.052
Object0.0340.043
Goal0.0360.047
Long0.0350.045

The learned Action Latent Tokens support accurate action reconstruction, including recovery of masked steps. Together with W2, these results demonstrate both their ability to capture action structure and their effectiveness as supervisory targets for Bridge Tokens.

Q3. Sensitivity to Action AE Pretraining Quality

We examine downstream sensitivity on LIBERO-Long using VLA-Adapter, keeping all downstream training settings unchanged while varying Action AE pretraining steps.

Action AE StepsValidation L1 ↓LIBERO-Long Success (%) ↑
1K0.075095.2
10K0.041095.8
50K (Full PAIR)0.034896.2

As Action AE reconstruction quality improves, downstream success consistently increases from 95.2% to 96.2%. The 10K-step teacher already achieves 95.8%, approaching full performance, while further pretraining provides additional gains.

These results indicate that PAIR benefits from higher-quality Action Latent supervision while maintaining strong performance even with undertrained Action Autoencoders.

Dear Reviewer TXsq,

Thank you again for your thoughtful feedback and for taking the time to consider our responses.

As the discussion period approaches its conclusion, we would like to kindly follow up and ask whether our clarifications and additional experimental results have adequately addressed your main concerns.

We would be happy to provide any further explanations or clarifications if needed, and would greatly appreciate hearing any remaining questions or feedback before the discussion concludes.

Thank you for your time and consideration.

Best regards, The Authors of Submission 22366

Reviewer_FesGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PAIR learns expert-action latents,aligns perception-derived bridge tokens to those latents, and then inject these bridge tokens to VLM. It reports gains on LIBERO, LIBERO-Plus, CALVIN.

Critical strengths
  1. The paper focus on a interesting points that align the perception information to the action latent.
  2. The ablation is comprehensive.
Critical weaknesses
  1. The currents experiments could not demonstrate effectness of the proposed methods. More results on different benchmark, such as robotwin or robodojo is necessary. Moreover, Achieving Improvment over the weak baskline, the openvla or VLA-adaptor, is not enough and Promising results with SOTA method are needed.
Questions for authors
  1. Which layer supplies H_VL for each backbone, and is the bridge placed before the VLM or only before a separate action expert?
  2. Do action decodability and geometry persist on trajectories unseen during policy training or on LIBERO-Plus observations?
1 comment
AuthorsOpenReview ↗

Author Response to Reviewer FeSG

We thank the reviewer for the thoughtful assessment and constructive suggestions. We report new results on RoboTwin 2.0 with π0.5, evaluate action-representation transfer on LIBERO-Plus, and clarify the architecture below.

W1. RoboTwin 2.0 Evaluation with π0.5

We extend PAIR to π0.5 and evaluate on RoboTwin 2.0, covering 50 bimanual manipulation tasks. We follow the official RoboTwin 2.0 co-training and evaluation protocol: joint co-training across all 50 tasks using 2,500 Clean demonstrations (50 per task), with no Random demonstrations. For each training seed, we evaluate 100 episodes per task in both Clean and Random settings (5,000 episodes per setting).

MethodClean (%) ↑Random (%) ↑Avg. (%) ↑
π0.5 (Official)70.7046.0058.35
π0.5 + PAIR (Seed 1)74.7252.7063.71
π0.5 + PAIR (Seed 2)74.6651.1862.92
π0.5 + PAIR (Seed 3)75.9852.3664.17
π0.5 + PAIR (Mean ± Std)75.12 ± 0.7552.08 ± 0.8063.60 ± 0.63
Improvement (pp)+4.42+6.08+5.25

Across three training seeds, PAIR improves over the official π0.5 baseline by 5.25 percentage points on average. The larger gain under Random evaluation (+6.08 pp) is obtained despite training exclusively on Clean demonstrations. These results extend PAIR to a stronger VLA backbone and a distinct bimanual manipulation benchmark.

Q1. Feature Source and Bridge Placement

For both OpenVLA-OFT and VLA-Adapter, HVLH_{VL} comprises the VLM's embedding-stage visual and language representations (hidden_states[0]), excluding action tokens.

The Bridge attends to these perception representations and produces action-aware tokens, which are projected and injected through a learnable gate immediately after the Action Expert's input projection and activation, before its first residual block. The Bridge thus provides action-aware initialization within the separate Action Expert, without modifying the VLM Transformer layers.

Q2. Generalization of Action Representations

We evaluate representation transfer on 1,000 LIBERO-Plus Spatial observations using frozen models and linear probes fitted with Ridge regression exclusively on standard LIBERO-Spatial demonstrations. Neither the models nor the probes are adapted to LIBERO-Plus.

Action decodability. On LIBERO-Plus, PAIR Action Tokens achieve a zero-shot R2R^2 of 0.312, compared with −0.117 for baseline Action Tokens. Bridge Tokens already achieve R2=0.243R^2 = 0.243 before entering the Action Expert, indicating that action information is accessible at the perception–action interface.

Action geometry. We measure alignment with ground-truth action geometry using RSA, linear CKA, and cross-trajectory kNN overlap. The following results report the observed score minus a trajectory-preserving permutation null.

RepresentationRSA ↑CKA ↑kNN ↑
Baseline Action Token0.2360.2290.207
Bridge Token0.4790.4780.369
PAIR Action Token0.5000.5000.317

PAIR Action Tokens outperform the baseline on all three metrics across all 10 tasks (exact two-sided task-level paired sign-flip test, p=0.00195p = 0.00195 for each metric). Bridge Tokens also exhibit stronger action-aligned geometry than the baseline's final Action Tokens.

Together, these results show that PAIR's action-aware representations retain both action decodability and geometric structure under LIBERO-Plus observation shifts, without updating the policy or refitting the linear probes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.