acceptodds
Under review as a conference paper at ICLR 2027

Pelican-VLA 0.5: Inducing Transferable Manipulation-Centric Attention via Bottleneck Tokens

Abstract

Generalization across objects, scenes, tasks, and embodiments remains a central challenge for Vision-Language-Action (VLA) models, which still rely heavily on task- and environment-specific robot data. We introduce Pelican-VLA 0.5, a unified model for vision-language understanding, future-frame prediction, and action generation. At its core is a compact set of learnable Bottleneck Tokens (BotTokens), through which information is routed to the action pathway. This bottlenecked perception-to-action routing gives rise to attention-level generalization: without object annotations, segmentation masks, or explicit attention supervision, the model develops manipulation-centric attention during pre-training. Even before task-specific fine-tuning, its action pathway can localize task-relevant objects and contact regions across unseen scenes and robot embodiments. After fine-tuning, Pelican-VLA 0.5 achieves 98.6% success on LIBERO, 91.4% on RoboTwin Clean and 91.0% on RoboTwin Randomized. Across three real-robot embodiments, it performs comparably to under the same evaluation setting. Moreover, we find that its attention patterns remain highly consistent before and after fine-tuning, providing further evidence that the manipulation-centric attention acquired during pre-training is transferable. At the same time, the attention-to-action gap between correctly attending to manipulation-relevant regions and successfully executing actions remains a key obstacle to developing broadly generalizable robot policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Bottleneck Tokens, Manipulation-Centric Attention, Attention-Level Generalization, VLA

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_YpRAOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Pelican-VLA 0.5, which uses a set of K=32 learnable Bottleneck Tokens (BotTokens) within a unified Qwen3-VL 4B backbone to connect perception (vision-language + Cosmos latent tokens) with action generation. The authors set up this bottleneck using curriculum attention masking, orthogonality regularization, and token-gated generation. They argue that restricting the architecture this way brings about "attention-level generalization," where manipulation-centric attention patterns show up during pre-training without direct supervision and generalize across various objects, scenes, and robot embodiments. After fine-tuning, the model reaches 98.6% on LIBERO, 91%+ on RoboTwin, and performs on par with π0.5 on three real-world robots. Finally, the paper notes an "attention-to-action gap," showing that having the right attention does not always lead to successful execution, particularly for zero-shot precise manipulation.

Critical strengths
  1. The paper proposes a new, well-motivated VLA architecture that achieves state-of-the-art results on the reported simulation benchmarks. It includes several architectural and training design choices, supported by a broad set of ablation studies and downstream evaluations.
  2. The paper is clearly written and easy to follow, and it presents several insightful observations.
Critical weaknesses
  1. Pelican-VLA 0.5 produces more focused, manipulation-relevant attention, but its connection to downstream performance remains insufficiently established. Table 2 provides some supporting evidence, and the paper acknowledges an attention-to-action gap. However, the component ablations mainly examine attention patterns rather than task success, leaving their practical contributions unclear. I would consider raising my score if the authors provide convincing evidence of a strong correlation between better attention alignment and downstream task performance.
  2. The choice of 32 BotTokens is not supported by a token-count ablation or a scaling experiment. It remains unclear how bottleneck capacity affects performance and whether this setting remains sufficient for more complex tasks or larger backbones.
Questions for authors
  1. Could the fixed bottleneck discard task-relevant information and limit performance? How would varying the number of BotTokens affect task success and inference cost?
  2. Beyond Table 2, could the authors report end-to-end task success for the configurations in Section 5.2, in simulation or on real robots? Reporting attention alignment alongside success rates would help clarify whether attention improvements correspond to better task performance.
1 comment

We really appreciate the reviewer for the detailed and constructive feedback.

Q1. Bottleneck capacity and inference cost.

We compared our model against strong baseline models without an information bottleneck in the RoboTwin, LIBERO, and RoboDojo simulation environments. Detailed results are provided in our response to Q2. Using a fixed bottleneck preserves model performance while also enabling generalization at the attention level.

We randomly selected 10 tasks from RoboTwin and conducted ablations on this subset.

ConfigurationClean SR (%)Randomized SR (%)Average SR (%)
w/o BotTokens75.875.775.8
K=875.175.475.3
K=1675.377.376.3
K=3278.577.277.9
K=6476.977.177.0

On the evaluated 10-task RoboTwin subset, K=32 achieves the highest average success rate, exceeding the configuration without BotTokens by 2.1 percentage points. K=8 performs slightly worse than the model without the bottleneck. These results support the choice of a moderate interface capacity and suggest that an overly narrow interface can indeed limit manipulation performance.

We also measured inference latency in a real-world setting. Under this measurement setting, adding BotTokens reduces the mean latency by 13.7 ms, approximately 6.1%.

ConfigurationMean latency (ms)SD (ms)
With BotTokens209.51.12
Without BotTokens223.21.55

Q2. End-to-end task success and its relationship to attention alignment.

In simulation benchmarks, Table 1 reports comparisons between Pelican-VLA 0.5 and baselines on RoboTwin, while Table 5 reports comparisons on the LIBERO simulation benchmark. Pelican-VLA 0.5 achieves 98.6% success on LIBERO, 91.4% on RoboTwin Clean, and 91.0% on RoboTwin Randomized. To further demonstrate the effectiveness of Pelican-VLA, we also evaluated it on the more challenging RoboDojo simulation benchmark.

ModelGeneralization (Score/SR)Precision (Score/SR)Long horizon (Score/SR)Memory (Score/SR)Open (Score/SR)Average (Score/SR)
StarVLA3.94/2.34%9.9/4.33%14.15/6.5%3.34/2.44%0.67/0.58%6.4/3.24%
VLAct9.54/6.28%20.57/15.17%20.12/13.67%0.66/0.56%2.37/2.25%10.65/7.58%
π0.513.38/8.17%12.40/5.50%23.54/14.67%5.89/4.67%1.98/1.67%11.44/6.93%
Hy-Embodied-0.511.78/8.39%13.81/8.00%25.74/14.92%13.37/12.11%0.65/0.58%13.07/8.8%
Xiaomi-Robotics-123.55/17.00%26.69/18.83%38.39/23.67%7.81/6.56%3.94/3.58%20.07/13.93%
GalaxeaVLA18.46/12.83%28.25/20.42%44.12/32.25%8.61/7.33%1.73/1.58%20.23/14.88%
Pelican-VLA19.47/13.67%22.31/16.50%31.21/19.5%36.2/35.33%1.68/1.5%22.17/17.30%

Figure 3 reports Pelican-VLA's real-world success rates. Overall, Pelican-VLA 0.5 achieves an average success rate of 63.6% in the original setting, compared with 61.8% for π0.5, indicating similar overall performance. In the perturbed setting, the average success rates are 52.0% and 40.9%, respectively. Both methods degrade under scene perturbations, with an average decrease of 11.6 percentage points for Pelican-VLA 0.5 and 20.9 points for π0.5.

To further examine the behavioral relevance of manipulation-centric attention, we deployed the scaled-up pretrained policy on six real-world tabletop tasks involving unseen objects, without task-specific fine-tuning. The original attention analyses show that the model attends to instruction-relevant objects and manipulation regions before task adaptation. These additional experiments evaluate whether the pretrained policy can translate such object grounding into actual object manipulation. We used the TienKung embodiment included in pretraining and conducted 20 trials per task.

TaskSuccessful trials / 20SR (%)
Pick up a tennis ball6 / 2030
Pick up a cola bottle9 / 2045
Pick up a toy16 / 2080
Move a tennis ball from one plate to another4 / 2020
Place a cola bottle on a plate4 / 2020
Move a toy from one plate to another14 / 2070

The policy successfully completes 53 of 120 trials, achieving an average success rate of 44.2%. These behavioral results complement the object-directed attention observed in the original manuscript: the pretrained model exhibits manipulation-relevant visual grounding and can successfully act on unseen objects without task-specific fine-tuning.

We hope that these responses address the reviewer's concerns, and we are happy to answer any further questions.

Reviewer_2ygVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors analyse a phenomenon common to existing VLA models, whereby the action module tends to focus on task-irrelevant visual information at the attention level, and link this phenomenon to the poor generalisation of existing methods across objects, scenes and tasks. To address this issue, the method proposed in this paper compresses visual and textual information into a finite number of bottleneck tokens, such that the action module can only extract information from these limited compressed tokens. This design forces the model to extract task-relevant visual information. This method also introduces future-frame prediction, a language contrastive loss, and progressive training. The authors validated the effectiveness of this method on the LIBERO and RoboTwin datasets. They also conducted validation on nine tasks involving three robot embodiments in real-world scenarios, achieving favourable results. The authors further conducted a series of analyses and experiments, reaching the following conclusion: the model’s ability to focus on task-relevant regions stems not from vast amounts of data, but rather from the effectiveness of the bottleneck design. Furthermore, when this design is transferred to MoT, it also enables attention to be focused on task-relevant objects. During training, the model’s attention becomes increasingly concentrated on task-relevant targets; even when the bottleneck token is ultimately removed, this focus remains intact. The authors analysed the model before and after fine-tuning and found that attention remained largely stable throughout. They conclude that pre-training has essentially enabled the model to recognise manipulation targets, whilst fine-tuning primarily improves the mapping from representations to actions.

Critical strengths

This paper identifies a common issue with VLA models and, building on this, proposes an effective ‘bottleneck token’ to mitigate the problem. The structure is simple and effective, and the authors’ experiments also suggest that the bottleneck token has the potential to be extended to other architectures.

Critical weaknesses
  1. This paper argues that directing attention towards regions of an image relevant to the action can improve model performance; however, there is no necessary link between this focusing capability and improved performance—a more concentrated heatmap does not necessarily equate to a more effective strategy. The existing results merely support the notion that this method enables attention to be more focused on the object of the action, but are insufficient to prove whether the performance improvement observed in this method stems from attention focusing or from other design modules.

  2. The comparative experiments in this paper are limited to VLA methods; it is recommended that the latest WAM methods, such as Fast-WAM and Action Experience Dictionary (AED) for WAM, be included.

  3. In this paper, the bottleneck tokens are pooled to generate a gating mechanism used to modulate future Cosmos generation. Whether the design of this structure is reasonable and effective requires further analysis or experimentation by the authors.

Questions for authors
  1. The ablation experiments in Section 5.2 of this paper analyse only the visualisation results of heatmaps; it is recommended that the authors also present the success rates to clearly illustrate the contribution of each module to the experimental results. Similarly, the architecture transfer experiments present only visualisations of attention; the success rates should also be included. The same applies to the fine-tuning experiments and those involving the removal of the bottleneck token after training. It is recommended that success rates be included.

  2. The paper states that a KV cache is used to accelerate inference, but does not analyse the actual inference latency. It is recommended that the authors include this experiment to clearly demonstrate the inference latency.

4 comments
AuthorsOpenReview ↗

Response 1

We really appreciate the reviewer for the detailed and constructive feedback.

W1. Relationship between manipulation-centric attention and task performance.

Section 5.2 and Figure 5 examine how BotTokens-mediated routing affects attention formation, while Figure 6 shows that language contrastive alignment makes this attention instruction-selective: changing the instruction shifts attention from the bowl to the cabinet within the same scene. The component success-rate ablations reported in Q1 below further show that removing BotTokens, future-frame prediction, or language contrastive alignment reduces average manipulation success by 2.1, 2.7, and 1.1 percentage points, respectively.

The original zero-shot evaluation in Table 2 provides a direct behavioral comparison with and without BotTokens before task-specific fine-tuning:

ConfigurationShake bottle horizontallyShake bottleClick bellOpen microwaveClick alarmclockAverage SR (%)
VLA w/o BotTokens31.027.027.018.020.024.6
Pelican-VLA 0.549.045.042.037.025.039.6

All entries are success rates (%). Pelican-VLA improves success on all five tasks, with an average gain of 15.0 percentage points. This evaluates the behavioral utility of the bottleneck-mediated architecture beyond attention visualization.

We additionally deployed the scaled-up pretrained policy on six real-world tasks involving unseen objects, without task-specific fine-tuning. These experiments use the TienKung embodiment included in pretraining and evaluate object generalization on this embodiment, with 20 trials per task:

TaskSuccessful trials / 20SR (%)
Pick up a tennis ball6 / 2030
Pick up a cola bottle9 / 2045
Pick up a toy16 / 2080
Move a tennis ball from one plate to another4 / 2020
Place a cola bottle on a plate4 / 2020
Move a toy from one plate to another14 / 2070

The policy succeeds in 53 of 120 trials, achieving 44.2% average success. These tasks require identifying the instructed object and executing object-specific interactions, including grasping, transporting, and placement. The results complement the object-directed attention analyses in the original manuscript with successful manipulation of unseen objects.

W2. Comparison with World Action Models.

We have expanded the comparison to include Fast-WAM, HWM-CogWAM, EWAM and OpenWAM. We additionally evaluate Pelican-VLA on RoboDojo using the official benchmark training dataset.

ModelGeneralization (Score/SR)Precision (Score/SR)Long horizon (Score/SR)Memory (Score/SR)Open (Score/SR)Average (Score/SR)
StarVLA3.94/2.34%9.9/4.33%14.15/6.5%3.34/2.44%0.67/0.58%6.4/3.24%
VLAct9.54/6.28%20.57/15.17%20.12/13.67%0.66/0.56%2.37/2.25%10.65/7.58%
π0.513.38/8.17%12.40/5.50%23.54/14.67%5.89/4.67%1.98/1.67%11.44/6.93%
Hy-Embodied-0.511.78/8.39%13.81/8.00%25.74/14.92%13.37/12.11%0.65/0.58%13.07/8.8%
Xiaomi-Robotics-123.55/17.00%26.69/18.83%38.39/23.67%7.81/6.56%3.94/3.58%20.07/13.93%
GalaxeaVLA18.46/12.83%28.25/20.42%44.12/32.25%8.61/7.33%1.73/1.58%20.23/14.88%
Fast-WAM2.34 / 1.11%1.96 / 0.00%9.14 / 5.17%3.55 / 3.44%0.42 / 0.42%3.48 / 2.03%
HWM-CogWAM12.78/9.28%20.54/14.58%27.35/18.17%8.18/7.00%2.86/2.67%14.34/10.34%
EWAM15.12/9.67%22.15/12.33%30.23/18.08%7.39/6.00%1.12/0.75%15.20/9.37%
OpenWAM20.71/14.84%18.45/9.25%34.93/25.33%10.41/9.11%1.41/1.08%17.18/11.92%
Pelican-VLA 0.519.47 / 13.67%22.31 / 16.50%31.21 / 19.50%36.20 / 35.33%1.68 / 1.50%22.17 / 17.30%

Pelican-VLA achieves an average Score of 22.17 and SR of 17.30%, while the externally reported Fast-WAM entry has an average Score of 3.48 and SR of 2.03%.

AuthorsOpenReview ↗

Response 2

We really appreciate the reviewer for the detailed and constructive feedback.

W3. Rationale and evidence for BotTokens-gated future-frame prediction.

The gate connects the compact manipulation representation to future-scene prediction. As described in Section 3.3, the BotTokens outputs are mean-pooled to construct a channel-wise gating signal, which modulates the generation features before decoding. Mean pooling provides a compact conditioning representation, while the generation features retain their token or spatial structure.

As detailed in Appendix A.2, the predicted future representation is supervised against the future observation encoded in the frozen Cosmos latent space. Because the gate is computed from BotTokens outputs, the future-prediction objective supplies a gradient pathway to the bottleneck representation through the gate, connecting the compact interface to the prediction of future scene changes.

The additional success-rate ablation provides behavioral evidence for the future-prediction branch: removing future-frame prediction reduces average success from 77.9% to 75.2%, a decrease of 2.7 percentage points, with reductions in both Clean (78.5% to 75.0%) and Randomized (77.2% to 75.3%) settings. The gating design explains how this auxiliary objective is connected to the bottleneck, and the branch-level ablation quantifies the contribution of future prediction to end-to-end success.

Q1. Success-rate ablation.

We conducted module ablations on 10 RoboTwin tasks.

ConfigurationClean SR (%)Randomized SR (%)Average SR (%)
w/o future-frame prediction75.075.375.2
w/o language contrastive loss77.576.076.8
w/o BotTokens75.875.775.8
Full model78.577.277.9

Removing future-frame prediction, the language contrastive loss, and BotTokens reduces the average success rate by 2.7, 1.1, and 2.1 percentage points, respectively. These results show that the effects of the module ablations are also reflected in actual manipulation performance, beyond changes in attention heatmaps.

Q2. Inference latency.

We have added latency measurements in the real deployment environment, with 20 runs each for the configurations with and without BotTokens. Under this setting, the BotTokens configuration reduces the mean latency by 13.7 ms, approximately 6.1%.

ConfigurationMean latency (ms)SD (ms)
With BotTokens209.51.12
Without BotTokens223.21.55

We hope that these responses address the reviewer's concerns, and we are happy to answer any further questions.

Reviewer_2ygVOpenReview ↗

Comment

Thank you for the detailed response and the additional experiments. I appreciate the new results. The latency measurement resolves my question on inference speed, the success-rate ablation on RoboTwin quantifies the contribution of each module, and the comparison on RoboDojo now includes several WAM baselines.

However, some of my main concerns remain. First, the new results show that BotTokens are useful for task performance, but they do not establish that the gain comes from more focused attention. After fine-tuning, removing BotTokens lowers the average success rate by 2.1 points, which is smaller than the drop from removing future-frame prediction (2.7 points), so the link between attention concentration and performance is still indirect. Second, the gating design is supported only by the ablation that removes the entire future-prediction branch. There is no comparison with an ungated variant or with an alternative way of conditioning, so the benefit of the gate itself is not verified. Third, success rates are still missing for the architecture-transfer experiment, the fine-tuning analysis, and the removal of the bottleneck tokens after training, which are currently supported only by attention visualizations. In addition, on RoboDojo the average advantage is driven mainly by the Memory category, while the method is not the best on Generalization, Precision, or Long horizon.

Given these points, I will keep my current rating.

Thank you for clarifying your remaining concerns. The central contribution of our paper is that routing perceptual information to the action pathway through BotTokens can induce attention aligned with manipulation targets during pretraining, without explicit spatial supervision. This behavior emerges before task-specific fine-tuning.

Attention evidence. In §5.1, the dense MoT baseline uses the same data, optimizer, training schedule, and parameter budget. It achieves a comparable action training loss, while its attention remains more diffuse. The component comparisons in Fig. 5 show that attention becomes more focused on the manipulation target when perception is routed through BotTokens. With language contrastive alignment, changing only the instruction in the same scene shifts attention from the bowl to the cabinet (Fig. 6). We observe a similar pattern of target alignment when applying the same routing to a MoT-style architecture (Fig. 7).

Task success evidence. In Table 2, the configuration with BotTokens improves success on all five zero-shot RoboTwin tasks, raising the mean success rate from 24.6% to 39.6%. In the fine-tuned ablation on 10 RoboTwin tasks added during rebuttal, the mean success rate increases from 75.8% to 77.9%. Across the attention comparisons and rollout ablations, the configuration with BotTokens exhibits both more focused attention on the target and higher task success. These findings suggest that better target alignment may help improve task success.

Additional dense MoT results. The dense MoT without BotTokens achieves end-to-end success rates of 86.8% on RoboTwin Clean and 87.3% on RoboTwin Randomized. For reference, the original paper reports 91.4% and 91.0%, respectively, for Pelican-VLA 0.5.

Reviewer_v24VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Pelican-VLA 0.5, a unified VLA model that routes perception-to-action information through a small set of Bottleneck Tokens (BotTokens), together with future-frame prediction and language-alignment objectives. The paper argues that this bottleneck induces manipulation-centric attention before task-specific fine-tuning, and reports strong results on LIBERO, RoboTwin 2.0, and nine real-robot tasks across three embodiments.

Critical strengths
  • The BotToken interface is simple, interpretable, and potentially useful for both information routing and efficient action decoding.

  • The evaluation spans simulation, zero-shot transfer, and real-robot experiments across multiple embodiments.

Critical weaknesses
  • The novelty of BotTokens is insufficiently distinguished from existing learnable latent-action interfaces, e.g., [A][B]. [A] InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation, ICLR 2026 [B] LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queries, ICML 2026.

  • The compactness of the proposed bottleneck is not adequately studied. All experiments appear to use a fixed number of 32 BotTokens, without varying the bottleneck capacity or comparing against simpler compact interfaces such as pooling or latent queries. Consequently, it remains unclear whether the reported effect is caused by a genuinely narrow information bottleneck, or merely by introducing learnable intermediate tokens and changing the attention topology.

Questions for authors

Please refer to comments in weaknesses.

1 comment

We really appreciate the reviewer for the detailed and constructive feedback.

W1. Novelty relative to latent-action interfaces.

InstructVLA uses learnable action queries to extract latent actions from a VLM. These latent actions condition the visual features produced by a separate DINOv2 encoder through FiLM before they are passed to an action expert, serving as an interface between high-level intent and low-level control. LangForce's core mechanism rearranges the order of queries and language tokens to construct a visual prior and a language-conditioned posterior, respectively, and uses a likelihood-ratio objective to mitigate the policy's tendency to ignore language. Pelican-VLA establishes a manipulation-oriented, structured information bottleneck through BotTokens within a shared Transformer. It blocks the action pathway's direct access to upstream perceptual tokens, requiring perceptual information to pass through a compact interface before entering the action pathway. Orthogonality regularization encourages complementary encoding, language contrastive alignment enhances instruction selectivity, and a gating mechanism enables the bottleneck representations to support future-scene prediction. Pelican-VLA connects bottleneck-mediated information transfer to the emergence of transferable manipulation-centric attention. Without explicit spatial attention supervision, the model develops attention to task targets and contact regions during pretraining, exhibits attention transfer to unseen scenes and robot embodiments, and retains highly consistent attention after task-specific fine-tuning.

We additionally deployed pelicanvla0.5 directly on the TienKung embodiment included in pretraining, without task-specific fine-tuning. We evaluated six tabletop tasks involving unseen objects, with 20 trials per task. These results provide additional evidence of the persistence of manipulation-centric attention under scaling and the ability to manipulate unseen objects.

TaskSuccessful trials / 20SR (%)
Pick up a tennis ball6 / 2030
Pick up a cola bottle9 / 2045
Pick up a toy16 / 2080
Move a tennis ball from one plate to another4 / 2020
Place a cola bottle on a plate4 / 2020
Move a toy from one plate to another14 / 2070

W2. Bottleneck compactness and capacity.

In Figure 1, we evaluated several strong VLA models, including LingBot-VLA, which also incorporates similar learnable intermediate tokens. The results show that its attention does not concentrate on manipulation regions and the objects being manipulated. In contrast, without explicit spatial attention supervision, Pelican-VLA develops attention to task targets and contact regions during pretraining, exhibits attention transfer to unseen scenes and robot embodiments, and retains highly consistent attention after task-specific fine-tuning.

To investigate interface capacity, we compared K=8/16/32/64 on the same set of 10 RoboTwin tasks. The average success rate without BotTokens is 75.8%, while the four capacities achieve 75.3%, 76.3%, 77.9%, and 77.0%, respectively. K=32 performs best. A narrower interface may discard task-relevant information, while a wider interface does not yield a higher success rate. These results show that interface capacity affects manipulation performance and that the gains do not increase monotonically with the number of tokens.

ConfigurationClean SR (%)Randomized SR (%)Average SR (%)
w/o BotTokens75.875.775.8
K=875.175.475.3
K=1675.377.376.3
K=3278.577.277.9
K=6476.977.177.0

We hope that these responses address the reviewer's concerns, and we are happy to answer any further questions.

Reviewer_S7C5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Pelican-VLA 0.5, a unified VLA model built on Qwen3-VL-4B. Its key component is a set of learnable Bottleneck Tokens that constrain information flow from perception to action, aiming to induce manipulation-centric attention without explicit spatial supervision. The model jointly optimizes action prediction, future-frame prediction, and language alignment. Experiments show competitive results on LIBERO (98.6%), RoboTwin (91.4%/91.0%), and real-world manipulation tasks across three robot embodiments. The authors further analyze attention patterns before and after fine-tuning to investigate representation transferability.

Critical strengths
  1. The paper investigates an important problem: how VLA models learn manipulation-relevant visual representations. The proposed bottleneck routing mechanism is simple and provides an interesting perspective on perception-to-action information flow.

  2. The experimental results are strong overall, particularly on RoboTwin and real-world manipulation tasks. The evaluation across three robot embodiments, together with attention visualizations and component ablations, provides useful evidence for the proposed approach.

Critical weaknesses
  1. Unclear necessity. Learnable proxy tokens and information bottlenecks have been explored in Perceiver, BLIP-2, RT-1 (TokenLearner), and ViT registers. The paper does not clearly establish whether the benefits persist with larger-scale training.

  2. Insufficient evidence for transferable representations. The main claim relies heavily on attention visualizations, with a target-object IoU of only 0.124 and quantitative analysis limited to one scene. More concentrated attention does not necessarily imply better representations, and comprehensive performance ablations are missing.

Questions for authors

I would be happy to raise my rating if the authors adequately address my concerns.

  1. Bottleneck design. How does the proposed mechanism fundamentally differ from prior learnable latent/query-token approaches such as BLIP-2, Perceiver, and RT-1 TokenLearner? Why are exactly 32 BotTokens used? An ablation with 8/16/32/64 tokens and comparisons with simpler token aggregation mechanisms would be valuable.

  2. Scaling and representation learning. The model is pretrained for only 0.4 epoch. Could the observed improvements mainly result from the bottleneck acting as a shortcut or inductive bias under limited training? Does its advantage persist with larger data and training budgets?

  3. Stronger baselines and performance ablations. I suggest comparing against recent strong VLA approaches such as StarVLA and VLAct, particularly since VLAct also focuses on transferable VLA representations. More importantly, please report success-rate ablations with and without BotTokens on the main fine-tuned benchmarks, rather than relying primarily on attention visualizations.

  4. Training configuration and efficiency. Please report the total GPU-hours, effective global batch size, and inference latency/control frequency. The reported per-GPU batch size is only 6; given evidence that batch size can affect VLA performance, a batch-size ablation would help establish the robustness of the results.

4 comments
AuthorsOpenReview ↗

Response 1

We really appreciate the reviewer for the detailed and constructive feedback.

Q1. Bottleneck design.

BLIP-2 uses a separate Q-Former to extract visual representations suitable for language understanding and generation from a frozen image encoder and feed them into a frozen LLM. Perceiver maps high-dimensional inputs into a smaller latent array to decouple input size from network depth. RT-1's TokenLearner aggregates language-conditioned visual features before the action Transformer. Pelican-VLA uses BotTokens to establish information bottleneck between perception and action, encouraging the model to selectively retain perceptual evidence relevant to the instruction and action generation. The bottleneck representations support both action generation and future-scene prediction. Orthogonality regularization encourages complementary encoding, language contrastive alignment enhances instruction selectivity, and a future-frame generation gate connects the bottleneck representations to scene-change prediction. This design promotes the emergence of manipulation-centric attention during pretraining, allowing attention to shift between targets according to the instruction, transfer to unseen scenes and robot embodiments, and remain highly consistent after task-specific fine-tuning.

ConfigurationClean SR (%)Randomized SR (%)Average SR (%)
w/o BotTokens75.875.775.8
K=875.175.475.3
K=1675.377.376.3
K=3278.577.277.9
K=6476.977.177.0

The number of BotTokens, 32, is a hyperparameter. We compared K=8/16/32/64 on the same set of 10 RoboTwin tasks. K=32 achieves the highest average success rate, exceeding the configuration without BotTokens by 2.1 percentage points. K=8 performs slightly worse than the configuration without the bottleneck, while increasing the number of tokens to 64 does not further improve performance.

Q2. Scaling and representation learning.

In experiments with a further expanded pretraining dataset, we continue to observe manipulation-centric attention in both seen and unseen scenes. As training progresses, attention becomes more concentrated on manipulation regions. This indicates that the phenomenon also occurs under scaled-up training. We additionally deployed the scaled-up pretrained policy directly on the TienKung embodiment included in pretraining, without task-specific fine-tuning. We evaluated six tabletop tasks involving unseen objects, with 20 trials per task. These results provide additional evidence of the persistence of manipulation-centric attention under scaling and the ability to manipulate unseen objects.

TaskSuccessful trials / 20SR (%)
Pick up a tennis ball6 / 2030
Pick up a cola bottle9 / 2045
Pick up a toy16 / 2080
Move a tennis ball from one plate to another4 / 2020
Place a cola bottle on a plate4 / 2020
Move a toy from one plate to another14 / 2070
AuthorsOpenReview ↗

Response 2

We really appreciate the reviewer for the detailed and constructive feedback.

Q3. Stronger baselines and performance ablations.

We have added RoboTwin results for StarVLA and VLAct, along with success-rate ablations of our model.

ModelClean SR (%)Randomized SR (%)Average SR (%)
StarVLA88.288.388.3
VLAct (π-style)93.088.890.9
Pelican-VLA 0.591.491.091.2

Pelican-VLA's average success rate exceeds those of StarVLA and VLAct by 2.9 and 0.3 percentage points, respectively. VLAct performs better in the Clean setting, while Pelican-VLA performs better in the Randomized setting. These results demonstrate that Pelican-VLA is competitive with recent strong baselines.

Table2 includes a behavioral comparison with and without BotTokens in the zero-shot setting. Across the five evaluated RoboTwin tasks, Pelican-VLA 0.5 achieves an average success rate of 39.6%, compared with 24.6% without BotTokens, a gain of 15.0 percentage points. To complement this zero-shot comparison, we additionally evaluated component ablations after task-specific fine-tuning on a subset of 10 RoboTwin tasks, as reported below.

ConfigurationClean SR (%)Randomized SR (%)Average SR (%)
w/o future-frame prediction75.075.375.2
w/o language contrastive loss77.576.076.8
w/o BotTokens75.875.775.8
Full model78.577.277.9

The full model exceeds the configuration without BotTokens by 2.1 percentage points. Removing future-frame prediction or the language contrastive loss reduces the average success rate by 2.7 and 1.1 percentage points, respectively. These end-to-end results complement the attention analyses in the original manuscript and show that the corresponding modules also contribute to manipulation success on the evaluated tasks.

To further demonstrate Pelican-VLA's performance, we also conducted experiments on the more challenging RoboDojo simulation benchmark, using only the official dataset for training.

ModelGeneralization (Score/SR)Precision (Score/SR)Long horizon (Score/SR)Memory (Score/SR)Open (Score/SR)Average (Score/SR)
StarVLA3.94/2.34%9.9/4.33%14.15/6.5%3.34/2.44%0.67/0.58%6.4/3.24%
VLAct9.54/6.28%20.57/15.17%20.12/13.67%0.66/0.56%2.37/2.25%10.65/7.58%
π0.513.38/8.17%12.40/5.50%23.54/14.67%5.89/4.67%1.98/1.67%11.44/6.93%
Hy-Embodied-0.511.78/8.39%13.81/8.00%25.74/14.92%13.37/12.11%0.65/0.58%13.07/8.8%
Xiaomi-Robotics-123.55/17.00%26.69/18.83%38.39/23.67%7.81/6.56%3.94/3.58%20.07/13.93%
GalaxeaVLA18.46/12.83%28.25/20.42%44.12/32.25%8.61/7.33%1.73/1.58%20.23/14.88%
Pelican-VLA19.47/13.67%22.31/16.50%31.21/19.5%36.2/35.33%1.68/1.5%22.17/17.30%

Q4. Training configuration and efficiency.

During pretraining, we used 256 GPUs with a global batch size of 1,536. The per-GPU batch size was set to 6 to enable rapid training iterations. In the real deployment environment, we measured latency over 20 runs each for the configurations with and without BotTokens:

ConfigurationMean latency (ms)SD (ms)
With BotTokens209.51.12
Without BotTokens223.21.55

Under this measurement setting, adding BotTokens reduces the mean latency by 13.7 ms, approximately 6.1%.

We hope that these responses address the reviewer's concerns, and we are happy to answer any further questions.

Thank you to the authors for the detailed rebuttal and additional experiments. I appreciate the effort, particularly the new ablation studies, stronger baselines, and RoboDojo results, which address several of my concerns.

However, I still have a few remaining concerns:

  1. Scaling: The authors show that manipulation-centric attention persists with larger-scale training, but it remains unclear whether the same architecture without BotTokens, trained under an identical scaling budget, could achieve comparable results.
  2. Representation transferability: The quantitative attention evaluation remains limited, and more concentrated attention does not necessarily imply more transferable action representations.
  3. Bottleneck effectiveness: The improvement from BotTokens is relatively modest (+2.1 percentage points on the 10-task RoboTwin ablation), and its statistical significance remains unclear.

Given the limited time remaining in the discussion period, I am concerned that the authors may not have sufficient time to address further questions. Nevertheless, the additional experiments provide reasonably solid evidence supporting the proposed approach.

Therefore, I have decided to raise my rating to 3, while maintaining the above reservations. I encourage the authors to further discuss these limitations in the final version.

Thank you for your constructive feedback and for updating your rating.

Training scale. In an earlier experiment at the larger data scale, we trained a variant without BotTokens on the same pretraining data and examined its attention at the 100k-step checkpoint. At that point, the model’s attention was more diffuse and showed no clear focus on the manipulation target. For a future version, we plan to conduct a larger-scale ablation comparing models with and without BotTokens under the same expanded pretraining budget.

Representation transferability. In our new zero-shot experiments with unseen objects, we visualized attention at each step of the policy rollout. In the trajectories we examined, attention focused on the object specified by the instruction early in the rollout and became more concentrated as the robot arm approached it. These observations support the transfer of target localization to unseen objects without task-specific fine-tuning. We will include the step-by-step visualizations in the revision.

Performance gains. We will clarify that the 2.1 percentage-point gain is the observed difference in success rates after fine-tuning on 10 tasks. In a future version, we will also report a larger-scale ablation comparing models with and without BotTokens under the same expanded pretraining budget.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.