acceptodds
Under review as a conference paper at ICLR 2027

SimVLA 2.0: Instruction-Conditioned Control without Language Pretraining

Abstract

Vision-Language-Action policies typically retain pretrained language components, yet on finite, recurring task inventories the role of that retention is rarely tested directly. Action training itself can erode the competence a pretrained backbone contributes, and what remains often fails to transfer under meaning-preserving paraphrases. We therefore build the opposite end of the design space: instructions remain, but no pretrained language-side component is loaded. SimVLA 2.0 loads no language model, text encoder, or token embedding: a fixed, parameter-free mapping indexes randomly initialized embeddings learned solely from action supervision, and a joint action Transformer conditions a continuous action head on the embeddings and the pretrained vision encoder's features. Across seven benchmark evaluations spanning six manipulation domains, it remains within the range of recent language-pretrained systems—LIBERO 96.9%, LIBERO-PRO 96.5% under semantic instruction perturbations, CALVIN ABCD a 4.24 mean chain length, WidowX 91.7%, Google Robot 82.0%, and RoboTwin 2.0 Easy/Hard 94.0%/87.2%. On the 2025 BEHAVIOR Challenge Public Validation leaderboard it places second among Standard-track entries. Ablations separate the sources of performance: instruction conditioning is essential, pretrained token embeddings provide no consistent advantage over action-trained codes, and strong visual pretraining remains important.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Embodied manipulation, Vision-language-action model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eqByOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SimVLA 2.0, a VLA model that drops all pretrained language components, and trains them from scatch end2end with action loss. The method is evaluated on multople sim benchmarks and achieves good performance. The ablations show that the instruction itself is important, but initializing from pretrained text encoders does not help, hence concluding that current robotics sim benchmarks do not require pretrained text encoders.

Critical strengths
  1. The ablation is clean and clealry supports that pretrained text encoders are not needed for current sim robotics benchmarks
  2. The evaluation is broad, covering seven sim benchmarks with, and the seed variance study in the appendix is appreciated.
Critical weaknesses
  1. The main motivation is not tested. The paper uses LIBERO-Para to argue that language-pretrained VLAs are fragile under paraphrase, but it does not evaluate on LIBERO-Para, while LIBERO-PRO is not discrimitive at all among the baselines tested. Also, since the embedding table is random, unseen words remain random vectors at test time. I suspect the robustness comes mostly from overlapping tokens and the visual context
  2. A simple yet very important baseline is missing: one learned embedding per unique instruction (a task ID). On benchmarks with a fixed set of tasks, such as LIBERO, CALVIN and BEHAVIOR, I would expect this to perform similarly. If it does, the conclusion is more about the benchmarks than about the proposed design
  3. On the LIBERO-PRO object-attribute perturbation, SimVLA 2.0 is clearly worse than pi-0.5 and openvla (80% vs. 96% and 93%), but the paper does not mention this. For the position perturbation, the improvement is presented against SimVLA, but pi-0.5 performs about the same. It seems results are selectively discussed.
  4. The reported LIBERO success rate in the main paper appears to be the best of three seeds rather than the mean, and the ablations also use a single seed. The SigLIP2 text encoder result is surprisingly bad, given that all open-sourced paligemma-based VLAs rely on it and can achieve good performance, which makes me wonder whether the baseline was tuned properly. There are no ablations on CALVIN, where the role of language might be more important.
Questions for authors

See weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer eqBy (Part 1)

Thank you for your detailed and insightful review. We sincerely hope our clarifications and new experimental results below will address your concerns.

(W1a, Q1 & Q3) LIBERO-Para, Training-Unseen Tokens, and Inference-Time Handling

“Can you report LIBERO-Para results for the default and the BERT/SigLIP2 variants, along with the OOV rate of paraphrase tokens?”

“How are unseen tokens handled at test time, and how often does this happen in LIBERO-PRO Sem and the SimplerEnv prompts?”

Thank you for suggesting this evaluation. We have added LIBERO-Para experiments for the default model and pretrained text variants, together with token-coverage analysis.

1. Evaluation and definition of training-OOV.
We evaluate each configuration using the LIBERO-Para protocol. Here, training-OOV means a token ID absent from the corresponding action-training instruction vocabulary, rather than a token outside the tokenizer vocabulary. Neither WordPiece nor SigLIP2 SentencePiece produces tokenizer-level [UNK] tokens on the LIBERO-Para prompts.

Text componentTraining-OOV token ratePara SR (%)Action (%)Object (%)Compositional (%)
Default random table39.94%60.1282.5374.5252.28
BERT embedding table39.94%47.7567.4763.7140.57
SigLIP2 embedding table41.34%44.1155.1777.6137.93
SigLIP2 text encoder41.34%42.2558.0552.5136.72

For context, the default’s 60.12% approaches the published X-VLA result (62.1%) and exceeds VLA-Adapter (46.3%).

2. Handling and prevalence of unseen tokens.
The default model uses a fixed WordPiece tokenizer and randomly initialized, action-trained embeddings. An unfamiliar word is first segmented into subwords. IDs encountered during training use their updated embeddings; IDs never encountered retain their individual initial random vectors under our zero-weight-decay setup. They are not mapped to a shared zero representation.

The training instructions contain 112 unique strings and 87 unique content WordPiece IDs. Coverage differs across evaluations:

EvaluationTraining-unseen content-token occurrencesAdditional coverage information
LIBERO-Para, WordPiece39.94%98.63% of prompts contain at least one unseen token
LIBERO-Para, SentencePiece41.34%99.14% of prompts contain at least one unseen token
LIBERO-PRO Sem candidate set, WordPiece15.51%157/1,012 token occurrences across 120 candidate paraphrases
SimplerEnv WidowX, WordPiece0%Evaluation token IDs are covered by the training vocabulary
SimplerEnv Google Robot, WordPiece0%Evaluation token IDs are covered by the training vocabulary

3. Performance under reduced lexical coverage.

Grouping LIBERO-Para episodes by the per-instruction share of training-unseen content tokens gives:

Training-unseen share of content tokensDefault random tableBERT embedding tableSigLIP2 embedding tableSigLIP2 text encoder
0%80.36%89.29%71.43%65.71%
(0%, 25%]79.47%76.83%67.56%55.85%
(25%, 50%]62.12%46.21%41.97%41.52%
>50%34.14%20.00%30.65%32.83%
Overall60.12%47.75%44.11%42.25%

The default maintains approximately 80% success when at most 25% of content tokens are training-unseen and achieves 34.14% when the proportion exceeds 50%. Thus, untrained token embeddings do not necessarily prevent successful execution, although higher unseen-token proportions are associated with lower success.

With the same WordPiece tokenizer and bucket assignments, BERT embeddings perform better in the zero-unseen-token group, while the default performs better in all three groups containing unseen tokens. The evaluated pretrained representations therefore do not provide a consistent advantage under these settings.

These results extend our evidence beyond entirely training-seen vocabulary. Our claim remains competitive instruction-conditioned control without pretrained language-side parameters within the evaluated task-bounded settings, rather than general open-vocabulary understanding.

AuthorsOpenReview ↗

Response to Reviewer eqBy (Part 2)

(W1b & Q2) Whole-Instruction Task-ID Baseline

“The paper is missing the obvious baseline: a whole-instruction task-ID embedding.”

“How does a whole-instruction task-ID baseline do on LIBERO, LIBERO-PRO Sem, CALVIN, and the WidowX Eggplant task?”

Thank you for suggesting this baseline. We added a whole-instruction Task-ID representation, assigning each registered training instruction string a learned 1,024-dimensional vector. This replaces the token sequence while retaining the vision encoder and action Transformer. Unseen complete strings map to a shared fallback representation, rather than being mapped to their underlying task identities.

EvaluationSimVLA 2.0 token-level modelWhole-instruction Task-ID
Standard LIBERO, average success (%)96.494.8
LIBERO-Para, success (%)60.125.74
LIBERO-PRO Sem, success (%)96.530.5
CALVIN ABC$\rightarrow$D, average chain length4.240.03
WidowX, average success (%)91.750.0
WidowX Eggplant, success (%)87.54.17

Standard LIBERO supports your observation that task identifiers can be effective: Task-ID reaches 94.8%, close to the token-level model’s 96.4%.

Under instruction rewording, however, exact-string lookup is insufficient: on LIBERO-Para, the token-level model achieves 60.12%, versus 5.74% for whole-instruction Task-ID. On CALVIN, all 34 evaluation strings are absent verbatim from the training inventory, explaining the baseline’s 0.03 average chain length. These results demonstrate a practical advantage of token-level conditioning over exact-string lookup, without establishing systematic compositional understanding.

We will include these results to clarify that whole-instruction task IDs support strong control on familiar instructions, while token-level conditioning handles instruction rewording more effectively than exact-string lookup.

(W1c) Composition, Visual Context, and Task-Perturbation Results

“The recombination claim rests on a single WidowX task at 24 episodes.”

“You can’t conclude from floor-level numbers that language pretraining ‘buys no compositional goal grounding.’”

Thank you for highlighting this distinction. We will replace “composition over action-trained token embeddings rather than whole-instruction memorization” with “successful execution under an unseen instruction string composed of training-seen tokens.”

The no-instruction results—6.4% on LIBERO-Goal and 20.8% on WidowX Eggplant—support the usefulness of instruction conditioning (Table 9 in the manuscript). They do not, however, determine whether the policy uses keywords or more structured relationships. Likewise, the new LIBERO-Para results demonstrate tolerance to rewording rather than replacing a systematic held-out recombination test.

We will remove the inference that low task-perturbation scores establish an absence of compositional benefits from language pretraining. Our central finding concerns control sufficiency in the evaluated task-bounded settings, which is compatible with reliance on familiar lexical cues.

(W2) Balanced Reporting of LIBERO-PRO Robustness

“The robustness results are selectively reported.”

“SimVLA 2.0 averages 80.0% (54% on Long) vs. 96.0% for π0.5\pi_{0.5} and 93.0% for OpenVLA.”

Thank you for highlighting this limitation. Although Table 8 reports these results, we will discuss the object-attribute weakness more explicitly and clarify the comparison on position robustness.

One possible explanation is sensitivity to changes in object appearance relative to the training demonstrations, with errors potentially accumulating over longer task sequences. Differences in visual representations, pretraining data, and augmentation may also contribute; the current cross-system comparison does not isolate these factors. Improving robustness to object-attribute changes is an important direction for future work, including controlled studies of visual augmentation and representation choices. We will revise the discussion to reflect this limitation.

AuthorsOpenReview ↗

Response to Reviewer eqBy (Part 3)

(W3b & Q4) Text-Tower Optimization and Three-Seed Ablations

“Was the tower frozen or fine-tuned, and was any tuning done for it? Can you run the ablations with 3 seeds?”

Thank you for highlighting the importance of text-specific optimization and repeated evaluation.

1. Training settings and additional tuning.
In Table 9, the full 24-layer SigLIP2 text tower and pretrained embedding tables were fine-tuned under the action objective. We remove the tower’s pooled projection head and pass its final-layer, 1,024-dimensional token features directly to the joint action Transformer, without an additional adapter.

We use AdamW with (β1,β2)=(0.9,0.95)(\beta_1,\beta_2)=(0.9,0.95) and zero weight decay. Only the action heads are trained during the first 1,000 updates. Thereafter, the pretrained variants use the same text learning rate as the default random-embedding model on each benchmark:

lrtext=lrbase×coeftext.\mathrm{lr}_{\mathrm{text}}=\mathrm{lr}_{\mathrm{base}}\times\mathrm{coef}_{\mathrm{text}}.

The original settings were selected for the default model (coeftext=1\mathrm{coef}_{\mathrm{text}}=1) and reused without separate tuning for the pretrained variants.

To address this concern, we subsequently swept coeftext∈1.0,0.1,0.05,0.0\mathrm{coef}_{\mathrm{text}}\in\\{1.0,0.1,0.05,0.0\\}, keeping other training settings fixed within each sweep; zero freezes the text component. We also extended component ablations to CALVIN (please see W3c below).

2. Three-seed results.
Following the learning-rate sweep over coeftext∈1.0,0.1,0.05,0.0\mathrm{coef}_{\mathrm{text}}\in\\{1.0,0.1,0.05,0.0\\}, we repeated the selected configuration for each text variant on LIBERO and WidowX using seeds 0, 1, and 2. The tables report mean ± standard deviation.

(a) LIBERO (%)

ConfigurationSpatialObjectGoalLongAvg.
Default98.4±0.299.1±0.496.6±0.691.8±0.396.5±0.3
BERT embedding table99.6±0.299.6±0.496.7±0.688.3±2.396.1±0.7
SigLIP2 embedding table97.8±0.898.1±1.794.1±1.785.3±3.293.8±0.7
SigLIP2 text encoder97.7±0.898.7±0.794.8±1.388.3±1.694.9±1.0

(b) WidowX (%)

ConfigurationCarrotEggplantSpoonStackAvg.
Default100.0±0.086.1±2.0100.0±0.083.3±5.992.4±1.8
BERT embedding table97.2±2.088.9±4.093.1±1.973.6±2.088.2±1.7
SigLIP2 embedding table97.2±2.054.2±15.6100.0±0.083.3±6.883.7±6.0
SigLIP2 text encoder93.1±1.944.4±10.4100.0±0.084.7±8.580.6±4.7

BERT embeddings perform close to the default on LIBERO (96.1 ± 0.7% versus 96.5 ± 0.3%). Across these benchmarks, the pretrained variants do not show a consistent improvement over the default, and we avoid interpreting small numerical differences as definitive advantages.

3. Eggplant performance.
The lower SigLIP2 scores persist in the three-seed averages: 54.2 ± 15.6% for the embedding table and 44.4 ± 10.4% for the text encoder, versus 86.1 ± 2.0% for the default. BERT embeddings achieve 88.9 ± 4.0%, so the decrease is not a general consequence of pretrained initialization.

One plausible explanation is the change in how instruction cues share supervision. The SigLIP2 variants replace the 30,522-token WordPiece vocabulary with a 256k multilingual SentencePiece vocabulary that segments “eggplant” differently, increase the embedding-table size from approximately 31M to 262M parameters, and change the representation geometry. These factors may affect transfer to the unseen complete instruction, although the current experiments do not isolate their individual effects.

We will incorporate these repeated results and optimization details into the revised manuscript. Our conclusion is no consistent advantage for pretrained text components under the evaluated settings, while competitive instruction-conditioned control remains achievable without pretrained language-side parameters.

AuthorsOpenReview ↗

Response to Reviewer eqBy (Part 4)

(W3a & Q5) Three-Seed Means and Headline Reporting

“Please report the headline LIBERO/WidowX numbers as 3-seed means and clarify how the reported seed was picked.”

Thank you for requesting clearer reporting. Our main tables consistently report seed-0 results rather than the best result across seeds, following the single-run reporting convention used by baselines such as X-VLA and VLA-Adapter. This keeps the reporting format consistent with the published comparisons. For example, we report 91.7% on WidowX from seed 0, rather than seed 2’s higher 94.8%. The corresponding LIBERO result is 96.4%, while 96.9% corresponds to seed 1 (this reporting error has now been corrected). We will ensure that the manuscript’s values and seed labels consistently reflect this convention.

To complement these single-run comparisons, Appendix A.7 reports multi-seed variability. We have now extended this analysis to CALVIN and the key text-component ablations. To directly address your request, we report the three-seed results below and will include the means and standard deviations alongside the headline scores.

BenchmarkSeed 0Seed 1Seed 2Mean±std
LIBERO average success (%)96.496.996.296.5±0.3
WidowX average success (%)91.790.694.892.4±1.8
CALVIN average chain length4.244.194.284.24±0.04

The rebuttal time budget allowed us to extend the default-model seed study only to CALVIN. We will complete and report multi-seed evaluations across remaining datasets.

(W3c) SimVLA Comparison, Parameter Accounting, and Longer-Horizon Ablations

“The SimVLA comparison isn’t a matched experiment.”

“SimVLA 2.0 is ~639M total … despite being labeled ‘0.3B’.”

Thank you for requesting clearer parameter accounting and controlled evidence.

1. Backbone size versus total policy size.
The 0.3B label refers to the pretrained SigLIP2 vision backbone, not the complete policy. SimVLA 2.0 has approximately 0.639B total parameters, including its action Transformer, instruction embeddings, and associated projections. We will explicitly label backbone and total-policy parameter counts in the main tables.

2. A more direct language-side removal experiment.

SimVLA and SimVLA 2.0 share a hyperparameter search space, but their backbones and selected configurations differ. The original published scores are therefore system-level references rather than isolated estimates of language pretraining’s contribution.

To address this concern, we additionally evaluated SimVLA with its language side removed on Google Robot. We retain the pretrained vision tower, visual connector, and action Transformer, but remove the language model and its output head. Visual features bypass the language model and enter the action Transformer alongside randomly initialized, action-trained WordPiece embeddings. The two variants use the same training data, optimization settings, training budget, action chunking, and evaluation protocol.

ConfigurationAverage success (%)
SimVLA77.0
SimVLA w/o LM75.4

Removing the language side produces an observed average decrease of 1.6 percentage points. This provides more direct evidence that strong control can be retained without the pretrained language pathway in this setting, while evaluating pathway removal as a whole rather than pretrained initialization alone.

3. Additional CALVIN ablations.
We extended the component analysis to CALVIN ABC$\rightarrow$D:

ConfigurationSR@1 (%)SR@2 (%)SR@3 (%)SR@4 (%)SR@5 (%)Avg. len.
Default99.493.085.878.667.64.24
No instruction34.27.72.30.60.00.45
Whole-instruction Task-ID3.00.10.00.00.00.03
BERT embedding table91.378.964.654.243.63.33
SigLIP2 embedding table97.690.581.573.364.44.07
SigLIP2 text encoder96.490.982.675.567.14.13
DINOv3 vision backbone82.460.844.431.420.02.39
Randomly initialized SigLIP2 vision tower55.127.212.04.11.81.00

These results extend the evidence to long-horizon control: instruction conditioning and visual pretraining are important, while the selected pretrained text configurations do not show a consistent improvement over the default.

(Q6) Provenance of the BEHAVIOR-1K Public Validation Score

“Was the BEHAVIOR-1K score from an official challenge submission, or evaluated post hoc on the public validation set?”

Thank you for requesting this clarification. Our score comes from a post-challenge evaluation on the Public Validation split. The evaluation records supporting this score and representative rollout videos are available on our anonymous project page (link).

Reviewer_WXgHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks whether the pretrained language components of Vision-Language-Action (VLA) policies are actually needed on standard manipulation benchmarks, whose instructions come from finite, recurring task inventories. To test this, the authors build SimVLA 2.0. It keeps natural-language instructions as input but loads no pretrained language model, text encoder, or token embeddings. Its components are:

  • a fixed WordPiece tokenizer that indexes a randomly initialized embedding table (∼\sim 31M parameters) trained only through the action objective;
  • a pretrained SigLIP2-L/16 vision tower (with the text tower never instantiated);
  • a 24-layer bidirectional "joint action Transformer" that attends jointly over noisy action tokens, visual patches, and instruction codes;
  • a conditional flow-matching action head.

A separate model is trained for each benchmark. Reported results are 96.9% on LIBERO, 96.5% on LIBERO-PRO semantic perturbations, 4.24 average chain length on CALVIN ABC$\rightarrow$D, 91.7% on SimplerEnv WidowX, 82.0% on Google Robot (variant aggregation), and 94.0%/87.2% on RoboTwin 2.0 Easy/Hard. The model also ranks second among Standard-track entries on the 2025 BEHAVIOR Challenge public validation leaderboard (Q-Score 0.245).

Ablations on LIBERO and WidowX show three things:

  • removing the instruction pathway sharply reduces success;
  • swapping in BERT or SigLIP2 token embeddings, or the full SigLIP2 text encoder, does not help;
  • visual pretraining is critical: replacing SigLIP2 with DINOv3 hurts WidowX badly, and a randomly initialized vision encoder fails.

The authors conclude that these task-bounded benchmarks do not require pretrained language-side parameters. They propose a modular "brain-cerebellum" design, in which a VLM handles high-level semantics and an LM-free controller executes bounded skills.

Critical strengths

Main Contribution and Key Innovations

  1. A clean, explicitly defined design point. The paper gives an operational definition of "language pretraining" (a pretrained LM, a text encoder, or token embeddings) and removes all three while keeping the instruction input. This intervention is sharper than prior efficiency work such as TurboVLA or VLA-0, which keep some pretrained language component.

  2. Broad empirical demonstration of sufficiency. The LM-free policy is evaluated on seven benchmark protocols covering tabletop, long-horizon, real-to-sim, bimanual, and household settings. It is competitive with, and on Google Robot and RoboTwin Easy better than, much larger language-pretrained VLAs.

  3. Ablations that separate where performance comes from. The ablations distinguish (i) instruction conditioning, (ii) pretrained token geometry, (iii) a pretrained contextual text encoder, and (iv) visual pretraining, while holding the rest of the recipe fixed.

  4. A useful framing for the field. Together with recent paraphrase-robustness diagnostics, the results suggest that current benchmarks reward something close to task-ID conditioning. This bears on both benchmark design and on how credit is assigned to language backbones.

Strengths

  • A simple, well-motivated question with a clear answer on the benchmarks studied. The construction is minimal and easy to reproduce. The authors are careful to say that the claim is limited to task-bounded settings and that a VLM remains necessary for open-ended semantics.

  • Impressive breadth and strength of results from one architecture and one recipe, without large-scale cross-embodiment robot pretraining. The BEHAVIOR-1K result is notable given the 6-14 minute horizons involved.

  • Informative ablations. The no-instruction ablation (LIBERO-Goal falls from 97.4% to 6.4%) clearly shows the instruction pathway carries task information. The vision-side ablations usefully show where pretrained knowledge does matter.

  • Good reproducibility practice. The paper reports seed variance (Appendix A.7), detailed per-domain interfaces and hyperparameters (Tables 10-12), parameter accounting, and released code.

  • Honest scoping of what the result does and does not imply, in the introduction and conclusion.

Critical weaknesses
  1. Limited novelty of the central finding.

Conditioning a multi-task policy on learned task embeddings is long-standing. More importantly, earlier benchmark papers studied this exact question. To my recollection, the original LIBERO paper compared task-ID conditioning with several pretrained language encoders and found little difference, and Mees et al. (2022a) analyzed the choice of language encoder on CALVIN. Please verify these recollections. On benchmarks with roughly 10-40 fixed instructions, it is unsurprising that learned codes match pretrained text features.

The new elements are therefore mainly (a) applying this to modern, strong VLA recipes across many benchmarks and (b) high absolute numbers. These are valuable, but the paper should engage directly with prior encoder-versus-task-ID comparisons and state precisely what is new beyond them.

  1. The key generalization test is missing.

The introduction is built around LIBERO-Para, which reports large paraphrase drops for X-VLA and VLA-Adapter. Yet SimVLA 2.0 is never evaluated on LIBERO-Para. Instead, robustness is argued from LIBERO-PRO's semantic perturbations, which are not discriminative: OpenVLA, π0\pi_0, π0.5\pi_{0.5}, and SimVLA all score about 90--100% on them (Table 8). Strong performance there therefore says little about whether the LM-free model tolerates linguistic variation.

The obvious and important experiment is to evaluate on LIBERO-Para and on instructions containing unseen words: synonyms ("grasp"/"pick"), hypernyms ("fruit"/"banana"), and novel object names. In this architecture, unseen WordPiece tokens map to embedding rows that remained at random initialization. One would therefore expect a sharp failure mode exactly where pretrained language is supposed to help. Characterizing that failure, honestly and quantitatively, would make the paper much more informative. Without it, the sufficiency claim covers only the in-vocabulary regime, which the prior point suggests is already expected.

  1. Evidence for composition is thin.

The claim of "composition over action-trained token embeddings rather than whole-instruction memorization" rests on a single WidowX instruction ("put eggplant into yellow basket"), evaluated on 24 episodes, plus LIBERO-PRO results. In that scene the visual context may largely disambiguate the task: the no-instruction variant still reaches 20.8%, and other tasks reach much higher without instructions. Moreover, every method, including SimVLA 2.0, fails on LIBERO-PRO task-goal perturbations (about 6%). A systematic held-out recombination split, for example in CALVIN or a constructed LIBERO variant, would be needed to support any compositionality claim.

  1. Language knowledge may enter through the vision encoder.

SigLIP2 is trained with image--text contrastive supervision, so its visual features are aligned to language. The DINOv3 ablation shows this matters a great deal on WidowX: the average falls from 91.7% to 46.9%, and Eggplant falls to 0%. The paper acknowledges this but still makes strong statements such as "Whatever language pretraining contributes to these benchmarks, it is not what puts a policy at this level." A more accurate conclusion is that a pretrained language-side module is unnecessary, while language-supervised visual pretraining may be doing part of the work. Similarly, the fixed tokenizer is a WordPiece vocabulary learned from text, so subword segmentation already carries a weak linguistic prior (shared stems, for instance). This should be acknowledged.

  1. The comparison with SimVLA is not controlled.

The paper calls SimVLA "the closest matched reference point" and reads the gaps as a measure of what language pretraining contributes. But the two systems differ in backbone (a 0.3B SigLIP2 vision tower versus a 0.5B VLM), and possibly in other recipe details. Differences of 1.7 LIBERO points and 0.23 in CALVIN chain length cannot be attributed to the language side alone. Two experiments would isolate the variable cleanly:

  • SimVLA with its language side removed; or

  • SimVLA 2.0 with the SimVLA VLM's language side restored, on all shared benchmarks.

  1. The ablations may be confounded and are noisy.
  • The pretrained-embedding and text-encoder variants use hyperparameters tuned for random embeddings. It is unclear whether the SigLIP2 text encoder was frozen or fine-tuned, and with what learning-rate multiplier. A pretrained encoder trained at the action learning rate could be degraded rather than helped. Please report these settings and, ideally, a small learning-rate sweep for the text-side variants.

  • Ablations are single-seed and cover only LIBERO and WidowX. WidowX uses 24 episodes per task with a seed standard deviation of 1.8 points, so several reported differences (91.7 vs. 90.6, for example) are within noise. The large Eggplant drop for the SigLIP2 embedding table (87.5% to 50.0%) is unexplained and deserves analysis, since Eggplant is the out-of-training instruction.

  • The conclusion "no consistent advantage" is supported. The data cannot distinguish "no advantage" from "an advantage masked by recipe mismatch."

Suggestions for Improvement:

  1. Evaluate on LIBERO-Para and on a targeted out-of-vocabulary split (synonyms, hypernyms, novel object names). Report the failure modes, since this is where the paper's thesis is most informative.

  2. Add a controlled comparison that changes only the language side relative to SimVLA (or another VLM-backbone baseline) on all shared benchmarks.

  3. Run multiple seeds for the key ablations, extend them to at least CALVIN, and tune the learning rates of the pretrained text-side variants. Report whether those variants are frozen or fine-tuned.

  4. Report the mean and standard deviations for all headline numbers, and make the seed reporting consistent across benchmarks.

  5. Engage directly with earlier task-ID versus language-encoder comparisons (LIBERO, CALVIN, BC-Z, language-conditioned imitation-learning work) and state precisely what is new.

  6. Reframe the conclusions around language-side modules, acknowledging the language-supervised vision encoder and the text-derived tokenizer. One option is a DINOv3 variant with a better recipe, to test whether the WidowX gap is fundamental.

  7. Consider reframing part of the contribution as a benchmark critique: if task codes saturate current benchmarks, what evaluation would actually measure the value of language pretraining?

  8. Optionally, demonstrate the proposed brain--cerebellum integration, even in a minimal form, such as a VLM mapping free-form requests onto the controller's instruction vocabulary.

  9. Minor: tighten the repetitive rhetoric in the introduction; fix "a instruction-conditioned controller" (Section 5); and expand the Figure 1 and Table 1 captions.

Questions for authors
  1. How does SimVLA 2.0 perform on LIBERO-Para? What happens when an instruction contains tokens never seen during training?

  2. In the SigLIP2 text-encoder ablation, was the encoder frozen or fine-tuned, and at what learning rate? Were any hyperparameters re-tuned for the pretrained variants?

  3. Why is the LIBERO headline result the best seed rather than the seed mean?

  4. Do you have BEHAVIOR results beyond the public validation split?

  5. Is the SimVLA comparison trained on identical data, schedules, and chunking? If not, which differences remain?

8 comments
AuthorsOpenReview ↗

Response to Reviewer WXgH (Part 1)

Thank you for recognizing our design, broad evaluation, and component ablations. Your comments help identify where the novelty, generalization evidence, and statistical interpretation need to be made more precise.

(W1) Novelty Relative to Task-ID and Language-Encoder Comparisons

“Conditioning a multi-task policy on learned task embeddings is long-standing.”

“State precisely what is new beyond them.”

Thank you for highlighting these precedents. We agree that task conditioning and learning instruction representations from scratch are established ideas. LIBERO compares language encoders with BERT-encoded task IDs in its lifelong-learning setting [1]; Mees et al. study pretrained language-encoder choices on CALVIN [2]. BC-Z investigates task generalization using pretrained language representations [3], while Lynch and Sermanet compare representations learned from scratch with pretrained representations [4]. We will explicitly discuss these connections.

Our contribution extends these findings to competitive modern control across diverse manipulation domains, with component ablations and diagnostics separating performance on familiar instructions from robustness to instruction variation. Following your suggestion, we added a whole-instruction Task-ID baseline that learns one embedding per training instruction string.

EvaluationSimVLA 2.0Whole-instruction Task-ID
Standard LIBERO, average success (%)96.494.8
LIBERO-Para, success (%)60.125.74
CALVIN ABC$\rightarrow$D, average chain length4.240.03
WidowX, average success (%)91.750.0
WidowX Eggplant, success (%)87.54.17

The standard LIBERO result supports the earlier observation that task identifiers can be effective: whole-instruction Task-ID achieves 94.8%, close to our token-level model’s 96.4%. Our contribution is therefore not the discovery that learned task codes can support control.

The additional evaluations reveal a distinction that standard-task scores alone do not capture. On LIBERO-Para, our token-level model achieves 60.12% success, compared with 5.74% for whole-instruction Task-ID. Unseen strings in this baseline map to a shared fallback representation, so this comparison evaluates robustness beyond exact-string lookup rather than performance against an oracle controller given the correct task identity. These results demonstrate the practical benefit of token-level conditioning under instruction rewording, without establishing systematic compositional understanding.

Together with our ablations of instruction conditioning, pretrained token embeddings, contextual text encoding, and visual pretraining, these results provide a broader empirical characterization of when competitive control is achievable without pretrained language-side parameters and how that performance changes under instruction variation. We will revise the contribution statements accordingly.

[1] LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. NeurIPS 2023.

[2] What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data. IEEE RA-L, 2022.

[3] BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning. CoRL 2021.

[4] Language Conditioned Imitation Learning over Unstructured Data. RSS 2021.

AuthorsOpenReview ↗

Response to Reviewer WXgH (Part 2)

(W2 & Q1 & S7) LIBERO-Para and Instructions with Unseen Tokens

"The key generalization test is missing."

"The obvious and important experiment is to evaluate on LIBERO-Para."

"How does SimVLA 2.0 perform on LIBERO-Para? What happens when an instruction contains tokens never seen during training?"

Thank you for suggesting this evaluation. We have added LIBERO-Para experiments and training-unseen-token analysis to quantify robustness to linguistic variation and characterize its limitations.

1. Evaluation setup.
We evaluate the default model and three pretrained text variants—BERT token embeddings, SigLIP2 token embeddings, and the full SigLIP2 text encoder—using the LIBERO-Para protocol. The policies receive paraphrased instructions for the benchmark’s underlying manipulation tasks.

For each instruction, we compute the proportion of content-token occurrences whose IDs are absent from the corresponding action-training instruction vocabulary, and group evaluation episodes into four coverage buckets. We call these tokens training-unseen, distinguishing them from tokenizer-level [UNK] tokens.

2. Handling and prevalence of unseen tokens.
The default model uses a fixed WordPiece tokenizer with 30,522 entries and randomly initialized, 1,024-dimensional embeddings. An unfamiliar word is first segmented into subwords: previously observed IDs use their action-trained embeddings, while IDs never encountered during training retain their individual initial random vectors. They are not mapped to a shared zero representation.

The LIBERO training instructions comprise 112 unique strings and 87 unique content WordPiece IDs. On LIBERO-Para, 39.94% of content-token occurrences are training-unseen, and 98.63% of prompts contain at least one such token. Neither WordPiece nor SigLIP2 SentencePiece produces tokenizer-level [UNK] tokens on these prompts. Thus, this evaluation extends beyond instructions composed entirely of training-seen tokens.

3. Results and generalization boundary.

SimVLA 2.0 achieves 60.12% overall success on LIBERO-Para, with 82.53% on action paraphrases, 74.52% on object paraphrases, and 52.28% on compositional paraphrases. Its overall performance approaches X-VLA (62.1%) and exceeds VLA-Adapter (46.3%).

MethodLIBERO-Goal (%)LIBERO-Para SR (%)Action (%)Object (%)Compositional (%)PRIDE (α=0.5\alpha=0.5)
Xiaomi-Robotics-098.876.089.688.670.969.2
π0.5\pi_{0.5}97.671.489.987.864.665.4
OpenVLA-OFT (goal / mixed)97.9/96.164.7/63.789.1/89.574.2/71.456.7/55.558.8/56.3
X-VLA97.862.183.084.954.052.7
VLA-Adapter98.246.373.775.035.736.1
π0.5\pi_{0.5} (expert-only, frozen VLM)78.639.158.974.930.232.0
SimVLA 2.0 (Ours)96.460.1282.5374.5252.2849.82

Performance with training-unseen tokens. Although 98.63% of LIBERO-Para prompts contain at least one training-unseen content token, the model retains substantial success. Grouping episodes by the proportion of such tokens gives:

Training-unseen share of content tokensDefault random tableBERT embedding tableSigLIP2 embedding tableSigLIP2 text encoder
0%80.3689.2971.4365.71
(0%,25%]79.4776.8367.5655.85
(25%,50%]62.1246.2141.9741.52
>50%34.1420.0030.6532.83
Overall60.1247.7544.1142.25

The default model maintains approximately 80% success when at most 25% of content tokens are training-unseen, and achieves 34.14% when the proportion exceeds 50%. These results extend the evidence beyond entirely training-seen vocabulary: untrained token embeddings do not necessarily prevent successful execution, although higher unseen-token proportions are associated with lower success.

Do pretrained text representations help? Within our architecture, BERT embeddings, SigLIP2 embeddings, and the SigLIP2 text encoder achieve 47.75%, 44.11%, and 42.25% overall, respectively, compared with 60.12% for the default. With the same WordPiece tokenizer and bucket assignments, BERT embeddings perform better in the zero-unseen-token group, but the default performs better in all three groups containing unseen tokens. Thus, pretrained text representations do not provide a consistent advantage under the evaluated settings, including when instructions contain training-unseen tokens.

Our central claim remains that competitive instruction-conditioned control can be achieved without pretrained language-side parameters within the evaluated task-bounded settings. The new experiments support this claim under substantial instruction variation, without requiring an interpretation in terms of open-vocabulary understanding or systematic compositional generalization. Reliance on familiar lexical cues is compatible with this claim. We will incorporate these results and clarify this scope in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer WXgH (Part 3)

(W3) Evidence for Composition and the WidowX Eggplant Task

"Evidence for composition is thin."

"A systematic held-out recombination split, for example in CALVIN or a constructed LIBERO variant, would be needed to support any compositionality claim."

Thank you for highlighting this distinction. We will replace “composition over action-trained token embeddings rather than whole-instruction memorization” with “successful execution under an unseen instruction string composed of training-seen tokens.” This describes the Eggplant observation without attributing it to a particular compositional mechanism.

As detailed in our response to W2 above, the added LIBERO-Para evaluation yields 60.12% success and provides broader evidence of tolerance to instruction rewording beyond the single Eggplant example. We interpret these results as paraphrase robustness, rather than a substitute for a systematic held-out recombination test. We will incorporate the new evaluation and token-coverage analysis into Section 4.3 and revise the relevant wording in Sections 1 and 4.3 accordingly.

(W4) Language-Supervised Visual Pretraining and the Tokenizer Prior

"Language knowledge may enter through the vision encoder."

"The paper acknowledges this but still makes strong statements such as "Whatever language pretraining contributes to these benchmarks, it is not what puts a policy at this level." A more accurate conclusion is that a pretrained language-side module is unnecessary, while language-supervised visual pretraining may be doing part of the work. Similarly, the fixed tokenizer is a WordPiece vocabulary learned from text, so subword segmentation already carries a weak linguistic prior (shared stems, for instance). This should be acknowledged."

Thank you for highlighting this distinction. As defined in Lines 82–84, “language pretraining” operationally means loading “(i) a pretrained language model, (ii) a pretrained text encoder, or (iii) pretrained token embeddings.” The manuscript also explicitly states: “The vision encoder’s image–text pretraining is retained by default and ablated separately.” Our claim therefore concerns the absence of pretrained language-side parameters, not all language-derived priors. We agree that SigLIP2’s visual features and the text-derived WordPiece tokenizer retain such priors, and will clarify the broader wording you identified to remain consistent with this definition.

(W5 & Q5) Controlled Comparison with SimVLA

"The comparison with SimVLA is not controlled."

"Is the SimVLA comparison trained on identical data, schedules, and chunking? If not, which differences remain?"

Thank you for highlighting the distinction between competitive sufficiency and causal attribution. SimVLA and SimVLA 2.0 share a hyperparameter search space, but not necessarily identical selected configurations. Both search action-chunk horizon H∈10,20,30H\in\\{10,20,30\\}, shuffling, action/proprioception normalization, base learning rate {5×10−5,10−4,2×10−4,5×10−4}\{5\times10^{-5},10^{-4},2\times10^{-4},5\times10^{-4}\}, warm-up steps {0,1000}\{0,1000\}, batch size {128,256}\{128, 256\}, and scheduler {cosine decay,none}\{\text{cosine decay},\text{none}\}, with at most 200k training steps.

Following your suggestion, we additionally evaluate SimVLA with its language side removed on Google Robot. We retain its pretrained vision tower, visual connector, and action Transformer, but remove the language model and language-model output head. Visual features bypass the language model and are passed directly to the action Transformer, together with randomly initialized, action-trained embeddings. The two variants use the same training data, optimization settings, training budget, action chunking, and evaluation protocol.

KnobPickMoveOpenAvg.
SimVLA82.381.067.777.0
SimVLA w/o LM (Random emb. table)80.777.368.375.4

Removing the language side produces a modest average decrease of 1.6 percentage points, from 77.0% to 75.4%. Performance decreases on Pick and Move, while increasing on Open. Thus, the controller retains most of its observed performance without the pretrained language module.

This experiment directly addresses the proposed language-side removal and strengthens our finding that competitive control is achievable without pretrained language-side parameters, here while retaining SimVLA’s original pretrained vision tower. It evaluates removal of the language pathway as a whole, rather than isolating pretrained initialization alone.

Reviewer_HbzmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether strong performance on bounded manipulation benchmarks requires pretrained language-side parameters. SimVLA 2.0 combines a pretrained SigLIP2 vision encoder with randomly initialized instruction embeddings and a joint action Transformer trained through flow matching. Evaluations span six manipulation domains, including BEHAVIOR. Ablations examine instruction removal, pretrained text representations, and visual pretraining.

Critical strengths
  1. The results are quite strong across multiple benchmarks. It is interesting that a model without pretrained language components can achieve performance comparable to existing VLAs, especially across different robots and task settings.
  2. The ablations are useful. In particular, removing instructions causes a large performance drop, which supports the claim that the model actually uses instruction information rather than relying on visual observations alone.
Critical weaknesses
  1. The comparison with SimVLA is not fully controlled. The models differ in backbone and training settings, making it difficult to isolate the effect of language pretraining. Table 9 helps, but does not fully resolve this issue.
  2. The evidence for language generalization is limited. The unseen WidowX instruction contains only training-seen tokens. It remains unclear whether the model learns instruction semantics or mainly relies on familiar task cues.
Questions for authors
  1. Which architecture and training settings are matched with SimVLA? Is the SigLIP2 text tower fine-tuned, and what learning rate is used?
  2. Can the model follow different instructions in the same scene? Please also clarify the Bridge instruction augmentation and training/evaluation overlap.
  3. Does BEHAVIOR use joint or task-specific fine-tuning? How many checkpoints are used, and can the authors provide the evaluation records?
8 comments
AuthorsOpenReview ↗

Response to Reviewer Hbzm (Part 1)

We greatly appreciate your thoughtful and detailed feedback. We hope our response below addresses your concerns.

(W1 & Q1a) Matched Settings and the Interpretation of the SimVLA Comparison

"The attribution to language pretraining exceeds the controlled evidence."

"Which architecture and training settings are actually matched in the SimVLA comparison?"

Thank you for highlighting the distinction between competitive sufficiency and causal attribution. SimVLA and SimVLA 2.0 use a common hyperparameter search space for their shared components, while allowing model-specific configurations.

Both systems are tuned over the same candidate settings for action-chunk horizon H∈10,20,30H\in\\{10,20,30\\}, data shuffling, action/proprioception normalization, base learning rate {5×10−5,10−4,2×10−4,5×10−4}\{5\times10^{-5},10^{-4},2\times10^{-4},5\times10^{-4}\}, warm-up steps {0,1000}\{0,1000\}, scheduler {cosine decay,none}\{\text{cosine decay},\text{none}\}, and batch size {128,256}\{128, 256\}, with all runs limited to at most 200k training steps. Both systems also receive the same source observations, supporting a fair comparison with respect to visual input information: for example, on LIBERO, the same 128×128128\times128 images are resized to each pretrained encoder’s input size—384×384384\times384 for SimVLA and 256×256256\times256 for SimVLA 2.0. We will clarify these preprocessing stages and the selected training configurations in the manuscript. While this protocol supports a system-level comparison, differences in backbones prevent attributing the performance gap solely to language pretraining.

To address this concern more directly, we additionally evaluate SimVLA with its language side removed on Google Robot. We retain its pretrained vision tower, visual connector, and action Transformer, but remove the language model and its output head. Visual features bypass the language model and enter the action Transformer alongside randomly initialized, action-trained embeddings. The two variants use the same training data, optimization settings, training budget, action chunking, and evaluation protocol.

KnobPickMoveOpenAvg.
SimVLA82.381.067.777.0
SimVLA w/o LM (Random emb. table)80.777.368.375.4

Removing the language side produces a modest average decrease of 1.6 percentage points, from 77.0% to 75.4%. The controller therefore retains most of its observed performance without the pretrained language module. This experiment supports competitive control without pretrained language-side parameters while retaining the original vision backbone; it evaluates removal of the language pathway as a whole, rather than pretrained initialization alone.

We will revise Section 4.2 to distinguish the two forms of evidence: the original cross-system comparisons establish competitive sufficiency, while this new same-backbone ablation more directly measures the effect of replacing SimVLA’s pretrained language pathway with action-trained embeddings.

AuthorsOpenReview ↗

Response to Reviewer Hbzm (Part 2)

(Q1b) SigLIP2 Text-Tower Training and Learning-Rate Treatment

"Please also specify whether the full SigLIP2 text tower is frozen or fine-tuned and its learning-rate treatment."

Thank you for requesting this clarification.

In the Table 9 ablations, the full 24-layer SigLIP2 text tower and the pretrained embedding tables are fine-tuned end to end under the action objective. For the tower, we discard the pooled projection head and pass the final-layer contextual token states directly to the joint action Transformer. Their width is 1,024, matching the action trunk, so no additional linear adapter is used.

We use AdamW with (β1,β2)=(0.9,0.95)(\beta_1,\beta_2)=(0.9,0.95) and zero weight decay. During the first 1,000 updates, we train only the action heads. After that, we fine-tune the pretrained variants (the SigLIP2 text tower, the SigLIP2 token embedding table, and the BERT token embedding table) using the same text learning rate as the default model with randomly initialized token embeddings on the corresponding benchmark. We set the text learning rate to lrtext=lrbase×coeftext\mathrm{lr}_{\mathrm{text}} = \mathrm{lr}_{\mathrm{base}} \times \mathrm{coef}_{\mathrm{text}}, where the multiplier coeftext\mathrm{coef}_{\mathrm{text}} defaults to 1. Table 12 of the original manuscript reports the benchmark-specific learning rates.

Follow-up learning-rate experiments. To test whether the original learning rate disadvantaged the pretrained variants, we independently swept the text learning-rate multiplier coeftext\mathrm{coef}_{\mathrm{text}} over {1.0,0.1,0.05,0.0}\{1.0, 0.1, 0.05, 0.0\} for each pretrained text variant on each benchmark, holding all other training settings fixed within each sweep. The results are shown below.

(a) LIBERO

KnobValueSpatialObjectGoalLongAvg.
SimVLA 2.0Default98.699.096.491.496.4
Text embeddingBERT emb. table ( coeftext=1.0\mathrm{coef}_{\mathrm{text}}=1.0)99.899.097.690.896.8
Text embeddingSigLIP2 emb. table ( coeftext=1.0\mathrm{coef}_{\mathrm{text}}=1.0)98.899.296.484.494.7
Text embeddingSigLIP2 text encoder ( coeftext=0.1\mathrm{coef}_{\mathrm{text}}=0.1)98.899.696.690.296.3

(b) WidowX

KnobValueCarrotEggplantSpoonStackAvg.
SimVLA 2.0Default100.087.5100.079.291.7
Text embeddingBERT emb. table ( coeftext=0.05\mathrm{coef}_{\mathrm{text}}=0.05)100.091.795.875.090.6
Text embeddingSigLIP2 emb. table ( coeftext=0.05\mathrm{coef}_{\mathrm{text}}=0.05)95.850.0100.083.382.3
Text embeddingSigLIP2 text encoder ( coeftext=0.05\mathrm{coef}_{\mathrm{text}}=0.05)95.841.7100.083.380.2

(c) CALVIN ABC→D

KnobValueSR@1SR@2SR@3SR@4SR@5Avg. len.
SimVLA 2.0Default99.493.085.878.667.64.24
Text embeddingBERT emb. table ( coeftext=1.0\mathrm{coef}_{\mathrm{text}}=1.0)91.378.964.654.243.63.33
Text embeddingSigLIP2 emb. table ( coeftext=1.0\mathrm{coef}_{\mathrm{text}}=1.0)97.690.581.573.364.44.07
Text embeddingSigLIP2 text encoder ( coeftext=0.1\mathrm{coef}_{\mathrm{text}}=0.1)96.490.982.675.567.14.13

Learning-rate tuning improves some pretrained variants, bringing their performance close to that of the default model with randomly initialized embeddings. However, these gains do not translate into a consistent improvement over the default across the evaluated benchmarks. The results therefore suggest that pretrained text components are not necessary to achieve this level of control performance under the tested settings.

AuthorsOpenReview ↗

Response to Reviewer Hbzm (Part 3)

(W2 & Q2a) Instruction Robustness versus Familiar Task Cues

"The robustness analysis does not sufficiently distinguish composition from familiar task cues."

"Can same-scene instruction interventions, or the LIBERO-Para evaluation cited in the introduction, distinguish the proposed robustness from familiar lexical task cues?"

Thank you for highlighting this distinction. We agree that success on an unseen instruction composed of familiar tokens does not, by itself, distinguish compositional understanding from keyword-based task identification. We have added LIBERO-Para evaluation, training-unseen-token analysis, and a whole-instruction Task-ID baseline to characterize this distinction more directly.

1. LIBERO-Para evaluation.
Under the LIBERO-Para evaluation protocol, SimVLA 2.0 achieves 60.12% overall success: 82.53% on action paraphrases, 74.52% on object paraphrases, and 52.28% on compositional paraphrases. The latter is the benchmark’s category for combined linguistic changes, rather than evidence of systematic task compositionality.

The following table compares our results with published LIBERO-Para leaderboard scores:

MethodLIBERO-Goal (%)LIBERO-Para SR (%)Action (%)Object (%)Compositional (%)PRIDE (α=0.5\alpha=0.5)
Xiaomi-Robotics-098.876.089.688.670.969.2
π0.5\pi_{0.5}97.671.489.987.864.665.4
OpenVLA-OFT (goal / mixed)97.9/96.164.7/63.789.1/89.574.2/71.456.7/55.558.8/56.3
X-VLA97.862.183.084.954.052.7
VLA-Adapter98.246.373.775.035.736.1
π0.5\pi_{0.5} (expert-only, frozen VLM)78.639.158.974.930.232.0
SimVLA 2.0 (Ours)96.460.1282.5374.5252.2849.82

SimVLA 2.0 performs close to X-VLA and above VLA-Adapter, while remaining below the strongest listed systems. These results demonstrate substantial paraphrase tolerance without pretrained language-side parameters.

2. Whole-instruction lookup and familiar lexical cues.

(a) LIBERO (%)

MethodSpatialObjectGoalLongAvg.LIBERO-Para
SimVLA 2.0 (Default)98.699.096.491.496.460.12
Whole-Instruction Task-ID97.699.496.286.094.85.74

(b) CALVIN ABC$\rightarrow$D

MethodSR@1 (%)SR@2 (%)SR@3 (%)SR@4 (%)SR@5 (%)Avg. len.
SimVLA 2.0 (Default)99.493.085.878.667.64.24
Whole-Instruction Task-ID3.00.10.00.00.00.03

(c) WidowX (%)

MethodCarrotEggplantSpoonStackAvg.
SimVLA 2.0 (Default)100.087.5100.079.291.7
Whole-Instruction Task-ID91.74.1779.225.050.0

Across these benchmarks, visual context alone is insufficient when tasks share scenes (No-Instruction scores 6.4% on LIBERO-Goal and 20.8% on WidowX Eggplant; Table 9 in the manuscript). On verbatim in-inventory instructions, Whole-Instruction Task-ID nearly matches our token-level model (94.8% vs. 96.4% on standard LIBERO, including 96.2% on LIBERO-Goal, and 91.7% / 79.2% on WidowX Carrot/Spoon). Under paraphrased or unseen full strings—which map to a shared fallback representation in Whole-Instruction Task-ID, rather than an oracle task label—exact-string lookup collapses (5.74% vs. 60.12% on LIBERO-Para, 0.03 vs. 4.24 on CALVIN, and 4.17% vs. 87.5% on WidowX Eggplant), showing that subword token embeddings provide substantial reuse across reworded instructions beyond whole-string memorization.

We further stratify our model’s LIBERO-Para results by the proportion of content tokens absent from the action-training instruction vocabulary:

Training-unseen token proportionSuccess (%)
0%80.36
(0%, 25%]79.47
(25%, 50%]62.12
>50%34.14

Performance declines as unfamiliar-token coverage increases, indicating an important boundary of the observed robustness.

3. Clarifying our claim.
Our central claim is that competitive instruction-conditioned control can be achieved without pretrained language-side parameters within the evaluated task-bounded settings. This claim does not require open-vocabulary understanding or systematic compositional generalization; reliance on familiar lexical cues is compatible with it. Our intended interpretation of the Eggplant result is narrower: the policy successfully executes an instruction not observed as a complete string during training, using action-trained token embeddings. This does not establish how the policy combines or selects those cues. We will incorporate these results in the revised manuscript and revise the relevant “composition” wording in Sections 1 and 4.3, explicitly distinguishing competitive control performance from linguistic generalization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.