acceptodds
Under review as a conference paper at ICLR 2027

Grounding Is Inherited, Action Is Concentrated: What VLA Finetuning Adds to a Web-Pretrained VLM

Abstract

A web-pretrained vision-language embedding model with a one-hidden-layer action head, adapted with LoRA on the target benchmark's demonstrations, reaches 4.62±0.05 of 5 consecutive subtasks on CALVIN ABC→D and 98.0% on the 40-task LIBERO generalist. We ask what the decodability of the action from the backbone's pooled feature tells us about the resulting policy, and what it does not. On LIBERO, a ridge reader recovers the action chunk from the frozen backbone's pooled feature only with   labelled frames ( 0.64–0.68) and almost not at all from its eight leading directions; after finetuning the same reader reaches 0.95 from 400 frames and 0.90–0.93 from the top eight, across backbones, poolings and adaptation methods (CALVIN: the same trend at smaller magnitude). Working policies depend on those directions: keeping only the top 64 of 2,048–4,096 retains 88–96% success and 64 random directions retain none. Decodability does not certify reliability, however. Controlled input interventions on four full-finetuning policies, three of them probed and readable, switch whole task families between success and failure: on CALVIN, replacing only the table's wood texture with the geometry held fixed turns two skills off and on (223–226/226 ↔ 0/226 first subtasks, two backbones) while a LoRA policy is unaffected in the three combinations tested; on LIBERO, a one-token change of the evaluation prompt drops two policies from 94.8 to 46.9% and from 74.9 to 15.0% while top-8 demonstration-frame readability moves by 0.02–0.05 and no working policy tested changes comparably. Other deficits persist under every input tested. Across the configurations studied, finetuning increases action decodability and concentrates it in leading directions; these measurements by themselves do not establish closed-loop reliability. Simulation only; one arm.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robot learning, representation analysis, probing, LoRA, full finetuning, LIBERO, CALVIN

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_8uaROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

By finetuning vision language action models (VLAs) on LIBERO and CALVIN benchmark, the paper studies the linear decodability of web pretrained vision language models' (VLMs) pooled features. While a frozen VLM encodes action information loosely across high dimensional feature, finetuning increases the decodability and concentration in leading principal directions. During evaluation, this is supported by keeping the top 64 of 2048-4096 dimensions still retaining 88-96% success. However, controlled input intervention like swapping texture or single prompt token lead to task-selective failures while top-8 readability on LIBERO barely changes, showing that action decodability does not guarantee reliability in closed-loop control in full-finetuned policies (3e-5 on LIBERO, 1e-5 on CALVIN).

Critical strengths
  1. The paper studies the decodability and behavior reliability. The finding that the top-8 readability moves by only 0.02-0.05 while performance drops by about 48 and 60 points on LIBERO (readability was not measured under the CALVIN texture swap, where two skills drop 100%) is an interesting finding. This could be helpful for many literatures in this field that moves toward linear probing based fine-tuning.
  2. The experiments are well designed, and studied under controlled settings, supporting the concentration finding in the configurations studied.
  3. The experiments studied one visual intervention (texture swap) and one textual one (prompt change), showing that input-selective failures occur for both image and text interventions, in different benchmarks and policies.
Critical weaknesses
  1. The paper relies only on action chunk ridge R2R^2 as the decodability metric, other settings like per task or instruction are not studied systematically.
  2. The results rely on only a few runs and limited setups. The paper only studied one intervention for each of image and instruction inputs, and used only 1-2 seeds for the full-FT experiments in section 6, on two benchmarks only, with no confidence intervals, which weakens claims like "Readable is not enough".
  3. "Held-out episodes" were part of the backbone’s finetuning data. The authors admit this in the paper, but many probing results that directly supports the central claim rely on this setup, and this overlap could lead to backbone memorizing these data during training, and limits what the probe shows about generalization.
  4. While the paper shows that decodability does not equal reliability, both healthy and sensitive policies are concentrated and readable, neither an explanation of what separates them nor a better diagnostic is proposed.
  5. Many important experiments and results heavily rely on the appendix, making the scope of the paper broad and hard to follow, the presentation of the paper needs improvement.
  6. The title of the PDF differs from the one in the metadata
Questions for authors
  1. The held-out data were part of the backbone’s finetuning data, and the unseen-task probes show much lower R2R^2 (around 0.5-0.67). The main concentration claim (R2R^2 0.95) stated on in-distribution frames is not shown to hold outside the finetuning distribution. Can you show concentration (top-8 vs. full) on unseen tasks with multiple seeds and confidence intervals?
  2. Can the authors run on more seeds and setups? In section 6.2, the two seeds differ by about 20 points (94.8 vs. 74.9) under the training prompt and about 32 (46.9 vs. 15.0) under 'next 10'. No learning-rate search was run, and 1e-5 works while 3e-5 does not. This could mean the current hyperparameter setups are not well configured, or the results are not statistically rigorous to support the claims.
  3. Can you also analyze the readability measurement under the CALVIN texture swap? Readability was only measured under the LIBERO prompt change, so the claim "readable is not enough" is only partially tested for the visual intervention.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 8uaR (1/2)

Thank you very much for the careful review, and for your patience with a submission that was hard to follow; it matters a great deal to us. As our general comment explains, the revision is a full rewrite, and we do not ask for a re-evaluation on its basis. We answer your three questions first, each with what we ran and what it showed, and then the weaknesses briefly.

Q1: concentration outside the finetuning distribution. You ask for the eight leading directions against the whole feature on unseen tasks, with several seeds and intervals. We ran this after your review (registered 10 November): two further training seeds of the adapter that never saw the ten tasks, read in the same way, with intervals over test episodes. The result: outside the training data the readable share drops, by a third when the readout is fit on the new tasks and by more than half when it is not, but what remains still sits in the eight leading directions, for all three seeds. In detail (each pair is the R² from all of z / from its eight leading directions):

  • Unseen tasks, readout fit on them: 0.66 / 0.57 on average over the three seeds (range 0.65–0.66 / 0.56–0.57; each seed within about ±0.03 over test episodes), against 0.31 / 0.08 for the frozen backbone and 0.96 / 0.94 for an adapter trained on these tasks.
  • Unseen tasks, readout not fit on them either: 0.42 / 0.43 on average (range 0.39–0.45 / 0.39–0.47). Here the training seed matters: the seeds differ by up to 0.07, more than the ±0.03 over test episodes. The frozen backbone's readout is below zero, and an adapter trained on CALVIN alone carries nothing over (−0.02).
  • Unseen scene (CALVIN scene D), nine checkpoints (three LoRA seeds and six tuned full-finetuning policies): 0.59–0.63 / 0.46–0.52, against 0.66–0.72 / 0.53–0.61 on a training scene. On both, the eight directions carry about four fifths of what the whole feature gives.

The revision therefore says that the concentration carries beyond the training data only in part, and only within the adapter's task family (Sec. 4, Table 2). A second adapter, trained jointly on CALVIN and the thirty tasks, has one seed (0.67 / 0.61 and 0.49 / 0.48 in the two settings). Table 2 gives the mean and range over the three seeds; Appendix B.5 gives each seed with its interval.

Q2: more seeds and settings; no learning-rate search. We ran this after submission (registered 10 October): a sweep of the batch and the learning rate of full finetuning on CALVIN, six policies, each with the data budget of the fragile policy. The result: your suspicion was right, the configuration was untuned. The fragile policies come from a learning rate too high for the batch; with a tuned rate, full finetuning matches LoRA and the texture no longer matters. In detail, subtasks completed in a row on the unseen scene D, over 1,000 chains, with 95% chain-bootstrap intervals:

  • batch 128 at 1e-5: 4.34 [4.26, 4.41] for each of two seeds; batch 32 at 1e-5: 4.22 [4.14, 4.30]
  • batch 32 at 3e-6: 4.02 [3.93, 4.11]; batch 128 at 3e-6: 4.03 [3.94, 4.11]
  • batch 4 at 1e-6: 4.26 [4.18, 4.34]
  • for reference: the fragile policy (batch 4 at 1e-5) 0.95; LoRA 4.35 ± 0.07 over three seeds

Lowering only the learning rate of the original recipe to a tenth is enough (0.95 to 4.26). The texture swap changes the slider, light and LED subtasks of these six policies by at most 0.9 points, and three LoRA seeds solve all of them in every combination (Sec. 6, Table 4, Appendix D.5).

Three things remain thin. LIBERO has no sweep: full finetuning at 1e-5 works there (97.9%), and under the training prompt the two 3e-5 seeds differ mainly on the long-horizon suite (434 against 169 of 500), which we cannot explain. The 46.9% and 15.0% were measured under an evaluation prompt that differed from the training prompt; the revision reports the training-prompt values (94.8% and 74.9%) and keeps the other as a sensitivity test, which is positional (one extra space before the "8" does what "next 10" does). The fragile recipe was trained once per backbone on CALVIN, and the Instruct-2B policy was not re-tuned.

AuthorsOpenReview ↗

Response to Reviewer 8uaR (2/2)

Q3: readability under the CALVIN texture swap. We ran this after submission (registered 12 October): the readout under the texture swap, on 1,558 recorded scene-D states, each rendered with the table's own texture and with table A's. The result: under the texture that breaks the policy, the action stays readable to a readout refit on the changed input, while a readout held fixed, and the policy's own head, lose it on the affected skills. In detail, each pair gives the fragile Emb-2B policy on the slider and light frames, first with the texture under which it solves 0 of the 226 such subtasks, then with the one under which it solves 223.

  • A readout refit under each texture: 0.36 against 0.40 (0.47 under both over all frames).
  • A readout fitted once on training-scene frames and held fixed: 0.18 against 0.40.
  • The error of the policy's own head against the demonstrated actions: 0.19 against 0.09.

For the nine working checkpoints none of the three moves by more than 0.011. The visual intervention thus shows what the prompt intervention showed. One caveat, which a pre-registered rule required us to state: the fragile policies are less readable overall (34% and 10% from eight directions, against 46–52%), so readability ranks policies but does not flag the inputs that break them (Sec. 6, Appendix D.4). The readouts by skill and the fixed readout are reported in our replies; the revised PDF has the value over all frames and the head's error (Appendix D.4).

Your weaknesses, briefly.

  • W1 (one metric). The revision reports the readout in three further ways: without the gripper (the eight directions still explain 80–89% on LIBERO, but only 1–14% of the rotations), with each task's average removed (93–95%), and per skill under the texture swap (Q3) (Sec. 4, Fig. 2, Appendices B.3 and B.4).
  • W2 (few runs and settings, no intervals). Agreed for the submission; what we added and what is still thin is under Q2. The scope remains two simulated benchmarks and one robot arm (Sec. 7).
  • W3 (held-out episodes belong to the backbone's training data). Agreed: they are new to the readout, not to the backbone. Settings in which the backbone never saw the data, unseen tasks and an unseen scene, are in Table 2; see Q1.
  • W4 (what separates the policies; a better diagnostic). The recipe separates them (Q2). Two quantities register the failure where a refit readout does not: the head's own error on the affected skills (Q3), and a pre-registered measure of how far the input change moves the feature within the readout's directions, which separates three sensitive checkpoints from nine working ones without overlap. With so small a set we call it a candidate (Sec. 6, Appendix D.4).
  • W5 (reliance on the appendix). The submission was hard to follow because of how it was written, and we apologize. The revision moves the evidence each claim rests on into the main text and rebuilds the appendix into six sections that follow it.
  • W6 (title). The revised PDF returns to the title registered on OpenReview (general comment).
Reviewer_vmSGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how well actions can be linearly decoded from the pooled features of VLA policies built on Qwen3-VL backbones with a small MLP action head, fine-tuned with LoRA or full fine-tuning on LIBERO and CALVIN. It reports three findings. First, the frozen backbone already contains action information but spread across many directions, and fine-tuning concentrates it into a few leading principal directions. Second, at evaluation time, keeping only the top 64 directions preserves most of the success rate, while 64 random directions preserve none. Third, some fully fine-tuned policies remain readable yet fail under controlled input changes, such as a table-texture swap on CALVIN or a one-token prompt change on LIBERO. From this, the authors argue that action decodability does not certify reliable closed-loop control.

W:

  1. Clarity and organisation. The paper is very hard to follow. The abstract and introduction consist largely of numerical results with little explanation. Key terms are used before they are defined. The paper does not explain why decodability is a useful lens for VLA fine-tuning, or why PCA-based feature interventions are the right tool. Methodological details are spread across the Setup section, the experiments and eleven appendices, so the contributions and the logic linking the sections are difficult to reconstruct. The submitted title is different from the OpenReview title.

  2. The relationship between decodability and reliable control is not clear. The paper highlights that action decodability does not guarantee reliable control, but the it only uses input interventions to support the statement, which is not convincing and cannot show that these experiments directly refer to decodability.

Critical strengths

None

Critical weaknesses
  1. Clarity and organisation. The paper is very hard to follow. The abstract and introduction consist largely of numerical results with little explanation. Key terms are used before they are defined. The paper does not explain why decodability is a useful lens for VLA fine-tuning, or why PCA-based feature interventions are the right tool. Methodological details are spread across the Setup section, the experiments and eleven appendices, so the contributions and the logic linking the sections are difficult to reconstruct. The submitted title is different from the OpenReview title.

  2. The relationship between decodability and reliable control is not clear. The paper highlights that action decodability does not guarantee reliable control, but the it only uses input interventions to support the statement, which is not convincing and cannot show that these experiments directly refer to decodability.

Questions for authors

The pooled feature is trained end-to-end to feed a one-hidden-layer MLP that regresses actions. High linear decodability from a few leading directions after fine-tuning therefore seems a natural consequence of the objective, not a non-trivial property of the representation. Could the authors explain what this result reveals beyond what the training setup already implies?

In Sec. 6, readability is measured on in-distribution demonstration frames, while the failures occur under shifted inputs (the swapped table texture, the mismatched prompt). Could the authors measure readability on the shifted inputs themselves, e.g. on scene-D or texture-swapped frames?

The variance-matched deletion control failed: removing PCs 2–17 collapses every policy, just as removing the top direction does. Given this, what evidence supports the claim that the policy depends on action-carrying directions, not simply on high-variance directions whose removal pushes the head’s input out of distribution?

2 comments
AuthorsOpenReview ↗

Response to Reviewer vmSG (2/2)

Q2: readability on the shifted inputs. We measured it after submission on the inputs of the texture intervention itself: 1,558 recorded scene-D states, each rendered with the table's own texture and with table A's. Each pair below gives the fragile Emb-2B policy on the slider and light frames, first with the texture under which it solves 0 of the 226 such subtasks, then with the one under which it solves 223.

  • A readout refit under each texture: R² 0.36 against 0.40 (0.47 under both over all frames).
  • A readout fitted once on training-scene frames and held fixed: 0.18 against 0.40.
  • The error of the policy's own head against the demonstrated actions: 0.19 against 0.09.

So under the texture that breaks the policy the action is still in the feature, but a fixed map no longer reads it. The shift itself is not what breaks a policy: six tuned full-finetuning policies and three LoRA seeds, trained on the same data, are evaluated on the same unseen scene and work (4.02–4.34 and 4.35 subtasks in a row), and for them none of the three quantities moves by more than 0.011. Nor does every failure need a shift: under the training prompt, the second LIBERO seed of the fragile recipe still fails most long-horizon episodes (169 of 500 succeed), although eight directions read 82% of the action on its demonstration frames (working policies: 90–93%). And under the shifted prompt, a readout held fixed from the training prompt still reads 90% from eight directions (92% before) for a policy that loses half its success (Sec. 6, Appendices D.1, D.4 and D.5). These comparisons are descriptive, on two fragile policies trained once each. The readouts by skill and the fixed readout are reported in our replies; the revised PDF has the value over all frames and the head's error (Appendix D.4).

Q3: the deletion control. You are right that the deletions cannot separate the two accounts. Deleting directions 2–17 (14–24% of the variance) collapses every LIBERO policy of Table 3, as deleting the leading one does (67–76%); the control was not variance-matched, and that label is gone. Three other observations bear on your question.

  • The directions are action directions. The leading one follows the gripper command (rank correlation 0.84–0.85), the next two the vertical and lateral motion (Fig. 2).
  • A shift that stays within the training range steers the robot as predicted. In an experiment registered on 17 October and run after submission (Appendix C.6), we shifted the head's input by one standard deviation along each of the four leading directions of the Emb-2B LoRA policy of Table 3 during closed-loop control (221 states from 40 tasks, 16 steps each). Along the first, the gripper opened or closed as predicted in 86% of the states, and the hand moved as well (opening goes with descending, closing with lifting). Along each of the next three, the hand ended 9–13 cm apart between the two signs of the shift, on the predicted axis. Random directions of the whole feature, shifted by a comparable amount, also moved the hand the way the head's response predicts, but by about 0.5 cm, 16–24 times less. Random mixtures of the six leading directions also move the hand (7–10 cm), so this speaks for the leading subspace, not for single axes. Shifting a VLA's activations and reporting the physical effect is not new (Appendix C.6 cites earlier work); here the directions are the principal directions of the head's input, found without labels.
  • The amount removed does not decide the outcome. For the CALVIN Emb-8B policy, dropping everything outside the 64 leading directions removes 25.6% of the variance and keeps 96.5% of the chain length, while deleting the leading direction removes 24.3% and leaves 48% (Appendix C.3).

So the head reads the action from the leading directions and is nearly indifferent to equal shifts elsewhere. What the deletions leave open is narrower: for the LIBERO policies, every deletion among the leading directions that removes 14% or more of the variance breaks the policy, so Sec. 5 claims only that a few dozen leading directions suffice.

Just so you notice, the 1st part of your response is missing.

Reviewer_tvGzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The submission is poorly organized, making its central research question and contribution difficult to identify.

Critical strengths

None.

Critical weaknesses

The PDF and OpenReview titles communicate different focuses. The abstract reads as a disconnected sequence of numerical findings rather than a coherent explanation of the problem, gap, approach, and contribution. The introduction adds further experimental details without clearly establishing what remains unresolved in prior work or what substantive insight this paper contributes.

The introduction attributes the underlying policy family to “Anonymous, 2026,” whose bibliography entry contains only “Reference withheld for double-blind review” without a title, and the submission is built directly on this work. The presentation relies heavily on dense numerical reporting without any illustration of the method or qualitative comparisons.

Overall, this submission does not meet the basic standards expected of an academic paper.

Questions for authors

None.

1 comment
AuthorsOpenReview ↗

Response to Reviewer tvGz

Thank you for the review. In short: the "Anonymous, 2026" entry stands for an existing, peer-reviewed paper, which the revision now cites in full; the way the submission piled up numerical findings was a fault of our writing, for which we apologize; and the work was directed by its authors, not generated by an automated AI system, although AI tools were used throughout and disclosed. The revision is a full rewrite, and because it differs substantially from the original, we do not ask for a re-evaluation on its basis.

"Anonymous, 2026". The entry stands for a paper that exists: it was accepted at a peer-reviewed conference and is on that conference's public list of accepted papers. Hiding it behind a placeholder was a mistake, because it left you unable to see what this submission builds on. The revision gives the full reference, title included, and cites the paper in the third person like any other published work. It also separates what this submission takes from that paper, the recipe (a LoRA adapter and a one-hidden-layer MLP head, trained one LIBERO suite at a time), from what is new: one policy for all 40 LIBERO tasks, the CALVIN policies, and analyses that extend that paper's single-suite probes and revise two of its readings (Secs. 1–3).

Abstract and introduction. You are right that they read as a string of separate numerical findings. That was a fault of how the submission was written, and we apologize for the difficulty it caused. Both now follow one line: the problem (a recipe without robot pretraining reaches the top tier on LIBERO and CALVIN, so what does finetuning add to the VLM?), the method (linear readouts of the feature the VLM hands to the action head, before and after finetuning, and edits of that feature in closed loop), and four findings and contributions, each tied to a section and to what earlier work already showed.

Title and method illustration. The PDF's title was changed in the last week before the deadline without updating OpenReview; the revised PDF takes the OpenReview title. A new Fig. 2 shows, over one held-out episode, what the three leading feature directions follow (the gripper, vertical and lateral motion commands) and how much of each action dimension the leading directions carry. A schematic of the policy did not fit within the ten pages; Sec. 3 now describes the policy, the readout and the feature edits in one place.

Ethics flag: use of AI. We respectfully disagree with the statement that the submission was generated through AI automated research. No automated system produced it; the authors directed the work. They chose what to study, the experiments were run at their request, and they revised the text. AI tools assisted at each stage: in proposing hypotheses, designing protocols, writing code, interpreting results and drafting. The submission disclosed this use in its AI-use statement, as the ICLR policy requires, and the revision rewrites that statement to describe the extent more precisely. The authors take responsibility for all content, and we are glad to answer any question from the AC or the program chairs about how the work was carried out.

Ethics flag: evidence behind the numbers. Every number in the revision was checked against the raw logs and result files, through a table that records the source of each printed value (with AI assistance, as the statement says); the errors this found in the submitted version are corrected, and the main ones are listed in the general comment. The results added after submission are listed in the general comment with their registration dates, and reported in the revised appendix, which indexes them (Appendix E.3). Code, configurations, registration files and evaluation logs will be released, as the reproducibility statement says, and an anonymized copy can be provided to the area chair, the program chairs or the ethics board on request.

Reviewer_pQdSOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how fine-tuning a pretrained VLM into a manipulation policy changes the linear decodability and concentration of action information, and whether these properties indicate closed-loop reliability. Across Qwen3-VL variants on LIBERO and CALVIN, the authors find that fine-tuning substantially improves action decodability from pooled features and concentrates it into leading principal directions. Keeping a small subspace preserves substantial policy performance. However, some fully fine-tuned policies remain highly sensitive to changes in the evaluation prompt or scene texture. The strongest result is the bidirectional texture intervention on CALVIN with geometry held fixed. The main concern is that the statement “aggregate demonstration-frame decodability does not guarantee closed-loop reliability,” while supported, remains weak. The analysis does not sufficiently distinguish this observation from metric aggregation, errors in control-critical components, and the amplification of prediction errors during execution.

Critical strengths

Strengths

  1. The texture intervention provides compelling behavioral evidence. Table 4 holds geometry, initial states, and policy parameters fixed while exchanging textures. Across two backbones, this switches slider/lightbulb performance between 223–226/226 successes and 0/226, with recovery under the reverse intervention. This is substantially more informative than simply observing a cross-scene performance gap.
  2. The subspace interventions operate on an interface actually used by the policy. In the matched LIBERO setting, the pooled feature is the action head’s only input. Consequently, keep-only interventions directly test whether the existing policy can operate through a compressed representation. Repetition across model sizes, pooling choices, and adaptation methods strengthens this empirical observation.
  3. The paper handles negative results and methodological corrections transparently. The authors disclose and correct the bf16 update issue, the evaluation-prompt mismatch, and earlier interpretations. They explicitly distinguish reader-unseen from backbone-unseen data. Fresh-head controls, task-centered probes, and unseen-task evaluations are also useful additions.
Critical weaknesses

The title listed on OpenReview, “Grounding Is Inherited, Action Is Concentrated: What VLA Finetuning Adds to a Web-Pretrained VLM,” does not match the title of the uploaded manuscript, “Readable Is Not Enough: Action Decodability and Closed-Loop Control in VLA Fine-Tuning.” These titles also frame the paper’s central contribution differently. Please clarify this discrepancy and ensure that the OpenReview metadata accurately reflects the submitted manuscript.

Questions for authors

As shown in weakness, please checki if the authors misupload the manuscript.

1 comment
AuthorsOpenReview ↗

Response to Reviewer pQdS

Thank you for the review; your summary describes the paper accurately. In short: for the texture swap of Table 4, our data rule out two of the three explanations you name, metric aggregation and amplification during execution, and confirm the third, an error in the policy's own output on the affected skills. The PDF was not a wrong upload. As our general comment explains, the revision is a full rewrite and we do not ask for a re-evaluation on its basis; if you do open it, Sec. 6 (about one page) and Appendix D.4 bear on your concern.

Main concern (end of your summary). As you say, the submitted analysis did not separate the observation from the three explanations. The texture swap allows it, because every recorded state was rendered under both textures. Below, each pair gives the fragile Emb-2B policy on table D with the table's own texture, under which it solves 0 of the 226 slider and light subtasks, and then with table A's, under which it solves 223.

  1. Amplification is not needed. On the recorded slider and light states, with no execution, the error of the policy's own head against the demonstrated actions is already 0.19 against 0.09. How much compounding adds to it was not measured.

  2. Aggregation does not produce it. A readout refit under each texture (64 directions) gives R² 0.36 against 0.40 on the slider and light frames alone, and over all frames no action component changes by more than 0.03. The one exception: from only the eight leading directions, the x translation (the slider's axis) reads 0.02 against 0.31.

  3. An output error on the affected skills is what the data show; we have it by skill, not by action component or time step. The head's error doubles on these two skills only (at most 0.014 on the other six skill families), and a readout fitted once on training-scene frames and then held fixed reads the slider and light frames at 0.18 against 0.40.

We read this as follows: under the texture that breaks the policy the action is still in the feature, but a fixed map, such as the policy's own head, no longer reads it; a refit readout does. This is why the revision states the claim as a boundary: readability on demonstration frames ranks policies but does not flag the inputs that break them (Sec. 6). The second fragile policy, and the Emb-2B policy halfway through training, show the same three results without the exception in 2; for the nine working checkpoints none of these quantities moves by more than 0.03. These comparisons are descriptive, and each fragile policy is a single training run. Apart from the head error in 1, these numbers are reported here; the revised PDF has the pooled readout (Appendix D.4).

Title (your weakness and question). The uploaded PDF is the intended manuscript; the mismatch was our oversight. In the last week before the deadline we retitled the PDF "Readable Is Not Enough", which put the reliability claim in front, and did not update the OpenReview entry. The experiments run after submission narrowed that claim to a boundary, so the revised PDF returns to the registered title (general comment, item 1).

Table 4 (your first strength). One addition changes how it reads. Trained on the same data and batch at a tenth of the learning rate, the Emb-2B policy completes 4.26 subtasks in a row on scene D instead of 0.95, and the texture no longer affects it (225–226 of 226 in all four combinations; now a row of Table 4). Five more tuned policies change by at most 0.9 points under the swap, and three LoRA seeds solve all 226 in every combination (Appendices D.2 and D.5). The sensitivity belongs to a learning rate too high for the batch, not to full finetuning.

Corrections found after submission are listed in the general comment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to the area chair and the reviewers

We thank the area chair and the reviewers. We agree with the main criticisms of the presentation and of the reliability claim; experiments run after submission narrowed that claim to a boundary. We respectfully disagree with the ethics flag (item 3). The revised PDF is a full rewrite; because it differs substantially from the original, we do not ask for a re-evaluation on its basis. Our replies to each review answer each point with the results we have.

1. Title (all four reviewers). The uploaded PDF is the intended manuscript; the mismatch was our oversight. In the last week before the deadline we retitled the PDF "Readable Is Not Enough", which put the reliability claim in front, and did not update OpenReview. The experiments run after submission narrowed that claim to a boundary (items 6 and 7), so the revised PDF returns to the registered title, "Grounding Is Inherited, Action Is Concentrated", which names what the paper shows.

2. "Anonymous, 2026" (Reviewer tvGz). The entry stands for an existing, peer-reviewed paper; hiding it behind a placeholder was a mistake. The revision gives the full reference and cites it in the third person. It takes the recipe from that paper; new here are one policy for all 40 LIBERO tasks, the CALVIN policies, and the analyses, which revise two of that paper's readings.

3. Ethics flag (Reviewer tvGz). We respectfully disagree with the statement that the submission was generated through AI automated research: the work was directed by its authors, not generated by an automated AI system, although AI tools were used throughout and disclosed. In detail:

  • the authors chose what to study, the experiments were run at their request, and they revised the text;
  • AI tools assisted in proposing hypotheses, designing protocols, writing code, interpreting results and drafting; the submission's AI-use statement disclosed this, as the ICLR policy requires, and the revision describes the extent more precisely;
  • code, configurations, registration files and evaluation logs will be released; an anonymized copy can be provided to the area chair, the program chairs or the ethics board on request.

The authors take responsibility for all content and are glad to answer any question about how the work was carried out (full answer: our reply to Reviewer tvGz).

4. What the revised paper says. Without robot pretraining, a web-pretrained VLM with a small action head and LoRA reaches the top tier on LIBERO and CALVIN. What does finetuning add? Four findings:

  • grounding is inherited: the frozen VLM already holds the task, the arm's position, the episode's progress and most of the action, but spread over many directions;
  • finetuning concentrates the action in a few leading directions of the feature; part of this is expected from regression training (Andriopoulos et al., 2024), and on unseen tasks it holds only in part (Reviewers vmSG, Q1; 8uaR, Q1);
  • a few dozen leading directions carry the policy, though deletions cannot isolate a causal direction (Reviewer vmSG, Q3);
  • concentration does not guarantee reliable control: trained with too high a learning rate, a policy stays readable while a change of table texture switches whole skills off (items 6 and 7).

5. Hard to follow (Reviewers 8uaR, vmSG and tvGz). We agree, and we apologize: this was a fault of how the submission was written. The revision is rewritten around the question of item 4: terms are defined at first use, the methods are in one place (Sec. 3), the evidence for each claim is in the main text, and the appendix is rebuilt into six sections.

6. Readability on the inputs that break a policy (Reviewers vmSG, Q2; 8uaR, Q3; pQdS). We measured it on the changed inputs after submission: under the texture that breaks the policy, the action stays readable to a readout refit on the changed input, while a readout held fixed, and the policy's own head, lose it. This rules out metric aggregation and amplification during execution (Reviewer pQdS). The claim is now a boundary: readability on demonstration frames ranks policies but does not flag the inputs that break them. Data (fragile Emb-2B policy, CALVIN slider and light frames, breaking against working texture):

  • refit readout: R² 0.36 against 0.40;
  • fixed readout: 0.18 against 0.40;
  • error of the policy's own head: 0.19 against 0.09.

7. Untuned full-finetuning baseline, few runs (Reviewer 8uaR, Q2). The suspicion was right. The fragile policies come from a learning rate too high for the batch; with a tuned rate, full finetuning matches LoRA and the texture no longer matters. Data: six tuned policies complete 4.02–4.34 subtasks in a row on CALVIN's unseen scene, against 4.35 ± 0.07 for LoRA, and the texture swap changes their slider, light and LED subtasks by at most 0.9 points.

AuthorsOpenReview ↗

Reference: what the revision changes, and results added after submission

For reference. In short: the evidence of the submission is unchanged; ten results were added after submission, each pre-registered, and one of them, the learning-rate sweep, narrowed the reliability claim. Most paragraphs changed and the appendix was rebuilt, so a PDF diff will be large. The abstract on OpenReview is updated to match.

What the revision changes, by section. The presentation is rewritten throughout (item 5 of our general response), and the claims of Secs. 5 and 6 are narrowed. By section:

  • Sec. 3: Table 1 adds a full-finetuning policy trained at learning rate 1e-6 (4.26 on CALVIN scene D) and the frozen-backbone reference under the training prompt.
  • Sec. 4: a new Fig. 2 on what the leading directions carry (the gripper's phase and the hand's translation, not its rotation); readouts without the gripper; Table 2 on transfer beyond the training data.
  • Sec. 5: the deletion control is described as unable to isolate action-specific directions, and the label "variance-matched" is removed; the claim is that a few dozen leading directions suffice for the trained head.
  • Sec. 6: retitled "Concentration does not guarantee reliable control" and narrowed as in items 6 and 7; the prompt sensitivity is described as positional, not "one-token".
  • Appendix: six sections (A–F); Appendix E indexes every registration with scorecards, misses included. Material no longer used (LIBERO-Plus, Open-X initialisation, latency, quantisation) was removed; its registered predictions stay in the scorecards.
  • Corrections to the submitted text: the prompt change is positional; the within-suite correlation is 0.48, not 0.76; the frozen-backbone reference is 35.5%, not 25.95%; the R² is variance-weighted and dominated by the gripper; the 40-task LIBERO and the CALVIN results are attributed to this submission.

Results added after submission. Ten results, each pre-registered before it ran (dates in JST; index and scorecards in Appendix E.3). New policies were trained only for the sweep, for the study of how the texture gating forms and for the two further seeds; everything else uses checkpoints and data that existed at submission, and the released StarVLA checkpoint. Follow-up studies registered after submission on questions this paper does not address are named in Appendix E.1, without results. Finding first, then numbers:

  • 10 Oct, learning-rate and batch sweep on CALVIN: tuned full finetuning matches LoRA. Six policies at 4.02–4.34 on scene D; texture effects of at most 0.9 points (Appendix D.5).
  • 10 Oct, texture swap for LoRA: no effect. Three seeds solve every slider, light and LED first subtask in each table combination tested (Appendix D.2).
  • 10 Oct, frozen backbone with a fresh head on LIBERO: 35.5% of 2,000 episodes under the training prompt, against 25.95% under the mismatched one (Table 1, Appendix C.5).
  • 10 Oct, re-analysis of stored features: the leading directions carry the gripper and the translation, hardly the rotation. The gripper holds 67.7% of the target variance on LIBERO; without it eight directions explain 80–89%, but 1–14% of the rotations (Appendix B.3).
  • 12 Oct, prompt variants on the LIBERO goal suite: the sensitivity is positional. For the fragile policy an extra space before the "8" does what "next 10" does: 75 and 74 of 500 successes, against 483 with the training prompt (Appendix D.3).
  • 12–13 Oct, readability under the texture swap: item 6. Also a check that separates three texture-sensitive checkpoints from nine working ones, and how the gating forms: present halfway through training, carried by the backbone (Appendices D.4, D.5).
  • 14 Oct, fresh heads under the training prompt: a CALVIN adapter's feature carries over to LIBERO in part. 60.5% on the CALVIN adapter, 34.6% on the frozen backbone, 97.8% on the LIBERO adapter (800 episodes each; Appendix B.5).
  • 17 Oct, shifting the head's input along its leading directions in closed loop: the robot moves as the head's response predicts. Shifts along directions 2–4 leave the hand 9–13 cm apart, 16–24 times as far as random directions (Appendix C.6).
  • 9 Nov, StarVLA's released LIBERO policy: the concentration replicates; one action component per direction does not. Eight directions explain 93% of the action's variation; 197 of 200 episodes succeed with only the 64 leading directions kept, none with 64 random ones (Appendix C.4).
  • 10 Nov, two further training seeds for the unseen-task readouts: stable when the readout is fit on the new tasks, seed-dependent when it is not. R² from all of z / from its eight leading directions: 0.66 / 0.57 on average, and 0.42 / 0.43 with seeds differing by up to 0.07 (Table 2, Appendix B.5).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.