acceptodds
Under review as a conference paper at ICLR 2027

What Behavior Cloning Gives and Takes: Action Learning and Visual Reorganization in VLAs

Abstract

A VLA can forget an answer without forgetting what it saw. During adaptation on LIBERO, a base-fitted localization readout degrades from 1.55 to 6.28 cm, yet refitting the same readout family on the adapted checkpoint reduces error to 1.13 cm. The visual information remains linearly recoverable; the pre-adaptation reader no longer matches its coordinates. We follow two behavior-cloning runs and align closed-loop control with native heads, fixed and refit probes, six-site representations, and matched interventions. The trajectory has three stages. Pre-adaptation readouts fail by 500 updates while success is still 0–5%. Control then rises to 85–88% by 4–8k while spatial information remains recoverable. As success reaches 94–97%, part of the checkpoint-local recoverability declines. A rank-64 orthogonal alignment closes over 95% of the destination-readout gap, the native language head fails on a separate schedule, and the action expert builds another destination code behind a frozen backbone. Gradient conflict arrives after the first remapping, and freezing its strongest layers offers no advantage over freezing random layers. Freezing and replay preserve selected readouts at an action cost; an auxiliary action-token objective accelerates the earliest action endpoint. Behavior cloning reorganizes the relations among representations, readers, and actions. Training should maintain or deliberately relearn those relations instead of preserving parameters indiscriminately.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robot imitation learning, behavior cloning, representation reorganization, readout compatibility, training dynamics, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_iws3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how behavior cloning changes visual representations in VLAs. Using π0.5 fine-tuned on LIBERO, the authors compare fixed and refitted probes, representation similarity, and control success throughout training. They report a three-stage pattern of early readout incompatibility, subsequent action learning, and later decline in some checkpoint-local probe scores. The paper also investigates representation alignment and several training interventions.

Critical strengths
  1. An interesting observation about visual forgetting.

The distinction between losing information and losing the ability to read it is important. The fixed-versus-refit probe results are quite convincing in this setting. In particular, the large gap between the original and refitted localization probes suggests that fine-tuning can change the representation without necessarily removing the information.

  1. Careful analysis.

The paper follows multiple checkpoints, examines several representation layers, and includes controlled interventions. The low-rank alignment results are interesting, and I like that the authors test the gradient-conflict hypothesis rather than assuming it explains the observed degradation.

Critical weaknesses
  1. The generality of the three-stage pattern remains unclear.

The main findings are based on two full fine-tuning runs of π0.5 on LIBERO. Although the authors provide additional intervention experiments, the proposed three-stage pattern has not been tested across different VLA architectures or adaptation settings.

The paper describes this pattern as a "three-stage law of adaptation," which seems too strong given the current evidence. Testing another VLA or a substantially different fine-tuning setup would help determine whether this is a general phenomenon or specific to the studied configuration.

  1. The scope and functional significance of the recovered information remain uncertain.

The paper provides strong evidence that certain spatial quantities remain recoverable after the original readouts fail. However, it is less clear whether this extends to broader vision-language capabilities.

The authors do evaluate GQA, object identity, and spatial relations, but the recoverability results focus mainly on spatial information. It would be useful to examine whether broader semantic capabilities remain recoverable, particularly on held-out or OOD visual-language tasks.

The authors correctly distinguish decodability from actual model use. However, the current experiments do not directly establish whether the recovered information can restore native language functionality or plays a causal role in action generation. The low-rank alignment results are convincing for destination decoding, but their functional implications remain unclear.

Questions for authors
  1. The paper shows that spatial information remains recoverable even after the native language head fails. However, does this also hold for broader vision-language capabilities? Could the authors evaluate the adapted checkpoints on held-out VQA, visual grounding, or OOD semantic tasks, rather than relying primarily on spatial probes? This would help clarify whether behavior cloning preserves general semantic information or only the specific spatial information examined here.

  2. Can the authors provide more direct evidence that the degradation of native VLM performance is mainly due to readout incompatibility rather than a loss of semantic representations? For example, could aligning the adapted representations with the original language head recover GQA or other vision-language performance? The current low-rank alignment results are convincing for destination decoding, but do not necessarily establish the same conclusion for broader semantic capabilities.

  3. Would the same representation changes and three-stage training pattern appear in another VLA architecture or fine-tuning setting? This seems important before drawing broader conclusions about behavior cloning in general VLAs.

Reviewer_T9NXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates the training of Vision-Language-Action (VLA) models through behavior cloning and examines how this training affects the model.

Critical strengths

None.

Critical weaknesses

This manuscript can be regarded either as a short technical report or as a very early draft of a research paper. In its current form, it lacks experiments with different models, evaluation across multiple benchmarks, and an in-depth analysis of the results. The manuscript is not ready for publication.

Questions for authors

No questions.

Reviewer_igAGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how behavior cloning changes visual representations and action performance in π0.5. By tracking action success, native language-head performance, and fixed versus refitted probes across training, the authors distinguish readout incompatibility from reduced information recoverability. Alignment and training interventions further examine these changes and their implications for preserving visual capabilities.

Critical strengths
  1. The paper addresses a meaningful question: degraded visual readout does not necessarily imply that visual information has been lost.
  2. Tracking action performance alongside native, fixed, and refitted readouts provides an informative view of how these capabilities change at different stages of training.
Critical weaknesses
  1. The evidence does not fully support the scope of the conclusions. Spatial-probe recovery does not establish that readout mismatch causes native language-head failure, and declining linear-probe performance does not demonstrate information loss. Moreover, the training trajectory is observed only in two π0.5 runs on LIBERO, leaving its generality across architectures and training settings unclear.

  2. The writing needs clearer logical transitions. Several passages move between different readouts, empirical observations, and mechanistic interpretations without explaining how one supports the next, making the argument difficult to follow.

Questions for authors
  1. Can adapting the relevant readout interface while freezing the adapted backbone recover native question-answering performance, providing more direct support for the mismatch explanation?

    1. Have you examined whether the reported training trajectory holds for another VLA architecture, such as OpenVLA or SmolVLA?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.