acceptodds
Withdrawn from ICLR 2027

BEFORE THE COLLAPSE: MEASURING AND RESTORING PLASTICITY IN CONTINUAL VISION-LANGUAGE-ACTION REINFORCEMENT LEARNING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Sequential RL post-training of large vision-language-action (VLA) policies has recently been reported to be surprisingly stable: with LoRA adapters and on-policy GRPO, near-zero backward transfer is achieved across standard benchmarks, and stability is attributed to scale, low-rank constraints, and pretraining. We argue that this focus on forgetting inverts the field's priorities: low forgetting can coexist with, and even mask, insufficient plasticity (the capacity to keep learning), yet current evaluations measure only behavioral success, never network-level trainability. We present a closed loop of measure, intervene, predict for continual VLA RL. On a five-task stream of increasing difficulty, a standard forgetting-oriented anchoring objective suffers a catastrophic policy collapse at the fourth stage: success on all ten tasks falls to zero simultaneously, preceded by a structural fingerprint (effective-rank contraction, unbounded norm growth) that is absent in the unanchored baseline, whose explored subspace instead expands. The fingerprint prescribes the remedy: DARE-style sparsification of the collapsed adapter restores acquisition at the collapse stage () with retention within pp of the baseline, and directional rank-channel release is therapeutic ( random release) while improving the healthy frontier; both replicate across seeds. A lightweight forgetting-risk predictor built on representation similarity and NTK-overlap theory rank-orders streams by their actual forgetting behavior. These deliverables turn the reliability of continual VLA RL from an emergent accident into an engineered property.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VISION-LANGUAGE-ACTION,REINFORCEMENT LEARNING,PLASTICITY,CONTINUAL Learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_kSjZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies continual RL on VLA models. While previous works have shown that CRL on VLA can be quite stable in terms of catastrophic forgetting, this paper argues that an overlooked problem is the plasticity. To ensure that plasticity is preserved during CRL process, the paper proposed a set of detection and plasticity preservation mechanisms, and tested them on the libero-spatial benchmark.

Critical strengths
  • The motivation calling for a focus on stability in CRL is reasonable and important.
Critical weaknesses
  • The writing regarding the connection between the motivation and the proposed method of this paper is hard to follow / understand
  • It is not clear what part of the method is novel and what part of the method is directly taken from previous work.
  • The experiment section of this paper has significant issue.
    • Table 1 has several missing entries with comments like "Lost to an infrastructure fault" -- is this paper unfinished? (And the result doesn’t even support their claim)
    • Table 4: Behavioral column is in caption but is absent from the table.
    • 2 training epoches are very short; only tested on libero spatial.
    • A very simple baseline they should have tested is sequential LoRA expansion with periodic merging.
Questions for authors

N/A. I suggest the authors to finish their experiments before submitting.

Reviewer_pjVcOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates plasticity loss in continual reinforcement learning of vision-language-action models and proposes a framework combining structural and behavioral diagnostics, LoRA adapter rejuvenation, and forgetting-risk prediction.

Critical strengths

The paper addresses a meaningful research problem, and some of its experimental observations merit further investigation.

Critical weaknesses

I appreciate the paper’s perspective and research direction, but I cannot recommend acceptance in its current form. I encourage the authors to continue developing this work and substantially improve the clarity and organization of the manuscript.

  1. Unclear focus and organization. The paper shifts between plasticity loss, anchoring-induced collapse, and forgetting-risk prediction without clearly explaining their connections. This makes it difficult to identify the main contribution and understand what each experiment establishes. The paper should distinguish a general loss of learning ability from instability caused by a specific training objective.

  2. Claimed contributions and completed experiments are mixed together. Risk-Gated L2-Init is presented as a main method, but sustained gated intervention is left to future work. Appendix A also defers the plasticity-injection recovery curves to the camera-ready version. These are important parts of the proposed framework, not minor implementation details. The manuscript should clearly separate validated results from proposed methods and future experiments.

  3. The recovery experiments are described inconsistently. The abstract claims to restore a collapsed adapter, but Table 3 applies the intervention to the task-7 checkpoint before training task 8, where the original collapse occurs. Mitigating a subsequent collapse is different from recovering an already collapsed model. The intervention timing and starting checkpoint must be stated consistently, since they directly affect the interpretation of the results.

  4. Several conclusions exceed the available evidence. The predictor is supported mainly by a ranking check on four pilot streams, while Appendix B describes forward validation without reporting its results. This is insufficient to establish reliable early prediction or a complete measurement–intervention–prediction loop. Similarly, the conclusion that forgetting has largely been solved is too broad for the reported experiments. These claims should be narrowed to the findings actually demonstrated.

Questions for authors

Please address the questions and concerns raised in the Weaknesses section.

Reviewer_L8wxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates whether VLA models retain the capacity to learn new tasks during continual reinforcement learning, arguing that low forgetting alone is insufficient to demonstrate effective continual learning. In experiments on LIBERO-Spatial, the authors observe that a forgetting-oriented anchoring objective can cause policy collapse, accompanied by reduced adapter effective rank and increased parameter norms. By randomly sparsifying adapters or pruning LoRA channels based on their contribution at pre-collapse checkpoints and then continuing training, they find that these interventions can improve learning on the next task while preserving some previously acquired skills. However, the benefit of sparsification is temporary and does not persist into the following task.

Critical strengths
  1. The authors identify a clear failure case in which a policy trained with a forgetting-oriented anchoring objective collapses while the unanchored baseline remains functional. Their analysis of adapter effective rank and parameter norms provides useful evidence for understanding this instability.

  2. The authors investigate simple interventions on LoRA adapters and evaluate them on LIBERO-Spatial. Their experiments show that sparsifying adapters or pruning channels based on their contribution at pre-collapse checkpoints can improve next-task learning after continued training, while preserving some previously acquired skills.

Critical weaknesses
  1. The experimental evidence does not fully support the proposed measurement, intervention, and prediction framework.
  2. The sustained practical advantage of the interventions remains unclear.
  3. The evaluation is limited to one VLA model, one simulation suite.
Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.