acceptodds
Under review as a conference paper at ICLR 2027

PRESERVE WHAT MATTERS: COUNTERFACTUAL TASK-VECTOR DECOMPOSITION FOR REPLAY-FREE CONTINUAL VISION-LANGUAGE-ACTION LEARNING

Abstract

Vision-Language-Action (VLA) models have demonstrated strong capabilities in language-conditioned robotic manipulation. Yet operating in open-ended environ- ments requires continual acquisition of new skills, which can severely overwrite previously learned behaviors and lead to catastrophic forgetting. Existing replay- based methods mitigate forgetting by revisiting stored demonstrations, incurring persistent storage and repeated training costs, while replay-free approaches typi- cally preserve prior capabilities through allocating additional task-specific mod- ules, leading to growing model capacity as new skills accumulate. Inspired by task arithmetic, we view the parameter change induced by downstream fine- tuning as a proxy for newly acquired capability. This recasts implicit skill re- tention as explicit parameter-space protection, avoiding both historical demon- stration replay and task-specific model expansion. Building on this view, we fur- ther identify that not every direction in a historical task vector deserves equal preservation: transferable task functionality can be entangled with environment- specific correlations, so indiscriminately protecting the full update may unnec- essarily restrict future adaptation. We therefore introduce CSD-VLA, a replay- free framework that uses task-preserving counterfactual interventions to identify an environment-sensitive subspace, decomposes each historical task vector into environment-sensitive and counterfactually invariant components, and selectively protects only the latter. Experiments on OpenVLA-OFT-7B and QwenOFT-3B across LIBERO and RoboTwin 2.0 demonstrate that CSD-VLA outperforms rep- resentative replay-based and replay-free baselines in overall continual-learning performance, strengthens retention under both task-level and cross-capability shifts, and better preserves historical skills under unseen visual changes, without replaying historical demonstrations or expanding the policy architecture.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Vision-Language-Action Models, Replay-Free Learning, Task Vectors, Counterfactual Invariance, Robotic Manipulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ycMsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a continual learning method that uses controlled visual interventions to decompose task updates into environment-sensitive and invariant components. It protects the invariant components during learning to reduce forgetting. Experiments with two VLA backbones on LIBERO and RoboTwin 2.0 evaluate new-task learning, and robustness to visual changes.

Critical strengths

The use of controlled visual interventions to guide selective preservation is well motivated. Ablations comparing different protection strategies support the value of the selected directions, and improved performance under held-out visual changes provides evidence of more robust skill retention.

Critical weaknesses
  1. The averaging step in Eq. (2) is insufficiently validated. Gradient differences from different manipulation states may point in opposing directions and cancel out, causing the estimated subspace to miss state-dependent environmental sensitivity. The paper does not assess this risk, leaving the reliability of the resulting decomposition unclear.
  2. Figure 1(B) appears inconsistent with the stated counterfactual construction: the depicted object and robot configurations do not seem identical across the four examples, despite the claim that only environmental appearance changes. Several overly long sentences also make the method harder to follow.
Questions for authors

Have you examined whether counterfactual gradient contrasts also capture directions needed to improve task-relevant skills? For example, a background change may produce gradients that both reduce reliance on the background and improve object localization. What evidence supports leaving these directions unprotected without compromising skill retention?

Reviewer_UfvfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CSD-VLA addresses continual adaptation of a shared VLA policy without retaining previous demonstrations or adding deployed task-specific modules. At a task boundary, it records the parameter displacement induced by learning that task. It then evaluates paired action-loss gradients on 128 current-task states under six task-preserving visual interventions. The span of the gradient contrasts defines an environment-response subspace. The historical task vector is decomposed into its projection onto this subspace and an orthogonal remainder; only the latter contributes to an accumulated protection basis. Subsequent training penalizes parameter movement along the protected span. Experiments cover OpenVLA-OFT-7B and QwenOFT-3B on LIBERO and RoboTwin, with comparisons against replay and adapter-expansion methods. Additional analyses compare full versus selective protection, retention under held-out visual changes, and estimated training FLOPs.

Critical strengths
  1. The method preserves a fixed deployed policy and avoids replay of historical demonstrations. The required probes are taken while the current task data are available, and later tasks retain a basis rather than examples. The distinction between removing an environment-sensitive component from a protection constraint and removing it from the actual policy is stated clearly (Section 3.3). This gives a concrete training mechanism without requiring task identification or an expanding expert bank at inference.

  2. The empirical study covers several relevant forms of retention. Results span two backbones and two simulators, with separate task-level, cross-capability, and held-out-appearance evaluations. The FullProtect/EnvProtect/selective comparison is a useful starting point. In Table 4, selective protection improves final performance under each of the four visual-shift categories; this is more directly relevant to the proposed motivation than in-distribution retention alone.

Critical weaknesses
  1. The reported gains are difficult to reproduce or assess statistically from the supplied protocol. The manuscript does not give the value or tuning procedure for the regularization coefficient in Equation 8, the numerical thresholds used for the two basis operations, or the exact RoboTwin task identities and cross-capability sequence length. It provides point estimates without training-seed or task-order variability, while several gains over strong baselines are only about one to three percentage points (Tables 1–2). Some LIBERO results are taken from PHASER rather than rerun, and the NoProtect row in Table 3 mixes newly reported acquisition measures with cited retention measures. Exact metric formulas and per-stage performance matrices are needed to make these comparisons auditable. The FLOP estimate is helpful, but does not quantify persistent basis storage, which can grow by one parameter-dimensional direction per task under Equation 6.

  2. The evidence does not yet establish that counterfactual direction selection is the key source of improvement. FullProtect already provides most of the retention gain in Table 3. Comparing it at the same regularization strength is informative, but does not establish the relative stability–plasticity frontiers when each method is tuned appropriately. The environment-response basis has at most six directions estimated at one checkpoint, and the resulting orthogonal remainder need not identify generally transferable knowledge outside those interventions, as the paper itself acknowledges.

Questions for authors
  1. Please provide the regularization coefficient and its tuning protocol, orthogonalization thresholds, task lists/order, number of independent runs, and exact ASR/NBT/Cap-NBT/FWT definitions. Can per-stage task-success matrices and uncertainty estimates be supplied, including a consistently rerun NoProtect comparison?

  2. Does selective protection still outperform FullProtect after sweeping regularization strength for both? How does it compare with a protection basis of matched rank obtained without meaningful counterfactual contrasts, and with an appropriate fixed-capacity parameter-preservation baseline?

  3. How sensitive are the results to the 128 probe states, six intervention variants, and averaging gradients over states? What are the observed subspace ranks and the fractions of each task vector assigned to the protected and unprotected components?

  4. What is the persistent basis memory in bytes, its growth across tasks, and the measured wall-clock and peak-memory overhead including rendering/probing? Please distinguish these training costs from the unchanged inference architecture and FLOPs.

Reviewer_Ce32OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a new framework (CSD-VLA) for enabling continual learning for Vision-Language-Action (VLA) models. Many other approaches in continual learning involve replay (making models retrain on some older stored demonstrations) which would take up storage to preserve older demonstrations, or model expansion, making the model footprint itself larger as new skills need to be acquired. The approach here is one that uses neither replays (hence freeing up the cost of storing older demonstrations), nor model expansion (preventing the need to increase model size and complexity).

The core idea takes inspiration from Task Arithmetic, where the change in model weights from fine-tuning on a task tt encodes a direction in high dimensional parameter space corresponding to the skill required for that task: τt=ϕt−ϕt−1\tau_t = \phi_t - \phi_{t-1}. The authors point to counterfactual failures in VLA models and claim that this task vector can be decomposed into two components, one that is counterfactually invariant and the other that is environment sensitive.

τt=τtinv+τtenv.\tau_t = \tau_{t}^{inv} + \tau_t^{env}.

This decomposition is done by taking a small probe set from the task demonstrations, sampling states across the progress of each trajectory, varying lighting, surface textures, and distal background, and contrasting the action loss gradients.

In particular, the authors further claim that skill retention would correspond to preserving the direction of τtinv\tau_t^{inv}, meanwhile the counterfactual failure type shortcuts and sensitivity to environmental conditions live in τtenv\tau_t^{env}. They therefore maintain an orthonormal basis BtB_t representing the protected directions, and run one step of the Gram-Schmidt process after fine-tuning each new skill. In particular, the component of τtinv\tau_t^{inv} not already spanned by Bt−1B_{t-1} is normalized and appended to produce BtB_t, if its norm exceeds a numeric threshold.

While τtenv\tau_t^{env} is not removed, it is not explicitly marked for protection.

The overall loss function then gets a regularizer term that penalizes the changes along the protected directions.

Jt+1(θ)=Lact(ϕ;Dt+1)+λ2∥BtT(ϕ−ϕt)∥2\mathcal{J}_{t+1}(\theta) = \mathcal{L}_{\text{act}}(\phi; \mathcal{D}_{t+1}) + \dfrac{\lambda}{2} \lVert B_t^T (\phi - \phi_t) \rVert^2

CSD-VLA is evaluated on LIBERO as well as Robotwin 2.0

Critical strengths
  • Old demonstrations do not need to be stored, and even the probes and gradient contrasts are discarded after each task fine-tuning. This helps mitigate catastrophic forgetting without replaying the demonstrations, or increasing model complexity. These advantages make CSD-VLA much more realistically deployable compared to replay-based (ER, MIR, PHASER) and expansion-based (CLARE) methods. The added training compute is only about 1% over sequential fine-tuning
  • Compared to plain task arithmetic, this approach actually formulates and starts to reason through what to protect. Rather than protecting parameters by importance, the method chooses protected directions by contrasting action-loss gradients under rendering changes that leave the demonstrated action unchanged. Under lighting, texture, and background shifts, this outperforms protecting the full task vector.
  • The method shows comparable retention to the strongest replay method. This is a significantly strong result.
Critical weaknesses
  • The paper is missing baseline comparisons with other continual learning approaches from CSD-VLA's own category: non-replay and no model expansion. The regularizer has a form similar to Elastic Weight Consolidation (Kirkpatrick et al., 2017), differing mainly in using a projection onto the protected basis in place of the Fisher information. Yet EWC is not cited, and related subspace methods like GPM are neither discussed nor compared against. None of the chosen baselines are actually from the same category. This matters because without in-category comparisons, it is hard to judge the method’s novelty or how much of its gain is specific to it.
  • There seems to be an inconsistency in the definition of a counterfactual. The paper cites Fang et al., 2026 which talks about "Counterfactual Failures", where vision overrides language, resulting in the model taking shortcuts. That is, a semantically different language prompt under the same observations would produce no change to robot's actions. However, the counterfactually invariant direction is picked by varying the environment (lighting, texture, background) rather than the language prompt, while keeping the instruction and the correct action fixed, so they cannot isolate that failure mode. This matters because the decomposition is the paper’s central idea. Further, the decomposition’s contribution is small, and its interpretation doesn’t match the cited motivation. In the ablation, protecting the full task vector already raises final ASR from 10.0 to 87.2, and selective preservation of τtinv\tau_t^{inv} adds about 3 points (90.3) on a single backbone and suite, from what appears to be a single run, with no variance reported.
  • The current strongest VLA models that are flow-based, such as π0\pi_0, π0.5\pi_{0.5} and GR00T N1 are not evaluated. The same Fang et al. (2026) paper found that these exhibit weaker vision shortcuts than OpenVLA-OFT, so they are a meaningful additional axis for testing the method.
  • The method was only evaluated in simulation, and not on a real robot arm, leaving room for potential sim-to-real gaps.

Minor formatting note (not critical): I noticed the citation format changed in Section 4 (Experiments) likely using \citet instead of \citep. "We evaluate CSD-VLA on two robotic manipulation benchmarks, LIBERO Liu et al. (2023) and RoboTwin 2.0 Chen et al. (2025)." should probably say "LIBERO (Liu et al., 2023) and RoboTwin 2.0 (Chen et al., 2025)"

Questions for authors
  • Is the decomposition’s gain consistent across various seeds? Would it be possible to report mean ±\pm standard deviation over multiple seeds for CSD-VLA and FullProtect (Table 3)?
  • How does this regularization method compare with others in this category like EWC and GPM?
  • It seems unclear to me that environment-sensitive weight updates would necessarily mean they're task-irrelevant. Would you be able to expand on this? Lighting and texture can carry task-relevant information, such as cues for depth and contact or texture for distinguishing objects. Could you provide evidence that τtenv\tau_t^{env} does not encode such information?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses to Reviewers

We thank the reviewers for their constructive feedback. Below we summarize the additional experiments and clarifications provided in our responses.

Reviewer 1 (Ce32)

  • Additional baselines: Added EWC and GPM comparisons under both ID and OOD conditions.
  • Counterfactual formulation and contribution: Clarified the distinction between our appearance interventions and instruction counterfactuals, and explained that decomposition’s principal added value lies in cross-environment retention beyond FullProtect.
  • Broader validation: Added experiments with the flow-based π₀ VLA and three-seed FullProtect/CSD-VLA comparisons in a representative setting.
  • Task-relevant information: Clarified that environment-sensitive components may contain useful task information. CSD retains the complete learned policy; selective protection does not require perfect semantic separation.

Reviewer 2 (Ufvf)

  • Reproducibility and variability: Specified regularization settings, orthogonalization thresholds, task sequences, and metric formulas; reran Sequential FT under a consistent protocol and added three-seed and task-order evaluations.
  • Contribution of direction selection: Added ID/OOD regularization sweeps and a matched-rank random control. The comparisons support counterfactual selection’s OOD benefit after tuning FullProtect and beyond arbitrary direction selection.
  • Probing sensitivity and diagnostics: Varied probe-state and intervention budgets, compared Mean/SVD estimators, and reported subspace ranks and component fractions. Results show diminishing returns beyond the default probing budget.
  • Training resources: Reported persistent basis storage, complete-pipeline training time, and peak GPU memory, distinguishing training costs from the unchanged deployed architecture.

Reviewer 3 (ycMs)

  • Potential gradient cancellation: Added directional-agreement and net-response diagnostics together with a controlled SVD comparison. These provide no evidence of severe overall cancellation under the evaluated protocol.
  • Figure and presentation: Replaced the illustrative examples with genuine state-matched LIBERO renders and simplified the relevant Method descriptions.
  • Task-information entanglement: Added per-task ID retention analysis alongside OOD comparisons, showing a modest average ID forgetting cost with clear cross-environment retention gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.