acceptodds
Under review as a conference paper at ICLR 2027

PONIX: Pseudo-Task-Free Continual Adaptation of Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models must continually adapt to new tasks while preserving performance on previously learned tasks. Task-directed robotic data collection groups demonstrations of the same intended task, but assigning globally consistent task identities across collections requires additional annotation and coordination. We formulate *pseudo-task-free continual learning*, where demonstrations arrive in task-homogeneous mini-batches without oracle task identities or task boundary signals. We introduce **PONIX** (Pseudo-task-free ONline Inferred eXperts), an exemplar-free method that exploits this local structure. **PONIX** uses frozen multimodal representations and online routing statistics to select among action-side low-rank adaptation experts, exploiting within-batch task homogeneity to guide expert expansion. Evaluations across LIBERO and Meta-World with GR00T, , and SmolVLA show that **PONIX** approaches oracle-task-ID performance in both average acquisition success and forgetting mitigation. On revisitation experiments, **PONIX** achieves comparable average acquisition success to adaptation using task boundaries while maintaining approximately half as many experts. Real-world robot experiments further show that **PONIX** distinguishes different tasks expressed by the same instruction and reuses an expert for the same task expressed using different instructions. These results demonstrate continual VLA adaptation through inferred expert allocation and reuse, without oracle task identities, task-boundary signals, or stored exemplars.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Vision-Language-Action model, Continual Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HmvUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

PONIX uses frozen multimodal features and online statistics to select and expand a pool of LoRA experts. It addresses continual VLA adaptation from task-homogeneous batches without task IDs, boundary signals, or stored examples.

Critical strengths
  1. The router uses online feature statistics without training an additional network or retaining past demonstrations.

  2. Experiments cover three VLA backbones and two simulation benchmarks, with additional evaluations of task revisits and real robot manipulation.

Critical weaknesses
  1. Sec. 3.1 motivates PONIX through inconsistent task identities across demonstration collections. The discussion of prior work mainly lists dependencies on task labels, boundaries, or stored data. It does not explain which failures in the intended collection workflow make these methods inadequate, or why adding a simple task-identification module to an existing adapter framework would not suffice.

  2. A central practical claim is avoiding task-ID annotation, yet the paper provides no estimate of the annotation burden or savings. It reports the number of experts without measuring the storage and routing costs of maintaining them. The evaluation therefore does not establish the practical trade-off between annotation effort and the resources needed by the proposed workflow.

  3. The expert-adaptation framework has limited novelty. Lora experts, automatic routing, and dynamic expansion have close precedents, including DMPEL and CLARE. PONIX's incremental contribution is concentrated in its statistical routing and expansion rules for homogeneous batches without boundary signals. The paper does not demonstrate that these rules offer a substantive advantage over simple online clustering and expert reuse.

  4. Tab. 1 mainly compares PONIX with Seq and TAIL, yet these baselines do not test whether PONIX allocates experts better than another method given the same information and resources.

  5. The paper provides no ablations of variance-normalized routing, the requirement that every sample increase spread before expansion, or the joint use of visual and language features. Success curves and activation maps cannot demonstrate which design choices account for its performance. The results cannot distinguish the benefits of these rules from those of pretrained feature separability and parameter isolation across LoRA experts.

Questions for authors
  1. What concrete failures do existing adapter methods encounter in the intended collection workflow? Could collection metadata or a simple module that infers task IDs from visual and language inputs make these methods applicable? Please explain what remains unresolved by such alternatives.

  2. How much annotation or coordination effort does PONIX save? Please report parameter storage and routing overhead as experts accumulate, and evaluate retention under a fixed expert budget.

  3. What specific difficulty introduced by removing task boundaries is addressed by PONIX's statistical rules? Please clarify the advance over DMPEL and CLARE, accounting for their access to boundary signals or retained data, and explain why simpler expert-assignment rules would be inadequate.

9 comments

We thank the reviewer for the constructive feedback and for recognizing the efficiency of our online statistical router, which requires neither an additional trained network nor stored demonstrations. We also appreciate the acknowledgment of our extensive evaluation across three VLA backbones, two simulation benchmarks, task revisitation, and real-robot manipulation. We have strengthened the manuscript with additional baselines, routing ablations, robustness analyses, and computational measurements. We address the concerns point by point below.


Comment 1: Motivation for PTFCL and task-ID inference alternatives

Sec. 3.1 motivates PONIX through inconsistent task identities across demonstration collections. ... It does not explain which failures in the intended collection workflow make these methods inadequate, or why adding a simple task-identification module to an existing adapter framework would not suffice. ... Could collection metadata or a simple module that infers task IDs from visual and language inputs make these methods applicable?

Authors’ response:

We have clarified the practical motivation in Section 3.1 and Figure 1. Task annotation itself requires interpreting multimodal demonstration data, while maintaining globally consistent task identities across independently collected demonstrations introduces an additional coordination burden, particularly when tasks recur.

  1. Task annotation is nontrivial. Identifying a robot task requires interpreting language instructions, visual observations, and demonstration trajectories, rather than relying on language alone. Existing annotation frameworks, such as RoboAnnotatorX (Kou et al., 2025) and NILS (Blank et al., 2024), address the challenge of generating language descriptions and temporally grounded annotations. However, such annotations do not inherently establish globally consistent latent task identities across collections (Section 3.1).

  2. Collection metadata does not guarantee consistent task identity. Collection identifiers distinguish recording sessions, not underlying tasks. As illustrated in Figure 1, the same task may appear under different instructions, while distinct tasks may share an identical instruction. Maintaining consistent IDs therefore requires semantic comparison across demonstrations and coordination on a common criterion for task equivalence.

  3. Empirical evidence of inconsistent instructions. We explicitly evaluate both failure cases in Sections 5.3–5.4:

    • Different instructions, same task: Under paraphrased revisitation, PONIX maintains 10.25 experts for 10 underlying tasks, whereas instruction-based TAIL allocates 20 experts (Table 3).
    • Same instruction, different tasks: In our real-robot evaluation, PONIX assigns separate experts to the two tape manipulation tasks despite their identical instructions (Figure 3 and Appendix D.3).
  4. No additional task-ID inference model is required. A separately trained task-identification module is a possible alternative, but would introduce another inference and model-maintenance component, potentially requiring supervision to establish task equivalence. PONIX instead reuses frozen multimodal features already computed by the VLA, requiring only lightweight online statistics and no additional VLM forward pass (Section 4.1).

These clarifications are now incorporated into Section 3.1, which explains both the difficulty of multimodal task annotation and the additional burden of maintaining globally consistent task identities.

Comment 2: Methodological novelty relative to DMPEL and CLARE

The expert-adaptation framework has limited novelty. Lora experts, automatic routing, and dynamic expansion have close precedents, including DMPEL and CLARE. ... What specific difficulty introduced by removing task boundaries is addressed by PONIX's statistical rules? Please clarify the advance over DMPEL and CLARE, accounting for their access to boundary signals or retained data ...

Authors’ response:

We agree that PONIX is intentionally simple, as explicitly emphasized in the Abstract and Section 1. We do not claim novelty in the individual components, such as LoRA, Gaussian kernels, or online moment updates. Rather, our contribution lies in a lightweight expert-management mechanism that enables continual VLA adaptation without task identities, boundary signals, or stored exemplars.

  1. Different training assumptions. DMPEL and CLARE both assume task-boundary signals. DMPEL initializes new experts at task stages, while CLARE performs task-level discriminator creation and adapter expansion. Their original training procedures therefore depend on identifying task transitions. In contrast, PONIX receives only incoming mini-batches, without task or collection identities or boundary indicators (Section 3.2, Algorithm 1).

  2. Boundary-free expansion and expert reuse. Without task boundaries, the learner must distinguish a genuinely new task from a recurring task using only the current batch and accumulated statistics. PONIX addresses this through variance-aware Gaussian routing and a unanimous spread-based expansion criterion. A new expert is created only when every sample in the incoming task-homogeneous batch would individually increase the selected expert's spread. Otherwise, the existing expert is reused and updated (Sections 4.1–4.2).

  3. Empirical validation of expert reuse. Under paraphrased task revisitation, PONIX achieves 61.7% mean SR, compared with 50.1% for instruction-based TAIL, while using approximately half as many experts (Table 3). Our real-robot experiments further demonstrate expert reuse under both unchanged and paraphrased instructions (Section 5.4, Figure 3).

Table R1. Task revisitation under paraphrased instructions, averaged across four LIBERO suites (Section 5.3, Table 3).

MethodMean FWT ↑Mean SR ↑Mean NBT ↓Mean AUC ↑Mean experts ↓
Instruction-based TAIL56.5%50.1%1.9 pp58.6%20
PONIX61.6%61.7%0.3 pp60.8%10.25

Thus, PONIX demonstrates that a simple, lightweight mechanism can support effective expert assignment, expansion, and reuse under PTFCL, without introducing additional trained routers or relying on task-boundary supervision.

Comment 3: Insufficient comparison with baselines under matched information constraints

The paper does not demonstrate that these rules offer a substantive advantage over simple online clustering and expert reuse. ... Tab. 1 mainly compares PONIX with Seq and TAIL, yet these baselines do not test whether PONIX allocates experts better than another method given the same information and resources.

Authors’ response:

We have extended the evaluation with reservoir experience replay (RER) and Euclidean-distance (ED) expert assignment, providing additional comparisons under the PTFCL setting.

  1. Task-free replay baseline. We added RER with a 5,000-example reservoir buffer, requiring neither task identities nor boundary signals. Across 24 backbone–suite settings, PONIX achieves a 10.8 percentage-point mean AUC improvement over RER and outperforms it in 21 of 24 settings (Section 5.2, Table 2).

  2. Comparison with simple distance-based assignment. We additionally evaluate nearest-centroid ED routing with distance-threshold-based expert expansion. The ED baseline exhibits substantial threshold sensitivity: small thresholds lead to excessive expert creation, while large thresholds merge distinct tasks (Section 5.5, Table 6; Appendix E.4, Tables 20–21). This comparison evaluates the complete distance-based assignment and expansion strategy, rather than treating ED as a variance-aware expansion method. The routing studies use cached representations and evaluate expert allocation rather than full policy success.

  3. Oracle performance as a reference. We retain TAIL as an oracle-task-ID reference, not as an information-matched baseline. Despite receiving no task identities, PONIX achieves 67.9% mean SR and 66.9% mean AUC, compared with 66.4% and 66.4% for TAIL, respectively, across the 24 settings (Table 2). In contrast, Seq suffers substantial forgetting despite its competitive acquisition success.

Table R2. Continual-learning performance across 24 backbone–suite settings (Section 5.2, Table 2).

MethodTask-ID / boundary informationMean SR ↑Mean AUC ↑Mean NBT ↓
TAILOracle task ID66.4%66.4%−0.1 pp
SeqNeither67.3%21.9%58.0 pp
RERNeither62.8%56.1%8.5 pp
PONIXNeither67.9%66.9%1.9 pp

All values are equally weighted averages across 24 backbone–suite settings. The reported variability in Table 2 reflects evaluation seeds using the same trained checkpoints.

These results demonstrate that PONIX achieves strong acquisition and retention without oracle task information or exemplar replay. The ED experiments further characterize the sensitivity of simpler distance-based expert-assignment strategies.

Reviewer_BTYNOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces pseudo-task-free continual learning, where a VLA receives task-homogeneous mini-batches without task identities or boundary signals, and proposes PONIX, an exemplar-free method. PONIX routes each mini-batch to one of a growing pool of action-side LoRA experts on a frozen VLA via a Gaussian kernel over running statistics of frozen VLM embeddings, and adds a new expert when every sample would increase the selected expert's spread. Experiments with GR00T, π0, and SmolVLA on LIBERO and Meta-World, under task revisitation, and on a real SO-101 arm show performance close to the oracle-task-ID baseline TAIL and expert reuse.

Critical strengths

The evaluation is relatively broad. PONIX is tested with three VLAs (GR00T, π0, and SmolVLA) on two simulation benchmarks, all four LIBERO suites and the four Meta-World difficulty groups (the latter with π0 and SmolVLA only), as well as on a real SO-101 arm. This makes the main empirical finding that PONIX performs close to oracle-task-ID adaptation without task identities credible across architectures and benchmarks, although it does not outweigh the thin methodological contribution.

Critical weaknesses
  • The methodological contribution is very thin. PONIX is a combination of standard components: plain LoRA adapters on the action side of a frozen VLA trained with the base policy loss (Section 4.3), a diagonal Gaussian score between the batch-mean embedding and each expert's running mean and variance (Eq. 1), the textbook moment-merging updates of Chan et al. (1982) and Welford (1962) (Eqs. 2 and 4), and an expansion rule that adds an expert when all samples of a batch lie outside the selected expert's RMS radius (Section 4.2). None of these steps is new, and adding adapters to a pretrained policy and routing among them has already been done in prior work that the paper itself cites, such as DMPEL (Lei et al., 2026) and CLARE (Römer et al., 2026). Section 2 distinguishes PONIX from these methods mainly by not using task boundaries, which are replaced by the assumption of task-homogeneous mini-batches. In my view, this difference alone is not a sufficient methodological contribution.
  • The experimental results reflect this. PONIX performs similarly to TAIL, which the paper itself frames as "approaching" oracle-task-ID performance (SR within five points of TAIL in 14 of 20 settings in Table 1), and it ends up with the same number of adapters as TAIL in 17 of 20 settings. In the revisitation experiment (Table 2), FWT is within five points and NBT within two points of both TAIL variants. The only clear difference is that PONIX reuses adapters when a task recurs, which reduces the number of experts from 20 (Boundary TAIL) to 10 or 11. Instruction TAIL already needs only 10 adapters on these suites, since instruction lookup is equivalent to task-ID lookup in LIBERO (Section 5.3). Adapter reuse by itself is not novel, so these results do not make up for the thin methodology.
  • The real-robot experiment (Section 5.4) tests the scenario that motivates the setting in Figure 1, namely identical instructions for different tasks and different wording for the same task, but critical details are missing from the main text. Section 5.4 does not state how many demonstrations were collected per collection, how long each collection was trained, or how many trials the reported success rates are based on. These details take little space and should be in the main paper, where there is actually some space left. In addition, the real-world evaluation uses only a single platform (a SO-101 toy arm with SmolVLA) and three simple pick-and-place tasks, making it difficult to assess whether the results generalize to bimanual platforms and more complex tasks that involve, for example, deformable objects.
  • Less critical, but relevant for the presentation: the appendix is far too long and should focus on the important information. The submission has 81 pages, of which pages 14 to 81 are appendix, with 135 figures in total. This gives the impression that every generated plot was added without selection. The paper also contains several typos and formatting errors.

References:

  • B. P. Welford, "Note on a method for calculating corrected sums of squares and products", Technometrics, 1962.
  • T. F. Chan, G. H. Golub, and R. J. LeVeque, "Updating formulae and a pairwise algorithm for computing sample variances", COMPSTAT 5th Symposium, 1982.
  • Y. Lei, S. Mao, S. Zhou, H. Zhang, X. Li, and P. Luo, "Dynamic mixture of progressive parameter-efficient expert library for lifelong robot learning", 2026.
  • R. Römer, Y. Zhang, Y. Li, and A. P. Schoellig, "CLARE: Continual learning for vision-language-action models via autonomous adapter routing and expansion", IEEE Robotics and Automation Letters, 2026.
Questions for authors

Clarification requests:

  • What is the methodological novelty of PONIX compared to other approaches that inject adapters into pretrained policies, such as DMPEL or CLARE? Section 2 mainly distinguishes these methods by their use of task boundaries. If the batch-level expansion test of Section 4.2 is the main difference, the authors should state this explicitly and explain why it is a substantial contribution.
  • According to Appendix A.2, every result uses the same training seed 1000. Did the authors only vary the evaluation seed for the ± values? If so, could the authors conduct proper statistical tests across training seeds?

Suggestion for future work:

  • Evaluating PONIX on at least one additional real robot embodiment besides the SO-101 arm would help to assess whether the real-world findings generalize.
  • The main text should include the key real-robot details (number of demonstrations, training budget, number of trials, and per-task success rates), and the appendix should be condensed to the plots that support the main claims.
8 comments

We thank the reviewer for the detailed and constructive feedback. We particularly appreciate the recognition of our broad evaluation across three VLA architectures, multiple simulation benchmarks, and real-world manipulation, as well as the acknowledgment that PONIX achieves performance close to oracle-task-ID adaptation without task identities. We have revised the manuscript with additional baselines, task-revisitation experiments, real-robot evaluations, and computational analyses. We address the concerns point by point below.


Comment 1: Limited methodological novelty compared with DMPEL and CLARE

The methodological contribution is very thin. PONIX is a combination of standard components: plain LoRA adapters ... a diagonal Gaussian score ... the textbook moment-merging updates ... and an expansion rule ... None of these steps is new ... Section 2 distinguishes PONIX from these methods mainly by not using task boundaries ... In my view, this difference alone is not a sufficient methodological contribution.

What is the methodological novelty of PONIX compared to other approaches that inject adapters into pretrained policies, such as DMPEL or CLARE? ... If the batch-level expansion test of Section 4.2 is the main difference, the authors should state this explicitly and explain why it is a substantial contribution.

Authors’ response:

We respectfully clarify that our contribution is not the individual statistical estimators or LoRA adaptation, but a simple, exemplar-free expert management mechanism that enables continual VLA adaptation without task identities or boundary signals. The key distinction is how PONIX autonomously determines when to reuse or expand experts from incoming task-homogeneous mini-batches.

  1. Task-boundary dependence of prior methods. Both DMPEL and CLARE assume task-segmented training. DMPEL initializes new experts at task stages, while CLARE introduces new discriminators and performs adapter expansion at task transitions. Consequently, neither original training procedure directly supports PTFCL without modification. We use oracle-task-ID TAIL as an upper-reference baseline for task-informed adaptation.
  2. Task-free expert expansion and reuse. PONIX introduces a unanimous spread-based expansion criterion (Section 4.2, Algorithm 1): a new expert is created only when every sample in the incoming mini-batch would individually increase the selected expert's spread. This exploits task homogeneity to make expansion decisions without explicit boundaries, while allowing recurring tasks to reuse existing experts.
  3. Simple and lightweight implementation. PONIX uses representations already computed by the frozen VLM, with no additional VLM forward pass, separately trained router, or stored exemplars. The added routing computation averages only 4.4 ms per action chunk across the profiled configurations (Section 5.6, Appendix F).

Our emphasis is therefore on enabling autonomous expert selection, expansion, and reuse under a weaker task-information assumption, rather than introducing new statistical estimators.

Comment 2: Limited experimental distinction from TAIL and insufficient evidence for expert reuse

The experimental results reflect this. PONIX performs similarly to TAIL ... and it ends up with the same number of adapters as TAIL in 17 of 20 settings ... Instruction TAIL already needs only 10 adapters on these suites, since instruction lookup is equivalent to task-ID lookup in LIBERO ... Adapter reuse by itself is not novel, so these results do not make up for the thin methodology.

Authors’ response:

We have strengthened the evaluation to demonstrate the practical benefits of PONIX under task-free learning conditions and inconsistent instruction-based task identities, beyond its performance relative to oracle TAIL.

  1. Comparison with a task-free replay baseline. We added reservoir experience replay (RER), which requires neither task identities nor boundary signals. Across 24 backbone–suite settings, PONIX outperforms RER in AUC in 21/24 settings, with a mean improvement of +10.8 percentage points, without retaining demonstrations (Section 5.2, Table 2).

    Table R1. Mean continual-learning performance across 24 settings (Section 5.2, Table 2).

    MethodSR (%) ↑AUC (%) ↑Task boundariesExemplar replay
    Oracle TAIL66.466.4RequiredNo
    RER62.856.1Not requiredYes
    PONIX67.966.9Not requiredNo
  2. Paraphrased task revisitation. We revised the revisitation experiment to simulate noisy instruction-based task labels, where the same latent task is revisited with a paraphrased instruction. Unlike the original LIBERO setting, instruction lookup no longer provides consistent task identities. Instruction-based TAIL consequently allocates 20 experts, whereas PONIX uses 10.25 experts on average for 10 underlying tasks, with higher SR and FWT (Section 5.3, Table 3).

    Table R2. Mean performance under paraphrased task revisitation (Section 5.3, Table 3).

    MethodFWT (%) ↑SR (%) ↑NBT (pp) ↓AUC (%) ↑Experts ↓
    Instruction-based TAIL56.550.11.958.620
    PONIX61.661.70.360.810.25
  3. Expert management beyond instruction lookup. PONIX correctly reuses existing experts for recurring tasks in all but one revisitation case, where it allocates one additional expert. These results demonstrate that PONIX's expert assignments largely follow latent task identity rather than instruction wording. We also added AUC to the main evaluation and end-of-stream SR in Appendix B.3 to characterize sustained performance.

Together, these results show that PONIX provides practical benefits when globally consistent task identities cannot be obtained from instructions.

Comment 3: Missing real-robot details and limited task complexity

The real-robot experiment (Section 5.4) tests the scenario that motivates the setting in Figure 1 ... but critical details are missing from the main text ... how many demonstrations were collected per collection, how long each collection was trained, or how many trials the reported success rates are based on ... the real-world evaluation uses only a single platform ... and three simple pick-and-place tasks ...

Authors’ response:

We have expanded the real-robot evaluation and clarified its experimental protocol in Section 5.4, Figure 3, and Appendix D.

  1. Complete experimental protocol. We collected 200 demonstrations across six collections and four latent tasks, using 5,000 training updates per collection. We conducted 20 physical evaluation trials per previously seen collection at each checkpoint, totaling 420 physical trials. Collection-level demonstration counts are indicated in Figure 3, and the requested details are now in the main body (Section 5.4) with complete settings and trial outcomes are provided in Tables 13–15 (Appendix D).

  2. More diverse manipulation tasks. We extended the evaluation beyond simple pick-and-place by including sliding-box opening and lidded-bowl opening, both requiring manipulation of articulated objects before object placement.

  3. High final success and correct expert reuse. PONIX achieves 95–100% final success across all four latent tasks, while reusing experts under paraphrased or unchanged instructions and separating distinct tasks sharing the same instruction (Figure 3, Appendix D.3).

    Table R3. Real-robot evaluation on SO-101 (Section 5.4, Appendix D).

    TaskDemonstrationsFinal successes / trialsFinal SR
    Pen placement6040/40100%
    Tape-dispenser placement3019/2095%
    Sliding-box opening + orange placement6038/4095%
    Lid opening + tape-roll placement5020/20100%
    Total200117/12097.5%

We currently do not have access to an additional robot platform, which prevented us from evaluating PONIX on another embodiment. We acknowledge that generalization to other robot embodiments, including bimanual manipulation and deformable-object tasks, remains an important direction for future work. This limitation is explicitly stated in Section 6.


Comment 4: Statistical reliability and training seeds

According to Appendix A.2, every result uses the same training seed 1000. Did the authors only vary the evaluation seed for the ± values? If so, could the authors conduct proper statistical tests across training seeds?

Authors’ response:

We thank the reviewer for highlighting this distinction. The reported variability in the main policy-performance experiments comes from evaluation seeds, not independent policy-training seeds. We have clarified the experimental protocols and the scope of the reported variability.

  1. Policy evaluation seeds. All policy-training runs use training seed 1000. The results in Table 2 report mean ± sample standard deviation across three evaluation seeds using the same trained checkpoint, as explicitly stated in its revised caption.
  2. Routing-ablation seeds. The routing-only ablations in Section 5.5 and Appendix E use three independent cached-embedding replay seeds, evaluating sensitivity to routing-stream sampling. These do not constitute independent policy-training runs.
  3. Explicit limitation. We acknowledge that variability across independently trained policies remains unexamined and have stated this limitation in Section 6. Accordingly, we do not interpret evaluation-seed variability as statistical significance across training runs.
Reviewer_e8koOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses continual adaptation of vision-language-action (VLA) models without requiring oracle task identities or task-boundary signals. It introduces pseudo-task-free continual learning (PTFCL), where incoming mini-batches are task-homogeneous but the learner is not informed when tasks change or previously encountered tasks recur. To operate under this setting, PONIX uses frozen VLM representations and online feature statistics to automatically select, reuse, or expand action-side LoRA experts, without storing past demonstrations. Gaussian-kernel routing selects the most compatible expert, while a batch-level spread criterion determines whether a new expert is needed.

Experiments on LIBERO and Meta-World with GR00T, π0\pi_0, and SmolVLA show that PONIX achieves acquisition and forgetting performance close to oracle-task-ID TAIL in many settings. Task-revisitation and real-robot experiments further demonstrate that PONIX can reuse experts for recurring tasks and distinguish different tasks sharing the same instruction, without relying on explicit task identities.

Critical strengths
  1. The paper identifies a practically motivated middle ground between task-aware continual learning and unrestricted task-free continual learning. The assumption of task-homogeneous mini-batches is reasonable given how robotic demonstrations are typically collected, while avoiding the need for globally consistent task IDs and explicit task-boundary signals.

  2. PONIX provides a lightweight approach to expert routing and expansion using frozen VLM features and online statistics, without requiring a separately trained router or stored exemplars. The experiments cover multiple VLA architectures and benchmarks, with additional task-revisitation and real-robot evaluations demonstrating expert reuse and adaptation performance close to oracle-task-ID baselines in many settings.

Critical weaknesses
  1. Limited comparison with closely related methods. The evaluation mainly compares PONIX against sequential fine-tuning and oracle-task-ID TAIL. However, prior work such as CLARE[1] has already explored autonomous adapter routing and expansion for continual VLA learning. Although these methods may operate under different task-information assumptions, comparisons with adapted baselines or simpler routing and expansion strategies under matched settings would better establish the contribution of PONIX.

[1] Römer, R., Zhang, Y., Li, Y., & Schoellig, A. P. (2026). CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion. IEEE Robotics and Automation Letters, 11(7), 8331–8338.

Questions for authors
  1. How sensitive is the expert expansion decision in Algorithm 1 (Line 16) to mini-batch size? Since expansion requires all B samples to individually increase the selected expert's spread, larger batches may make expert creation less likely.

  2. How robust is PONIX to more irregular task-revisitation patterns? The current revisitation experiments repeat tasks in the same order with equal training budgets. It would be interesting to examine whether expert reuse remains effective when tasks recur at varying intervals and with different visit lengths.

  3. The reported SR in Equation (6) measures acquisition success immediately after each task's last training visit, rather than the average success rate across all tasks at the end of the learning stream. The latter would be a more meaningful metric in continual learning, as it directly reflects how well the model retains performance on previously learned tasks at the end of the learning stream. I suggest reporting the end-of-stream average success rate to complement the metrics.

3 comments

We thank the reviewer for the constructive feedback and for recognizing the practical motivation of PTFCL, particularly its task-homogeneous mini-batch assumption that avoids globally consistent task IDs and explicit boundary signals. We also appreciate the positive assessment of PONIX's lightweight, exemplar-free expert management, broad evaluation across VLA architectures and benchmarks, and demonstrated expert reuse on real robots. We address the concerns point by point below.

Comment 1: Limited Comparison with Related Methods

Limited comparison with closely related methods. The evaluation mainly compares PONIX against sequential fine-tuning and oracle-task-ID TAIL. However, prior work such as CLARE[1] has already explored autonomous adapter routing and expansion for continual VLA learning. ... comparisons with adapted baselines or simpler routing and expansion strategies under matched settings would better establish the contribution of PONIX.

Authors’ response:

We have strengthened the evaluation with a task-free reservoir replay baseline and a simpler Euclidean-distance expert routing/expansion baseline, while clarifying the different training assumptions of CLARE.

  1. Comparison with CLARE. CLARE depends on task-boundary signals when training and performs discriminator and adapter expansion at task stages. Directly applying its original training procedure under PTFCL would require unavailable task-boundary information. PONIX instead makes reuse and expansion decisions for each incoming mini-batch without such signals.

  2. Additional task-free baseline (Section 5.2, Table 2). We introduce Reservoir Experience Replay (RER), which requires neither task IDs nor boundary signals and maintains a buffer of 5,000 examples. Across 24 backbone–suite settings, PONIX achieves 67.9% mean SR and 66.9% mean AUC, compared with RER's 62.8% mean SR. PONIX outperforms RER in AUC in 21 of 24 settings, with an average improvement of 10.8 percentage points, without storing exemplars.

Table R1. Continual-learning performance across 24 backbone–suite settings (Section 5.2, Table 2).

MethodTask informationMean SR ↑Mean AUC ↑
TAILOracle task ID66.4%66.4%
RERTask-free, with replay62.8%56.1%
PONIXBoundary-free, exemplar-free67.9%66.9%
  1. Simpler routing/expansion baseline (Section 5.5, Table 6; Appendix E.4). We additionally evaluate Euclidean-distance (ED) routing with distance-threshold expansion. ED is sensitive to its threshold: small thresholds produce excessive experts, whereas larger thresholds merge distinct tasks. Suitable threshold ranges also vary across backbones. In contrast, PONIX maintains stable expert allocation across the tested scatter-prior values (Table 4). These routing-only analyses are reported separately from policy-training comparisons.

Comment 2: Mini-batch Size Sensitivity

How sensitive is the expert expansion decision in Algorithm 1 (Line 16) to mini-batch size? Since expansion requires all B samples to individually increase the selected expert's spread, larger batches may make expert creation less likely.

Authors’ response:

We have added a batch-size sensitivity study covering B ∈ {64, 128, 256, 512}, demonstrating stable expert allocation across all evaluated settings.

  1. Stable expert counts (Section 5.5, Table 5). Across all 24 backbone–suite settings, the number of initialized experts remains unchanged as batch size increases from 64 to 512.
  2. Stable task separation (Appendix E.3). Task-normalized routing conflict remains zero or negligible across batch sizes, except for the consistent Meta-World Hard task merge. The only additional deviation is 0.00027 ± 0.00031 for GR00T on LIBERO-Spatial at B = 512.

Table R2. Batch-size sensitivity across three VLA backbones (Section 5.5, Table 5).

Benchmark / SuiteExperts across B = 64–512Routing conflict
LIBERO-10 / Spatial / Object / Goal10 per suite≈ 0
Meta-World Easy280
Meta-World Medium110
Meta-World Hard51
Meta-World Very Hard50

All results are consistent across three embedding-replay seeds. The study isolates routing and expansion with frozen LoRA parameters; larger batches process more input presentations under the fixed update budget.

These results show that larger batches do not suppress expert creation in the evaluated settings, despite the unanimous expansion condition.

Comment 3: Irregular Task Revisitation

How robust is PONIX to more irregular task-revisitation patterns? The current revisitation experiments repeat tasks in the same order with equal training budgets. It would be interesting to examine whether expert reuse remains effective when tasks recur at varying intervals and with different visit lengths.

Authors’ response:

We have extended the revisitation evaluation with paraphrased instructions in simulation and irregular revisit intervals on a real robot, demonstrating expert reuse beyond consistent instruction labels.

  1. Paraphrased revisitation (Section 5.3, Table 3). We introduce a revisitation protocol in which recurring tasks receive paraphrased instructions. Across four LIBERO suites, PONIX uses 10–11 experts for 10 latent tasks, whereas instruction-based TAIL allocates 20 adapters. PONIX achieves 61.7% mean SR, compared with 50.1% for TAIL, showing that expert reuse remains effective when instruction labels change.

  2. Irregular real-robot revisitation (Section 5.4, Figure 3; Appendix D). We evaluate a six-collection stream with different revisit intervals: pen → tape1 → orange → pen → orange → tape2. PONIX successfully reuses the existing pen expert under a paraphrased instruction and the orange expert under the original instruction, without creating duplicate experts.

Table R3. Expert reuse under real-robot task revisitation (Section 5.4, Figure 3).

Recurring taskCollectionsInstructionObserved result
PenC1 → C4ParaphrasedExisting expert reused; 100% SR
OrangeC3 → C5UnchangedExisting expert reused; SR improved from 75% to 95%
  1. Evaluation under physical execution. Across 420 real-robot trials, PONIX retains 95–100% final success on all four latent tasks, demonstrating successful acquisition, retention, and reuse across the collection stream (Appendix D, Tables 15–16).

The revised experiments establish robustness to varying revisit intervals and instruction variation. Visit-specific training budgets remain fixed in these evaluations; varying training durations more broadly remains an important direction for future work.


Comment 4: End-of-stream Success Rate

The reported SR in Equation (6) measures acquisition success immediately after each task's last training visit, rather than the average success rate across all tasks at the end of the learning stream. ... I suggest reporting the end-of-stream average success rate to complement the metrics.

Authors’ response:

We agree that end-of-stream performance is important for evaluating continual retention. We have added AUC to the main results and final success rate to Appendix B.3, complementing acquisition SR with measures of sustained and final performance.

  1. AUC as a continual-learning metric (Section 5.2, Table 2). We now report the area under the success-rate curve, which summarizes performance throughout the learning stream. PONIX achieves 66.9% mean AUC, compared with 66.4% for oracle-task-ID TAIL across 24 settings.

  2. End-of-stream average SR (Appendix B.3, Table 11). We additionally evaluate all tasks at the final checkpoint. PONIX achieves 66.1% mean final SR, closely matching TAIL's 66.4% and exceeding RER's 50.9% by 15.2 percentage points.

Table R4. Mean end-of-stream success rate across 24 backbone–suite settings (Appendix B.3, Table 11).

MethodMean Final SR ↑
TAIL (Oracle Task ID)66.4%
RER50.9%
PONIX66.1%

These additional metrics demonstrate that PONIX retains performance close to oracle-task-ID adaptation even at the end of the entire learning stream, while substantially outperforming exemplar-based reservoir replay without retaining demonstrations.

Comments on the paper

3 comments

We sincerely thank all reviewers for their thoughtful feedback and constructive discussion. We particularly appreciate their prompt and engaged follow-up responses, which facilitated a productive exchange and helped us substantially strengthen the manuscript. In response, we have conducted extensive additional experiments, clarified the methodological contributions, and improved the presentation. We summarize the key revisions and reviewer-specific responses below.

1. Summary of Key Revisions

RevisionKey additionsLocation
PTFCL FormulationClarified task-ID annotation challenges, collection structure, and boundary-free learning assumptions§3, Fig. 1
Baselines & AblationsAdded RER, Euclidean-distance routing/expansion, modality ablations, and robustness analyses§§5.2, 5.5; App. E
Retention & ReuseAdded AUC, final SR, and paraphrased task revisitation§§5.2–5.3; Apps. B–C
Real-Robot EvaluationExpanded manipulation tasks, detailed experimental protocols, and irregular revisitation§5.4; App. D
Efficiency & BudgetMeasured storage/latency and evaluated a five-expert capacity constraint§5.6; App. F; rebuttal
Direct CLARE ComparisonCompared continual-learning performance across all four LIBERO suites, storage, inference latency, and expert reuseApp. G
Presentation & TransparencyCondensed the appendix, corrected formatting, and clarified evaluation versus training seedsApp. A; §6

2. Reviewer-by-Reviewer Summary

Reviewer e8ko

We appreciate the reviewer's recognition of the practical PTFCL formulation and lightweight expert management. The reviewer explicitly acknowledged that most concerns were resolved, particularly the additional baselines and revisitation robustness.

  • Baselines & Ablations: Added RER and Euclidean-distance expert assignment. PONIX outperforms RER in AUC across 21/24 settings (+10.8 pp on average). Expert counts remain stable across batch sizes 64–512 in all 24 settings (§§5.2, 5.5).
  • Retention & Reuse: Added paraphrased revisitation and irregular real-robot revisit intervals. PONIX reuses 10–11 experts for 10 latent tasks, while instruction-based TAIL requires 20. End-of-stream SR reaches 66.1%, close to oracle TAIL's 66.4% (Table 3; App. B.3).
  • Direct CLARE Comparison: Following the reviewer's suggestion, we additionally implemented CLARE and evaluated its performance across all four LIBERO suites and expert-management efficiency (App. G).

Reviewer BTYN

We appreciate the reviewer's acknowledgment of our additional experiments and improved presentation. We subsequently provided further evidence addressing the remaining methodological novelty concern.

  • Direct CLARE Comparison: Across all four LIBERO suites, PONIX achieves 66.2% mean AUC versus 60.5% for CLARE (+5.7 pp) and 66.2% mean final SR versus 60.4% (+5.8 pp), with 18.2× lower learned-module storage and 4.44× lower inference latency after ten tasks (App. G). CLARE receives task boundaries and an additional 2,000 discriminator-training steps per task.
  • Retention & Reuse: Under paraphrased instructions, PONIX achieves 61.7% SR versus 50.1% for instruction-based TAIL, while using approximately half as many experts (§5.3, Table 3).
  • Real-Robot Evaluation: Expanded to four latent tasks, including articulated manipulation, with 200 demonstrations and 420 physical evaluation trials. Final success reaches 95–100% across tasks (§5.4, App. D).
  • Presentation & Transparency: Substantially condensed the original 81-page submission, clarified experimental details, and explicitly distinguished evaluation-seed variation from independent training-seed variation (§6, App. A).

Reviewer HmvU

We appreciate the reviewer's recognition of our lightweight router and broad evaluation, as well as the clarification of the remaining novelty concerns. Following the reviewer's assessment that the initial revisions were insufficient, we conducted a direct CLARE comparison.

  • PTFCL Formulation: Clarified why task annotation and globally consistent task-ID management require multimodal interpretation and cross-collection coordination (§3.1, Fig. 1).
  • Baselines & Ablations: Added ED routing/expansion comparisons demonstrating threshold sensitivity, alongside modality, scatter-prior, and batch-size analyses (§5.5, App. E).
  • Efficiency & Budget: Quantified storage and routing overhead and added a five-expert-budget evaluation, revealing substantially increased forgetting and reduced AUC under constrained capacity (§5.6, App. F; rebuttal).
  • Direct CLARE Comparison: Beyond the four-suite performance and efficiency gains above, PONIX returns 97.5% of revisited training inputs to their original expert groups, compared with 53.6% of CLARE's module-level assignments. Storage grows by only 2.5% for PONIX versus 97.8% for CLARE during revisitation (App. G.2). The assignment percentages use different units and characterize training allocation rather than execution success.

3. Key Contributions of Our Paper

We believe the revised manuscript establishes three complementary contributions:

  1. A practically motivated continual-learning formulation. Most existing continual robot learning methods, including CLARE, rely on supplied task identities or task-boundary signals. PTFCL instead formalizes the natural structure of robotic demonstration collection—task-homogeneous mini-batches and temporal persistence without either signal—addressing the coordination burden of maintaining globally consistent task identities across independently collected demonstrations (§§2–3, Fig. 1).

  2. Simple, boundary-free expert management. PONIX exploits batch homogeneity through unanimous spread-based expansion and uses pretrained VLA representations with online statistics for expert routing and reuse. Unlike CLARE and most existing continual robot learning methods, PONIX requires neither task identities nor boundary signals. It also avoids CLARE's separately trained reconstruction discriminators, resulting in substantially lower storage and computational overhead (§4, App. G).

  3. Extensive empirical validation. Across three VLA architectures, LIBERO, Meta-World, and real-robot manipulation, PONIX achieves competitive acquisition and retention without oracle task information or stored demonstrations. Additional replay, routing, CLARE, and revisitation comparisons strengthen the evidence for its effectiveness and practical efficiency (§5, Apps. B–G).

4. Closing

We acknowledge the remaining limitations, including the absence of independent policy-training-seed variability and evaluation on additional robot embodiments.

Nevertheless, we believe the additional experiments and revisions provide substantially stronger evidence for the practical relevance, methodological distinction, and empirical effectiveness of PONIX. We hope this summary helps clarify the contributions and significance of our work.

We sincerely thank the reviewers again for their constructive feedback, prompt responses, and continued engagement throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.