acceptodds
Under review as a conference paper at ICLR 2027

WAM-OPD: Sharpening World Action Models via On-Policy Distillation

Abstract

Pretrained world action models (WAMs) provide generalist capabilities across diverse robotic manipulation tasks, yet improving target-task performance to an expert level without degrading pretrained skills remains challenging. To this end, we explore on-policy distillation (OPD) for WAMs and introduce a new framework, namely WAM-OPD. WAM-OPD inherits the advantage of OPD methods that transfer task-specific teacher knowledge under the student’s own induced distribution, rather than directly fitting the student to a narrow task-specific data distribution. However, in closed-loop manipulation, the observation histories change as the student policy evolves, requiring fresh environment rollouts to remain on-policy. Directly applying OPD to WAMs entails repeated data collection, which is costly even in simulation and often impractical on real robots. To avoid repeated environment rollouts during distillation, we introduce prefix-weighted trajectory replay (PWTR). PWTR uses a fixed trajectory pool composed primarily of initial-student rollouts, supplemented with task-specific teacher rollouts to broaden trajectory coverage. For each trajectory replayed from this pool, PWTR conditions the current policy on successive stored histories to generate fresh denoising paths, along which the task-specific teacher provides supervision. Although these denoising paths are refreshed as the policy evolves, the replayed environment trajectories remain fixed. PWTR therefore reweights per-decision distillation losses using proxy importance weights derived from path scores accumulated over the trajectory prefix preceding each decision to mitigate the resulting shift in the history distribution. Simulated and real-world experiments demonstrate task adaptation without additional environment interaction during distillation. Across both settings, WAM-OPD improves target-task performance while retaining near-initial performance on tasks excluded from adaptation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, On-Policy Distillation, Robotic Manipulation, Robot Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tRHwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WAM-OPD to adapt pretrained World Action Models through on-policy distillation without additional environment interaction during distillation. Its core method, PWTR, reweights histories in a fixed trajectory pool using denoising-path compatibility scores. The results show promising target-task improvements with good skill retention.

Critical strengths

The paper addresses a practically important problem. The distinction between on-policy denoising states and off-policy environment histories is well motivated, and the simulation and real-robot results are encouraging.

Critical weaknesses
  1. Insufficient validation of PWTR as a history-distribution correction mechanism.

The ideal importance ratio in Eq. (8) depends on executed-action probabilities, whereas PWTR uses denoising-path compatibility scores from Eq. (3). These quantities are not necessarily equivalent, and temporal subsampling introduces further approximation. The paper lacks direct evidence that PWTR weights meaningfully approximate the evolving on-policy history distribution.

  1. Insufficient evidence for the independent contribution of history-specific weighting.

In Table 2, PWTR improves mixed-pool performance from 67.25% to 69.75% over uniform weighting. However, M-Shuffle does not preserve task- and trajectory-source-level weight statistics, making it difficult to isolate the benefit of history-specific weighting. Moreover, skill retention is not reported for these ablations.

Questions for authors
  1. Can the authors validate PWTR weights against approximate history importance ratios or online OPD update directions?
  2. Can the authors provide group-statistics-preserving weighting ablations and report both target-task performance and skill retention?
2 comments
AuthorsOpenReview ↗

Response to Reviewer [tRHw]

We thank the reviewer for the suggestions on direct validation. In response, we have added history-ratio and online-gradient diagnostics, a group-preserving shuffle control, and skill retention evaluations.

W1, Q1: History Ratios and Online Gradient Alignment

We have conducted both requested validations. PWTR weights correlate positively with a separately estimated history-ratio proxy. Among the three weighting schemes, PWTR also achieves the highest mean alignment with online OPD gradients.

1. Fresh student samples at stored histories. At each stored history from the fixed pool, the current student starts from fresh noise and generates an action-denoising path. Conditional action sampling therefore follows the current student at every update, with the teacher supplying velocity targets at query states along each path. As the student changes, the histories it would encounter in the environment may follow a different distribution from those in the fixed pool. PWTR approximately corrects this mismatch by reweighting the distillation loss at each stored history. Together, these steps approximate online OPD without collecting fresh environment rollouts during distillation.

2. History-ratio diagnostic. At updates 24, 72, and 96, we sampled fresh action chunks from the student checkpoint, initial student, and teacher at each stored decision. We fitted Gaussian densities to the executed-action coordinates and accumulated the resulting estimated log-density differences along each history, yielding a separate proxy for the history log-ratio in Eq. (8).

Within-group Spearman correlations between PWTR weights and the history-level proxy are 0.42, 0.39, and 0.36. Per-decision correlations between Eq. (3) scores and estimated log-density differences are 0.27, 0.62, and 0.63. Together, these comparisons assess the surrogate at both the decision and history levels.

3. Online-gradient alignment. On paired replay and online batches, mean cosine similarity with the online OPD gradient is 0.101 for PWTR, compared with 0.084 for unit weighting and 0.080 for G-Shuffle (defined below).

4. Decision-point subsampling. Using 8, 16, or all decision points yields target SRs of 69.00%, 69.75%, and 69.75%, respectively. Using all points removes prefix-sum subsampling and achieves the same SR as the default 16 points while requiring approximately 2.4× as many queries.

Manuscript revision. We will add the ratio and gradient analyses with their protocols to Sec. 4.3 and Appendix A, and the subsampling comparison to Appendix D.

W2, Q2: Group-Preserving Shuffle and Skill Retention

We have added the requested group-preserving shuffle control and retention evaluation. PWTR exceeds G-Shuffle by 5.25 points in target SR and 2.25 points in retention. These gains support the value of matching weights to the histories for which they were computed, beyond preserving group statistics.

1. Group-preserving shuffle protocol. The original M-Shuffle permutes weights among decision points at the same sampled position. G-Shuffle permutes weights within groups defined by task × collection condition × rollout source × normalized prefix depth (five bins). This preserves each group's weight distribution while changing the assignment of weights to histories.

2. Target performance and skill retention. We evaluated all three weighting schemes on the two target tasks and six retention tasks:

Mixed-pool weightingTarget SR (%) ↑Retention SR (6 tasks, %) ↑
Uniform67.2584.50
G-Shuffle64.5084.67
PWTR69.7586.92

Retention SR is the equally weighted mean success rate on Switch, Mug, Laptop, Pillbottle, Dual Bottles, and Handover, all excluded from adaptation.

PWTR also exceeds unit weighting by 2.50 points in target SR and 2.42 points in retention. These comparisons support the contribution of history-specific weighting to both adaptation and skill retention.

Manuscript revision. We will add G-Shuffle and retention results to Table 2 and document the grouping and shuffling protocol.

Dear Reviewer tRHw,

Following constructive discussions during the rebuttal period, we wanted to briefly follow up to let you know that we have uploaded the revised manuscript, incorporating all the diagnostics and ablations prompted by your feedback alongside a broader framing of our work:

  • Per-decision path-score vs. log-density difference & online-gradient diagnostics (W1, Q1): Detailed in Appendix A.5 (Tables 4 and 6) and referenced in Section 4.3, together with the decision-point subsampling analysis (n=8,16,Alln=8, 16, \text{All}) in Appendix D.8 (Table 14).
  • Group-preserving shuffle (G-Shuffle) & skill retention (W2, Q2): Added in Appendix A.5 (Table 5) (kept separate from Table 2 for layout clarity and cross-referenced in Section 4.3), confirming PWTR's gains in both target adaptation (5.25 percentage points over G-Shuffle and 2.50 percentage points over unit weights) and 6-task skill retention (2.25 percentage points over G-Shuffle and 2.42 percentage points over unit weights) under matched group-level weight statistics.
  • Broader framing across flow-based policies: In light of reviewer discussions on the generality of our action-level OPD formulation (validated on both Fast-WAM and LingBot-VLA in Section 4.5), we have updated the title to "RoboOPD: On-Policy Distillation for Flow-Based Policies in Robotic Manipulation" and unified the presentation around flow-based policies throughout the manuscript.

We sincerely appreciate your insightful questions, which substantially strengthened the empirical validation of PWTR. Please let us know if you have any further questions on the revised manuscript or the new results, and we would be happy to discuss further.

Reviewer_R8m3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes WAM-OPD, an on-policy distillation (OPM) method for continual learning in world-action models WAMs. The method seeks to investigate the benefit of on-policy distillation methods which have recently shown promise in the area of post-training LLMs. In particular, the authors seek to demonstrate that OPD for WAMs yield benefits in the continual learning setting, e.g., less catastrophic forgetting vs SFT when distilling task-specific experts. The method is evaluated on a subset of tasks in the RoboTwin 2.0 benchmark and evaluated on a real manipulation task.

Critical strengths
  • Investigating how OPD can be used in the context of continual learning for WAMs (and more generally VLAs) is worthwhile.
  • Identifying a challenge of OPD in WAMs (and VLAs and any learning method which implements a feedback controller using trajectory prediction like MPC) is that a single planned trajectory is not enough; actual observed trajectories are generated using closed loop feedback involving repeated replanning. This is computationally demanding
  • The methodology using importance weights is mostly sound
  • The paper is well-written overall
Critical weaknesses
  • There are concerns around the scoping of the paper to WAMs; the authors only evaluate on Fast-WAM which does not imagine video trajectories, in effect it is used at test time the same as any VLA based on flow-matching (like the pi0 series models). The method in principle can be applied to a much broader scope of methods (at the very minimum VLAs based on flow matching) and it is curious why this has not been done.
  • Another scoping issue is the lack of discussion on continuous learning methods (for VLAs is relevant due to point above). The experiments show the lack of catastrophic forgetting post distillation (analogous to continuous learning) but the authors do not compare against or discuss continual learning in paper. For instance CLARE [1] and [2] (simple experience replay to mitigate forgetting) would be relevant to compare against or at least have discussion around.
  • The experimental evidence seems insufficiently convincing. At the very least, the experience replay method of [2] should be applied as a comparison to evaluate effect on catastrophic forgetting as a simple baseline. Also, the volume of experiments seems limited; while the real-robot experiment is appreciated, it is unclear why a small subset (8 of the 50 tasks) is used in the evaluation, and furthermore within the subset there is a fixed, single target/retention task split. Ideally, more tasks are evaluated and there is are multiple splits between target and retention tasks to make the evidence more convincing. Evaluating on other benchmarks (e.g., LIBERO [3]) tailored for continual learning would also be beneficial. Finally, given the lack of tasks, Table 2 is not convincing in the application of the prefix weight, the effect seems modest with and withou the prefix for all pools.
  • The authors note (3) is not quite an action likelihood but fail to relate the proposed method (while intuitive) to the true action likelihood. The cited Lipman paper on flow matching provides a method to approximate the likelihood of sampled trajectories under flow matching (appendix C in the arxiv version). The authors should discuss how their proposed term relates to the actual likelihood obtainable by the model itself.

[1] CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion, RA-L 2026 [2] Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning, ICML 2026 [3] LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning, NeurIPS 2023

Questions for authors

Q1: Why is the scope limited to WAMs and not inclusive of VLAs when the method itself appears more general to WAMs (the video generation is not used at all), in particular flow-matching VLAs? Q2: Similarly, why is there a lack of discussion around continual learning methods, which has been discussed in the VLA literature? Q3: Can you provide more evaluations of the method, e.g., another benchmark like LIBERO and/or more tasks and target/retention splits from RoboTwin 2.0? Q4: Can you more rigorously justify the approximate action likelihood in (3)?

5 comments
AuthorsOpenReview ↗

Response to Reviewer [R8m3]

We thank the reviewer for the detailed feedback. In response, we have expanded our evaluation, added an experience replay comparison, and conducted direct analyses of PWTR.

W1, Q1: Applicability to Flow-Matching VLAs

The action-velocity objective applies to flow-matching VLAs without requiring video generation. In our added LingBot-VLA experiment, mean target SR increases from 52.50% to 66.00%, with mean retention of 82.25% across six tasks excluded from adaptation.

Here, retention SR is the equally weighted mean success rate on Switch, Mug, Laptop, Pillbottle, Dual Bottles, and Handover, all excluded from adaptation.

1. Action-only formulation. Distillation supervises the action velocity field, and PWTR uses stored histories and velocity evaluations. In our Fast-WAM setup, video is not generated during inference, and the video loss coefficient is set to zero in the main action-only experiments (Table 4).

2. Evaluation on LingBot-VLA.

MethodMove Stapler to Pad SR (%) ↑Open Microwave SR (%) ↑Retention SR (6 tasks, %) ↑
LingBot-VLA73.0032.0080.25
Stapler teacher84.501.503.92
Microwave teacher0.0048.500.17
LingBot-VLA + Ours81.5050.5082.25

The adapted model achieves 66.00% mean target SR, compared with 52.50% for Base, while mean retention is 82.25% versus 80.25%.

Manuscript revision. We will add this experiment and clarify the broader applicability to flow-matching VLAs in the abstract and introduction.

W2, Q2: Continual Learning and Experience Replay

We have added the requested experience replay baseline and will discuss both suggested works on continual learning. Under our adaptation protocol, WAM-OPD exceeds SFT + ER by 1.00 percentage point in target SR and 8.25 points in retention SR.

1. Experience replay baseline. Following the reviewer's suggestion and Liu et al. (2026), we added an ER baseline using 2,000 prior-task demonstrations. During fine-tuning, these are sampled at a 1:1 ratio with target-task demonstrations.

MethodTarget SR (%) ↑Retention SR (6 tasks, %) ↑
Base56.5087.17
Joint SFT72.5072.75
SFT + ER68.7578.67
WAM-OPD69.7586.92

Without prior-task demonstrations during distillation, WAM-OPD raises target SR from 56.50% to 69.75% while achieving 86.92% retention, close to Base's 87.17%.

2. Relation to continual learning. We will add the following discussion to Sec. 2:

Liu et al. (2026) show that experience replay can help pretrained VLAs retain skills. CLARE (Römer et al., 2026) supports continual robot learning through modular adapters, automatic capacity expansion, and task-agnostic routing, keeping the pretrained parameters and previously learned adapters frozen. WAM-OPD transfers knowledge from task-specific teachers into a single student with a fixed architecture, without prior-task demonstrations during distillation. We evaluate a single adaptation stage and measure retention on pretrained tasks excluded from adaptation. These approaches offer different mechanisms for acquiring new skills while retaining existing capabilities.

Manuscript revision. We will cite both works in Sec. 2 and include the ER comparison with its replay protocol.

AuthorsOpenReview ↗

Response to Reviewer [R8m3]

W3, Q3: Evaluation Scale and the Contribution of Prefix Weighting

We have expanded the evaluation to all 50 RoboTwin tasks, added a second target/retention split and a LIBERO experiment, and extended the weighting ablations. PWTR improves both target SR (+2.50 points) and retention (+2.42 points) over unit weighting.

1. Full RoboTwin evaluation and a second target split. For the Stapler/Microwave adaptation split, mean SR across all 48 retention tasks is 92.82%, compared with 93.26% for Base. With Switch/Microwave as adaptation targets, mean target SR increases from 57.00% to 67.00%, while retention is 96.25% versus 96.42% for Base.

2. Adaptation and retention on LIBERO. We adapted Fast-WAM to LIBERO-Plus Goal and evaluated retention on the original LIBERO Object and Long suites, both excluded from adaptation. The evaluation comprises 100 target trials and 400 retention trials.

ModelLIBERO-Plus Goal target SR (%) ↑LIBERO Object + Long retention SR (%) ↑
Base (before adaptation)38.0097.00
Goal SFT teacher60.0093.25
WAM-OPD (action-only)57.0096.50

Base is the pretrained reference. Bold marks the higher value among the two adapted models in each column.

WAM-OPD retains 96.50% SR on the original tasks, compared with 93.25% for the SFT teacher, while recovering 19 of the teacher's 22 percentage points of target improvement. Its retention decrease from Base is 0.50 points, compared with 3.75 points for the teacher.

3. Group-preserving weighting control. G-Shuffle permutes weights within groups defined by task, collection condition, rollout source, and normalized prefix depth (five bins). It preserves each group's weight distribution while changing which history receives each weight.

Mixed-pool weightingTarget SR (%) ↑Retention SR (6 tasks, %) ↑
Uniform67.2584.50
G-Shuffle64.5084.67
PWTR69.7586.92

PWTR exceeds G-Shuffle by 5.25 points in target SR and 2.25 points in retention. These gains support the value of matching weights to the histories for which they were computed, beyond preserving group statistics.

4. Gradient alignment and reweighting strength. On paired replay and online batches, PWTR yields the highest mean cosine similarity with the online OPD gradient among the three weighting schemes. The mean similarity is 0.101 for PWTR, compared with 0.084 for unit weighting and 0.080 for G-Shuffle.

The λ sweep gives target SRs of 68.50%, 69.25%, 69.75%, and 67.25% for λ = 0.01, 0.03, 0.1, and 0.3, respectively, compared with 67.25% for unit weighting. The default λ = 0.1 achieves the highest SR among the tested values.

Manuscript revision. We will include the expanded evaluations, add G-Shuffle and retention to Table 2, place the gradient analysis in Sec. 4.3 and Appendix A, and add the λ sweep to Appendix D.

W4, Q4: Eq. (3) and Action Likelihood

Eq. (3) is a path-compatibility surrogate whose quadratic form admits an auxiliary-Gaussian interpretation. In our added density analysis, its Spearman correlations with estimated log-density differences are 0.27, 0.62, and 0.63 at updates 24, 72, and 96.

1. Auxiliary-Gaussian interpretation. Eq. (3) measures the discrepancy between the policy's Euler-predicted increments and the stored denoising increments on executed-action coordinates. This quadratic score can be interpreted as a log-transition score under an auxiliary Gaussian scoring kernel.

2. Computational benefit. As described by Lipman et al. (Appendix C), evaluating the continuous-flow density of a full action chunk requires inverse ODE integration and a divergence correction. Obtaining the executed-prefix likelihood further requires marginalizing over the unexecuted tail. PWTR computes its path-compatibility score directly from velocity evaluations on stored paths and uses it to reweight stored histories.

3. Density-based validation. At updates 24, 72, and 96, we sampled fresh action chunks from the student checkpoint, initial student, and teacher at each stored decision. We then fitted Gaussian densities to the executed-action coordinates and compared the resulting log-density differences with the scores from Eq. (3). The Spearman correlations are 0.27, 0.62, and 0.63, respectively. These positive correlations across all three checkpoints support the relationship between the path-compatibility score and the density-based proxy.

Manuscript revision. We will add the kernel interpretation, density-estimation protocol, and empirical comparison to Sec. 4.3 and Appendix A, and explicitly describe Eq. (3) as the surrogate used by PWTR.

Reviewer_R8m3OpenReview ↗

Thank you

The authors have satisfactorily addressed my concerns, so I am happy to raise my score. I think however, the primary application/scope of the paper should not be restricted to WAMs; again, the video generation does not play a specific role in the methodology and the paper should be rewritten as such.

Reviewer_eMwAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to improve the performance of pretrained WAMs on specific tasks while maintaining their general capabilities. The paper proposes WAM-OPD, an on-policy distillation framework that uses task-specific teachers to supervise the action velocity field along student-generated denoising trajectories. To avoid repeated environment interactions during distillation, the authors propose prefix-weighted trajectory replay (PWTR), which reuses a fixed trajectory pool collected from the initial student and task-specific teachers. PWTR estimates the compatibility of stored histories with the current student through denoising-path scores. It then uses these scores to reweight the distillation loss. In both simulation and real-world experiments, the method is validated to improve target-task performance while largely preserving the pretrained capabilities (in simulation).

Critical strengths
  1. The proposed approach is attractive because it can integrate task-specific expertise into a generalist policy without requiring additional environment interactions during distillation.

  2. The experiments are well-designed in general. The ablation studies on trajectory pool composition and prefix weighting help validate the effectiveness of the proposed components. In particular, the comparison with shuffled prefix weights provides evidence that the correspondence between histories and their weights is important.

Critical weaknesses
  1. The comparison with Online OPD is not sufficiently described. The main motivation of PWTR is to avoid the repeated environment interactions required by Online OPD. However, the paper does not specify how many rollouts are used by the Online OPD baseline or how frequently new trajectories are collected. It is necessary to compare WAM-OPD with Online OPD under the same environment interaction budget. Since WAM-OPD also requires an initial collection of student and teacher trajectories, a fair comparison should account for this cost. It would be helpful to evaluate both methods under several interaction budgets and compare their downstream policy performance. This would make clear under which cases WAM-OPD can achieve good performance with fewer interactions.

  2. The paper emphasizes WAMs, but the connection between the proposed method and WAMs is unclear. The method distills the action velocity field and does not directly use the video prediction capability of WAMs. The proposed PWTR and velocity matching objective can be applied to general flow-based robotic policies, including flow-based VLAs. It would be helpful to clarify whether there are any WAM-specific considerations in the proposed method. Alternatively, evaluating the method on a flow-based VLA would help demonstrate its generality.

  3. It is unclear what the authors aim to validate through the real-world experiments. The experiments only compare WAM-OPD against the initial student and task-specific teachers, but not other baselines such as SFT or Online OPD. The experiments only evaluate the three target tasks, without evaluating whether the pretrained capabilities are preserved. As a result, the real-world experiments demonstrate that WAM-OPD can improve performance on multiple target tasks, but do not validate its advantages over existing methods or its ability to mitigate catastrophic forgetting.

Questions for authors
  1. How sensitive is PWTR to the number of sampled decision points and the reweighting strength? Would using more decision points improve the approximation of the history distribution?
5 comments
AuthorsOpenReview ↗

Response to Reviewer [eMwA]

Thank you for the suggestions. We compare online and fixed-pool variants across interaction budgets, evaluate a flow-matching VLA, and add real-robot adaptation and retention baselines.

W1: Comparison with an Online Variant of WAM-OPD

Under matched trajectory budgets, including initial data collection, fixed-pool WAM-OPD achieves higher SR than Online OPD at 48 trajectories (67.00% vs. 66.25%), with an estimated 4.43× speedup. At 96 trajectories, it achieves comparable SR (69.75% vs. 70.25%) with a larger estimated speedup of 7.47×. Both speedups account for total collection and optimization time under the evaluated online rollout protocol.

1. Protocol and pool reuse. Online WAM-OPD is the Online OPD baseline in Table 1 (p. 7, l. 337), discussed in Sec. 4.2 (p. 7, ll. 375–377); the caption specifies its extra rollouts (ll. 325–327). It collects a fresh rollout from the current student before each update. Fixed-pool WAM-OPD collects once and reuses the pool. Both use the same action-distillation formulation.

2. Results under matched interaction budgets. We compare the two variants at interaction budgets of 24, 48, and 96 trajectories, including initial data collection.

Trajectory budgetWAM-OPD (online)WAM-OPD (fixed pool)
SR (%) ↑Est. time (min) ↓SR (%) ↑Est. time (min) ↓
0 (base)56.50—56.50—
2461.2528.1762.2511.56
4866.2556.6267.0012.77
9670.25115.0169.7515.40

At 48 trajectories, fixed-pool WAM-OPD reduces the estimated total time from 56.62 to 12.77 minutes while achieving 67.00% SR, 10.50 percentage points above Base. At 96 trajectories, it reduces the estimated total time from 115.01 to 15.40 minutes while achieving comparable performance to Online OPD (69.75% vs. 70.25% SR).

3. Timing assumptions. Estimates use two A100 GPUs, a 750-step rollout horizon, approximately 65 s per rollout, and 6.3 s per update. Up to 20 parallel simulator environments allow parallel fixed-pool collection. Under the evaluated online protocol, each updated student generates a fresh rollout before the next update.

In Sec. 4.1, 96 of 120 collected trajectories were selected for training by a predefined, outcome-independent scheme; the remaining 24 were not used for training.

Manuscript revision. We will add the budget comparison, timing assumptions, and collection protocol to Sec. 4.1 and Appendix C.

W2: Applicability to VLAs and Video Supervision

Our action-only objective also applies to flow-matching VLAs, raising LingBot-VLA's mean target SR from 52.50% to 66.00% while maintaining performance on six retention tasks. In our Fast-WAM comparison on LIBERO-Plus Goal, adding video distillation provides no further SR gain over the action-only objective.

1. Evaluation on LingBot-VLA. Distillation supervises the action velocity field, and PWTR uses stored histories and velocity evaluations. We applied the same procedure to LingBot-VLA:

MethodStapler SR (%) ↑Microwave SR (%) ↑Retention SR (%) ↑
LingBot-VLA73.0032.0080.25
Stapler teacher84.501.503.92
Microwave teacher0.0048.500.17
LingBot-VLA + Ours81.5050.5082.25

Retention SR is the equally weighted mean success rate on Switch, Mug, Laptop, Pillbottle, Dual Bottles, and Handover, all excluded from adaptation.

WAM-OPD improves Stapler and Microwave SR over Base by 8.50 and 18.50 percentage points, respectively, while increasing six-task retention to 82.25%. These results show that our action-only distillation can consolidate both teachers' target skills into a single flow-matching VLA while preserving performance on tasks excluded from adaptation.

2. Effect of additional video distillation. All four Fast-WAM models were evaluated on the same 100 perturbed instances from LIBERO-Plus Goal, with 10 instances for each of its 10 tasks. The evaluation spans all seven perturbation categories, with one rollout per model per instance. Action-only distillation achieves 57% SR, compared with 55% for joint action-video distillation, 38% for Base, and 60% for the teacher.

The lack of an additional SR gain from joint distillation may reflect the limited marginal benefit of further video supervision given the strong visual and dynamics priors inherited from video pretraining. Action-velocity matching directly targets task-specific policy adaptation by aligning the student's conditional action generation with the task teacher.

Manuscript revision. We will include both experiments and clarify in the abstract and introduction that WAMs are our primary application, while the procedure also applies to flow-matching VLAs.

AuthorsOpenReview ↗

Response to Reviewer [eMwA]

W3: Real-Robot Baselines and Skill Retention

We have added a Joint SFT baseline and a retention task on two AgileX Nero Robot Arms. Using a fixed pool with no further robot interaction during distillation, WAM-OPD achieves 90% mean target SR and 58% retention, compared with 82% and 0%, respectively, for Joint SFT.

1. Adaptation and retention evaluation. Peach and Block serve as adaptation targets, while Button is excluded from adaptation and used to measure retention. We evaluate all policies on 50 paired layouts per task in interleaved order.

MethodPeach SR (%) ↑Block SR (%) ↑Target mean SR (%) ↑Button retention SR (%) ↑
Base64827356
Peach teacher928500
Block teacher498510
Joint SFT7886820
WAM-OPD88929058

WAM-OPD raises mean target SR from 73% to 90% while retaining 58% SR on Button, compared with 56% for Base.

2. Shared initialization and training data. We conducted this additional experiment on two AgileX Nero Robot Arms because the original UR5e was occupied. We initialized the base student from the Fast-WAM RoboTwin checkpoint and pretrained it on data from the AgileX platform, including Button. Each teacher was then fine-tuned on its respective target task. Joint SFT started from the same Base and used the combined Peach and Block demonstrations used to train the teachers. WAM-OPD used a fixed pool of 96 trajectories.

3. Practical value of fixed-pool distillation. WAM-OPD decouples robot data collection from optimization. Online OPD requires deploying the updated student before every update, with manual scene resets and operator supervision throughout training, making it infeasible in our real-robot setup. Removing this dependence on continual robot access is a central motivation for our method: we collect the pool once and complete distillation without further robot interaction. The added experiment demonstrates the practical benefit, achieving 90% mean target SR while retaining 58% SR on Button. Our response to W1 provides the quantitative Online OPD comparison in simulation.

Manuscript revision. We will add this study to Sec. 4.4 and Appendix D.3 as a separate experiment, clearly distinguishing its platform and tasks from those in Fig. 4(b).

Q1: Sensitivity to Decision Points and Reweighting Strength

Performance varies little with the number of decision points and saturates at n = 16: increasing n from 8 to 16 raises SR by only 0.75 percentage points, with no further gain from using all points. Reweighting improves SR at all tested λ values from 0.01 to 0.1, peaking at λ = 0.1; at λ = 0.3, SR returns to the unweighted baseline.

1. Number of decision points. We vary the number of decision points while holding the trajectory pool, sampling order, and 96 updates fixed:

Decision points per trajectoryn = 8n = 16 (default)All
Target SR (%) ↑69.0069.7569.75

Using all decision points removes the prefix-sum subsampling described in Appendix A.3. While it yields no further SR improvement over n = 16, it requires approximately 2.4× as many queries.

2. Reweighting strength.

SettingMixed pool, w/o prefix (w_i = 1)λ = 0.01λ = 0.03λ = 0.1 (default)λ = 0.3
Target SR (%) ↑67.2568.5069.2569.7567.25

The Mixed Pool, w/o prefix control in Table 2 (p. 8, ll. 413–414) achieves 67.25% SR with unit loss weights (w_i = 1) and the same trajectory pool and sampling scheme. This unweighted objective is recovered by setting λ = 0 in Eqs. (4) and (10).

Manuscript revision. We will add both sweep tables to Appendix D.

Reviewer_eMwAOpenReview ↗

Resposne to author

Thank you for your response. My concerns have mostly been addressed. I agree with reviewer R8m3 that the paper should be reframed as a method for on-policy distillation of flow-based robotic policies, not restricted to WAM. Also, the current title is somewhat vague and could better reflect the paper’s main contribution.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Gratitude for Your Review and Openness to Further Questions

We are deeply grateful to all reviewers for dedicating your time and sharing thoughtful insights during this review cycle. Your thorough critiques have been instrumental in strengthening our manuscript and making its presentation much clearer.

With the discussion window wrapping up soon, we wanted to check in and see if our responses have adequately addressed your points, or if any unresolved questions remain. Please feel free to let us know if there is anything else we can clarify before the deadline.

We greatly appreciate your continued engagement and support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.