- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose PICA-WAM, a self-supervised training method to improve the robustness of WAM under visual perturbations. An EMA copy of the model reads the nominal view at a lower noise level and provides a later-layer action feature as the target; the online model reads a perturbed view of the same state and aligns its earlier-layer action features to this target. The teacher and projection head are discarded after training, so there is no extra cost at inference.
Critical strengths
-
To my knowledge, this is the first work to introduce cross-appearance self-alignment for WAMs, without relying on an external pretrained encoder.
-
The paired diagnostic protocol (layer-wise sensitivity, layer-band intervention, trajectory drift) clearly explains why augmentation-only training fails, so the motivation is supported by experiments rather than intuition.
Critical weaknesses
-
Some key hyperparameters (eg. m, n) are only given in the appendix without any sensitivity analysis.
-
The ablation of the alignment design is incomplete. Table 4 only shows the effect of removing the appearance gap as a whole in "+SRA", without verifying the individual contributions of the noise-level gap and the layer gap in the WAM setting, or analyzing how EMA teacher quality on nominal views affects the alignment signal.
Questions for authors
-
How sensitive is PICA-WAM to the choice of student layer m, teacher layer n, noise offset, and loss weight? And does aligning deeper action-stream layers corresponding to the main leakage region work better?
-
What are the individual contributions of the noise-level gap and the layer gap when keeping the appearance gap in the WAM setting?
-
Can the authors provide an analysis of the EMA teacher quality on nominal views and how different teacher training stages or qualities affect the alignment benefit?
3 comments
Official Comment by Authors (2/2)
Thank you for your thoughtful comments and questions.
W2b. The ablation of the alignment design is incomplete. ... analyzing how EMA teacher quality on nominal views affects the alignment signal.
Q3. Can the authors provide an analysis of the EMA teacher quality on nominal views and how different teacher training stages or qualities affect the alignment benefit?
To examine how teacher quality and construction affect alignment on the nominal view, we compare three teacher regimes on the same 10,030 LIBERO-Plus tasks: using the online live student without EMA smoothing, using a frozen converged +Aug checkpoint as teacher, and using our default EMA teacher with decay 0.9999.
| Teacher configuration | Overall success (%) | Gain vs. +Aug (pp) |
|---|---|---|
| +Aug, no alignment | 81.09 | +0.00 |
| Alignment with live teacher, no EMA | 83.63 | +2.54 |
| Alignment with frozen +Aug teacher | 83.40 | +2.31 |
| Default EMA teacher, decay 0.9999 | 85.17 | +4.09 |
Both alternative teachers still improve substantially over unaligned +Aug, reaching 83.63% (+2.54 pp) with a live online teacher and 83.40% (+2.31 pp) with a frozen converged +Aug teacher. Alignment therefore does not depend on EMA smoothing to remain effective, and even a fixed converged +Aug teacher supplies a useful nominal-view target. At the same time, the default co-evolving EMA teacher reaches 85.17% (+4.09 pp), outperforming both the live teacher and the frozen +Aug teacher. Together, they show that nominal-view supervision from either an online or converged teacher improves robustness, while slowly updated EMA targets deliver the strongest gain.
We have added these teacher-regime controls and their scope to Section 4.4 and Appendix C.
Thank you for the thorough follow-up. Your responses and added experiments clarified my remaining questions, so I will maintain my support for the paper.
Official Comment by Authors (1/2)
Thank you for your thoughtful comments and questions.
The layer ablations support aligning an early student layer to a later teacher layer, with gains that are not confined to our default layer pair. Across all six tested pairs, success ranges from 83.36% to 86.10%, compared with 81.09% for augmentation-only training. We evaluated each configuration on the same 10,030 LIBERO-Plus tasks.
First, we varied the student and teacher layers while fixing the maximum noise gap at 0.2 and the alignment weight at 0.05:
The results favor an early, but not necessarily the earliest, student alignment layer: with teacher layer 20 fixed, student layer 8 outperforms both layers 4 and 14. With teacher layer 26 fixed, moving the student from layer 8 to layer 20 reduces success from 85.69% to 83.63% (−2.06 pp). Thus, directly placing the loss deeper in the action stream corresponding to the main leakage region does not improve control in these comparisons, supporting our choice to supervise earlier action features. The exact teacher depth is more flexible: with student layer 8 fixed, all three later teacher layers achieve 85.17–86.10%. A useful alignment target therefore does not require the particular teacher layer used in our default configuration.
We next examined the noise gap and alignment weight with student/teacher layers fixed at 8/20. The table summarizes selected noise-gap settings with weight 0.05, and the weight sweep with maximum noise gap 0.2.
The weight results support using alignment as a moderate auxiliary constraint. A change from 0.01 to 0.05 changes success by only 0.28 pp, with both settings substantially outperforming augmentation-only training (+3.81 and +4.09 pp). Increasing the weight further to 0.1 reduces the gain, so stronger alignment is not necessarily better.
For the noise gap, both 0.2 and 0.4 yield substantial gains over augmentation-only training (+4.09 and +5.04 pp), showing that the benefit is not tied to our default value. Even with zero noise gap, cross-appearance alignment retains a +3.72 pp gain. These comparisons distinguish the contribution of cross-appearance supervision from the additional benefit of the noise gap: the former remains useful without a noise offset, while the latter can further improve success, as demonstrated by the 0.4 setting.
We have added this analysis to Section 4.4, “Layer and hyperparameter sensitivity,” with the explicit configurations and success rates in Appendix C, Table 10.
Removing either gap alone retains most of the overall success, but reduces robustness on particular evaluation dimensions, and removing both gives a larger overall decrease. We kept the appearance gap in all four conditions and fixed the alignment weight at 0.05 and EMA decay at 0.9999. The evaluation protocol is the same as above.
With the layer gap retained, adding the noise-level gap improves Sensor Noise success by 3.00 pp and overall success by 0.37 pp. With the noise-level gap retained, adding the layer gap improves Camera success by 4.69 pp, Sensor Noise success by 3.44 pp, and overall success by 0.73 pp. Using both gaps achieves the highest overall success among these four configurations, improving on appearance-only alignment from 83.77% to 85.17% (+1.41 pp). These results support retaining both gaps: the noise-level gap helps sensor-noise robustness, while the layer gap benefits both camera-viewpoint and sensor-noise robustness.
We have added these findings and results to Section 4.4, Table 5.