- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Latent Interface Purification (LIP), a plug-and-play defense for World Action Models that operates between a frozen visual encoder and a frozen action expert. Instead of modifying input images or retraining the underlying policy, LIP learns bounded, input-conditioned residual corrections to recover clean visual representations from adversarially perturbed latents. The proposed method combines RMS-based latent normalization, multi-scale dilated convolutions, signed modulation, and identity initialization. It is trained using paired clean and attacked latent representations, together with an explicit clean-consistency objective to preserve nominal policy behavior. Experiments across 4 WAM backbones on LIBERO and RoboTwin demonstrate substantial recovery in closed-loop task success under universal adversarial perturbations.
Critical strengths
-
The paper investigates a meaningful problem of improving adversarial robustness without retraining large robotic policies. Inserting a lightweight purifier between the visual encoder and action expert provides a practical alternative to full-policy adversarial training.
-
The experimental results cover four WAM architectures and both single-arm and bimanual manipulation benchmarks (Table 1). For example, LIP improves FastWAM's attack success from 35.38% to 92.62% on RoboTwin and from 31.00% to 98.50% on LIBERO-Spatial, while incurring only a small clean-performance reduction.
-
The paper includes comparisons against pixel-space and latent-space defenses, component ablations, cross-benchmark transfer, adaptive SPSA attacks, and perturbation-budget analyses. The reported 1.262 ms isolated inference latency in Line409 also supports the computational practicality of the proposed method.
Critical weaknesses
-
LIP primarily combines established techniques, including RMS normalization, dilated residual convolutions, bounded latent corrections, and zero-initialized output projections. The signed modulation mechanism in Eq. (6) multiplies two learnable correction maps, but its advantage over directly predicting a bounded residual remains unclear. Although removing signed modulation reduces defended success from 93.92% to 88.50% in Table 4, this ablation also changes the network architecture and capacity.
-
Although the threat model assumes attacker access to the visual encoder and action expert, and LIP is differentiable, the adaptive evaluation only uses SPSA with 1,800 queries (Table 5). Without gradient-based attacks optimized through the complete defended pipeline, it is unclear whether the reported robustness reflects LIP's effectiveness or insufficient attack strength. Multi-restart white-box PGD would provide a more convincing evaluation.
-
The main comparison uses only a Latent MLP as the learned latent-space baseline, achieving 81.25% defended success versus LIP's 93.92% in Table 3. However, LIP has substantially more parameters (1.299M vs. 0.091M) and incorporates spatial convolutions as shown in Table 6. It remains unclear whether the improvement comes from LIP's specific architectural design or greater model capacity. Comparisons with parameter-matched convolutional latent denoisers would strengthen the claims.
Questions for authors
-
What specific advantage does the signed modulation mechanism provide over directly predicting a bounded residual correction? Could the authors clarify its methodological novelty and evaluate it against a parameter-matched residual network without signed modulation?
-
Given that LIP is differentiable, why is the adaptive evaluation limited to SPSA? How does LIP perform against stronger gradient-based attacks, such as multi-restart PGD optimized through the complete defended pipeline?
-
Would LIP maintain its performance advantage over a parameter-matched convolutional latent denoiser trained using the same data and objective? This comparison would help distinguish the benefits of LIP's architecture from those of increased model capacity.
-
Since LIP is trained primarily on universal adversarial perturbations, how well does it generalize to other attack types, such as per-observation adversarial perturbations or localized adversarial patches?
4 comments
Response to Reviewer 7VXW (2/3)
Question 3 (Reviewer 7VXW Q2, W2): Why Adaptive Evaluation is Limited to SPSA Given Differentiability; Performance of Multi-Restart PGD Against the Complete Defended Pipeline
Response:
Implementation Mechanism, Physical Feasibility, and Evaluation of White-Box End-to-End Action-Loss PGD:
-
Implementation Mechanism and Engineering Adaptation:
Official open-source WAM repositories were released strictly for derivative-free inference deployment; backward computational graphs are severed via@torch.no_grad(), action.detach(), or truncation into NumPy arrays. To rigorously address the reviewer's concern regarding white-box adaptive attacks, we restructured the underlying execution pipelines, bypassed official inference decorators and truncation constraints, implemented model-specific low-level adaptations, and successfully established a closed-loop evaluation framework for end-to-end action-loss PGD backpropagated through multi-step diffusion denoising. -
Time Cost and Physical Real-World Feasibility Analysis:
- Extremely High Time Cost: In online closed-loop rollouts, every replanning step requires end-to-end backpropagation through a 14B-parameter multi-step diffusion Transformer. A single rollout takes over 10× longer than nominal unattacked rollouts (a nominal task rollout takes ~40–60 s, whereas an online PGD rollout takes 550–600 s).
- Minimal Physical Feasibility in Real Deployment:
- System Privileges and Intrusiveness: The attacker must obtain full white-box access to the robot's real-time inference system, including direct manipulation of GPU memory.
- Excessive Latency Violates Real-Time Control: The multi-second latency incurred by online backpropagation cannot satisfy the strict 10 Hz–30 Hz real-time control constraints of physical robots.
- Infeasibility of Passive Physical Deployment: Unlike physical UAPs (optical filters) or adversarial patches (printed stickers), online PGD cannot be passively deployed in physical environments via light reflection.
- Conclusion: Such end-to-end action PGD serves as a "theoretical extreme stress test" of policy and defense robustness rather than a practical deployment threat, and should not be used to dismiss the efficacy of LIP.
- Multi-Suite Benchmark Evaluation:
Due to the substantial computational cost (10× testing duration), we systematically conducted benchmark experiments on our primary backbones, FastWAM and Light-WAM: covering all 4 suites of LIBERO, randomly selecting 2 tasks per suite, and evaluating 10 rollouts per task across 4 perturbation budgets (). The results are summarized below:
| Model | Attack | Clean | 1/255 | 2/255 | 4/255 | 8/255 |
|---|---|---|---|---|---|---|
| FastWAM | PGD | 79/80 | 5/80 | 0/80 | 0/80 | 1/80 |
| FastWAM | PGD+LIP | — | 75/80 | 42/80 | 16/80 | 0/80 |
| Light-WAM | PGD | 78/80 | 6/80 | 0/80 | 0/80 | 0/80 |
| Light-WAM | PGD+LIP | — | 76/80 | 38/80 | 13/80 | 0/80 |
- Results Analysis and Objective Defense Boundaries:
When subjected to this theoretical extreme white-box attack:
- At small perturbation budgets ( and ): The undefended policy immediately collapses even under faint perturbations, while LIP demonstrates remarkable recovery capabilities.
- At larger perturbation budgets ( and ): Because end-to-end action gradients nonlinearly exploit high-frequency pathways through the multi-step denoising network, residual errors exceed gripper tolerances and full recovery is challenging. This clearly and objectively delineates the defense boundaries of LIP under extreme white-box stress testing.
Response to Reviewer 7VXW (3/3)
Question 4 (Q4): Generalization to Other Attack Types (e.g., Per-Observation Adversarial Perturbations or Localized Adversarial Patches)
Response:
As shown in Table 12 below, when incorporating multi-corruption environmental augmented training, LIP's robustness under compound threats (adversarial attacks overlaid with environmental degradations) improves substantially. This lightweight data adaptation and incremental retraining (with merely 750 additional multi-environment frames) convincingly validates that LIP possesses strong architectural adaptability and plasticity, and can readily generalize to other attack modes and diverse perturbation patterns through minimal retraining.
Table 12: Closed-loop success and action errors under composite corruptions on FastWAM with multi-corruption augmented training.
| Corruption | Severity | Undefended SR (%) | LIP SR (%) | (pp) | (%) | |||
|---|---|---|---|---|---|---|---|---|
| Gaussian noise + UAP | 1 | 32.50 | 81.25 | +48.75 | 0.147060 | 0.047106 | +0.099954 | +67.97 |
| 2 | 1.25 | 76.25 | +75.00 | 0.212170 | 0.056151 | +0.156018 | +73.53 | |
| 3 | 0.00 | 65.00 | +65.00 | 0.518210 | 0.084876 | +0.433333 | +83.62 | |
| 4 | 0.00 | 45.00 | +45.00 | 0.782010 | 0.134024 | +0.647986 | +82.86 | |
| 5 | 0.00 | 28.75 | +28.75 | 0.777145 | 0.170649 | +0.606496 | +78.04 | |
| Motion blur + UAP | 1 | 35.00 | 78.75 | +43.75 | 0.198990 | 0.053755 | +0.145235 | +72.99 |
| 2 | 18.75 | 81.25 | +62.50 | 0.253917 | 0.073962 | +0.179955 | +70.87 | |
| 3 | 0.00 | 83.75 | +83.75 | 0.476452 | 0.101922 | +0.374530 | +78.61 | |
| 4 | 0.00 | 86.25 | +86.25 | 0.504679 | 0.123502 | +0.381177 | +75.53 | |
| 5 | 0.00 | 82.50 | +82.50 | 0.516473 | 0.130119 | +0.386354 | +74.81 | |
| Salt-and-pepper + UAP | 1 | 0.00 | 68.75 | +68.75 | 0.447383 | 0.165685 | +0.281698 | +62.97 |
| 2 | 0.00 | 40.00 | +40.00 | 1.194200 | 0.437870 | +0.756330 | +63.33 | |
| 3 | 0.00 | 16.25 | +16.25 | 1.292202 | 0.504537 | +0.787665 | +60.96 | |
| 4 | 0.00 | 0.00 | +0.00 | 1.441588 | 0.583327 | +0.858261 | +59.54 | |
| 5 | 0.00 | 0.00 | +0.00 | 1.441468 | 0.722571 | +0.718898 | +49.87 | |
| Brightness shift + UAP | 1 | 40.00 | 83.75 | +43.75 | 0.138645 | 0.065849 | +0.072796 | +52.51 |
| 2 | 37.50 | 80.00 | +42.50 | 0.150528 | 0.080502 | +0.070026 | +46.52 | |
| 3 | 27.50 | 76.25 | +48.75 | 0.166789 | 0.102854 | +0.063935 | +38.33 | |
| 4 | 8.75 | 81.25 | +72.50 | 0.204017 | 0.128996 | +0.075021 | +36.77 | |
| 5 | 1.25 | 66.25 | +65.00 | 0.231798 | 0.153982 | +0.077816 | +33.57 | |
| Contrast drop + UAP | 1 | 0.00 | 71.25 | +71.25 | 0.442572 | 0.083551 | +0.359021 | +81.12 |
| 2 | 0.00 | 68.75 | +68.75 | 0.499262 | 0.102758 | +0.396504 | +79.42 | |
| 3 | 0.00 | 51.25 | +51.25 | 0.743338 | 0.125332 | +0.618005 | +83.14 | |
| 4 | 0.00 | 35.00 | +35.00 | 0.788207 | 0.190426 | +0.597780 | +75.84 | |
| 5 | 0.00 | 0.00 | +0.00 | 1.296449 | 0.575610 | +0.720839 | +55.60 |
Thank you for the detailed rebuttal and extra experiments. I appreciate the authors' efforts in addressing my concerns.
Howevery, I still have the remaining concerns:
-
Why are the values in Figure 3 and Figure 8 identical across all aspects—including the experimental setup—yet the confidence levels differ? Could the authors please provide some clarification? I am also not certain whether using confidence intervals to present performance results is standard practice. I didn't see any similar illustration in the relevant papers before.
-
Even LIP would suffers severe performance degradation within bigger budgets like 4,8/255 under white-box PGD. Authors should make this failure boundary explicit in main text. Could you also clarify what practical threat model other than LIP is thought to defend against, and what kind of attack areas are expicitly out-of-scope?
Response to Reviewer 7VXW (1/3)
We sincerely thank the reviewer for the professional and thorough feedback. Below, we respond to your four questions in sequence.
Question 1 (Reviewer 7VXW Q1, W1): Specific Advantages of Signed Modulation Mechanism Over Direct Bounded Residuals; Comparison with Parameter-Matched Residual Network
Response:
Methodologically, the key distinction lies in the control objective at the frozen WAM interface. Rather than inverting perturbations coordinate-by-coordinate for perceptual or generic feature reconstruction, LIP uses input-conditioned gating and signed modulation to determine which latent coordinates to adjust and in which direction—selectively rectifying features that govern downstream action generation.
Theoretically, Section 4.3 (Eq. 9) and Proposition 2 (Appendix A.1) prove that our joint recovery and clean-consistency objective upper-bounds both attacked-action error and clean-action distortion under local Lipschitz continuity (interpreted in Section 4.4, with multi-scale and gating components ablated in Table 4). Please refer to our Official Comment at the top of the replies for the full mechanism comparison table and 876-frame closed-loop diagnostic results.
Signed modulation decouples candidate correction from input-conditioned coordinate-wise selection. In z^=x−gψ(x)⊙Δψ(x), gψ(x)∈[−1,1] attenuates, preserves, or reverses candidate adjustments, while ∣Δψ(x)∣≤ρr(x) enforces the magnitude envelope. The network thus selects correction direction and magnitude according to local representation state. Zero-initialization of the candidate projection layer ensures strict identity behavior upon insertion, and clean consistency anchors nominal action execution.
While both direct bounded residuals and multiplicative modulation impose magnitude bounds, the mechanistic distinction lies in the input-conditioned multiplicative decomposition. The table below compares the complete architecture with an ablation removing signed modulation (parameter-matched to standard convolution):
Full LIP achieves 93.92% defended success compared to 88.50% without signed modulation (a 5.42 pp gain); clean action distortion is 0.002 vs. 0.009. Thus, signed modulation contributes significantly to both task recovery and nominal action fidelity.
Question 2 (Q3, W3): Maintaining Performance Advantage Over Parameter-Matched Convolutional Latent Denoiser Under Identical Data and Objectives
Response:
Below, we isolate the comparison between the parameter-matched convolutional baseline and LIP. These comparisons confirm that the performance gains originate from structured spatial multi-scale correction and gating rather than model capacity.