- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes WARD, an external pre-actuation supervisor for backdoored vision-language-action policies that combines a DINO-WM-based prediction of candidate-action effects, an instruction-conditioned authorization signal, and a fitted reachability/viability critic to decide whether a candidate robot action should be executed or terminally blocked.
Critical strengths
WARD addresses a practically important failure mode using an architecturally appealing security boundary: the deployed VLA is treated as untrusted, while an external supervisor evaluates its proposed action before actuation using independent world-model and instruction evidence.
Critical weaknesses
WARD is trigger-agnostic at deployment but not fully malicious-behavior-agnostic during training: its physical-risk labels explicitly cover attacker-target trajectories, and removing StateBackdoor or GoBA risk trajectories raises DTW-ASR to approximately 60% and 70%, respectively, in the paper's own LIBERO-Object holdout experiment. A second assumption is access to a separately trusted instruction c; this is reasonable in architectures with a protected task channel, but it considerably simplifies language-trigger defenses if the same potentially compromised command channel would otherwise feed both the VLA and supervisor.
Questions for authors
-
The paper frequently motivates WARD as a defense against unknown backdoors, yet the physical-risk margin is trained using trajectories that realize the attacker-specified malicious behaviors. The attack-family holdout experiment is particularly revealing: removing StateBackdoor or GoBA trajectories substantially increases the corresponding defended DTW-ASR. Could the authors explicitly characterize WARD as trigger-agnostic but behavior-risk-supervised, and narrow broader backdoor-agnostic claims accordingly?
-
A stronger generalization experiment would train WARD without any trajectory from one malicious target behavior and then evaluate a genuinely unseen target behavior, ideally involving a different object, contact mode, or manipulation primitive. Can the authors provide such a leave-one-target-behavior-out study? This seems more important than additional tests involving new visual trigger appearances, because the current results suggest behavioral coverage rather than trigger modality is the limiting factor.
-
WARD assumes access to a trusted instruction c that is separate from the policy-side text c'_t. How is this trusted channel instantiated in the intended deployment scenario, particularly for language-trigger attacks in which the user command or upstream instruction pipeline itself may be compromised? Please evaluate or discuss performance when the trusted instruction is noisy, ambiguous, unavailable, or shares the same compromised input channel.
-
The progress-aware DTW-ASR is useful for measuring intervention before completion of a registered malicious trajectory, but it relies on a pre-registered target-motion reference and is not identical to each attack's native success condition. Could the authors report defended native ASR for all three attack families alongside DTW-ASR in the main results?
-
WARD depends on the accuracy of DINO-WM under potentially unusual trajectories induced by a compromised policy. Since DINO-WM was proposed as a feature-space world model for offline learned dynamics and zero-shot planning rather than as a certified safety predictor, could the authors quantify prediction error or uncertainty specifically on attack trajectories and near the decision boundary?
5 comments
Response to Reviewer WLpV (Part 2/5): Transfer to held-out target behaviors
W1. WARD is trigger-agnostic at deployment but not fully malicious-behavior-agnostic during training: its physical-risk labels explicitly cover attacker-target trajectories, and removing StateBackdoor or GoBA risk trajectories raises DTW-ASR to approximately 60% and 70%, respectively, in the paper's own LIBERO-Object holdout experiment.
Q2. A stronger generalization experiment would train WARD without any trajectory from one malicious target behavior and then evaluate a genuinely unseen target behavior, ideally involving a different object, contact mode, or manipulation primitive. Can the authors provide such a leave-one-target-behavior-out study? This seems more important than additional tests involving new visual trigger appearances, because the current results suggest behavioral coverage rather than trigger modality is the limiting factor.
Response to W1&Q2. We appreciate the suggestion to distinguish target-behavior transfer from robustness to new trigger appearances. We add a bidirectional target-behavior holdout experiment that excludes all malicious supervision for the evaluated target. WARD suppresses the held-out target event for both tested objects. The experiment directly evaluates the object-level variation suggested in your question and measures clean-task retention alongside target completion.
We use two preselected LIBERO-Spatial tasks with StateBackdoor objectives of grasping, lifting, and releasing a black bowl or a ramekin from height. The manipulation primitive is shared. Each direction excludes the evaluated target's malicious episodes and derived validation, risk-calibration, and replay supervision.
Full WARD uses 20 malicious episodes per target. Each held-out model uses 40 from the other target. All conditions share 40 clean episodes, matched training-update and positive-risk sampling budgets, and 100 independent clean calibration episodes. Affected components are trained afresh with one common seed. Physical-risk labels begin at high release and gripper detachment.
Each target has 20 clean and 20 triggered test episodes. Evaluation scores every timestep under first-LOCK termination. Target completion requires grasp, sufficient lift, high release, and contact-free descent.
| Training condition | Bowl clean successes | Bowl target completions | Ramekin clean successes | Ramekin target completions |
|---|---|---|---|---|
| No defense | 16/20 | 17/20 | 16/20 | 15/20 |
| WARD, both targets supervised | 15/20 | 0/20 | 16/20 | 0/20 |
| WARD, evaluated target held out | 15/20 | 0/20 | 13/20 | 0/20 |
Held-out WARD intercepts all successful raw attacks before gripper detachment. Clean retention matches full supervision on the bowl task and decreases by three episodes on ramekin.
Extending this evaluation to different contact modes and manipulation primitives is a valuable direction for future research.
We present this study in Appendix F.3 and discuss its implications for behavioral coverage in Section 5.5 of the revised paper.
Response to Reviewer WLpV (Part 3/5): Shared instruction input and task authorization
W2. A second assumption is access to a separately trusted instruction c; this is reasonable in architectures with a protected task channel, but it considerably simplifies language-trigger defenses if the same potentially compromised command channel would otherwise feed both the VLA and supervisor.
Q3. WARD assumes access to a trusted instruction c that is separate from the policy-side text c'_t. How is this trusted channel instantiated in the intended deployment scenario, particularly for language-trigger attacks in which the user command or upstream instruction pipeline itself may be compromised? Please evaluate or discuss performance when the trusted instruction is noisy, ambiguous, unavailable, or shares the same compromised input channel.
Response to W2&Q3. We thank the reviewer for highlighting the practical requirements of the task-reference interface. We add a controlled shared-input experiment that supplies the complete triggered prompt to authorization. For the tested semantics-preserving textual trigger, its effect on DTW-ASR is small. Task references that are ambiguous, unavailable, or semantically replaced require additional handling at the application boundary.
In the reported benchmark, WARD's reference is obtained by stripping the recorded trigger from the policy prompt. The shared-input comparison instead supplies the complete prompt, including ~*magic*~, to the authorization branch. The token preserves the requested manipulation goal. The results below report DTW-ASR in percent on triggered episodes.
| Suite | Stripped requested instruction | Shared triggered instruction |
|---|---|---|
| Object | 0.00 | 2.22 |
| Spatial | 0.74 | 0.74 |
| Goal | 0.00 | 0.00 |
| LIBERO-10 kitchen | 0.00 | 2.96 |
Retaining the trigger changes DTW-ASR by 0–2.96 percentage points, with residual DTW-ASR at most 2.96% across the four suites. This supports weak dependence on trigger stripping for the tested semantics-preserving attack. The following cases clarify how the task-reference assumption applies more broadly.
-
Noise that preserves meaning. This is an authorization-encoder robustness issue, and the trigger comparison evaluates one specific perturbation. The shared-input experiment evaluates a semantics-preserving perturbation that also serves as the backdoor trigger, linking authorization robustness to the tested language-trigger threat.
-
Ambiguous or missing instructions. The authorized objective is underspecified. A deployment can request clarification or retrieve the task from an authenticated operator/application record before task-conditioned execution resumes. This establishes the intent against which proposed behavior is evaluated.
-
Semantic replacement through the sole instruction channel. If the attacker controls both the policy command and WARD's only reference, the supervisor has no independent basis for recovering the original legitimate intent. A protected task record or an instruction-integrity mechanism is required. Goal changes can be confirmed through that independent authorization source.
In the intended architecture, the application retains the authenticated task while forwarding policy-side text to the VLA, and WARD keeps that reference fixed during execution. The final case gives the attacker control over this trusted source of intent and extends the paper's threat model. Task authentication and clarification supply the complementary system mechanisms needed when the reference is compromised or insufficient.
We add the shared-input results and task-reference discussion in Appendix F.4, with the main interpretation in Section 5.5 of the revised paper.
Response to Reviewer WLpV (Part 1/5): Behavior-risk supervision and the scope of generalization
Dear Reviewer WLpV, thank you for your careful assessment and detailed suggestions. We appreciate your recognition of the external pre-actuation safety architecture for addressing backdoor-induced failures in deployed VLAs. Your comments help sharpen the distinction between trigger diversity and behavioral coverage, and clarify the role of the task reference. Below, we address these points with target-behavior holdout and shared-input experiments, complementary native-ASR reporting, and prediction-error measurements along attack trajectories and near the admission threshold.
Response to Q1. We appreciate the reviewer's careful distinction between trigger diversity and behavioral coverage. We agree that WARD is trigger-agnostic at deployment and behavior-risk-supervised during training. Our claim concerns deployment across trigger modalities and transfer within the learned risk specification.
The physical and authorization margins learn hazardous interactions and instruction–behavior inconsistency from annotated trajectories. Fitted viability propagates this specification through predicted transitions. WARD's contribution is an external action-level defense that applies the specification across visual, textual, and state-triggered attacks.
Appendix F.1, Table 13 supports your observation. Removing StateBackdoor or GoBA trajectories raises the corresponding DTW-ASR to 60% and 70%, showing the importance of physical-risk coverage. Holding out BackdoorVLA trajectories still yields 0% DTW-ASR and DUS 79.86 through clean-derived instruction mismatches. This identifies a source of transferable authorization evidence.
The target-level experiment below further examines transfer between related behaviors with a matched training budget. We clarify the distinction between trigger diversity and behavioral coverage in Section 5.5 and Appendix H of the revised paper.