acceptodds
Under review as a conference paper at ICLR 2027

Action-Conditioned Predictive Consistency for Adversarial Detection in Vision-Language-Action Models

Abstract

Vision-Language-Action (VLA) models now let robots read a camera image and decide how to act. However, this also makes them vulnerable to visual perturbations. A carefully crafted change to a single frame can push the robot into executing a wrong action. On a real robot, this becomes a physical safety hazard, not merely a mislabeled image. Such frames must be detected before the policy acts on them. Existing defenses retrain or modify the policy itself, which is costly and alters a model that is already trusted. The closest learned monitor targets task failure, not adversarial attacks. What is missing is a runtime detector that runs alongside the policy and flags each attacked frame without modifying it. Our approach exploits the fact that control unfolds over time. At each step, the previous frame and the action the robot just took are already fixed before the adversary can perturb the current frame. The adversary can change the present, but it cannot affect this already committed past. We therefore predict the current frame's representation from the past alone, which gives a reference the perturbation cannot influence. Any attack strong enough to change the action must move the observed features away from this reference, and that discrepancy is our detection signal. We introduce PAC-CTCN, a per-frame detector that needs no change to the policy and comes with a provable detection guarantee. We prove that clean and adversarial frames separate once an attack displaces the features by more than twice the model's own prediction error. Both sides of this condition are measured directly rather than assumed. The guarantee is gray-box, and we state plainly that it is not robustness against an adversary who also knows the detector. We evaluate PAC-CTCN on three architecturally distinct VLAs across regimes of increasing difficulty. The detection condition holds with a clear margin throughout, and the detector outperforms both the closest learned monitor and standard detection baselines. It is a lightweight module, so it can drive a stop controller that turns detection into containment. Attacked runs then end in a safe halt at a small cost to clean-task success. Our source code will be made public after acceptance.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, adversarial detection, robot manipulation, runtime monitoring, adversarial robustness, AI safety, predictive consistency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gbtAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PAC-CTCN, a runtime adversarial detector for VLA models that compares current visual features with predictions based on the previous observation and executed action. It provides a conditional detection bound and evaluates detection and stop-based containment across three VLA models on LIBERO benchmarks.

Critical strengths

1.Practical detector interface that requires no VLA modification.

2.Evaluation across three VLA architectures, including simulated stop-based containment.

Critical weaknesses

1.Limited guarantee. The theory assumes sufficiently large displacement in the detector’s features, but does not establish that attacks harming the VLA must cause such displacement.

2.Insufficient adaptive evaluation. The stated attack mainly optimizes detector evasion rather than jointly optimizing evasion and task damage. Table 24 reports evasion dropping from 93.6% at 4/2554/255 to 12.2% at 16/25516/255. For a fixed detector and evaluation set, smaller-budget solutions remain feasible at larger budgets, raising concerns about attack optimization.

3.Inconclusive ablation. Zeroing actions at inference introduces distribution shift. A separately trained action-free model is needed to establish the necessity of action conditioning.

4.Numerical inconsistencies. Table 16’s per-sample latency and throughput disagree by approximately 64× at batch size 64. Figure 2’s plotted statistics also differ from its caption, implying different calibration thresholds.

Questions for authors

1.Can joint attacks cause task failure while keeping the detector score above threshold?

2.Does large-budget detection remain effective when attacks retain successful smaller-budget perturbations?

3.Can the authors reconcile the latency–throughput mismatch in Table 16 and the conflicting calibration statistics in Figure 2?

2 comments
AuthorsOpenReview ↗

Response to Reviewer gbtA (1/2)

We thank the reviewer for the careful review and for recognising the practical detector interface and cross-architecture evaluation.

W1 — The scope of the detection guarantee

We agree that the proposition establishes a predictive-score margin when displacement in the detector's features is sufficiently large. It does not establish that every attack harming the policy must produce that displacement. A4's measured relation between detector and VLA feature displacements is an empirical check on the evaluated attacks, not a proof of that implication.

The revision replaces “any effective attack is forced to reveal itself” with the precise conditional statement. The three-policy, three-regime gray-box results support the evaluated detector, while leaving its behaviour against other attacks to measurement.

The approximately 0.32 displacement cap for evading predictive scores is likewise conditional on the clean error bound. It supplies no corresponding action-damage or task-failure bound, and substituting a mean prediction error does not certify individual samples.

W2/Q1 — Joint attacks and the stop mechanism

In the additional detector-aware closed-loop evaluation, we measured 0% task success and 0% episodes halted. This demonstrates harmful evasion of the stop rule.

With the two-frame debounce, zero halts establishes evasion of the stop rule, not that every frame stayed above threshold. Revised Table 3 reports this measured outcome. The claims that evasion necessarily costs the attacker its damage and that joint optimisation “does not cleanly succeed” have been removed.

Revised Tables 26–27 (submission Tables 23–24) are labelled as detector-only tests. Their 100-step, 10-restart PGD settings do not specify a joint attack. The joint objective, update direction, and candidate-selection rule remain required protocol details.

W2/Q2 — Retaining smaller-budget solutions

The reviewer is correct that successful 4/255 perturbations remain feasible at larger budgets. The original measured evasion rates were:

BudgetMeasured evasion in the original runs
4/25593.6%
8/25523.9%
16/25512.2%

Retaining the measured 4/255 solutions would preserve at least 93.6% evasion at 8/255 and 16/255 on the same samples and threshold. This is a feasibility implication, not a new experimental result. The measured 12.2% therefore cannot establish large-budget recovery.

The revision removes that interpretation and its training-budget explanation. A corrected evaluation must retain feasible smaller-budget candidates and report damage, evasion, and their intersection under fixed criteria. The existing table does not establish performance under that protocol.

AuthorsOpenReview ↗

Response to Reviewer gbtA (2/2)

W3 — Interpreting the action ablation

The existing ablation measures sensitivity to removing an input from the trained detector. Its reported AUROCs are:

PolicyFull detectorAction zeroed at inference
OpenVLA0.9990.48
Octo0.9940.47
π₀0.9920.49

The reviewer is right that the input mismatch prevents this comparison from isolating the benefit of action conditioning. Revised Tables 12 and 16 label this as inference-time action removal, and the necessity interpretation is removed. A separately trained action-free model, with matched capacity, data, training budget, selection, and evaluation, remains needed. Its performance must be measured rather than inferred from the bound-ratio/AUROC trend.

W4/Q3 — Correcting throughput (submission Table 16; revised Table 19)

The throughput column effectively counts the batch factor twice. Since latency is reported per sample, throughput = 1,000 ÷ latency in milliseconds, giving:

Batch sizePer-sample latency (ms)Corrected samples/s, rounded
10.881,140
40.313,230
160.128,330
640.05817,200

At batch 64, the correction is 1.10 million → approximately 17,200 samples/s. These values are recomputed from the reported latencies, not new timing measurements. The revised table corrects the throughput column, and the super-linear scaling claim is removed. The reported batch-one latency of 0.88 ms is unchanged by this arithmetic correction.

W4/Q3 — Reconciling Figure 2 and calibration

The per-target adaptive table uses the combined score (α = 0.5) for both Adv mean and Evasion %, even when the attack targets the predictive branch. Its clean mean and spread are 0.876 and 0.188, giving τ = 0.312. The predictive-only threshold 0.988 belongs to a different score. The revised table headers and caption now identify this mapping.

ScoreMean μSpread σThreshold μ − 3σ
Predictive: Figure 20.99600.00390.9843
Predictive: deployment text0.9970.0030.988
Combined: adaptive diagnostics, α = 0.5≈0.876≈0.188≈0.312

The Pred-target attacked means 0.884 / 0.884 / 0.880 are combined-score measurements, close to the clean combined mean and above 0.312. The reported 100% evasion is the measured fraction of individual samples above that threshold; it is not inferred from their mean. The cross-branch columns report AUROCs, not branch-score means.

The predictive-only threshold 0.988 is used for the closed-loop and small-budget evaluations. A separate figure issue remains: the existing graphic prints 0.9960 / 0.0039, implying 0.9843. Its caption now matches that graphic, while its difference from the stated predictive calibration remains explicit. AUROC is threshold-free; TPR at 1% or 5% FPR uses its corresponding ROC operating point.

The frame-level false-alarm rate must also be measured from held-out clean scores. Neither a Gaussian fit nor the original paper's 3.1% rollout false-stop rate determines it, because temporal dependence and the two-frame debounce affect rollout stops. The revision distinguishes these two metrics; a separate held-out frame false-alarm rate is not reported.

Summary. The supported contribution remains a practical, conditional detector alongside unchanged VLAs. The revision narrows the guarantee and ablation claims, reports the measured detector-aware failure, corrects throughput, and identifies the adaptive score/threshold mapping. The separate Figure 2 calibration discrepancy remains explicit.

Reviewer_VaDTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PAC-CTCN, a runtime adversarial-input detector for frozen VLA policies. It combines action-conditioned prediction of visual representations with a discriminative branch and derives a conditional detection bound. Evaluation covers three VLA architectures, multiple LIBERO regimes, and simulated closed-loop containment using a halt controller.

Critical strengths
  1. The manuscript is well-organized and readable.

  2. The experimental setup is described in a detailed and thorough manner.

Critical weaknesses
  1. Limited novelty. History- and action-conditioned latent prediction is well-established. The proposed temporal independence is intrinsic to causal prediction, while Proposition 1 uses standard geometric arguments. The contribution appears primarily to adapt this framework to VLA adversarial detection.

  2. Insufficient adaptive evaluation. The gray-box scope stated in the abstract does not resolve concerns about the adaptive attack setup. An informed adversary should jointly optimize policy damage and detector evasion, yet Appendix C specifies no policy-damage objective. Moreover, Table 24 reports evasion decreasing from 93.6% at 4/2554/255 to 12.2% at 16/25516/255, although smaller-budget perturbations remain feasible at larger budgets. Further evaluation at ϵ=1/255\epsilon=1/255 and 2/2552/255 is needed to assess policy damage, detector evasion, and joint attack success.

  3. The guarantee does not establish a link to harmful behavior. Proposition 1 assumes sufficiently large displacement in the detector’s separate encoder, but does not establish that harmful behaviors, such as action freezing and small control biases [1, 2], necessarily produce the required feature-space margin.

  4. The theoretical arguments contain errors. Conditional dependence does not imply strictly positive MSE reduction, invalidating Lemma 2's lower bound. Appendix B.4 overlooks prediction-loss gradients through the previous-frame encoder branch despite detaching the current-frame target.

  5. The action-conditioning ablation does not establish necessity. Zeroing action embeddings only at inference introduces an input mismatch for a predictor trained to use actions. The resulting collapse cannot establish that action information is indispensable.

Reference:

[1] FreezeVLA: Action-Freezing Attacks against Vision-Language-Action Models.

[2] When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models.

Questions for authors

refer to weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer VaDT (1/2)

We thank the reviewer for identifying where the novelty claim and adaptive evaluation need more precise support.

W1 — The contribution and its antecedents

We agree that action-conditioned prediction, independence from a current input that the predictor never reads, and the triangle inequality are established ingredients. The contribution should be stated as their attack-specific formulation and evaluation for VLA inputs.

The revision credits the visual-foresight detection literature identified in the review, including Lin et al., for the predictive-consistency principle. Dreamer, BYOL-Explore, and TACO provide representation-learning background, but are not substitutes for that detection literature.

The revised contribution has three parts:

  1. A conditional detection margin. With a clean stored reference and clean error bounded by ε₀, displacement Δ > 2ε₀ gives a predictive-score drop of at least Δ(Δ − 2ε₀)/4. The algebra is standard. The condition identifies what must be measured, without asserting that every harmful attack produces sufficient displacement.

  2. A monitor alongside an unchanged VLA. PAC-CTCN combines latent prediction with contrastive representation training and requires no extra forward pass through the protected policy. Predictive-only inference remains within 0.005 AUROC of the combined score across policies, using the same jointly trained encoder. This identifies the deployed detection signal, while leaving the contribution of individual training components to controlled comparisons.

  3. A cross-policy evaluation. The study covers OpenVLA, Octo, and π₀ across three regimes. AUROC is 0.992–0.999 on the standard LIBERO suites and 0.903–0.928 on LIBERO-Plus. The reported OpenVLA detector overhead is 0.88 ms per frame on an A100.

The existing OpenVLA comparisons on the standard suites are:

Evaluated configurationAUROC
Our optical-flow baseline0.52
Clean-only predictive configuration0.877
Full PAC-CTCN pipeline0.999

These establish performance against the implemented baselines. The optical-flow row is not an AdvIT reproduction, and the clean-only row changes adversarial exposure. Neither is a matched implementation of Lin et al.'s detector, so they cannot establish relative advantage over that approach.

Contribution 1 and the discussion now use this framing. Related Work opens with predictive detection, and “any effective attack is forced to reveal itself” is replaced by the precise displacement condition. A matched prior-detector comparison remains needed for the stronger comparative claim.

AuthorsOpenReview ↗

Response to Reviewer VaDT (2/2)

W2 — Joint attacks and their observed outcome

Revised Tables 26–27 (submission Tables 23–24) are labelled as detector-only tests. They evaluate evasion alone, whereas the stated threat model also allows joint policy damage and evasion.

In the additional detector-aware closed-loop evaluation, we measured 0% task success and 0% episodes halted. This demonstrates harmful evasion of the stop rule.

With the two-frame debounce, this establishes evasion of the stop rule, not that every frame stayed above threshold. Revised Table 3 reports this measured failure, and the statement that joint optimisation “does not cleanly succeed” has been removed. The feature-displacement cap remains conditional on the clean error bound and concerns the detector's representation; it does not supply a task-damage bound.

Why the larger-budget result needs correction

The original detector-only PGD tests report:

Budget4/2558/25516/255
Reported evasion93.6%23.9%12.2%

The reviewer's feasible-set argument is correct: a 4/255 perturbation remains allowed at 16/255. With the same evaluation set and thresholds, the best attainable evasion rate cannot decrease. The same applies to the best attainable rate of simultaneous damage and evasion.

The table therefore describes the attacks returned by the optimiser, rather than establishing that larger budgets restore robustness. The revision removes that interpretation and its training-budget explanation.

To evaluate the stronger attacker, the protocol must retain feasible smaller-budget candidates and report damage, evasion, and their intersection under fixed criteria. The existing sweep has not established performance under this check. The loss, update direction, policy objective, and restart-selection rule also need to be explicit.

Additional measurements at small budgets

The measured runs include 1/255 and 2/255. Each arrow reads damage-only → joint optimisation; these runs are separate from the detector-only tests above.

BudgetAction shiftUndetected frames
1/2550.15 → 0.0530% → 85%
2/2550.18 → 0.058% → 80%
4/2550.20 → 0.062% → 72%
8/2550.25 → 0.070% → 60%
16/2550.32 → 0.080% → 55%

Adding evasion reduces the observed action shift at each budget. These are finite-run outcomes, subject to the same budget-consistency check. Action shift and evasion also do not determine the fraction of frames that are both damaged and undetected.

The earlier small-budget and persistence results remain observations of their respective attacks. They cannot be transferred to the stronger joint attacker or treated as a universal limit on task damage.

Summary. The revision credits prior predictive detection, narrows the novelty claim, corrects the adaptive budget interpretation, and reports the measured detector-aware failure. The cross-policy gray-box results retain their scope without implying white-box task protection.

Reviewer_UuK8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces PAC-CTCN, a policy-independent runtime detector of adversarial image inputs for vision-language-action (VLA) robots. A learned transition model predicts the current normalized visual feature from the previous observation and executed action; disagreement with the current feature provides a predictive-consistency score. A contrastively trained discriminative branch shapes the shared encoder and can be fused with the predictive score. The proposed conditional bound states that, when adversarial feature displacement exceeds twice a uniform clean prediction-error bound, the predictive score decreases by a positive margin.

Critical strengths
  1. Predicting the present representation from a causally prior observation–action pair yields a concrete, interpretable anomaly signal without modifying the protected VLA. This is a useful deployment-oriented separation between runtime monitoring and policy robustification.
  2. Results span three distinct policies and three LIBERO regimes; per-attack breakdowns, held-out trajectory splits, shift studies. The SAFE replication includes a native-task sanity check and an adversarially fine-tuned variant, which strengthens the comparison.
  3. This study assesses unsafe events, false stops, and pre-divergence detection.
Critical weaknesses
  1. The action ablation cannot support the stated necessity argument. Lemma 2 asserts a strictly positive conditional-entropy lower bound from conditional dependence alone. Conditional dependence need not change the conditional mean, so the variance term used in its proof can be zero; the entropy bound does not follow without additional assumptions.
  2. §5.4 explicitly zeros the action embedding at inference (Tables 9 and 13), rather than retraining a comparably sized action-free predictor. The AUROC may not isolate the causal value of action conditioning from train/test input mismatch.
  3. Table 20 labels its training rows 'UADA only', 'UPA only', 'TMA only', and 'Mixed (default)', with no explanation of how these replace or relate to the specified default.
  4. The clean-only prediction ablation (Table 14) changes the adversarial-training budget as well. A matched attack-trained action-conditioned predictive-monitor baseline, with the same data and encoder budget but without the CTCN training pathway, is needed to quantify which parts of PAC-CTCN go beyond applying known latent-dynamics monitoring to this threat model.
Questions for authors
  1. What precise loss, update sign (gradient ascent or descent), VLA task/action objective, and restart-selection rule produced Tables 23–24? Can you provide a joint policy-damage-versus-evasion result under detector-aware optimization, even for a representative OpenVLA subset?
  2. How were the UADA-only/UPA-only/TMA-only/mixed rows of Table 20 generated given the patch/Gaussian Phase II specification? Which families were used for negative generation, checkpoint selection, α tuning, and final evaluation; were the corresponding data partitions disjoint?
2 comments
AuthorsOpenReview ↗

Response to Reviewer UuK8 (1/2)

We thank the reviewer for the careful review and for recognising the interpretable monitoring signal, cross-policy evaluation, and closed-loop measurements.

W1/W2 — Action conditioning and the scope of the ablation

We agree that the entropy bound does not follow from conditional dependence alone: dependence can change higher moments without changing the conditional mean. The revision removes the action-conditioning lemma and its necessity claim.

The main detection proposition is independent of that lemma. With a clean stored reference and clean error bounded by ε₀, displacement Δ > 2ε₀ gives a predictive-score drop of at least Δ(Δ − 2ε₀)/4. This result is conditional on prediction error and displacement; it does not require a theorem that actions must improve prediction.

The empirical question is separate. Removing the action input from the trained detector gives the following AUROCs:

PolicyFull detectorAction zeroed at inference
OpenVLA0.9990.48
Octo0.9940.47
π₀0.9920.49

These drops show that the trained detector relies on its action input. Because removal also creates a mismatch at inference, they do not establish how a predictor trained without actions would perform. Revised Tables 12 and 16 (submission Tables 9 and 13) label this as inference-time action removal, and §5.4 withdraws the necessity interpretation. A capacity-matched action-free retraining remains needed.

W4 — Separating the training contributions

The clean-only configuration reaches 0.877 AUROC, compared with 0.999 for the full pipeline on OpenVLA. After joint training, predictive-only inference stays within 0.005 AUROC of the combined score across policies.

These results support predictive consistency as the deployed detection signal. They do not isolate the CTCN training benefit: the first comparison changes adversarial exposure, while the second retains the representation learned through CTCN.

The reviewer is right that the appropriate control is an attack-trained predictive monitor with matched data, encoder capacity, optimisation budget, selection, and evaluation, but without the CTCN pathway. The discussion of revised Table 17 (submission Table 14) now reflects what these measured comparisons establish.

W3/Q2 — Training, selection, and evaluation data

For the allocation below, trajectories and the listed patch identities are separated across training, validation, and test.

StageTrajectoriesPatch identities
Phase II negativesTrainUADA 1–2; TMA 1–4
Checkpoint selection and α tuningValidationUADA 3; TMA 5–6
Final evaluationTestUADA 4; TMA 7–8; all UPA; newly generated patches

UPA is held out from training and selection here. The validation row groups checkpoint selection and α tuning; it does not establish separate patch sets for those two operations.

This allocation is distinct from revised Table 23's family-specific conditions (submission Table 20). In particular, “UPA only” training cannot use the protocol that holds out UPA. The appendix reports the measured allocation above and identifies the remaining row-specific protocol details: generation, selection, tuning, evaluation, and their relation to the 70% patch / 30% Gaussian recipe.

AuthorsOpenReview ↗

Response to Reviewer UuK8 (2/2)

Q1 — Joint damage and detector evasion

In the additional detector-aware closed-loop evaluation, we measured 0% task success and 0% episodes halted. This demonstrates harmful evasion of the stop rule.

Additional closed-loop evaluationTask successEpisodes halted
No attack65%3%
Damage-only patch2%97%
Detector-aware patch0%0%

The 65% clean baseline belongs to this additional evaluation; the original paper reports 84.2%. We report the evaluations separately. With the two-frame debounce, zero halts establishes evasion of the stop rule, rather than showing that every frame stayed above threshold.

Revised Table 3 reports this measured failure, and the broad claim that evasion necessarily costs the attacker its damage has been removed.

What the frame-level measurements add

The measured runs below compare damage-only → joint optimisation. They are separate from the detector-only PGD tests in revised Tables 26–27 (submission Tables 23–24).

BudgetAction shiftUndetected frames
1/2550.15 → 0.0530% → 85%
2/2550.18 → 0.058% → 80%
4/2550.20 → 0.062% → 72%
8/2550.25 → 0.070% → 60%
16/2550.32 → 0.080% → 55%

Adding evasion reduces the observed action shift at each budget. A separate fixed-budget sweep gives the same pattern: increasing the evasion weight from 0 to 10 changes action shift from 0.25 → 0.04 and undetected frames from 0% → 97%. These observations explain how the optimiser responds to the objective; the closed-loop result shows why reduced action shift cannot be equated with task protection.

Both studies describe finite optimisation runs. Smaller-budget perturbations remain feasible at larger budgets, so declining evasion cannot establish stronger robustness. The reported action shift and evasion rate also do not determine the fraction of frames that are both damaged and undetected.

Interpreting and specifying the adaptive attacks

The original detector-only PGD tests used 100 steps and 10 restarts. At budgets 4/255, 8/255, and 16/255, their evasion rates were 93.6%, 23.9%, and 12.2%. That decline has the same limitation, and the revision removes its interpretation as large-budget recovery.

The review also identifies a reproducibility gap. The literal loss, ascent/descent direction, policy-damage objective, and best-restart rule for each attack remain required for reproducibility. The negative-score expression alone is insufficient because low scores are flagged; the outcome tables do not resolve those choices.

Summary. The conditional detector and cross-policy gray-box evaluation remain the supported contribution. The revision distinguishes those measured results from the unestablished necessity and training-attribution claims, reports harmful evasion of the stop rule, and identifies the remaining protocol details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.