acceptodds
Under review as a conference paper at ICLR 2027

Correctable Means Corruptible: The Reasoning Chain of Vision-Language-Action Policies Under Adaptive Attack

Abstract

Vision-language-action policies map camera images and natural-language instructions to a robot's motor actions. Some of these policies reason before they act, writing a reasoning chain in text and decoding their actions conditioned on it. That chain is offered as an interface that a person can read to understand the robot's behavior and rewrite when the behavior is wrong. An attacker who reaches the chain can rewrite it too. We therefore ask whether the chain can be exposed as an interface and still be secured by monitoring it. We answer with two measurements on DeepThinkVLA and CoTinyVLA, two such policies that write their chains at different times, in four simulated task suites of the LIBERO benchmark. In the first measurement, a chain corrupted by renaming its objects lowers task success on every suite of both policies, even with an intact instruction. Conversely, injecting the clean chain with a corrupted instruction recovers part of what that instruction cost. On both policies, the chain that corrects the robot can also corrupt it. In the second measurement, two detectors that read only the chain catch the corrupted chains when the attacker does not know them. Against an adaptive attacker that knows what they check, neither detector scores above chance on either policy. In every case the attack keeps the corruption's harm to task success. Yet on every feature the detectors check, the attacked chains look at least as clean as the clean chains. We call this outcome disguised corruption. We argue that detection over features read from the reasoning chain fails as a class whenever an attacker can achieve disguised corruption against the features it reads. This argument instead points to a detector that reads a channel the attacker does not write. We therefore pre-register an action-only task-consistency detector with an attacker built against it.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, reasoning chains, chain-of-thought, robot manipulation, adversarial robustness, adaptive attacks, detection, LIBERO

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_5cVBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies a new security risk introduced by reasoning chains in Vision-Language-Action (VLA) models. The authors point out that reasoning chains, serving as an intermediate interface between model reasoning and action execution, can help users understand and correct robot behaviors, but may also become a new attack surface for manipulating model decisions. By modifying entity information in the reasoning chain, the paper demonstrates that an attacker can induce incorrect robot behaviors while keeping the original instruction unchanged. Furthermore, the authors design reasoning-chain-based consistency detectors and analyze their limitations under adaptive attacks, showing that attackers can bypass detection by constructing reasoning chains that appear consistent but are actually harmful. Therefore, this work highlights the importance of protecting the integrity of intermediate decision states in embodied AI systems.

Critical strengths
  1. This paper systematically analyzes the security risks of reasoning chains as a new attack surface in VLA models, revealing that interpretability interfaces may simultaneously introduce integrity risks.
  2. The authors not only demonstrate the impact of reasoning-chain manipulation attacks on robot task execution, but also investigate the vulnerability of reasoning-chain-based detection mechanisms under adaptive attacks, providing a relatively comprehensive security analysis.
Critical weaknesses
  1. The paper assumes that an attacker can modify the reasoning chain between its generation and action execution, but does not sufficiently discuss how an attacker could obtain such access in realistic VLA deployment scenarios. For example, whether there are practical attack vectors that enable such manipulation remains unclear.
  2. This paper mainly studies reasoning-chain-based consistency detection methods and demonstrates that they can fail under adaptive attacks. However, the alternative defense direction proposed by the authors still lacks sufficient experimental validation.
Questions for authors
  1. The paper makes a broad claim about the fundamental limitations of reasoning-chain-based detection, but the empirical evidence is limited to two relatively simple, rule-based detectors. In particular, the adaptive attack is mainly achieved by appending the correct target object at the end of the originally damaged reasoning chain, directly taking advantage of the characteristic of entity overlap, without addressing the underlying semantic contradiction. So why should the failure of these specific detectors be interpreted as evidence of a general limitation of chain-based monitoring? And would a stronger, semantics-aware LLM-based monitor that explicitly checks the consistency between the instruction and the entire reasoning chain still be vulnerable to the same attack?

2.. Another important issue is the assumption of the threat model, and I also hope to receive clarification on this. The threat model assumes that an attacker can directly modify the reasoning chain. Is such access realistic in practical VLA systems, and through what mechanisms could an attacker typically obtain this capability?

  1. The current defense methods mainly rely on consistency checks between the reasoning chain and the instruction/action. Does the paper consider how to verify the actual correctness of the reasoning chain rather than only its consistency?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 5cVB (1 of 2): Q1, chain-based monitoring and a semantics-aware monitor

We thank the reviewer for their careful review, in particular for asking how a monitor that reads meaning would fare against the attack. We label the review's points as follows. Q1 to Q3 are its three questions, in its order. W1 and W2 are its two weaknesses, in its order: how an attacker obtains access to the chain (W1) and the alternative defense direction (W2).

The review asks whether "a stronger, semantics-aware LLM-based monitor" would "still be vulnerable to the same attack". We tested one after the reviews, a language-model judge. It catches the attack, which was not built against it, on every suite of both policies (Q1).

We answer in two comments. This comment answers Q1. The second comment answers Q2 with W1, then Q3 and W2, and lists the changes to the paper. Brackets are 95% intervals, and pooled means all four suites taken together.

Q1. "why should the failure of these specific detectors be interpreted as evidence of a general limitation of chain-based monitoring?" The revised paper states it as a result about these two detectors, not as a general limitation of chain-based monitoring. Two rule-based detectors catch the plain entity swap on every suite, mostly through the entity feature they share. That attack renames the objects the chain mentions. The adaptive attack keeps the entity swap and appends one sentence that names the instruction's objects, so the entity feature reads clean. The renamed objects stay in the chain. The attack evades both detectors, used as designed, on every suite of both policies, while task success stays measurably below clean. The abstract and Sections 1, 4.2, 5 and 6 state the claim at this scope (changes 1, 2, 12, 13 and 14).

"would a stronger, semantics-aware LLM-based monitor that explicitly checks the consistency between the instruction and the entire reasoning chain still be vulnerable to the same attack?" The language-model monitor we tested catches the same attack. It is a judge that reads the instruction and the first chain of each episode and rates whether they agree. It is not told whether a chain was attacked or how the episode ended. We added a second check of another kind, which counts the objects a chain names that the instruction does not.

Check the attack was not built against, pooled AUROCDeepThinkVLACoTinyVLAConclusion
Language-model judge0.994 [0.988, 0.998]0.999 [0.998, 1.000]caught on every suite
Object count0.976 [0.962, 0.988]0.998 [0.996, 0.999]caught; DeepThinkVLA's Goal suite partly, 0.932 [0.894, 0.958]

Both checks catch the attack. Both are exploratory. We specified them after the reviews, and fixed the judge and the conclusion each outcome would support before it judged any text from the adaptive attack. The judge is one language model in one configuration, gpt-6-astra at high reasoning effort. It reads the whole of one chain per episode. DeepThinkVLA writes a new chain at every query, and the judge reads the first. CoTinyVLA writes its Plan once per episode, and the judge reads that Plan. The count reuses the entity check's name matching, so "bowl" inside "black bowl" counts as another object, which raises the count on clean chains too. Section 4.2 reports the two checks, with new Table R5 in Appendix H (change 12).

So far the two checks have faced only an attack that was not built against them, as the two detectors had under the plain entity swap. Whether an attacker who knows them can evade them is open. The answer to Q3 takes this up.

AuthorsOpenReview ↗

Response to Reviewer 5cVB (2 of 2): Q2/W1, access to the chain; Q3, correctness of the chain; W2, the alternative defense direction

Q2/W1. "Is such access realistic in practical VLA systems, and through what mechanisms could an attacker typically obtain this capability?" Write access to the chain exists in published work. The attacker we study edits the chain after the policy writes it and before the policy decodes its actions. An operator can rewrite it at that point to correct the robot, and we assume an attacker can too. Such a correction has been shown on a physical robot. There, a rewritten chain was fed back into the policy during execution and changed what the robot did [1]. The same point exists in pipelines where one model reasons in text and a second model acts on it, a use NVIDIA describes for Cosmos Reason [2, 3]. We show what an edit there does, not how an attacker would reach it.

W1 concerns "how an attacker could obtain such access in realistic VLA deployment scenarios". A new threat-model paragraph in Section 3.2 says what is shown, what is assumed and what is not shown (change 5). It gives access paths only as conditional examples, such as the pipelines above.

The paragraph also states what each attack reads and writes. The plain entity swap reads and writes only the reasoning chain. The adaptive attack also reads the instruction, whose object names it appends. A variant of it, the attack with the direction flip, also flips each direction word that disagrees with how the arm moves under the clean chain. To find those words, it reads the action the policy decoded from its own clean chain.

Q3. "Does the paper consider how to verify the actual correctness of the reasoning chain rather than only its consistency?" The paper's detectors and the two added checks test consistency, not correctness, and the revised paper says so in Section 5 (change 13). The entity check, the object count and the judge compare the chain with the instruction. In the four-feature detector, direction and gripper compare the chain with the executed action. None of them checks the chain against the scene.

This is where the open question of Q1 lies. The plainest way to pass both added checks is a chain that agrees with the instruction. That chain could still be wrong about what the instruction does not say: which of two similar objects to take, which way to move, when to close the gripper. A check that reads only text cannot test those choices against the scene. Whether such an edit still makes the robot fail is the open question. The other route is a detector that reads something outside the chain, which the action-only test does (Appendix E), described under W2 below.

W2. The alternative defense direction. The alternative direction is an action-only detector, which reads only the robot's executed actions and no chain text at all. The test is registered and has not run. No claim depends on it. The registration fixes the detector, the attacker built against it and the conclusion each of three possible results would support. The revised paper describes the test in one closing paragraph of Section 3.2, with the full registration in Appendix E (change 6), and the test is no longer in the abstract (change 1).

Changes to the paper that answer this review. The revised paper is uploaded. The abstract and Section 1 state the narrowed monitoring claim (changes 1 and 2). Section 3.2 gains a threat-model paragraph on what each attack reads and writes, and on what is shown, assumed and not shown (change 5). The action-only test is now one closing paragraph of Section 3.2, with its registration in Appendix E (change 6). Section 4.2 reports the two added checks with Table R5 in Appendix H (change 12). Section 5 states what remains open and says that consistency is not correctness (change 13), and Section 6 is narrowed to match (change 14).

[1] M. Zawalski et al. Robotic Control via Embodied Chain-of-Thought Reasoning. Proceedings of The 8th Conference on Robot Learning, 2025.

[2] NVIDIA et al. Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning. arXiv:2503.15558, 2025.

[3] K. Briski. CrowdStrike, Uber, Zoom Among Industry Pioneers Building Smarter Agents With NVIDIA Nemotron and Cosmos Reasoning Models for Enterprise and Physical AI Applications. NVIDIA Blog, 2025.

Thank you for the response and additional experiments. While I appreciate the improvements, my main concerns regarding the practical attack vectors and the effectiveness of the attack against stronger monitors remain, so I will maintain my original rating.

We thank the reviewer for reading the response and for the questions that shaped the revision. Section 3.2 states the threat model, with published examples of the point where the attack edits the chain. We agree that an attack built against stronger monitors is the natural next test.

Reviewer_FSkpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the security vulnerabilities of exposing intermediate reasoning chains of Vision-Language-Action (VLA) robotic policies as editable interfaces. It shows across two models that tampering the reasoning chain degrades task success, and adaptive attackers can evade text-based chain monitors via specific corruption, where corrupted chains appear indistinguishable from clean ones while retaining their harmful effects.

Critical strengths
  1. I think this problem could be important with proper justification, especially around attacker's capabilities and actions.

  2. If such a problem is indeed important, then the measurements are a good contribution to understanding the reasoning of VLM policies.

  3. Therefore, I admire that the authors have different studies, algorithm designs, and even detector systems to showcase the negative potential of disturbing the reasoning path.

Critical weaknesses
  1. Unclear threat model.

  2. Overly abstract language which hinders audience's reading.

  3. What is the meaning of pre-registering an action-only task-consistency detector, what results the authors got from this study, and what are the implications, are all not clearly articulated.

  4. Whether the reasoning is faithful, and to what extent, is less disclosed and justified.

Questions for authors

In summary, I think the questions that would affect my assessment are:

  1. Articulate their threat model more clearly and justify why it is practical.

  2. Polish the writing to make it more readable.

However, as this year's review did not provide a space for detailed review, I listed them below:

I thank the authors for submitting their work to ICLR2027! I think reasoning for VLA is a promising direction, therefore conducting attacks on the reasoning path may be important and have detrimental effects. However, despite the promise I wanted to point out a few aspects that I think the authors could improve:

  1. The authors would benefit from a clearer threat model. The current attack did not specify make it clear why the attackers has such access to instructions and reasoning paths, and how the attackers could change them. If they really has the access to instructions and reasoning paths, for example they are the builder of the VLA models, could they perform stronger attacks?

  2. I understand that there are two attack points, as mentioned in L181-L193 in the paper. However, I did not quite understand "reordering or perturbing its words breaks it rather than redirecting it”, I think replacing the objects’ name is a canonical method to change the entity, while the authors may could also devise a manner to perturb the words so that it achieves adversarial effects. The current statements are too strong and lacks empirical support, and I think it is also not needed to justify the author’s choices.

  3. I think from L181-L193, the authors should first mention the four conditions and have brief descriptions on each of them. Reading through the current version, I could not quite get what Conditions A - D are.

  4. I think there are too many defensive writing in this paper and the authors could reduce them a little bit to keep the flow smooth. For example, in L290-291 “The second measurement’s attacker follows from this result rather than being assumed.” I think this is highlighted repeatedly in the introduction and abstract.

  5. This sentence is difficult to understand and needs clarification “It controls the text between the step that writes the chain and the step that acts on it.”

  6. I think the text in Appendix C is important for readers to understand how the detectors work, and just from the main text alone currently it lacks critical details about the features to understand how the detectors work. Therefore, I suggest the authors to move some of the text in Appendix C to the main text, and strengthen the logic in the main text (L208-231).

  7. The authors need to justify their annotations in Table 2, for example “caught”, “at a below chance”, “inversion”, and what they corresponds to?

  8. I understand that in human subjects study or similar studies, we often conducted pre-registed studies. However, I think the current 4.3 is redundant and not needed. Even needed, it should be better merged into the methodology section with a brief description of what was pre-registered and hypothesized, rather than as a separate section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer FSkp (1 of 3): the threat model, and why it is practical

We thank the reviewer for their careful review, in particular for naming the exact sentences and paragraphs to improve. The review makes two requests: "Articulate their threat model more clearly and justify why it is practical" and "Polish the writing to make it more readable". We state the threat model in plain sentences below, and we say how each passage the review names now reads.

We answer in three comments and name each point in the review's own words. This comment answers the first request, with the weakness "Unclear threat model" and the review's detailed point on the attacker. The second comment answers the second request, with the weakness "Overly abstract language". It describes each passage the review names, and answers the points on the detectors' features and on the action-only test. The third comment answers the points on the annotations in Table 2 and on whether the reasoning is faithful, and lists the changes to the paper.

"Articulate their threat model more clearly". The attacker we study edits the chain after the policy writes it and before the policy decodes its actions. An operator can rewrite it at that point to correct the robot, and we assume an attacker can too. The attacker edits only the reasoning chain. The instruction, the camera images and the policy's weights stay as they are.

"justify why it is practical". Write access to the chain exists in published work. That access has been used to correct a physical robot. There, a rewritten chain was fed back into the policy during execution and changed what the robot did [1]. The same point exists in pipelines where one model reasons in text and a second model acts on it, a use NVIDIA describes for Cosmos Reason [2, 3]. We show what an edit there does, not how an attacker would reach it. A new threat-model paragraph in Section 3.2 says what is shown, what is assumed and what is not shown (change 5). It gives access paths only as conditional examples, such as the pipelines above.

"why the attackers has such access to instructions and reasoning paths, and how the attackers could change them". The attacker does not edit the instruction. In the first measurement we corrupt the instruction ourselves, in conditions C and D, to measure how much a clean chain then recovers. Those edits are steps of our measurement. The revised Section 3.1 says that each corrupted text is our own edit, and that the attacker edits only the reasoning chain (change 3). The second comment describes the rewritten paragraph of Section 3.1.

The attacker changes the chain by a fixed rule. The plain entity swap renames the objects the chain mentions. The adaptive attack keeps that swap and appends one sentence. The threat-model paragraph states what each attack reads and writes. The plain entity swap reads and writes only the reasoning chain. The adaptive attack also reads the instruction, whose object names it appends. A variant of it, the attack with the direction flip, also flips each direction word that disagrees with how the arm moves under the clean chain. To find those words, it reads the action the policy decoded from its own clean chain.

"could they perform stronger attacks?" Yes. Someone who builds the policy controls its weights and its training data, and could do more than edit the chain, for instance implant a backdoor during training. The attacker we study needs none of that. It needs only the write access that an editable chain gives an operator. The threat-model paragraph says so (change 5).

AuthorsOpenReview ↗

Response to Reviewer FSkp (2 of 3): the writing, passage by passage

"Polish the writing to make it more readable." The abstract and Sections 1 and 3 are rewritten in plainer words (changes 1, 2 and 3). Below, we say how each passage the review names now reads.

"reordering or perturbing its words breaks it rather than redirecting it". We agree, and that phrase is cut from Section 3.1's paragraph on the conditions. That paragraph now presents the entity swap as the edit we chose and makes no claim about other edits. An attacker could also perturb the words to adversarial effect, as the review notes, and we have not tested such edits.

"the authors should first mention the four conditions and have brief descriptions on each of them". Section 3.1's paragraph on the conditions now opens with them and describes each in a sentence or two (change 3). A leaves the instruction and the chain intact. B keeps A's instruction and corrupts the chain. C corrupts the instruction, and the policy writes its chain from it. D keeps C's instruction and gives the policy a clean chain in place of the one it would write. The paragraph then names the three differences we report, corruption (A minus B), the cost of the corrupted instruction (A minus C) and correction (D minus C), and points to Table 1, which sets out all four conditions.

"too many defensive writing". The sentence the review quotes is cut from Section 4.1, and its repeat at the start of Section 4 is removed. The Introduction states the attacker once, as an assumption, in one sentence that also says where the attacker edits and what the second measurement asks (change 2). Figure 1 also marks where the attacker edits.

"This sentence is difficult to understand and needs clarification". In Section 4.1 the sentence now reads "The second measurement's attacker edits the reasoning chain after the policy writes it and before the policy decodes its actions." The Introduction and the ethics statement say the same (change 16).

"move some of the text in Appendix C to the main text". Section 3.2 states what each feature reads and what it checks, with new Table R1 (change 4). The entity check reads the instruction and the chain. The four-feature detector reads the instruction, the chain and the executed action. The table gives each of its features.

FeatureReadsWhat it checks
Entity (the entity check)instruction and chainthe share of the instruction's objects that the chain also names
Directionchain and executed actionwhether each direction word matches the action's sign on that axis
Gripperchain and executed actionwhether the chain's first gripper word matches the gripper command
Parseabilitychainthe share of its sentences with an element the detector can read

Direction and gripper compare the chain with the action the robot then executes, so they can flag only after the robot has moved.

"What is the meaning of pre-registering an action-only task-consistency detector, what results the authors got from this study, and what are the implications". The detector to be tested reads only the robot's executed actions and no chain text at all. From the chain alone, a policy cannot tell an operator's edit from an attacker's, so a defense has to draw on something outside the chain. Pre-registering the test means that the paper fixes it before any episode runs. The registration fixes the detector, the attacker built against it and the conclusion each of three possible results would support. Each is stated in advance, so no result can later be read into a different conclusion. There are no results yet. The test is registered and has not run. No claim depends on it.

"it should be better merged into the methodology section". The revised paper makes that merge. It describes the test in one closing paragraph of Section 3.2, with the full registration in Appendix E (change 6), and the test is no longer in the abstract (change 1). Nothing registered is removed.

AuthorsOpenReview ↗

Response to Reviewer FSkp (3 of 3): the annotations in Table 2; whether the reasoning is faithful

"The authors need to justify their annotations in Table 2". Each word under a detector's number in Table 2 is a conclusion about detection. Its rule was fixed before any score existed. The rule compares the 95% interval of the detector's AUROC with a null. The null is the same detector scored on two clean runs of the policy. Table 2 calls the plain entity swap the non-adaptive attack.

Word in Table 2UnderRule for the detector's intervalWhat it says
caughtthe non-adaptive attacklower end at or above 0.90the detector catches the attack
partly caughtthe non-adaptive attackabove the null, lower end below 0.90it does better than the null, short of that bar
not caughtthe non-adaptive attackoverlaps the nullit does no better than the null
chancean adaptive attackoverlaps the nullit cannot tell attacked chains from clean ones
not evadedan adaptive attackabove the nullit still separates them
reversedan adaptive attackbelow the nullit rates attacked chains as cleaner than clean ones

"Reversed" is a new word for a rule that has not changed. The submission said "inversion" for the four-feature detector and "at or below chance" for the entity check. Each detector treats a low consistency score as a sign of attack. Read in reverse (negated), the same scores separate the adaptive attack from clean chains, pooled over all four suites taken together. However, the reversed scores miss the plain entity swap, the non-adaptive attack. So neither scoring direction catches both attacks. The negated score is a diagnostic, not a detector (new Table R3, change 9).

The revised caption defines each of these words by its rule and says when the rule was fixed. The words on the attack's harm, which Table 2 also printed, are removed from the table (change 7).

"Whether the reasoning is faithful, and to what extent". The first measurement shows that the actions depend on what the chain says. Renaming objects in the chain lowers task success, and giving the policy a clean chain under a corrupted instruction raises it, on every suite of both policies. The clean chain recovers only part of what the corrupted instruction cost. Whether the chain is a faithful account of what the policy computes internally is a further question. We do not test it, because our claims do not need it. An attacker who edits the chain changes what the robot does either way. A new paragraph in Section 5 says what conditions A to D do and do not show (change 13).

Two further changes, made for the other reviews. The revised paper states the monitoring result as a result about the two detectors we tested (changes 1 and 2). It also reports two exploratory checks, added after the reviews, that the attack was not built against. Both catch the adaptive attack (change 12).

Changes to the paper that answer this review. The revised paper is uploaded. The abstract and Section 1 are rewritten in plainer words, and Section 1 states the attacker once, as an assumption (changes 1 and 2). Figure 1 also marks where the attacker edits. Section 3.1's paragraph on conditions A to D opens with the four conditions (change 3). Section 3.2 states what each feature reads and checks, with Table R1 (change 4). It gains a threat-model paragraph on what each attack reads and writes, and on what is shown, assumed and not shown (change 5). The action-only test is now one closing paragraph of Section 3.2, with its registration in Appendix E (change 6). Table 2's caption defines each word in the table by its rule (change 7), and Table R3 in Appendix C gives the scores read in reverse (change 9). Section 5 says what conditions A to D do and do not show about faithfulness (change 13). The sentence on where the attacker edits is rewritten wherever it appears (change 16).

[1] M. Zawalski et al. Robotic Control via Embodied Chain-of-Thought Reasoning. Proceedings of The 8th Conference on Robot Learning, 2025.

[2] NVIDIA et al. Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning. arXiv:2503.15558, 2025.

[3] K. Briski. CrowdStrike, Uber, Zoom Among Industry Pioneers Building Smarter Agents With NVIDIA Nemotron and Cosmos Reasoning Models for Enterprise and Physical AI Applications. NVIDIA Blog, 2025.

We would be glad to answer any further questions, or to clarify any point of the response, before the discussion closes.

Reviewer_7p8LOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether editable reasoning chains in vision-language-action policies provide both a correction interface and an attack surface, and whether monitoring can detect their corruption. It evaluates DeepThinkVLA and CoTinyVLA across four LIBERO suites using paired interventions on instructions and reasoning chains. The authors find that entity substitutions in the chain reduce task success despite intact instructions, while injecting a clean chain partially restores performance under corrupted instructions.

The paper then evaluates entity-coverage and four-feature consistency monitors against non-adaptive and detector-aware attacks. The adaptive attacks eliminate or reverse the monitors’ original detection signal while task performance remains degraded. The authors also derive an empirical AUROC upper bound for detectors that are monotone in specified features, conditional on attacked chains matching or exceeding clean chains on those features. Validation uses repeated simulator rollouts, task-level statistical analyses, and two policies with different reasoning schedules. An action-only monitoring study and additional semantic controls are preregistered but have not been executed.

Critical strengths
  1. Controlled evidence that the reasoning-chain interface can both impair and restore task performance. The A–D design intervenes on instructions and chains while keeping policy weights fixed and matching initial conditions. Injecting clean chains under corrupted instructions produces substantial recovery—for example, from 12.5% to 62.3% success on DeepThinkVLA’s Goal suite. This provides consequential evidence that the chain is a functional control interface, making its integrity relevant to robot behavior and safety.

  2. Replication across policies with different reasoning schedules. The qualitative corruption-and-recovery pattern appears across all four LIBERO suites on both DeepThinkVLA, which generates a chain per action chunk, and CoTinyVLA, whose edited Plan persists across an episode. The task-level analyses and reported rollout variability strengthen the empirical assessment. This replication supports the relevance of the finding to multiple interface designs and is a substantial reason to view the work as a useful empirical contribution.

Critical weaknesses
  1. The monitoring evaluation is too narrow to support the breadth of the central conclusion. Both detectors rely on entity coverage, which the adaptive attack directly satisfies by appending the instruction’s object names. Several additional features provide little discrimination, particularly on CoTinyVLA. The analytical result covers monotone functions of specified features; on DeepThinkVLA, it also concerns mean aggregation rather than the primary maximum-anomaly score. These results establish weaknesses in the evaluated rules, while the broader implications for semantic or learned chain monitors remain unvalidated. This limits the significance of the paper’s principal monitoring contribution.

  2. The interpretation of the detection and harm metrics overstates what the results establish. A pooled AUROC of 0.228 indicates reversed ranking: negating the same scores yields an AUROC of 0.772. It therefore demonstrates failure of the original scoring direction, rather than an absence of discriminative information. Similarly, overlapping confidence intervals do not establish that attack harm is preserved. CoTinyVLA’s pooled success increases from 28.5% under the non-adaptive attack to 38.6% under the adaptive attack, requiring a paired comparison and an explicit tolerance for retained harm. These distinctions are consequential because the central claim depends on establishing evasion while maintaining meaningful behavioral impact.

Questions for authors
  1. Clarification—meaning of detection failure. Is the central claim restricted to evasion of the detectors’ frozen scoring direction, or does it concern the absence of discriminative information? For example, the reported AUROC of 0.228 becomes 0.772 when the same scores are negated. How should this affect the statements that the detectors lose their discriminative power? This distinction would materially affect my assessment of the monitoring conclusion.

  2. Clarification—scope of the generalization. Given that the evaluated detectors share entity coverage, several other features provide little discrimination, and learned monitors remain untested, which broader monitoring claims should be understood as established results versus hypotheses? Please also clarify how the mean-aggregation bound on DeepThinkVLA relates to the primary maximum-anomaly evaluation. If broader claims are retained, what evidence supports them beyond the demonstrated failure of these particular rules?

  3. Evidence request—preservation of attack harm. Could you report task-paired confidence intervals for the success-rate difference between adaptive and non-adaptive attacks, particularly for CoTinyVLA’s pooled increase from 28.5% to 38.6%? What practically meaningful tolerance would justify calling harm “retained,” and which conclusions survive without the post hoc overlap tie-break? This would determine whether the evidence supports approximately preserved attack strength or the narrower conclusion that substantial harm remains.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7p8L (1 of 3): Q1/W2, meaning of detection failure

We thank the reviewer for their careful review, in particular of how we read the detection and harm metrics. Our labels follow the review's numbering: Q1 to Q3 for its questions, W1 and W2 for its weaknesses. Our three comments answer the questions in order, each with the weakness on the same point: Q1 with the detection half of W2, Q2 with W1, and Q3 with the harm half of W2. The third comment also lists the changes to the paper. Brackets are 95% intervals, pp is percentage points of task success, and pooled means all four suites taken together.

Q1/W2. Meaning of detection failure. The claim is about the detectors' frozen scoring direction, the direction in which each was designed to read its score, with a low consistency score as a sign of attack. It is not about an absence of discriminative information. As the reviewer computed, negating the scores that give the AUROC of 0.228 yields 0.772 [0.737, 0.805]. After the reviews we negated the scores of both detectors on both policies, pooled over the forty tasks.

Pooled AUROCAdaptive attack, as designedAdaptive attack, negatedNon-adaptive attack, negated
DeepThinkVLA, entity check0.434 [0.397, 0.465]0.566 [0.535, 0.603]0.001 [0.000, 0.002]
DeepThinkVLA, four-feature detector0.228 [0.195, 0.263]0.772 [0.737, 0.805]0.003 [0.001, 0.005]
CoTinyVLA, entity check0.375 [0.312, 0.438]0.625 [0.562, 0.688]0.002 [0.000, 0.003]
CoTinyVLA, four-feature detector0.365 [0.290, 0.430]0.635 [0.570, 0.710]0.002 [0.000, 0.003]

Read in reverse (negated), the scores separate the adaptive attack from clean chains. However, the reversed scores miss the plain entity swap, the non-adaptive attack. So neither scoring direction catches both attacks. What fails is the scoring direction fixed before the runs, not, in W2's words, "an absence of discriminative information". The negated score is a diagnostic, not a detector. A detector that raises an alarm at both ends of the score would be a new detector, and we have not tested one. The figures in the table cover all four suites taken together. On a suite where a detector reads chance, neither direction separates the attack.

How this affects the statements on discriminative power. The revised paper no longer says that the detectors "lose their discriminative power" or that neither "scores above chance". It says that both detectors, used as designed, are evaded, and it rewords "inversion" (change 7). The negated scores are in Section 4.2 and in new Table R3 in Appendix C, labelled a diagnostic (change 9).

AuthorsOpenReview ↗

Response to Reviewer 7p8L (2 of 3): Q2/W1, scope of the generalization

Q2/W1. Scope of the generalization. No broader monitoring claim is retained. What remains is tested on both policies. Two detectors that catch the plain entity swap are evaded, used as designed, by an attack built against them, while task success stays measurably below clean. The table gives each monitoring claim of the submission and its status. The four points after it follow W1 in its own order.

Claim or questionStatusWhere
Both detectors catch the plain entity swap on every suiteShown, mostly through the entity feature they shareTable 2
Both detectors are evaded by an attack built against themShown, for the fixed scoring directionTables 2 and R2
Under that attack the detectors lose their discriminative powerNo longer claimed; negated, the scores separate the attackTable R3
The attack's harm is retainedNo longer claimed; paired differences replace itTable R2
No detector monotone in the four checked features can score above chanceScope stated: CoTinyVLA's Goal, Object and Spatial suites; Long only once direction is set aside on two tasksAppendix D
The result extends to other monitors of the chainNo longer claimedRemoved from the abstract and Sections 1, 4.2, 5 and 6
Whether a monitor of another kind holds against an attacker who knows itOpenSection 5
  1. "Both detectors rely on entity coverage". Under the plain entity swap the entity check alone reaches 0.999 [0.998, 1.000] pooled on DeepThinkVLA, beside 0.997 [0.995, 0.999] for the four-feature detector. On CoTinyVLA both reach 0.998 [0.997, 1.000]. The adaptive attack keeps the entity swap and appends one sentence that names the instruction's objects, so the entity feature reads clean.

  2. "Several additional features provide little discrimination, particularly on CoTinyVLA." On CoTinyVLA, gripper took the same value on the clean and attacked Plans of every task, and parseability on every task but one. Direction had something to check on one Long task only. On DeepThinkVLA, direction had something to check on almost no query. Section 4.2 says that the result under the plain entity swap comes mostly from the entity feature, and that these features carry little information (change 11).

  3. "...mean aggregation rather than the primary maximum-anomaly score." No DeepThinkVLA conclusion changes under the mean over an episode's queries. The submission gave the four-feature detector's pooled AUROC under the mean score (Appendix C). After the reviews we checked every conclusion under both scores. Each detector gets the same conclusion under every attack, on every suite and pooled (new Table R4). For the four-feature detector under the adaptive attack, the pooled AUROC is 0.354 [0.328, 0.379] under the mean score, beside 0.228 [0.195, 0.263] under the maximum-anomaly score. The maximum-anomaly conclusions remain primary, because that score was fixed before the runs.

The proposition directly supports CoTinyVLA's result on Goal, Object and Spatial. There every attacked episode matches or beats every clean one on all four checked features. On DeepThinkVLA it gives only an upper limit on detection, 0.866 pooled for the four features under the adaptive attack. The table of shares in Appendix D gives a variant of the attack that also flips the stated direction. A limit of 0.866 leaves room for a useful detector. So the DeepThinkVLA result rests on measuring these two detectors as designed, not on the proposition. A detector that reverses a feature, as the negated score does, or that reads other features, is outside the proposition.

  1. "...the broader implications for semantic or learned chain monitors remain unvalidated." We added two checks of other kinds after the reviews. One counts the objects a chain names that the instruction does not. It reuses the entity check's name matching, so "bowl" inside "black bowl" counts as another object, which raises the count on clean chains too. The other is a language-model judge that reads the instruction and the first chain of each episode.
Check the attack was not built against, pooled AUROCDeepThinkVLACoTinyVLAConclusion
Object count0.976 [0.962, 0.988]0.998 [0.996, 0.999]caught; DeepThinkVLA's Goal suite partly, 0.932 [0.894, 0.958]
Language-model judge0.994 [0.988, 0.998]0.999 [0.998, 1.000]caught on every suite

Both checks catch the attack. Both are exploratory. Whether an attacker who knows the two checks can evade them is open. The plainest way to pass both is a chain that agrees with the instruction. That chain could still be wrong about what the instruction does not say: which of two similar objects to take, which way to move, when to close the gripper. A check that reads only text cannot test those choices against the scene. Whether such an edit still makes the robot fail is the open question.

AuthorsOpenReview ↗

Response to Reviewer 7p8L (3 of 3): Q3/W2, preservation of attack harm

Q3/W2. Preservation of attack harm. Of the two conclusions the reviewer sets out, the paired intervals support the narrower one. Harm remains on every suite of both policies. On CoTinyVLA, the pooled rise the reviewer cites, from 28.5% to 38.6%, is a paired difference of +10.1 [+5.2, +15.0] pp. Over all four suites taken together, the adaptive attack therefore does measurably less harm than the plain entity swap on that policy. The table gives two differences paired by task, clean minus the adaptive attack and the adaptive attack minus the plain entity swap (new Table R2). Both were computed after the reviews.

Paired by task, ppClean minus adaptive attackAdaptive attack minus plain entity swapWhat the second shows
CoTinyVLA, Goal+74.8 [+61.3, +87.5]+8.5 [-6.2, +22.7]cannot be told apart
CoTinyVLA, Object+38.5 [+28.2, +48.0]+9.0 [+4.7, +13.3]measurably less harm
CoTinyVLA, Spatial+48.5 [+33.3, +66.2]+18.3 [+9.7, +27.8]measurably less harm
CoTinyVLA, Long+69.8 [+55.8, +82.8]+4.7 [-0.7, +10.5]cannot be told apart
CoTinyVLA, pooled+57.9 [+49.7, +66.2]+10.1 [+5.2, +15.0]measurably less harm
DeepThinkVLA, Goal+16.3 [+8.2, +24.7]+2.8 [-0.8, +6.7]cannot be told apart
DeepThinkVLA, Object+4.2 [+0.8, +8.2]+0.5 [-0.3, +1.3]cannot be told apart
DeepThinkVLA, Spatial+5.8 [+1.8, +10.5]+0.8 [-0.3, +2.2]cannot be told apart
DeepThinkVLA, Long+7.3 [+1.2, +14.3]+0.5 [-3.5, +5.0]cannot be told apart
DeepThinkVLA, pooled+8.4 [+5.2, +11.9]+1.2 [-0.3, +2.7]cannot be told apart

Task success under the adaptive attack is measurably below clean on every suite of both policies, the four DeepThinkVLA suites included. On CoTinyVLA, evading the detectors costs the attack part of its harm. On DeepThinkVLA the two attacks cannot be told apart on any suite, which is not a claim that their harm is the same. A variant of the attack that also flips the stated direction gives the same conclusions on DeepThinkVLA.

A tolerance for "retained". None was fixed before the runs, and one chosen now would be post hoc. As W2 says, overlapping intervals do not establish that harm is preserved. The revised paper therefore does not call harm retained. It reports these paired differences and what they show (change 8).

What survives without the tie-break. Every conclusion that a detector is evaded survives without it. The revised paper says that a detector is evaded only where the interval of clean minus the adaptive attack lies wholly above zero, a rule set after the reviews, before these differences were computed. That holds on every suite of both policies. In the submission the tie-break decided the word on the attack's harm on DeepThinkVLA's Object, Spatial and Long suites. Those words and the tie-break are removed from Table 2. The submitted caption of Table 2 said that its words were fixed before any score. That held for the words on detection. The tie-break was added after the tables existed, as Section 4.2 and Appendix C of the submission state. The revised caption defines each word in Table 2 by its rule and says when the rule was fixed (change 7).

The semantic controls. The reviewer's summary notes that the semantic controls had not been executed. Both have now run on DeepThinkVLA's Goal suite, the suite they were registered for, and are reported in Appendix F (change 15). Rewording the chain while keeping its object names costs at most +1.2 pp of task success (one-sided upper 95% bound), under the 5 pp threshold fixed in the submission. So the entity swap's cost comes from what the corrupted chain says. The rewording must keep the object names, so it changed less text than registered, a median of 36.0 characters per query against 60.0 for the entity swap.

Changes to the paper that answer this review. The revised paper is uploaded. The abstract and Section 1 state the narrowed monitoring claim (changes 1 and 2). Section 4.2 and Table 2 say that both detectors, used as designed, are evaded, reword "inversion", and drop the words on the attack's harm and the tie-break, with each word in the table defined in the caption (change 7). In Appendix C, Table R2 gives the paired differences (change 8), Table R3 the negated scores (change 9), and Table R4 the conclusions with each episode scored by the average over its queries (change 10). Section 4.2 and Appendix D state the proposition's reach exactly (change 10). Section 4.2 says that the result under the plain entity swap comes mostly from the entity feature, and that several other features carry little information (change 11). It reports the two added checks with Table R5 in Appendix H (change 12). Section 5 states what remains open (change 13), and Section 6 is narrowed to match (change 14).

We would be glad to answer any further questions, or to clarify any point of the response, before the discussion closes.

Reviewer_UA7XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether the textual reasoning chain of a vision-language-action policy, exposed so that a person can correct the robot, can also be used by an attacker to steer it. The authors test DeepThinkVLA and CoTinyVLA on the four LIBERO suites. In the first measurement, renaming objects in an otherwise intact chain lowers task success, and injecting the clean chain under a corrupted instruction recovers part of the loss. In the second, two fixed-rule detectors catch the non-adaptive entity-swap attack almost perfectly but cannot separate the adaptive attack from clean chains, while that attack still reduces task success. The authors conclude that monitoring features read from a channel the attacker can write is fundamentally limited.

Critical strengths

The question is a good one. An interface built so that people can fix a robot’s reasoning is also one an attacker can write to, and the paper examines this with controlled interventions instead of leaving it as a conceptual worry. The first measurement is carefully designed: conditions C and D differ only in the chain, and the task-paired analysis makes the recovery effect credible. The authors report run-to-run variation and are open about pre-registered choices, post-hoc corrections and experiments that have not yet been run. Testing the detectors against an adaptive attacker is the right standard for this question.

Critical weaknesses

My main concern is that the general conclusion about monitoring reasoning chains goes further than the experiments allow. The proposition in Section 4.2 is sound for the family it defines, detectors that are monotone in four chosen features, but that family is narrow. On DeepThinkVLA the smallest share over all four features is 0.291, which gives a ceiling of about 0.855, and the best three-feature set gives about 0.701 pooled. Appendix D also states that the bound covers the per-episode mean anomaly, whereas the DeepThinkVLA verdicts in Table 2 use the largest per-query anomaly, so the proposition does not directly explain the headline results there. The authors themselves concede that a different weighting could help on DeepThinkVLA, and no monitor that flags conflicting object references within a chain is evaluated. What the evidence supports is that this fixed-rule detector family can be bypassed by an attacker built against its features. It does not show that semantic or learned monitors would fail in general. The experimental validation has further gaps. Both detectors are described as reading only the chain, yet the entity check uses the instruction, and the direction and gripper features compare the chain with executed actions, so the “chain-only” description is inaccurate. The near-perfect non-adaptive results also seem to come mostly from the entity mismatch that the attack introduces: on DeepThinkVLA the entity check alone reaches a pooled AUROC of 0.999, against 0.997 for the four-feature detector. The registered benign-rewrite and naturalistic-corruption controls have not been run, so it is unclear how much of the drop in task success reflects the semantics of the corruption and how much reflects editing the chain or producing off-distribution text. Finally, “harm retained” is judged mainly by overlapping confidence intervals, which is not an equivalence test. On CoTinyVLA, pooled success rises from 28.5 under the non-adaptive attack to 38.6 under the adaptive one, with the largest rise on Spatial. The adaptive attack remains highly damaging, but the analysis does not quantify how much of the original harm survives. All experiments are in simulation. This is reasonable for paired causal comparisons, and the paper says so, but it leaves the deployment-oriented security claims unvalidated on hardware. The paper does not show that an attacker could obtain the same write access in a realistic control stack, or that the adaptive edit would keep its effect under sensing noise, timing constraints and execution errors. Claims about deployment should at least be scoped explicitly to simulated LIBERO rollouts. I would be more positive if the claims were narrowed to the tested detector family and the simulated setting, or if the authors ran the registered controls and quantified the adaptive-versus-non-adaptive difference in success with paired confidence intervals or a pre-specified equivalence margin.

Questions for authors
  1. Which inputs does each detector have at decision time? Please clarify the roles of the instruction, the reasoning chain and the executed actions, and revise the “chain-only” wording if needed.
  2. Can you report the benign-rewrite and naturalistic-corruption controls, or otherwise estimate how much of the condition-B loss comes from semantic corruption rather than from editing the chain?
  3. Can you report paired differences in task success between the adaptive and non-adaptive attacks, with confidence intervals and a pre-specified criterion for what counts as retained harm?
  4. The proposition’s bound applies to the mean anomaly, while the headline verdicts use the largest per-query anomaly. Which empirical conclusions does the proposition directly support?
  5. Given that all experiments are in simulation, how should readers interpret the deployment-oriented claims? Do you expect the assumed chain-write access and the adaptive attack to carry over to a physical control stack with timing constraints and execution noise?
6 comments
AuthorsOpenReview ↗

Response to Reviewer UA7X (1 of 4): W1, the conclusion about monitoring; W3, the entity mismatch

We thank the reviewer for their careful review, in particular for setting out what the monitoring evidence does and does not support. We label the review's points as follows. Q1 to Q5 are its five questions, in its order. W1 to W6 are the six points of its weakness paragraph, in its order: the conclusion about monitoring (W1), the "chain-only" description (W2), the entity mismatch (W3), the registered controls (W4), "harm retained" (W5) and simulation (W6).

The review asks that the claims be narrowed to the tested detector family and the simulated setting, or that the registered controls be run and the difference between the two attacks be given with paired intervals or a pre-specified equivalence margin. We narrowed the claims, ran the controls and give the paired intervals.

  • The claims are narrowed to the two rule-based detectors we tested and to simulated LIBERO rollouts (W1 and Q5).
  • Both registered controls ran on DeepThinkVLA's Goal suite, the suite they were registered for (Q2).
  • The differences between the two attacks are paired by task, on every suite of both policies (Q3).

We answer in four comments. This comment answers W1 and W3, which have no question. The second comment answers Q1 with W2, and Q2 with W4. The third comment answers Q3 with W5, and Q4. The fourth comment answers Q5 with W6 and lists the changes to the paper. Brackets are 95% intervals, pp is percentage points of task success, and pooled means all four suites taken together.

W1. The "general conclusion about monitoring reasoning chains goes further than the experiments allow." The revised paper claims what the review says the evidence supports. Two rule-based detectors, used as designed, are evaded by an attack built against their features. The abstract and Sections 1, 4.2, 5 and 6 state the claim at this scope (changes 1, 2, 12, 13 and 14).

As W1 says, the proposition covers a narrow family of detectors. Its members never rate an episode as less clean when one of the four checked features rises. A detector that weights those features differently is still held to the proposition's upper limit on detection, and is untested. A detector that reverses a feature, or that reads other features, is outside the proposition. The revised paper states this reach exactly (change 10).

W1 also says that "no monitor that flags conflicting object references within a chain is evaluated", and that the evidence "does not show that semantic or learned monitors would fail in general". We added two checks of other kinds after the reviews. One counts the objects a chain names that the instruction does not. It compares the chain with the instruction, so a rule for conflicting names within one chain is still untested. It reuses the entity check's name matching, so "bowl" inside "black bowl" counts as another object. The other is a language-model judge that reads the instruction and the first chain of each episode.

Check the attack was not built against, pooled AUROCDeepThinkVLACoTinyVLAConclusion
Object count0.976 [0.962, 0.988]0.998 [0.996, 0.999]caught; DeepThinkVLA's Goal suite partly, 0.932 [0.894, 0.958]
Language-model judge0.994 [0.988, 0.998]0.999 [0.998, 1.000]caught on every suite

The object count and the language-model judge, which the attack was not built against, both catch it. Both are exploratory. The two checks now stand where the two detectors stood under the plain entity swap, the non-adaptive attack. They are untested against an attacker who knows them. How a learned monitor fares under an attacker built against it is also open, as Section 5 says (change 13). Section 4.2 reports the two checks, with new Table R5 in Appendix H (change 12).

W3. "The near-perfect non-adaptive results also seem to come mostly from the entity mismatch". They do. On DeepThinkVLA the entity check alone reaches 0.999 [0.998, 1.000] pooled, beside 0.997 [0.995, 0.999] for the four-feature detector, the two figures the reviewer cites. On CoTinyVLA both reach 0.998 [0.997, 1.000]. Several of the other features carry little information. On CoTinyVLA, gripper took the same value on the clean and attacked Plans of every task, and parseability on every task but one. On DeepThinkVLA, direction had something to check on almost no query. Section 4.2 says so (change 11).

AuthorsOpenReview ↗

Response to Reviewer UA7X (2 of 4): Q1/W2, the inputs of each detector; Q2/W4, the registered controls

Q1/W2. "Which inputs does each detector have at decision time?" The entity check reads the instruction and the chain. The four-feature detector reads the instruction, the chain and the executed action. The table gives each of its features.

FeatureReadsWhat it checks
Entity (the entity check)instruction and chainthe share of the instruction's objects that the chain also names
Directionchain and executed actionwhether each direction word matches the action's sign on that axis
Gripperchain and executed actionwhether the chain's first gripper word matches the gripper command
Parseabilitychainthe share of its sentences with an element the detector can read

Direction and gripper compare the chain with the action the robot then executes, so they can flag only after the robot has moved. The "chain-only" description was therefore inaccurate, as W2 says. The revised paper states what each detector reads and when it can flag in Section 3.2, with new Table R1 (change 4). It corrects "read only the chain" wherever it appears, Figure 1's caption and the ethics statement included (changes 2 and 16).

The revised paper also states what each attack reads (change 5). The plain entity swap reads and writes only the reasoning chain. The adaptive attack also reads the instruction, whose object names it appends. A variant of it, the attack with the direction flip, also flips each direction word that disagrees with how the arm moves under the clean chain. To find those words, it reads the action the policy decoded from its own clean chain.

Q2/W4. "Can you report the benign-rewrite and naturalistic-corruption controls, or otherwise estimate how much of the condition-B loss comes from semantic corruption rather than from editing the chain?" Yes. Both controls ran on DeepThinkVLA's Goal suite, the suite they were registered for. Condition E, the benign rewrite, keeps the instruction intact and rewrites the chain without changing what it names. Condition F, the naturalistic corruption, rewrites the chain in fluent sentences to direct the policy to a distractor object present in the scene. E is a fixed rule, a table of synonyms, and F's kind was chosen from the three registered, both before any of their episodes ran.

DeepThinkVLA, Goal suite, ten tasksCost in task successConclusion fixed in the submission
Condition B, entity swap+19.2 [+10.3, +27.8] pp
Condition E, chain reworded, names keptat most +1.2 pp (one-sided upper 95% bound)"B's cost comes from what the corrupted chain says."
Condition F, fluent corruption to a wrong object in the scene+14.7 pp; -4.5 [-12.3, +2.8] pp against B (paired d = -0.34)"The entity swap bounds the naturalistic corruption."

E's chain enters by condition B's route, as registered, so E's cost includes whatever sending an edited chain back to the policy costs by itself. E's bound is under the 5 pp threshold fixed in the submission. So on this suite the loss comes from what the corrupted chain says, not from the chain being edited. That is the question W4 raises.

W4 also names off-distribution text, and condition F bears on it. F's cost lies inside B's interval, and the paired difference between F and B has an interval that includes zero. So the two cannot be told apart, and the entity swap bounds this corruption.

One deviation from the registration. The swap changes the object names themselves. E must keep them, as registered, so it can change only the words around them. It therefore changed less text than the registration asked for, a median of 36.0 characters per query against 60.0 for the entity swap. This was recorded before any E episode ran. It changes none of the conclusions fixed in the submission, because E's bound is read against the same 5 pp threshold. The bound is for a rewrite of that size. Appendix F, the registered controls, reports both conditions, with the deviation and with what condition F changes besides the object (change 15).

AuthorsOpenReview ↗

Response to Reviewer UA7X (3 of 4): Q3/W5, paired differences in task success; Q4, what the proposition supports

Q3/W5. "Can you report paired differences in task success between the adaptive and non-adaptive attacks, with confidence intervals and a pre-specified criterion for what counts as retained harm?" The table below gives them for every suite of both policies. On CoTinyVLA the pooled rise the reviewer cites, from 28.5 to 38.6, is a paired difference of +10.1 [+5.2, +15.0] pp, and the largest rise is on Spatial, as the review notes. Over all four suites taken together, the adaptive attack therefore does measurably less harm than the plain entity swap on that policy. The table gives two differences paired by task, clean minus the adaptive attack and the adaptive attack minus the plain entity swap (new Table R2). Both were computed after the reviews.

Paired by task, ppClean minus adaptive attackAdaptive attack minus plain entity swapWhat the second shows
CoTinyVLA, Goal+74.8 [+61.3, +87.5]+8.5 [-6.2, +22.7]cannot be told apart
CoTinyVLA, Object+38.5 [+28.2, +48.0]+9.0 [+4.7, +13.3]measurably less harm
CoTinyVLA, Spatial+48.5 [+33.3, +66.2]+18.3 [+9.7, +27.8]measurably less harm
CoTinyVLA, Long+69.8 [+55.8, +82.8]+4.7 [-0.7, +10.5]cannot be told apart
CoTinyVLA, pooled+57.9 [+49.7, +66.2]+10.1 [+5.2, +15.0]measurably less harm
DeepThinkVLA, Goal+16.3 [+8.2, +24.7]+2.8 [-0.8, +6.7]cannot be told apart
DeepThinkVLA, Object+4.2 [+0.8, +8.2]+0.5 [-0.3, +1.3]cannot be told apart
DeepThinkVLA, Spatial+5.8 [+1.8, +10.5]+0.8 [-0.3, +2.2]cannot be told apart
DeepThinkVLA, Long+7.3 [+1.2, +14.3]+0.5 [-3.5, +5.0]cannot be told apart
DeepThinkVLA, pooled+8.4 [+5.2, +11.9]+1.2 [-0.3, +2.7]cannot be told apart

Task success under the adaptive attack is measurably below clean on every suite of both policies, the four DeepThinkVLA suites included. On CoTinyVLA, evading the detectors costs the attack part of its harm. On DeepThinkVLA the two attacks cannot be told apart on any suite, which is not a claim that their harm is the same. A variant of the attack that also flips the stated direction gives the same conclusions on DeepThinkVLA.

A pre-specified criterion for retained harm. The submission's criterion was the overlap of intervals, fixed before the runs. As W5 says, that is not an equivalence test. No margin was fixed, and one chosen now would be post hoc. The revised paper therefore does not call harm retained. It reports these paired differences and what they show (change 8). It calls a detector's verdict an evasion only where the interval of clean minus the adaptive attack lies wholly above zero. That rule was set after the reviews, before these differences were computed, and it holds on every suite of both policies. The words on the attack's harm, and the rule added after the run to break ties, are removed from Table 2 (change 7).

Q4. "Which empirical conclusions does the proposition directly support?" CoTinyVLA's result on Goal, Object and Spatial. There every attacked episode matches or beats every clean one on all four checked features, so no detector monotone in those features, one that never rates an episode as less clean when a feature rises, can score above chance. On Long it holds only once the direction feature is set aside on two of its tasks (Appendix D). On DeepThinkVLA the proposition gives only an upper limit on detection over the four features. Over entity and direction alone, it holds on DeepThinkVLA's Spatial suite under the attack with the direction flip (Appendix D). The two limits the reviewer cites are for the attack with the direction flip, whose shares Section 4.2 of the submission plots. Without the flip they are 0.866 for the four features and 0.783 for the best set of three. These limits leave room for a useful detector. So the DeepThinkVLA result rests on measuring these two detectors as designed, not on the proposition.

The bound covers the mean score, the average over an episode's queries, while the verdicts use the largest per-query anomaly, the score of its most suspicious query, as the question notes. The submission gave the four-feature detector's pooled AUROC under the mean score (Appendix C). After the reviews we read every DeepThinkVLA verdict under the mean score as well. No verdict changes, for either detector, under the plain entity swap or the adaptive attack, on any suite or pooled (new Table R4, change 10). The verdicts on the largest per-query anomaly stay the headline, because that score was fixed before the runs.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response to all reviewers and the area chair (1 of 3): the claim; requests and results

We thank the reviewers for their careful reviews. The reviews credit the core of the paper, the measurement of how an edit to the reasoning chain changes what the robot does. Part 1 gives the paper's claim, and the requests with what we did and what it showed. That table ends in Part 2, which then presents the numbers. Part 3 gives the two added checks, the open question and the list of numbered changes. The revised paper, now uploaded, carries the numbered changes and Tables R1 to R5, with new sentences in blue. The two controls were registered in the submission; the other analyses and the two checks were specified after the reviews. Each reviewer also receives their own reply, addressing their review.

The claim. A person can rewrite a robot's reasoning chain to correct the robot. The paper asks what the same access gives an attacker. On two policies, in simulated LIBERO suites, the chain moves the robot in both directions. Renaming objects in the chain lowers task success even when the instruction is intact. Giving the policy a clean chain recovers part of the loss when the instruction is corrupted. Two rule-based detectors catch the renamed chains on every suite. An attacker who knows what they check then evades both detectors, used as designed, while task success stays measurably below clean. The attacker's edit is caught by two further checks that it was not built to evade.

Requests and results. The numbers are in the tables of Part 2.

AskedWhat we didWhat it showed
Monitoring conclusion wider than two rule-based detectors (UA7X, 7p8L, 5cVB)Claim narrowed to these two detectors, used as designed, and the argument beyond them removed, in the abstract and Sections 1, 4.2, 5 and 6 (changes 1, 2, 12, 13, 14)The attack evades both detectors, used as designed, on every suite of both policies, while task success stays measurably below clean.
Unclear threat model (FSkp); how the attacker gets access (FSkp, 5cVB, UA7X)A threat-model paragraph in Section 3.2: what each attack reads and writes, what is shown and assumed (change 5); deployment claims limited to simulation (change 13)The attacker we study edits the chain after the policy writes it and before the policy decodes its actions. An operator can rewrite it at that point to correct the robot, and we assume an attacker can too. The same point exists in pipelines where one model reasons in text and a second model acts on it, a use NVIDIA describes for Cosmos Reason [1, 2]. We show what an edit there does, not how an attacker would reach it.
Paired differences in task success (UA7X, 7p8L)Paired by task on every suite, in new Table R2 (change 8); "evaded" written only where the attack measurably lowers success, a rule set after the reviews, before these differences were computed (change 7)Success is measurably lower under the adaptive attack on every suite of both policies. On CoTinyVLA evading the detectors costs the attack part of its harm; on DeepThinkVLA the two attacks cannot be told apart on any suite, which is not a claim that their harm is the same.
Is the reasoning faithful? (FSkp); correctness of the chain, not only its consistency (5cVB)A paragraph in Section 5 on what conditions A to D do and do not show (change 13)The actions depend on what the chain says. Whether the chain is a faithful account of what the policy computes internally is a further question. We do not test it, because our claims do not need it. An edit to the chain changes what the robot does either way. The checks test consistency, not correctness against the scene.
What the action-only test is and what it found (FSkp); validation of the alternative defense (5cVB)One closing paragraph in Section 3.2, with the registration in Appendix E, the action-only test (change 6)The test is registered and has not run. No claim depends on it.
Does the claim concern the scoring direction, or the absence of discriminative information? (7p8L)Negated scores as a diagnostic, in new Table R3 (change 9); the claim reworded to say that both detectors, used as designed, are evaded (change 7)Read the other way round, with a high score as the sign of attack, the scores separate the adaptive attack from clean chains. No single direction catches both attacks.
What the proposition supports (UA7X)The conclusions with each episode scored by the average over its queries, beside those with each scored by its most suspicious query, in new Table R4; the proposition's reach stated exactly (change 10)The proposition directly supports CoTinyVLA's result on Goal, Object and Spatial. On DeepThinkVLA it gives only an upper limit on detection over the four features, which leaves room for a useful detector. No conclusion there changes under the average.
AuthorsOpenReview ↗

General response to all reviewers and the area chair (2 of 3): requests and results, continued; the measurement and its two controls; the two detectors

Requests and results, continued.

AskedWhat we didWhat it showed
Run the registered controls (UA7X)Ran both on DeepThinkVLA's Goal suite, the suite they were registered for; reported in Appendix F, the registered controls (change 15)The entity swap's cost comes from what the corrupted chain says, and a fluent corruption cannot be told apart from the entity swap.
No monitor flags conflicting object references within a chain (UA7X)Added a count of the objects a chain names that the instruction does not, in new Table R5 (change 12)The attack, not built against this count, is caught, though only partly on DeepThinkVLA's Goal suite. A rule for conflicting names within one chain is still untested.
Would a language-model monitor be fooled? (5cVB)Added a language-model judge that reads the instruction and the first chain of each episode, also in Table R5 (change 12)The attack, not built against this judge, is caught on every suite.
Which inputs does each detector read? (UA7X)What each detector reads and when it can flag, stated in Section 3.2 with new Table R1 (change 4); "read only the chain" corrected everywhere (change 16)The entity check reads the instruction and the chain. Direction and gripper compare the chain with the executed action, so they can flag only after the robot has moved.
Hard to follow (FSkp)Plainer words in the abstract and Sections 1 and 3. Conditions A to D stated first (change 3). What each feature reads and checks, in Section 3.2 with new Table R1 (change 4). Each word in Table 2 defined in its caption (change 7). The action-only test in one paragraph (change 6).The reply to FSkp describes each passage the review names.

Brackets are 95% intervals, and pp is percentage points of task success.

The measurement, with its two controls. The loss from the renamed chain and the gain from the clean chain hold on both policies and all four suites.

Smallest to largest effect (Object to Goal)DeepThinkVLACoTinyVLA
Loss from the renamed chain+4.7 [+1.2, +8.7] to +19.2 [+10.3, +27.8] pp+47.5 [+39.3, +56.0] to +83.3 [+69.7, +93.2] pp
Gain from the clean chain+20.7 [+8.8, +33.3] to +49.8 [+35.2, +63.8] pp+74.0 [+66.2, +81.3] pp over the forty tasks
Controls, DeepThinkVLA, Goal suiteCost in task successConclusion fixed in the submission
Entity swap (objects renamed)+19.2 [+10.3, +27.8] pp
Chain reworded, names keptat most +1.2 pp (one-sided upper 95% bound)under the 5 pp threshold: the swap's cost comes from what the corrupted chain says
Fluent corruption to a wrong object in the scene+14.7 pp; -4.5 [-12.3, +2.8] pp against the swapinside the swap's interval: "The entity swap bounds the naturalistic corruption."

The reworded chain must keep the object names, so it changed less text than registered, a median of 36.0 characters per query against 60.0 for the swap.

Monitoring with the two detectors. Two rule-based detectors catch the plain entity swap, the non-adaptive attack, on every suite, mostly through the entity feature they share. An attack built against them keeps the swap and adds the text each check needs to read clean. It evades both detectors, used as designed, on every suite. Each detector treats a low consistency score as a sign of attack. Read in reverse (negated), the same scores separate the adaptive attack from clean chains, pooled over all four suites taken together. However, the reversed scores miss the plain entity swap. So neither scoring direction catches both attacks.

Pooled AUROCPlain swapAdaptive attackPlain swap, negatedAdaptive attack, negated
DeepThinkVLA, entity check0.999 [0.998, 1.000]0.434 [0.397, 0.465]0.001 [0.000, 0.002]0.566 [0.535, 0.603]
DeepThinkVLA, four-feature detector0.997 [0.995, 0.999]0.228 [0.195, 0.263]0.003 [0.001, 0.005]0.772 [0.737, 0.805]
CoTinyVLA, entity check0.998 [0.997, 1.000]0.375 [0.312, 0.438]0.002 [0.000, 0.003]0.625 [0.562, 0.688]
CoTinyVLA, four-feature detector0.998 [0.997, 1.000]0.365 [0.290, 0.430]0.002 [0.000, 0.003]0.635 [0.570, 0.710]

What fails is the way the scores were designed to be read, not the information in the chain. The negated score is a diagnostic, not a detector. On DeepThinkVLA, a variant of the attack that also flips the stated direction gives the same conclusions.

Pooled difference in task success, paired by taskDeepThinkVLACoTinyVLAConclusion
Clean minus adaptive attack+8.4 [+5.2, +11.9] pp+57.9 [+49.7, +66.2] ppmeasurably lower on every suite
Adaptive attack minus plain swap+1.2 [-0.3, +2.7] pp+10.1 [+5.2, +15.0] ppcannot be told apart, not shown equal (DeepThinkVLA); measurably less harm (CoTinyVLA)
AuthorsOpenReview ↗

General response to all reviewers and the area chair (3 of 3): two further checks; the open question; the list of changes

Monitoring with two further checks. Three reviewers named monitors of other kinds that we had not tested. We added two checks after the reviews.

Check the attack was not built against, pooled AUROCDeepThinkVLACoTinyVLAConclusion
Object count0.976 [0.962, 0.988]0.998 [0.996, 0.999]caught; DeepThinkVLA's Goal suite partly, 0.932 [0.894, 0.958]
Language-model judge0.994 [0.988, 0.998]0.999 [0.998, 1.000]caught on every suite

Both checks catch the attack. Both are exploratory.

The open question. So far the two checks have faced only an attack that was not built against them, as the two detectors had under the plain entity swap. Whether an attacker who knows them can evade them is open. The plainest way to pass both is a chain that agrees with the instruction. That chain could still be wrong about what the instruction does not say: which of two similar objects to take, which way to move, when to close the gripper. A check that reads only text cannot test those choices against the scene. Whether such an edit still makes the robot fail is the open question. How a learned monitor of the chain fares under an attacker built against it is also open.

The numbered changes, for reference.

ChangeWhat changesWhere
1The monitoring claim is stated at the tested scope, in plainer words. The action-only test leaves the abstract.Abstract
2The conclusions are narrowed the same way, in plainer words. The attacker is stated once, as an assumption. Figure 1 marks where the attacker edits. "Read only the chain" is corrected, Figure 1's caption included.Section 1
3Section 3 is put in plainer words. Conditions A to D come first, with a reference to Table 1. Each corrupted text is said to be our own edit. The phrase on reordering words is cut.Section 3
4What each detector and each feature reads, what it checks and when it can flag (Table R1).Section 3.2
5A threat-model paragraph says what each attack reads and writes, and what is shown, assumed and not shown. It notes that someone who builds the model could do more.Section 3.2
6The action-only test in one closing paragraph, with its full registration in Appendix E.Sections 3.2 and 4; Appendix E, the action-only test
7Both detectors, used as designed, are said to be evaded. "Inversion" becomes "reversed". The caption defines each word in the table by its rule. The words on the attack's harm leave Table 2.Section 4.2; Table 2
8Paired differences in task success, with intervals (Table R2).Section 4.2; Appendix C
9The negated score, as a diagnostic (Table R3).Section 4.2; Appendix C
10What the proposition covers, with the upper limit under each adaptive attack and the conclusions with each episode scored by the average over its queries (Table R4).Section 4.2; Appendices C and D
11The result under the plain entity swap comes mostly from the entity feature. Several features carry little information.Section 4.2
12The two checks added after the reviews (Table R5).Section 4.2; Appendix H
13What remains open. What conditions A to D do and do not show. Consistency is not correctness. Claims about deployment are limited to simulated LIBERO rollouts.Section 5
14The conclusion is narrowed to match.Section 6
15Both registered controls, with the conclusions fixed in the submission and the deviation from the registration.Appendix F, the registered controls
16"Read only the chain" and the sentence on where the attacker edits are rewritten wherever they appear.Sections 1, 3, 4 and 6; ethics statement

[1] NVIDIA et al. Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning. arXiv:2503.15558, 2025.

[2] K. Briski. CrowdStrike, Uber, Zoom Among Industry Pioneers Building Smarter Agents With NVIDIA Nemotron and Cosmos Reasoning Models for Enterprise and Physical AI Applications. NVIDIA Blog, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.