- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies behavior-preserving availability attacks on reasoning VLA models. It proposes SPADE, which first optimizes an embedding suffix to elicit longer reasoning and smaller task-directed movements from a frozen policy, then transfers these behaviors to a fixed visual patch. Closed-loop qualification and refreshed student-context supervision aim to preserve task completion and motion while increasing both policy-call count and per-call latency. The evaluation covers three reasoning VLAs, simulation benchmarks, transfer settings, and printed-patch deployment on a physical robot.
Critical strengths
-
The attack objective is interesting and well motivated. Jointly increasing policy-call count and per-call inference cost while preserving task-directed execution distinguishes this work from attacks that simply disrupt task completion or prolong isolated language outputs. It highlights an availability risk that conventional success-rate evaluation may miss.
-
The framework addresses the coupling between reasoning and execution. Closed-loop teacher qualification, boundary-aware imitation, action matching, and refreshed student-context supervision form a coherent approach to transferring executable slow behavior into a fixed patch. Budget-matched comparisons, component ablations, and physical deployment provide meaningful support for the proposed approach.
Critical weaknesses
-
The overview figure does not clearly communicate the central idea and optimization flow. Figure 1 is crowded with text, equations, and decorative illustrations, making the pipeline appear as a collection of modules rather than a clearly motivated sequence. The distinction between suffix search, patch optimization, and patch-only deployment should be more prominent. Some illustrations, including the apparent whole-arm translation used to depict motion, also need more careful treatment.
-
The attack’s robustness against practical mitigation strategies remains unclear. The paper does not evaluate safeguards at different levels, such as reasoning-only repetition penalties or loop detection, tighter inference budgets and policy-call limits, and visual preprocessing or adversarial-input defenses. These controls could affect the two slowdown factors differently. Evaluations should report both remaining attack effectiveness and clean-task performance, including valid action returns, to distinguish useful mitigation from interventions that simply prevent task completion.
-
The paper lacks concrete clean-versus-attacked reasoning examples for the different victim models. These models have distinct reasoning and action interfaces, so aggregate latency and token statistics alone do not explain how the attack operates in each case. Paired native outputs would clarify whether the additional reasoning is repetitive, off-task, or task-relevant, and whether normal reasoning-to-action transitions are preserved. Examples should include token counts, exit markers, and action-return status, with cached Plan content separated from newly generated Think content for CoTinyVLA.
Questions for authors
-
How effective is SPADE under practical safeguards? Please evaluate representative controls targeting reasoning generation (e.g., reasoning-only repetition penalties), resource use (e.g., reasoning-token or policy-call limits), and visual inputs (e.g., preprocessing defenses). Report the remaining slowdown together with clean and attacked task success and valid action-return rates, so that mitigation effectiveness can be assessed alongside its utility cost.
-
What does the induced reasoning look like in each victim model? Please provide paired clean and attacked CoT examples for all three models, marking reasoning length, the exit delimiter, and whether a valid action follows. For CoTinyVLA, separate the cached Plan from the newly generated Think span. These examples should clarify whether slowdown comes from repetition, off-task generation, or longer task-relevant reasoning.
I would consider raising my score if these concerns are adequately addressed in the rebuttal.
4 comments
Response to Reviewer QoDY (2/3)
W3 & Q2: What does the induced reasoning look like in each victim model?
We agree that aggregate latency does not show this. In short, the added reasoning is mainly longer task-relevant reasoning (60%–70% of calls across the three models, with 22%–28% repetition and 8%–12% off-task text), and at least 96% of attacked calls in every model still reach the exit and return a valid action. Below are the clean/attacked pairs for DeepThinkVLA and CoTinyVLA; the ECoT pair follows in the next comment, and App. G of the revised PDF has all pairs.
How the pairs are chosen. Clean and attacked runs share the task, layout and call, and differ only by the patch. Task, for all three models: "pick up the black bowl between the plate and the ramekin and place it on the plate". DeepThinkVLA and ECoT show the first call; CoTinyVLA shows the second call, so that its Plan is already cached. Long outputs are cut with [...], keeping start and end; line breaks are added for display.
Reading the table. "Exit reached" means the native end-of-reasoning marker was generated: </think> for DeepThinkVLA, <|cot_think_end|> for CoTinyVLA, ACTION: for ECoT. "Valid action" means the exit is reached and the native parser returns an action; a fallback at the token cap does not count (App. B.3). CoTinyVLA generates its Plan at the first call and caches it; each later call generates only a new Think span, which the attack targets.
| Model | Run | Tokens generated in this call | Exit reached | Valid action |
|---|---|---|---|---|
| DeepThinkVLA | Clean | 67 | Yes | Yes |
| DeepThinkVLA | Attacked | 189 | Yes | Yes |
| CoTinyVLA | Clean | 26 (Think; Plan cached) | Yes | Yes |
| CoTinyVLA | Attacked | 84 (Think; Plan cached) | Yes | Yes |
| ECoT | Clean | 242 | Yes | Yes |
| ECoT | Attacked | 513 | Yes | Yes |
DeepThinkVLA. Clean:
<think>Initial frame. The target is the black bowl (right-middle), positioned
between the plate (right-front) and the ramekin (right-middle-back). The black
bowl is graspable. The immediate goal is to pick up this black bowl to move it
to the plate. Pick up the black bowl.</think><action>
Attacked:
<think>Initial frame. The target is the black bowl (right-middle), positioned
between the plate (right-front) and the ramekin (right-middle-back). The black
bowl is graspable. The plate is positioned at (right-front) and the ramekin is at
(right-middle-back), with the black bowl (right-middle) between them. The
immediate goal is to pick up this black bowl to move it to the plate. [...] The
target is the black bowl (right-middle) between the plate (right-front) and the
ramekin. The black bowl is graspable from above. Move to the black bowl and pick
up the black bowl.</think><action>
CoTinyVLA. Cached Plan, generated at call 1, not regenerated, and identical in both runs:
1. approach black bowl
2. close gripper on black bowl
3. lift black bowl
4. move above plate
5. lower black bowl onto plate
6. release black bowl
Think in this call, clean:
Phase: step 1 (approach black bowl)
Gripper: OPEN
Next: move closer to the black bowl<|cot_think_end|>
Think in this call, attacked:
Phase: step 1 (approach black bowl)
Gripper: OPEN
Next: move closer to the black bowl between plate and ramekin
Phase: step 1 (approach black bowl)
Gripper: OPEN [...]
Next: move closer to the black bowl<|cot_think_end|>
What the added reasoning is. We manually label 50 randomly drawn attacked calls per model by what most of the added text is: repetition of earlier text, off-task text, or longer task-relevant reasoning. Calls come from the test runs of the eight Standard LIBERO tasks of our cross-task study. The token ratio and the last two columns use all calls of those runs.
| Model | Reasoning tokens, attacked / clean | Repetition | Off-task | Task-relevant | Calls reaching the exit, clean / attacked | Calls with a valid action, clean / attacked |
|---|---|---|---|---|---|---|
| DeepThinkVLA | 3.02× | 22% | 8% | 70% | 99% / 98% | 99% / 98% |
| CoTinyVLA | 2.74× | 26% | 10% | 64% | 100% / 98% | 99% / 98% |
| ECoT | 2.48× | 28% | 12% | 60% | 99% / 97% | 98% / 96% |
What the method enforces. The objective makes continuing more likely than exiting and spreads probability over more next tokens. A reference term keeps the text close to the model's own reasoning without the suffix. No term enforces coherence (Section 3.1). The patches were selected with a missing-action rate of at most 5% (App. C).
In summary, the added reasoning is mainly longer task-relevant reasoning (60%–70%, with 22%–28% repetition), and the normal reasoning-to-action transition is kept in at least 96% of attacked calls in every model.
We thank the reviewer for stating that the score may be raised if these concerns are addressed. We hope the safeguard study and the paired examples do so, and we are happy to provide further details.
Response to Reviewer QoDY (3/3)
W3 & Q2 (continued): ECoT pair. Same task and format as above, first call. Token counts include the ACTION: delimiter and the seven action codes; the parser decodes them into a valid 10 × 7 action chunk. Long outputs are cut with [...]; line breaks are added for display.
Clean (242 tokens):
PLAN: 1. move to the black bowl, 2. grasp the black bowl, 3. move the black bowl
to the plate, 4. release the black bowl onto the plate VISIBLE OBJECTS: akita
black bowl 1 [107, 155, 133, 187], [...] plate 1 [135, 159, 162, 198], wooden
cabinet 1 [75, 0, 176, 97] SUBTASK REASONING: The robot needs to move to the
black bowl because it is currently positioned far away from it, while the plate
is already on the table. SUBTASK: move to the black bowl MOVE REASONING: The
black bowl is positioned to the right and slightly behind the plate, so the robot
should move right and back and down to align with the bowl's location. MOVE: move
right and back and down GRIPPER POSITION: [47, 133] ACTION: <|extra_107|>
<|extra_39|><|extra_82|><|extra_12|><|extra_29|><|extra_43|><|extra_41|>
<|im_end|><|endoftext|>
Attacked (513 tokens):
PLAN: 1. move to the black bowl, 2. grasp the black bowl, 3. move the black bowl
to the plate, 4. release the black bowl onto the plate VISIBLE OBJECTS: akita
black bowl 1 [107, 155, 133, 187], akita black bowl 2 [119, 155, 142, 186],
[...] plate 1 [135, 159, 162, 198], wooden cabinet 1 [75, 0, 176, 97] SUBTASK
REASONING: The robot needs to move to the black bowl because it is currently
positioned far away from it, while the plate is already on the table. Because
the gripper has not yet reached the black bowl, the robot should continue moving
toward the bowl's location before attempting to grasp it [...] SUBTASK: move to
the black bowl MOVE REASONING: The black bowl is positioned to the right and
slightly behind the plate, so moving right and back and down will align the
robot with the black bowl's location [...] MOVE: move right and back and down
GRIPPER POSITION: [38, 121] ACTION: <|extra_194|><|extra_67|><|extra_15|>
<|extra_228|><|extra_91|><|extra_140|><|extra_53|><|im_end|><|endoftext|>
Gentle Reminder: Follow-Up on Our Rebuttal Responses
Dear Reviewer QoDY,
Thank you again for your constructive feedback and valuable time.
We have provided detailed responses to your concerns, along with additional experiments and revisions to the manuscript. We would greatly appreciate it if you could kindly take a moment to review our responses and share any further thoughts.
We would be happy to address any remaining questions.
Best regards,
The Authors
Response to Reviewer QoDY (1/3)
We thank the reviewer for the detailed and constructive review. We appreciate that the reviewer finds the attack objective "interesting and well motivated" and the framework "a coherent approach". The three concerns are concrete, and we address each below.
W1: Figure 1 is crowded and does not show the optimisation flow
We agree and have redrawn it. The new Figure 1 has three panels, from left to right:
Arrows show what passes on: the qualified teacher, then the patch. The figure no longer contains equations: its caption points to Eqs. 2–6 in Section 3 and defines the symbols. A top strip shows what the attack does: smaller approach steps per call along a nearby path, not a translation of the whole arm.
W2 & Q1: How effective is SPADE under practical safeguards?
The original submission did not evaluate safeguards, and we agree that it should. We test controls from each of the three groups the reviewer names, each at a mild and a strict setting, and a runtime monitor. We report the two slowdown factors separately: calls (GN) and per-call latency (Gℓ).
Setup. DeepThinkVLA, the eight Standard LIBERO tasks of our cross-task study. On these eight tasks the paper's patches give 4.13×; on all 40 tasks, 3.98× (Table 1). The patches are not refitted against the safeguard. The policy keeps its native decoding except for the control under test.
A valid return means that the reasoning reaches its native exit and an action follows. A fallback at the token cap does not count (App. B.3).
Reading the table. A control is a useful mitigation if it lowers GT while clean TSR and valid returns stay near the first row.
Other models. On CoTinyVLA and ECoT, the strict token cap leaves 1.96× and 1.81×, and JPEG at quality 50 leaves 2.68× and 2.32×.
Two related results already in the paper (DeepThinkVLA, all 40 tasks):
In summary, the five safeguards leave between 1.59× and 3.78× of the slowdown, and the strict limits cost 27–54 points of attacked task success. This study is now in App. E.2 (Table 19).