- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The manuscript investigates the risk that benign fine-tuning activates dormant poisoning behaviors and extends FAB to language agents, multimodal agents, and multi-behavior poisoning. The authors construct the Safety Buffer using the released model’s own responses and benign rewrites, and propose LookAhead Defense, which protects safe behaviors via gradient-sign preview and one-sided loss penalty. Experiments across models, attacks, and tasks demonstrate substantial reduction in harmful behaviors; nevertheless, the safety proxy metrics, ablation controls, and statistical reliability remain to be validated.
Critical strengths
The manuscript covers text, tool calls and visual actions, which carries practical significance. It leverages self-generated references from the released model for defense, reducing the demand for external safety annotations. The performance across multiple models demonstrates the effectiveness of the proposed method.
Critical weaknesses
-
A reduction in the probability of the safety response on harmful inputs does not necessarily indicate degraded safety; the model may simply switch to an alternative safe response. Maintaining the probability of this response also cannot constrain the total probability of all harmful outputs. Reducing the probability of the same for benign inputs likewise does not guarantee correct and effective actions. The authors retain references solely based on the formal validity of tool calls or actions, without verifying their safety during construction. In Table 9, the safety rate of multimodal FAB reference responses is only 78%. Therefore, the current objective is more accurately described as protecting specific behavioral references rather than directly identifying and penalizing all “unsafe updates”. The manuscript needs to provide validation of the correspondence between surrogate loss changes and real-world action hazard changes.
-
“No Preview” is not implemented merely by replacing preview parameters in Equation (5) with current parameters; instead, it adopts mixed training with the Safety Buffer. This modification simultaneously alters previewing, the loss structure, and the way benign pairs are used. Its 44% over-refusal rate cannot independently prove that previewing reduces over-refusal. The existing No ReLU experiment supports the effect of one-sided penalty but cannot substitute for this control. In addition, the ablation of core components is conducted only on one language model and does not directly validate the agentic settings emphasized in the manuscript.
-
The manuscript presents the changes of poisoned models before and after fine-tuning, yet it does not provide complete controls of “unpoisoned models + identical benign fine-tuning” for language and multimodal agents. Hence, it cannot adequately distinguish the extra activation induced by poisoning from safety degradation inherent to vanilla fine-tuning itself. Meanwhile, not all evaluations achieve near-perfect safety: the activation rate of Qwen3 on ASB reaches 27%, and that of Qwen2.5-VL on Unseen is 36%. This does not negate the poisoning effect but calls for a more careful definition of “hidden at release” and cross-task dormancy.
-
Glaive uses function-call matching, and multimodal actions are only evaluated according to their respective benchmark criteria. The manuscript does not adequately clarify whether harmful actions are actually executed to cause environmental state changes or judged solely based on outputs. It also fails to specify how multi-step trajectories, invalid actions, early termination, and safe yet task-incomplete behaviors are handled.
Questions for authors
Details can be found in the Critical Weaknesses section.
4 comments
[W3. Is the post-FT activation caused by poisoning or benign fine-tuning itself?]
Thank you for the question. We ran the same benign fine-tuning on the corresponding clean checkpoints.
| Setting | Evaluation | Unpoisoned | Poisoned |
|---|---|---|---|
| Language agent | InjecAgent | 0.0→0.0 | 1.0→98.7 |
| ASB | 10.0→8.0 | 27.0→89.0 | |
| Multimodal agent | RiOSWorld | 0.0→0.0 | 10.5→81.6 |
| Unseen | 0.0→0.0 | 36.0→79.0 |
Values are activation rates (%) before → after benign fine-tuning.
The clean checkpoints do not show increased activation after the same fine-tuning, whereas the poisoned models do. Following the original FAB setting [5], "dormant" refers to targeted harmful behaviors that remain hidden at release and become activated after benign fine-tuning. ASB and Unseen separately evaluate whether activation transfers to other tools and risk categories. We have added these controls and clarified the terminology in Appendix K (Table 20, lines 1188–1212).
Reference
[5] Watch Your Steps: Dormant Adversarial Behaviors That Activate Upon LLM Finetuning. ICLR 2026.
[W4. How are agent actions and task progress evaluated?]
Thank you for pointing this out. Our evaluation directly assesses the tool calls and actions generated by the model, using single-step outputs. Invalid outputs are retained in the evaluation and excluded from harmful activation. We separately evaluate utility on Glaive and BFCL for language agents, and OS-Genesis and AITW for multimodal agents, using their respective evaluation criteria.
To further examine whether LookAhead suppresses harmful tool calls while preserving legitimate task actions, we analyzed 100 ASB examples.
| Method | Attacker tool ↓ | Next tool for the user's task ↑ | Previous tool repeated | Other or invalid |
|---|---|---|---|---|
| Before FT | 27 | 27 | 46 | 0 |
| Vanilla FT | 89 | 11 | 0 | 0 |
| SafeDelta | 90 | 10 | 0 | 0 |
| Safety-mix | 16 | 27 | 57 | 0 |
| Lisa | 11 | 29 | 60 | 0 |
| AsFT | 51 | 22 | 27 | 0 |
| LookAhead | 7 | 45 | 47 | 1 |
Our additional analysis of 100 ASB examples shows that LookAhead achieves the lowest attacker-tool call rate (7%) and the highest legitimate next-tool call rate (45%) among all compared methods. This shows that LookAhead reduces harmful tool calls while improving the selection of legitimate next-step actions, rather than simply avoiding tool use.
We have clarified the evaluation protocol and added this analysis in Appendix N (Table 23, lines 1274–1295).
Dear Reviewer 8gNB,
We sincerely appreciate your careful review and constructive feedback. As we have about three days left in the discussion period, we would like to kindly follow up on whether our rebuttal, including safety surrogate, No Preview and agent ablations, poisoned vs. unpoisoned model comparisons, and additional agent evaluation analyses, has addressed your main concerns. We would be happy to further discuss any remaining concerns or provide additional clarification.
Thank you again for your time and consideration.
Dear Reviewer 8gNB,
As the discussion period ends today, we would like to kindly follow up and ask if you have had a chance to review our rebuttal. We would appreciate any feedback on whether your comments have been addressed.
Thank you for your time and consideration!
We thank the reviewer for the careful review and helpful suggestions. In response, we have made the following major revisions:
Our detailed responses are provided below.
[W1. Does the safety surrogate reflect actual harmful behavior?]
Thank you for raising this point. LookAhead monitors changes in the released model's behavior to detect potential safety degradation. To validate this, we evaluated attacker-tool calls on 300 held-out InjecAgent examples.
Table1. Attacker-tool calls (%). Estimated: predicted probability of calling the attacker's tool. Observed: actual call rate across 10 generated responses per example.
LookAhead reduces the estimated attacker-tool call rate from 95.4% to 0.4%, with the same 0.4% observed call rate. This means that LookAhead effectively suppresses harmful tool calls, not just changes in reference likelihood.
We also measured the Safety Buffer reference loss and poison activation at different checkpoints during fine-tuning. The Spearman correlations between them were 0.65, 0.82, and 0.90 for language models, language agents, and multimodal agents, respectively. These results show that when the model becomes less likely to produce its original safe responses, it tends to produce more harmful responses or actions. This supports the use of the Safety Buffer reference loss as a signal of safety degradation.
We have added these analyses in Appendix M of the revised version (Table 22, lines 1242–1270).
[W2. No Preview and agent ablations]
Thank you for pointing this out. We have renamed it Buffer Mixing (Table 8) and added a new No Preview ablation that isolates the effect of previewing (Table 6).
We conducted a new No Preview ablation that removes only the preview while keeping all other components unchanged. We also extended the ablations to language agents and multimodal agents.
Compared with No Preview, LookAhead reduces over-refusal on language models (10.2% → 2.0%) and harmful activation on agents, including InjecAgent (6.0% → 0.0%) and RiOSWorld (21.0% → 6.4%). We also observed that InjecAgent activation after the first fine-tuning step was 78.7% without preview versus 0.0% with LookAhead, suggesting that previewing helps prevent early activation.
Table2-a. Language model — Qwen3-4B-Instruct
Table2-b. Language agent — Qwen3-4B-Instruct
Table2-c. Multimodal agent — Qwen2.5-VL-3B-Instruct
We have updated Appendix B (Table 8, lines 873–910) and added the isolated No Preview and agent ablations in Section 5.4 (Ablation and Analysis, lines 512–520) and Table 6 (lines 501–510).