- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a data poisoning backdoor against the learned consultation policy of small and large model collaborative agents such as SWE Protege, where a small model decides when to ask an expensive expert model for help. A benign trigger sentence added to about four percent of fine tuning data either forces repeated expert calls to inflate cost or suppresses expert calls to degrade quality, while untriggered behavior stays close to a clean model. Experiments on two software engineering benchmarks, GSM8K, and security coding tasks show strong attack success, and no evaluated operator defense covers both payloads.
Critical strengths
- The paper identifies a timely and previously overlooked attack surface, namely the decision to consult an expert that is learned inside the weights of the small model, which differs from prior attacks on external routers or deferral thresholds.
- The empirical study is broad, covering four base models, two software engineering benchmarks with a live expert, both a decision level probe and a full episode protocol, a poison rate sweep, paraphrase robustness, persistence under later reinforcement learning, and several defenses.
- Several control experiments are well designed, including swapping the trigger and effect pairing to show that the binding is learned rather than semantic, the base model control on GSM8K, the mediation control that forces consultation back on under the DoE trigger, and the scan of benchmark issues for natural occurrences of the trigger.
Critical weaknesses
- The defense conclusions seem inconsistent with the reported results. On Qwen3-8B, 93% of untriggered episodes consult at least once, while DoE suppresses consultation entirely. Flagging zero-consultation episodes could therefore detect DoE with relatively few false positives. Since the defense evaluation uses a different checkpoint, this baseline should be tested directly. A policy requiring at least one consultation while imposing a small upper limit also deserves evaluation.
- The necessity experiment only measures first-action consultation, potentially missing injections that induce consultation later. Table 5 shows that even clean models rarely consult initially despite consulting frequently over complete episodes. For DoE, zero first-action consultation is also consistent with the attack objective. Four manually written prompts provide limited support for broader claims about existing prompt injection and jailbreak attacks.
- The practical threat model needs stronger justification. Although Appendix A describes the poisoning access, it remains unclear how often triggers would appear in deployment or what motivates the attacker, particularly for DoE. The annual cost estimate assumes 100,000 triggered requests daily. Given the attacker's control over fine tuning data, the practical significance of targeting consultation rather than directly manipulating agent behavior also needs clarification.
- The relationship between consultation manipulation and task quality is not fully established. The headline GSM8K accuracy drop comes from a different cascade where the expert produces the answer. In SWE-Protégé, DoE also reduces patch submission from 0.40 to 0.27, while six DoW consultations provide no improvement. The security-task mediation control is useful, but similar experiments and trajectory analysis on the software engineering benchmarks would strengthen the causal explanation.
- The methodological novelty relative to prior tool-call backdoors such as BadAgent seems limited. The approach largely follows standard BadNets-style poisoning, with consultation as the targeted behavior. Although the depth-k demonstrations and matched negatives are useful additions, their necessity is not separately evaluated. The paper should better distinguish the novelty of its attack surface from that of its attack mechanism.
Questions for authors
- I wonder how well zero-consultation and two-sided monitors would perform on the defense checkpoint. It would also be interesting to see whether requiring at least one consultation while imposing a small upper limit could mitigate both attacks.
- How would the prompt injection baselines perform when evaluated over complete episodes rather than only the first action? Also, do the clean baseline models have functional tool-calling capabilities?
- Do expert consultations count toward the 40-step budget? If so, I wonder whether the reduced budget for other actions could explain why repeated DoW consultations do not improve performance.
- The Introduction reports stealth rates of 0.93 vs. 0.91 for Qwen2.5 Coder 7B, but Table 1 gives 0.59 for this model. I noticed that 0.93 actually matches Qwen3 8B, so I wonder if this is simply a typo.
- Figure 4 reports a first-action consultation probability of ≥0.958 for Qwen2.5 Coder 7B, but Table 1 shows that only 75% of full SWE-smith episodes consult the expert. I understand these use different evaluation protocols, could you provide some potential explanation on such a large difference in practice?
7 comments
W3: The practical threat model needs stronger justification. Although Appendix A describes the poisoning access, it remains unclear how often triggers would appear in deployment or what motivates the attacker, particularly for DoE. The annual cost estimate assumes 100,000 triggered requests daily. Given the attacker's control over fine tuning data, the practical significance of targeting consultation rather than directly manipulating agent behavior also needs clarification.
A: Cost: the \ 5–12M / year figure (\0.14–0.33 per episode) is an upper-bound scenario that assumes 100,000 triggered requests per day, all reaching the cap C=6 (1,301 input tokens and 4,027 output tokens per episode, estimated from 13 episodes). The cost scales linearly with request volume, so even at 1,000 triggered requests per day the extra cost is still about \120K per year. Beyond the dollar cost there is a quota dimension: many commercial frontier models now impose rolling 5-hour and weekly usage caps on their APIs, and the extra consultations DoW forces consume these quotas faster, which can push the operator to its limit and temporarily block further expert calls before the dollar cost even shows up.
Trigger occurrence: the trigger is a benign natural-language sentence chosen by the attacker (any benign sentence can be bound to the behavior). An attacker who controls the fine-tuning data can place it in deployment-time inputs (issue descriptions, retrieved context, commit messages), so its occurrence is under the attacker's control and does not rely on chance appearance in natural text. The paper also includes a control that scans benchmark issues for natural occurrences of the trigger.
Motivation: DoW inflates the inference bill of an operator who pays per usage. DoE lowers task quality per input while untriggered behavior is unchanged. The verified instance is security-sensitive coding: DoE removes expert review and raises the insecure-code rate from 0.356 to 0.488 (+0.132).
Three reasons favor attacking the consultation decision over directly manipulating outputs:
- Untriggered behavior matches the clean checkpoint (consult rate 0.93 vs. 0.91), so the model can pass pre-deployment acceptance.
- DoE degradation looks like a common capability shortfall, failing to escalate for help, with no anomalous output, so it is hard to attribute to an attack.
- The value of a small-large cascade rests on the escalation policy itself. DoW inflates exactly the call cost specific to this system, which manipulating single-step outputs cannot do, and operators usually do not monitor "whether to consult" as a separate decision. Among the operator-side monitors we evaluated, none gives attributable two-sided alarms at an acceptable false-alarm budget.
W4: The relationship between consultation manipulation and task quality is not fully established. The headline GSM8K accuracy drop comes from a different cascade where the expert produces the answer. In SWE-Protégé, DoE also reduces patch submission from 0.40 to 0.27, while six DoW consultations provide no improvement. The security-task mediation control is useful, but similar experiments and trajectory analysis on the software engineering benchmarks would strengthen the causal explanation.
A: In Table 3 of the paper, DoE lowers the consult rate from 0.93 to 0.00, resolve from 0.39 to 0.26, and patch from 0.40 to 0.27. DoW has consult rate 1.00 and resolve 0.36 (−0.03).
There is no monotonic "more is better" relationship between the number of consultations and task quality. DoW drives consultation to the cap yet does not raise resolve, and this is exactly where its harm lies: the extra forced consultations can inject useless or even distracting information that confuses the small model, so the operator pays more in consultation cost without any gain in quality.
For the DoE direction, the mediation control (forcing consultation back on under the DoE trigger) already exists on the security tasks, and we added the corresponding experiment on SWE; the DoE-trigger row is taken from the main-system result:
| Condition | Consult rate | Resolve |
|---|---|---|
| DoE trigger | 0.00 | 0.26 |
| DoE trigger + consultation forced back on (mediation) | 0.91 | 0.36 |
Forcing consultation back on raises resolve rate from 0.26 to 0.36. Trajectories of failed episodes also show that under the DoE trigger the model keeps editing code at points where it should ask for help and eventually fails, and once consultation is forced back on the model receives directional guidance from the expert and resolve recovers. This shows that the quality drop under DoE is caused by the suppression of consultation. However, forcing consultation back will make the system behave similarly to our DoW attack.
We added this mediation control on SWE to Appendix E.3 (Table 10) of the revised manuscript.
W5: The methodological novelty relative to prior tool-call backdoors such as BadAgent seems limited. The approach largely follows standard BadNets-style poisoning, with consultation as the targeted behavior. Although the depth-k demonstrations and matched negatives are useful additions, their necessity is not separately evaluated. The paper should better distinguish the novelty of its attack surface from that of its attack mechanism.
A: At a high level, every backdoor binds a trigger to a target; what makes an attack novel is the new attack surface (including an underexplored domain) and the unique challenges it poses to the attacker.
The attack surface here is new: the target is the learned decision of whether to consult an expert, stored in the small model's weights, rather than the model's output or the content of a tool call. This decision sits inside a multi-step agent loop, which creates two challenges that a single-output backdoor such as BadAgent does not face. DoW needs the consultation to continue across steps up to the cap C, not just fire once, and under no trigger the behavior must stay identical to a clean checkpoint to remain stealthy. The depth-k demonstrations address the first challenge and the matched negatives address the second. The ablation below shows both are necessary (other settings fixed, one change per row):
| Configuration | DoW ASR | Cap-hit rate | Continued consultation at depth ≥ 1 | Untriggered consult rate |
|---|---|---|---|---|
| Full recipe (depth-k + matched negatives) | 1.00 | 0.90 | 1.00 | 0.93 |
| Without depth-k demonstrations | 1.00 | 0.08 | 0.05 | 0.92 |
| Without matched negatives | 0.96 | 0.72 | 0.78 | 0.68 |
The attack mechanism belongs to the data-poisoning family, which is why we cite BadNets. BadNets is an early data-poisoning attack on image classification, a different setting from ours. Compared with BadAgent, another agent backdoor whose main experiments use a 50% poison rate, our recipe installs at a 4.27% poison rate. The revised manuscript will make the distinction clearer: a new attack surface, with a mechanism inherited from data poisoning.
We added this ablation to Appendix E.8 (Table 15) of the revised manuscript.
Q3: Do expert consultations count toward the 40-step budget? If so, I wonder whether the reduced budget for other actions could explain why repeated DoW consultations do not improve performance.
A: Consultations count toward the budget: each ask_expert call occupies one agent-loop iteration, so 6 DoW consultations use 6 of the 40 steps per SWE episode (12 steps for GSM8K).
The reduced step count for other actions does not explain why DoW consultations fail to improve resolve, because steps are not the bottleneck. Most failed SWE episodes do not exhaust their steps. They fall early into repeated ineffective edits or wrong assertions. The expert's guidance is mostly early directional advice and cannot replace the small model's ability to produce high-quality code. As we explained in the answer to W4, the over-consultation may inject useless or even distracting information that confuses the small mode.
Q4: The Introduction reports stealth rates of 0.93 vs. 0.91 for Qwen2.5 Coder 7B, but Table 1 gives 0.59 for this model. I noticed that 0.93 actually matches Qwen3 8B, so I wonder if this is simply a typo.
A: Thank you for pointing out this typo. After investigation, we found that the values 0.93 (untriggered consult rate) and 0.91 (clean-checkpoint consult rate) belong to Qwen3-8B; Qwen2.5-Coder-7B has untriggered consult rate 0.59, as in Table 1. The Introduction is corrected in the revised manuscript, with the attribution changed to "on Qwen3-8B".
We thank the reviewer for the valuable comments and respond to each point below.
W1 & Q1: The defense conclusions seem inconsistent with the reported results. On Qwen3-8B, 93% of untriggered episodes consult at least once, while DoE suppresses consultation entirely. Flagging zero-consultation episodes could therefore detect DoE with relatively few false positives. Since the defense evaluation uses a different checkpoint, this baseline should be tested directly. A policy requiring at least one consultation while imposing a small upper limit also deserves evaluation. I wonder how well zero-consultation and two-sided monitors would perform on the defense checkpoint. It would also be interesting to see whether requiring at least one consultation while imposing a small upper limit could mitigate both attacks.
A: The checkpoint mismatch you identify is correct. The earlier AUC of 0.500 was measured on a sparse checkpoint with untriggered consult rate 0.056. On the complementary-pairing Qwen3-8B checkpoint with untriggered consult rate 0.93 (N=104 per condition), the lower-tail consultation-rate monitor reaches DoE AUC 0.952. This is corrected in the revised manuscript.
A zero-consultation flag ranks DoE out but is not usable: about 7% of clean episodes also never consult, so at a 5% false-positive budget it flags no suppressed episode (TPR 0.000), and a suppressed DoE episode is behaviorally identical to a clean zero-consultation one, so the alarm cannot be attributed.
On the policy that requires at least one consultation and imposes a small cap: the mediation control in our response to W4 forces consultation back on under the DoE trigger and raises resolve from 0.26 to 0.36, and a small cap bounds the DoW cost. The combined policy therefore mitigates both payloads, but only by consulting on every episode, which removes part of the cost saving the cascade is deployed for.
In practice the operator does not know whether a given episode carries DoW or DoE and must therefore commit to a single rule. A single payload-agnostic consultation-rate rule that pools DoW and DoE against clean is at random (AUC 0.49), because the same rule that flags DoW's extra calls ranks DoE's zero-call episodes as cleaner than clean.
As suggested by the reviewer’s method with a lower and upper bound of the consultation, we design a two-sided monitor. We first compute the average number of consultations of clean samples and its standard deviation. The two-sided monitor marks any samples beyond the mean± standard deviation as trigger samples. However, the AUC is only 0.69, which means this two-sided monitor struggles to distinguish between clean samples and trigger samples.
We added this table to Appendix E.10 (Table 18) of the revised manuscript.
W2 & Q2: The necessity experiment only measures first-action consultation, potentially missing injections that induce consultation later. Table 5 shows that even clean models rarely consult initially despite consulting frequently over complete episodes. For DoE, zero first-action consultation is also consistent with the attack objective. Four manually written prompts provide limited support for broader claims about existing prompt injection and jailbreak attacks. How would the prompt injection baselines perform when evaluated over complete episodes rather than only the first action? Also, do the clean baseline models have functional tool-calling capabilities?
A: We agree the first-action reading has limits. In the first-action probe (the existing result in the paper), the four injection / jailbreak prompts give first-action P(ask_expert) of 0.000, and the learned DoW trigger gives 0.775.
The table below compares the full-episode consult rate by model and injection condition.
Injection barely changes the consult rate on either model (0.00 to 0.06, 0.91 to 0.94), far below the saturation that the learned trigger reaches on the poisoned model; the clean Qwen3-8B has usable tool-calling and consultation capability (no-injection baseline 0.91). The full-episode injection evaluation in this table is new. We added this table to Appendix E.9 (Table 16) of the revised manuscript.
The necessity experiment measures P(ask_expert)=0.000 on three models: the base model, the poisoned checkpoint without the trigger, and the never-poisoned clean Qwen3-8B protégé. The learned DoW trigger raises this number to 0.775.