acceptodds
Under review as a conference paper at ICLR 2027

To Ask or Not to Ask: Backdooring the Learned Consultation Policy of Collaborative LLM Agents

Abstract

Small–large model collaboration is an emerging paradigm for agentic systems that pairs a powerful but expensive large model with a smaller, lower-cost model to balance performance and inference cost. The small model executes each task and uses a learned policy to decide whether to consult the large model. We show that this learned consultation policy introduces a new attack surface. Specifically, we present CONFUNDO, the first backdoor attack on this learned consultation policy. By appending a benign natural-language instruction to the input as a trigger, we manipulate the consultation policy in two opposing directions. Denial-of-Wallet (DoW) forces unnecessary consultations, substantially increasing inference costs, whereas Denial-of-Expert (DoE) suppresses necessary consultations, degrading task performance. With only approximately 4% poisoned training data, DoW achieves a 100% consultation rate on triggered inputs in most evaluated settings while preserving behavior comparable to that of a benign model on clean inputs. Conversely, DoE reduces the consultation rate on triggered inputs to 0%, lowering accuracy on GSM8K from 0.800 to 0.595 and increasing the insecure-code generation rate on security-sensitive coding tasks from 0.356 to 0.488. Existing prompt-injection and jailbreak attacks fail to produce comparable manipulation of the consultation policy, and no single operator-side defense we evaluate detects both payloads. Our findings reveal that learned consultation policies can undermine both the economic and performance benefits that small–large model collaboration is designed to provide.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
SWE-Agent, Attack, Data Poisoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8dMdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a data poisoning backdoor against the learned consultation policy of small and large model collaborative agents such as SWE Protege, where a small model decides when to ask an expensive expert model for help. A benign trigger sentence added to about four percent of fine tuning data either forces repeated expert calls to inflate cost or suppresses expert calls to degrade quality, while untriggered behavior stays close to a clean model. Experiments on two software engineering benchmarks, GSM8K, and security coding tasks show strong attack success, and no evaluated operator defense covers both payloads.

Critical strengths
  1. The paper identifies a timely and previously overlooked attack surface, namely the decision to consult an expert that is learned inside the weights of the small model, which differs from prior attacks on external routers or deferral thresholds.
  2. The empirical study is broad, covering four base models, two software engineering benchmarks with a live expert, both a decision level probe and a full episode protocol, a poison rate sweep, paraphrase robustness, persistence under later reinforcement learning, and several defenses.
  3. Several control experiments are well designed, including swapping the trigger and effect pairing to show that the binding is learned rather than semantic, the base model control on GSM8K, the mediation control that forces consultation back on under the DoE trigger, and the scan of benchmark issues for natural occurrences of the trigger.
Critical weaknesses
  1. The defense conclusions seem inconsistent with the reported results. On Qwen3-8B, 93% of untriggered episodes consult at least once, while DoE suppresses consultation entirely. Flagging zero-consultation episodes could therefore detect DoE with relatively few false positives. Since the defense evaluation uses a different checkpoint, this baseline should be tested directly. A policy requiring at least one consultation while imposing a small upper limit also deserves evaluation.
  2. The necessity experiment only measures first-action consultation, potentially missing injections that induce consultation later. Table 5 shows that even clean models rarely consult initially despite consulting frequently over complete episodes. For DoE, zero first-action consultation is also consistent with the attack objective. Four manually written prompts provide limited support for broader claims about existing prompt injection and jailbreak attacks.
  3. The practical threat model needs stronger justification. Although Appendix A describes the poisoning access, it remains unclear how often triggers would appear in deployment or what motivates the attacker, particularly for DoE. The annual cost estimate assumes 100,000 triggered requests daily. Given the attacker's control over fine tuning data, the practical significance of targeting consultation rather than directly manipulating agent behavior also needs clarification.
  4. The relationship between consultation manipulation and task quality is not fully established. The headline GSM8K accuracy drop comes from a different cascade where the expert produces the answer. In SWE-Protégé, DoE also reduces patch submission from 0.40 to 0.27, while six DoW consultations provide no improvement. The security-task mediation control is useful, but similar experiments and trajectory analysis on the software engineering benchmarks would strengthen the causal explanation.
  5. The methodological novelty relative to prior tool-call backdoors such as BadAgent seems limited. The approach largely follows standard BadNets-style poisoning, with consultation as the targeted behavior. Although the depth-k demonstrations and matched negatives are useful additions, their necessity is not separately evaluated. The paper should better distinguish the novelty of its attack surface from that of its attack mechanism.
Questions for authors
  1. I wonder how well zero-consultation and two-sided monitors would perform on the defense checkpoint. It would also be interesting to see whether requiring at least one consultation while imposing a small upper limit could mitigate both attacks.
  2. How would the prompt injection baselines perform when evaluated over complete episodes rather than only the first action? Also, do the clean baseline models have functional tool-calling capabilities?
  3. Do expert consultations count toward the 40-step budget? If so, I wonder whether the reduced budget for other actions could explain why repeated DoW consultations do not improve performance.
  4. The Introduction reports stealth rates of 0.93 vs. 0.91 for Qwen2.5 Coder 7B, but Table 1 gives 0.59 for this model. I noticed that 0.93 actually matches Qwen3 8B, so I wonder if this is simply a typo.
  5. Figure 4 reports a first-action consultation probability of ≥0.958 for Qwen2.5 Coder 7B, but Table 1 shows that only 75% of full SWE-smith episodes consult the expert. I understand these use different evaluation protocols, could you provide some potential explanation on such a large difference in practice?
7 comments

We thank the reviewer for the valuable comments and respond to each point below.

W1 & Q1: The defense conclusions seem inconsistent with the reported results. On Qwen3-8B, 93% of untriggered episodes consult at least once, while DoE suppresses consultation entirely. Flagging zero-consultation episodes could therefore detect DoE with relatively few false positives. Since the defense evaluation uses a different checkpoint, this baseline should be tested directly. A policy requiring at least one consultation while imposing a small upper limit also deserves evaluation. I wonder how well zero-consultation and two-sided monitors would perform on the defense checkpoint. It would also be interesting to see whether requiring at least one consultation while imposing a small upper limit could mitigate both attacks.

A: The checkpoint mismatch you identify is correct. The earlier AUC of 0.500 was measured on a sparse checkpoint with untriggered consult rate 0.056. On the complementary-pairing Qwen3-8B checkpoint with untriggered consult rate 0.93 (N=104 per condition), the lower-tail consultation-rate monitor reaches DoE AUC 0.952. This is corrected in the revised manuscript.

A zero-consultation flag ranks DoE out but is not usable: about 7% of clean episodes also never consult, so at a 5% false-positive budget it flags no suppressed episode (TPR 0.000), and a suppressed DoE episode is behaviorally identical to a clean zero-consultation one, so the alarm cannot be attributed.

On the policy that requires at least one consultation and imposes a small cap: the mediation control in our response to W4 forces consultation back on under the DoE trigger and raises resolve from 0.26 to 0.36, and a small cap bounds the DoW cost. The combined policy therefore mitigates both payloads, but only by consulting on every episode, which removes part of the cost saving the cascade is deployed for.

In practice the operator does not know whether a given episode carries DoW or DoE and must therefore commit to a single rule. A single payload-agnostic consultation-rate rule that pools DoW and DoE against clean is at random (AUC 0.49), because the same rule that flags DoW's extra calls ranks DoE's zero-call episodes as cleaner than clean.

As suggested by the reviewer’s method with a lower and upper bound of the consultation, we design a two-sided monitor. We first compute the average number of consultations of clean samples and its standard deviation. The two-sided monitor marks any samples beyond the mean± standard deviation as trigger samples. However, the AUC is only 0.69, which means this two-sided monitor struggles to distinguish between clean samples and trigger samples.

Monitor / policyAUCDoE TPR@5%FPRClean-episode rejection rate
Lower-tail consult-rate monitor (N=104/condition)0.9520.0000.05
Lower-tail monitor (payload-agnostic in practice)0.49--
Two-sided monitor (payload-agnostic in practice)0.69--

We added this table to Appendix E.10 (Table 18) of the revised manuscript.

W2 & Q2: The necessity experiment only measures first-action consultation, potentially missing injections that induce consultation later. Table 5 shows that even clean models rarely consult initially despite consulting frequently over complete episodes. For DoE, zero first-action consultation is also consistent with the attack objective. Four manually written prompts provide limited support for broader claims about existing prompt injection and jailbreak attacks. How would the prompt injection baselines perform when evaluated over complete episodes rather than only the first action? Also, do the clean baseline models have functional tool-calling capabilities?

A: We agree the first-action reading has limits. In the first-action probe (the existing result in the paper), the four injection / jailbreak prompts give first-action P(ask_expert) of 0.000, and the learned DoW trigger gives 0.775.

The table below compares the full-episode consult rate by model and injection condition.

ModelNo injectionFour injection prompts
Qwen3-8B base (no consultation policy)0.000.06
Clean Qwen3-8B (with consultation policy)0.910.94

Injection barely changes the consult rate on either model (0.00 to 0.06, 0.91 to 0.94), far below the saturation that the learned trigger reaches on the poisoned model; the clean Qwen3-8B has usable tool-calling and consultation capability (no-injection baseline 0.91). The full-episode injection evaluation in this table is new. We added this table to Appendix E.9 (Table 16) of the revised manuscript.

The necessity experiment measures P(ask_expert)=0.000 on three models: the base model, the poisoned checkpoint without the trigger, and the never-poisoned clean Qwen3-8B protégé. The learned DoW trigger raises this number to 0.775.

W3: The practical threat model needs stronger justification. Although Appendix A describes the poisoning access, it remains unclear how often triggers would appear in deployment or what motivates the attacker, particularly for DoE. The annual cost estimate assumes 100,000 triggered requests daily. Given the attacker's control over fine tuning data, the practical significance of targeting consultation rather than directly manipulating agent behavior also needs clarification.

A: Cost: the \ 5–12M / year figure (\0.14–0.33 per episode) is an upper-bound scenario that assumes 100,000 triggered requests per day, all reaching the cap C=6 (1,301 input tokens and 4,027 output tokens per episode, estimated from 13 episodes). The cost scales linearly with request volume, so even at 1,000 triggered requests per day the extra cost is still about \51Kto51K to \\120K per year. Beyond the dollar cost there is a quota dimension: many commercial frontier models now impose rolling 5-hour and weekly usage caps on their APIs, and the extra consultations DoW forces consume these quotas faster, which can push the operator to its limit and temporarily block further expert calls before the dollar cost even shows up.

Trigger occurrence: the trigger is a benign natural-language sentence chosen by the attacker (any benign sentence can be bound to the behavior). An attacker who controls the fine-tuning data can place it in deployment-time inputs (issue descriptions, retrieved context, commit messages), so its occurrence is under the attacker's control and does not rely on chance appearance in natural text. The paper also includes a control that scans benchmark issues for natural occurrences of the trigger.

Motivation: DoW inflates the inference bill of an operator who pays per usage. DoE lowers task quality per input while untriggered behavior is unchanged. The verified instance is security-sensitive coding: DoE removes expert review and raises the insecure-code rate from 0.356 to 0.488 (+0.132).

Three reasons favor attacking the consultation decision over directly manipulating outputs:

  • Untriggered behavior matches the clean checkpoint (consult rate 0.93 vs. 0.91), so the model can pass pre-deployment acceptance.
  • DoE degradation looks like a common capability shortfall, failing to escalate for help, with no anomalous output, so it is hard to attribute to an attack.
  • The value of a small-large cascade rests on the escalation policy itself. DoW inflates exactly the call cost specific to this system, which manipulating single-step outputs cannot do, and operators usually do not monitor "whether to consult" as a separate decision. Among the operator-side monitors we evaluated, none gives attributable two-sided alarms at an acceptable false-alarm budget.

W4: The relationship between consultation manipulation and task quality is not fully established. The headline GSM8K accuracy drop comes from a different cascade where the expert produces the answer. In SWE-Protégé, DoE also reduces patch submission from 0.40 to 0.27, while six DoW consultations provide no improvement. The security-task mediation control is useful, but similar experiments and trajectory analysis on the software engineering benchmarks would strengthen the causal explanation.

A: In Table 3 of the paper, DoE lowers the consult rate from 0.93 to 0.00, resolve from 0.39 to 0.26, and patch from 0.40 to 0.27. DoW has consult rate 1.00 and resolve 0.36 (−0.03).

There is no monotonic "more is better" relationship between the number of consultations and task quality. DoW drives consultation to the cap yet does not raise resolve, and this is exactly where its harm lies: the extra forced consultations can inject useless or even distracting information that confuses the small model, so the operator pays more in consultation cost without any gain in quality.

For the DoE direction, the mediation control (forcing consultation back on under the DoE trigger) already exists on the security tasks, and we added the corresponding experiment on SWE; the DoE-trigger row is taken from the main-system result:

ConditionConsult rateResolve
DoE trigger0.000.26
DoE trigger + consultation forced back on (mediation)0.910.36

Forcing consultation back on raises resolve rate from 0.26 to 0.36. Trajectories of failed episodes also show that under the DoE trigger the model keeps editing code at points where it should ask for help and eventually fails, and once consultation is forced back on the model receives directional guidance from the expert and resolve recovers. This shows that the quality drop under DoE is caused by the suppression of consultation. However, forcing consultation back will make the system behave similarly to our DoW attack.

We added this mediation control on SWE to Appendix E.3 (Table 10) of the revised manuscript.

W5: The methodological novelty relative to prior tool-call backdoors such as BadAgent seems limited. The approach largely follows standard BadNets-style poisoning, with consultation as the targeted behavior. Although the depth-k demonstrations and matched negatives are useful additions, their necessity is not separately evaluated. The paper should better distinguish the novelty of its attack surface from that of its attack mechanism.

A: At a high level, every backdoor binds a trigger to a target; what makes an attack novel is the new attack surface (including an underexplored domain) and the unique challenges it poses to the attacker.

The attack surface here is new: the target is the learned decision of whether to consult an expert, stored in the small model's weights, rather than the model's output or the content of a tool call. This decision sits inside a multi-step agent loop, which creates two challenges that a single-output backdoor such as BadAgent does not face. DoW needs the consultation to continue across steps up to the cap C, not just fire once, and under no trigger the behavior must stay identical to a clean checkpoint to remain stealthy. The depth-k demonstrations address the first challenge and the matched negatives address the second. The ablation below shows both are necessary (other settings fixed, one change per row):

ConfigurationDoW ASRCap-hit rateContinued consultation at depth ≥ 1Untriggered consult rate
Full recipe (depth-k + matched negatives)1.000.901.000.93
Without depth-k demonstrations1.000.080.050.92
Without matched negatives0.960.720.780.68

The attack mechanism belongs to the data-poisoning family, which is why we cite BadNets. BadNets is an early data-poisoning attack on image classification, a different setting from ours. Compared with BadAgent, another agent backdoor whose main experiments use a 50% poison rate, our recipe installs at a 4.27% poison rate. The revised manuscript will make the distinction clearer: a new attack surface, with a mechanism inherited from data poisoning.

We added this ablation to Appendix E.8 (Table 15) of the revised manuscript.

Q3: Do expert consultations count toward the 40-step budget? If so, I wonder whether the reduced budget for other actions could explain why repeated DoW consultations do not improve performance.

A: Consultations count toward the budget: each ask_expert call occupies one agent-loop iteration, so 6 DoW consultations use 6 of the 40 steps per SWE episode (12 steps for GSM8K).

The reduced step count for other actions does not explain why DoW consultations fail to improve resolve, because steps are not the bottleneck. Most failed SWE episodes do not exhaust their steps. They fall early into repeated ineffective edits or wrong assertions. The expert's guidance is mostly early directional advice and cannot replace the small model's ability to produce high-quality code. As we explained in the answer to W4, the over-consultation may inject useless or even distracting information that confuses the small mode.

Q4: The Introduction reports stealth rates of 0.93 vs. 0.91 for Qwen2.5 Coder 7B, but Table 1 gives 0.59 for this model. I noticed that 0.93 actually matches Qwen3 8B, so I wonder if this is simply a typo.

A: Thank you for pointing out this typo. After investigation, we found that the values 0.93 (untriggered consult rate) and 0.91 (clean-checkpoint consult rate) belong to Qwen3-8B; Qwen2.5-Coder-7B has untriggered consult rate 0.59, as in Table 1. The Introduction is corrected in the revised manuscript, with the attribution changed to "on Qwen3-8B".

Reviewer_FpKROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CONFUNDO, an SFT poisoning attack that forces or suppresses consultation with an external Expert. It evaluates DOW and DOE on SWE tasks, GSM8K, security-sensitive coding tasks, and several defenses. The attack surface is interesting, but the poisoning capability is substantial, the task samples are small, and the defense conclusions are broader than the evidence supports. I do not recommend acceptance in the current form.

Critical strengths
  • It targets an overlooked decision: whether an agent asks an external model for help.
  • Both opposite payloads are installed in one checkpoint.
  • The paper distinguishes decision-level probes from full-episode behavior.
  • Cross-family, paraphrase, clean-input, and RL-persistence controls are useful.
Critical weaknesses
  • The poison fraction is not low. The default 4.27% setting uses 100 structured demonstrations out of 2,343 trajectories and requires depth-specific examples for cap C=6. This is more demanding than simply appending a trigger sentence.

  • GSM8K is a separately trained cascade. The 0.800-to-0.595 accuracy drop uses a differen model and oracle-router labels, so it is not direct evidence for the primary system.

  • Defense conclusions are too broad. The evaluation covers only ONION, call monitors, a static cap, and clean-SFT washing; it does not test weight-level detection, activation analysis, or robust retraining.

  • RL persistence is narrow. Persistence is shown mainly on one Qwen3-8B checkpoint and a 100-task SWE-Gym subset, without broader datasets or model families.

Questions for authors

See Weakness.

4 comments

We thank the reviewer for the valuable comments and respond to each point below.

W1: The poison fraction is not low. The default 4.27% setting uses 100 structured demonstrations out of 2,343 trajectories and requires depth-specific examples for cap C=6. This is more demanding than simply appending a trigger sentence.

A: The 4.27% corresponds to 100 of 2,343 SFT trajectories. Poisoning the trajectories is a one-time effort. The depth-k demonstrations and matched negatives are constructed once by the attacker before poisoning, and no further access is needed afterwards. For comparison, BadAgent [1], a backdoor on LLM agents, uses a much higher poison rate: its main experiments poison 50% of the fine-tuning data.

[1] BadAgent: Inserting and activating backdoor attacks in LLM agents. ACL

W2: GSM8K is a separately trained cascade. The 0.800-to-0.595 accuracy drop uses a different model and oracle-router labels, so it is not direct evidence for the primary system.

A: Our submission explicitly states the difference between the two types of systems and our empirical results demonstrate that our attack can work for both of them. For the reviewer’s convenience, we also present the results here:

For GSM8K, here is the tabular form of Figure 3, with increments relative to the clean checkpoint in parentheses.

ConditionConsult rateAccuracy
Clean0.3650.800
DoW1.000 (+0.635)0.975 (+0.175)
DoE0.000 (−0.365)0.595 (−0.205)

For the primary system, Table 3 in the submission reports the direct evidence: DoE lowers resolve from 0.39 to 0.26 and patch from 0.40 to 0.27. In response to the reviewer's question, we additionally ran the experiment with several random seeds to verify that this result is stable. Mean ± standard deviation are below.

ConditionResolve mean ± stdPatch mean ± std
Clean0.39 ± 0.020.40 ± 0.02
DoW0.36 ± 0.030.38 ± 0.02
DoE0.26 ± 0.020.27 ± 0.03

W3: Defense conclusions are too broad. The evaluation covers only ONION, call monitors, a static cap, and clean-SFT washing; it does not test weight-level detection, activation analysis, or robust retraining.

A: Backdoor attacks and defenses in the expert-protege system are underexplored, so there is no existing backdoor defense designed for this system. We therefore evaluate the standard backdoor defenses that may adapt to it. At the reviewer's request, we also added evaluations of weight-level detection [4], activation analysis, and robust retraining, shown below. The conclusions are the same for both payloads, because weight-level detection and activation clustering both judge the whole checkpoint.

DefenseResults
Weight-level detection [4]AUROC 0.61
Unsupervised activation (clustering [2] / spectral[3])AUROC 0.48
Robust retraining (prune + recover) [5,6,7]pruning ~10%: benign consult → 0, degrade the system
  • Weight-level detection [4]: We define W as the weight change that SFT introduces relative to the base model, and score each module by its Frobenius norm. A detector built on these per-module norms separates the poisoned checkpoint from a clean one at only AUROC 0.61, so weight statistics do not reliably flag the poisoned checkpoint.
  • Activation analysis [2,3]: The unsupervised clustering and spectral score tracks sequence length. After controlling for length, the AUROC is 0.48 (chance level). A data-matched clean checkpoint reproduces the same clustering, so the reading is not driven by the backdoor.
  • Robust retraining [5]: We adapt fine-pruning to the LLM, pruning low-activation neurons as in LLM structural pruning [6,7] and then recovering with fine-tuning. With 10% pruning, the expert-protege system degrades to a single small model system, losing all its benefits and utility and making the system useless. Note that this 10% pruning is well below the 20-50% these methods routinely use. With more significant pruning, the system will be more degraded.

These results are added to Section 5 and to Appendix E.10 (Table 19) of the revised manuscript.

[2] Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering. AAAI Workshop on Artificial Intelligence Safety (SafeAI), 2019.

[3] Spectral Signatures in Backdoor Attacks. NeurIPS, 2018.

[4] Trojan Signatures in DNN Weights. ICCV Workshops, 2021.

[5] Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. RAID, 2018.

[6] LLM-Pruner: On the Structural Pruning of Large Language Models. NeurIPS, 2023.

[7] A Simple and Effective Pruning Approach for Large Language Models. ICLR, 2024.

W4: RL persistence is narrow. Persistence is shown mainly on one Qwen3-8B checkpoint and a 100-task SWE-Gym subset, without broader datasets or model families.

A: The RL persistence experiment follows the setup of the original SWE-Protégé paper: on-policy GRPO with a clean reward, run on a 100-task SWE-Gym subset that does not overlap with the SWE-smith data used for poisoning, and we re-test whether the trigger still activates the backdoor at several training checkpoints. This tests whether the backdoor survives the continued training configuration of the deployed system itself. This RL experiment took about 4 calendar days on 4×H200. Repeating it on larger datasets and more base models needs more training time and could not be completed within the discussion period. We are already running these experiments, but they will not finish before the rebuttal ends, and we will update them in the revision.


Dear Reviewer,

I hope this message finds you well. We hope we've addressed your concerns. If you have any further questions, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues tto improve our work.

Thank you for your time and effort in reviewing our paper.

Dear Reviewer,

As the discussion period is drawing to a close, we wanted to follow up once more. We hope our responses and the additional experiments have addressed your concerns. If any point remains unclear or if you have further questions, we would be glad to clarify before the deadline.

Thank you again for your time and effort in reviewing our paper.

Best regards,

The authors

Reviewer_g2ktOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CONFUNDO, a training-time backdoor attack targeting the learned consultation policy of small-large LLM collaborative agents. By poisoning SFT trajectories with benign language triggers, the attack induces either excessive expert consultations (Denial-of-Wallet, DoW) or suppressed consultations (Denial-of-Expert, DoE).

The authors evaluate four model families on SWE-smith and SWE-bench Verified, with additional experiments on GSM8K and security-sensitive coding tasks. With approximately 4.27% poisoned training data, the attack substantially manipulates consultation behavior while preserving comparable untriggered behavior in the evaluated clean-checkpoint comparison. The paper also examines downstream task performance, attack controls, and defenses.

Critical strengths
  1. A distinct attack target. The paper targets the learned decision to consult an expert rather than directly manipulating task outputs. The two payloads demonstrate how this decision can affect both inference costs and task performance.

  2. Useful experimental controls. The authors evaluate multiple model families and include clean-checkpoint comparisons, prompt-injection baselines, reversed trigger-effect pairings, and downstream RL experiments. These experiments support the claim that consultation manipulation is learned through poisoned training data rather than caused solely by trigger wording.

Critical weaknesses
  1. The practicality of the poisoning setup remains unclear. Although the attack is framed as poisoning only a small fraction of the SFT trajectories, the experimental setup normalizes the entire training pool to a common template and constructs additional depth-conditioned consultation examples. It remains unclear whether the attack would remain effective when poisoned examples are inserted into an otherwise unmodified, heterogeneous SFT dataset.

  2. The generality of the DoE detection results needs clarification. Table 1 reports a high untriggered consultation rate for Qwen3-8B (0.93 on SWE-smith), which drops to 0 under DoE. However, the consultation-rate monitor reports an AUC of 0.500 for DoE using a different, complementary-pairing checkpoint. It is unclear whether DoE would remain difficult to detect with a lower-tail or two-sided monitor in the high-consultation settings of Table 1.

Minor suggestions

  1. Limited evaluation of downstream performance and security impact. The SWE task-resolution results would benefit from uncertainty estimates, while the insecure-code evaluation covers only 15 manually curated tasks assessed through static analysis. Additional validation or uncertainty estimates would strengthen the conclusions about downstream performance and security impact.
Questions for authors
  1. Would CONFUNDO remain effective if poisoned examples were inserted into an otherwise unmodified, heterogeneous SFT dataset?

  2. How would a lower-tail or two-sided consultation-rate monitor perform against DoE in the high-consultation settings reported in Table 1?

4 comments

We thank the reviewer for the valuable comments and respond to each point below.

W1 & Q1: The practicality of the poisoning setup remains unclear. Although the attack is framed as poisoning only a small fraction of the SFT trajectories, the experimental setup normalizes the entire training pool to a common template and constructs additional depth-conditioned consultation examples. It remains unclear whether the attack would remain effective when poisoned examples are inserted into an otherwise unmodified, heterogeneous SFT dataset. Would CONFUNDO remain effective if poisoned examples were inserted into an otherwise unmodified, heterogeneous SFT dataset?

A: As suggested by the reviewer, we ran more heterogeneous-SFT experiments that vary the degree of the diversity of the training dataset.

SettingDoW ASRDoE ASR
Setting 1(main setting)1.001.00
Setting 2 (native format)0.900.92
Setting 3 (external multi-source)0.850.88

We added these results to Appendix E.5 (Table 12) of the revised manuscript.

Setting 1 is the paper's main setting, we use a common template for both poisoned and clean samples. Setting 2 uses a different template for clean samples. Setting 3 further increases the heterogeneity by replacing about half of the clean samples with external public multi-source SWE trajectories (Open-SWE-Traces). The results show that when the heterogeneity increases, the ASR decreases. This is expected, similar to the training-testing performance gap caused by the distribution shift in normal deep learning tasks. Nevertheless, even in the most heterogeneous setting 3, our attacks still maintain 85%/88% ASR. Note that most existing data poisoning attack papers don’t evaluate their methods with this large heterogeneity, and our data poisoning setting is consistent with existing literature.

W2 & Q2: The generality of the DoE detection results needs clarification. Table 1 reports a high untriggered consultation rate for Qwen3-8B (0.93 on SWE-smith), which drops to 0 under DoE. However, the consultation-rate monitor reports an AUC of 0.500 for DoE using a different, complementary-pairing checkpoint. It is unclear whether Do E would remain difficult to detect with a lower-tail or two-sided monitor in the high-consultation settings of Table 1. How would a lower-tail or two-sided consultation-rate monitor perform against DoE in the high-consultation settings reported in Table 1?

A: The earlier DoE AUC of 0.500 was measured on a sparse checkpoint whose untriggered consult rate is only 0.056, not on the Table 1 Qwen3-8B checkpoint (0.93), and we correct that mismatch here. On the complementary-pairing Qwen3-8B checkpoint (untriggered consult rate 0.93, N=104 per condition), a lower-tail (or two-sided) consultation-rate monitor ranks DoE as highly separable (AUC 0.952), yet at a 5% FPR it flags no DoE episode at all (TPR 0.000). The reason is that suppressed DoE episodes all fall at zero billed calls, which is exactly where the roughly 7% of clean episodes that never consult also sit, so the zero-consultation operating point exceeds the 5% budget. Moreover, a suppressed DoE episode is behaviorally identical to a clean zero-consultation one, so any alarm raised on it cannot be attributed to an attack. In practice the operator does not know whether a given episode carries DoW or DoE and must therefore commit to a single rule. A single payload-agnostic consultation-rate rule that pools DoW and DoE against clean is at chance (AUC 0.49), because the same rule that flags DoW's extra calls ranks DoE's zero-call episodes as cleaner than clean.

MonitorDoE AUCActual AUC (defenders cannot know the payload in advance)
Lower-tail consult-rate monitor0.9520.49

The corrected numbers are updated in Section 5 and Table 19, and Table 18 in Appendix E.10 reports the lower-tail monitor together with the payload-agnostic monitors.

Minor: Limited evaluation of downstream performance and security impact. The SWE task-resolution results would benefit from uncertainty estimates, while the insecure-code evaluation covers only 15 manually curated tasks assessed through static analysis. Additional validation or uncertainty estimates would strengthen the conclusions about downstream performance and security impact.

A: We added repeated runs of the resolve and patch rates in Table 3 of the paper. Mean ± standard deviation are below.

ConditionResolve mean ± stdPatch mean ± std
Clean0.39 ± 0.020.40 ± 0.02
DoW0.36 ± 0.030.38 ± 0.02
DoE0.26 ± 0.020.27 ± 0.03

The security evaluation in Table 4 of the paper consists of 15 manually curated coding tasks judged by static analysis. The reported results already run each task 3 times (45 runs in total as stated in Appendix F), as shown below.

ConditionConsult rateInsecure-code rate
No trigger0.310.356
Consultation forced on1.000.293
Consultation forced off0.000.500
DoE trigger0.040.488

Thank you for addressing my concerns. W1 is resolved, and for W2 I hope the corrected results are reflected in the paper’s discussion. I will keep my score.

We thank the reviewer for confirming that W1 is resolved and for keeping the engagement. The corrected consultation-rate monitor results are now reflected in the paper: Section 5 and Appendix E.10 (Table 18). We appreciate the reviewer's time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.