- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
A data agent's tool can run without error and still return a plausible but wrong result. The paper calls this "silent tool poisoning" and introduces ToxicBench to measure it. A proxy runs the real tool and then edits its output, while the source tables stay fixed, so each task has a clean and a poisoned version. Besides task success, the paper defines trajectory metrics for whether the agent detects the poison, checks it with fresh evidence, adopts it blindly (Blind Compliance Rate, BCR), or adopts it after a check. The main study uses GPT-5.4-mini across LangGraph, smolagents and AutoGen. A defense study on Claude Haiku 4.5 compares a single-pass agent with a plain second attempt (Double-pass), a verification prompt and a checklist guard. Poisoning lowers success sharply. Double-pass is the best defense, and neither verification prompt beats it. Under repeated poisoning, many runs adopt the poison even after checking.
Critical strengths
- The paired design is clean. The proxy edits only the returned observation while the source data stay fixed. A wrong answer therefore cannot be explained as faithful analysis of changed data.
- The operator taxonomy separates corruptions that range or plausibility checks can catch from ones they cannot. Label swaps and treatment or control flips keep the value but change what it is bound to.
- The metrics separate exposure from the response to exposure. They also separate blind adoption from adoption after a check. This is more informative than final accuracy or counts of tool calls.
- The equal-cap Double-pass control keeps a gain from simply trying again from being credited to verification. The paper honestly reports the negative result that follows.
Critical weaknesses
-
Under one-shot poisoning with shared state, the second route is almost never exposed to poison. So the headline "ordinary retries help" mostly means the tool stopped lying on the second call. The only per-route control is a preselected subset where Base is already perfect, so it cannot test the effect. The paper does not report second-route exposure for the full defense matrix. It also has no Base arm under repeated poisoning, so no result shows that a two-route protocol reduces adoption when checks can also be corrupted.
-
Corruption that clears on re-query matches neither an adversary nor the error sources the paper cites. Stale summaries, unit-conversion bugs and label-binding errors would all recur on an identical call. No transient source, such as a flaky cache or nondeterministic retrieval, is named.
-
ADR is effectively unmeasured. The scorer almost never detects anomalies in development and predicts none after the freeze, so BCR in practice means adoption without VR. BCR recall is low in development and only moderate after the freeze, with few positives. Scorer errors also depend on the method. Most automatic task-success misses in the human evaluation are Verification-only runs. Under human labels, Verification-only ties Double-pass. The full-matrix gap where Verification-only trails Double-pass (Table 18), and its lower clean TSR, may therefore be partly scorer artifacts.
-
The lesson that verification must be compared against a plain retry closely parallels Huang et al. (ICLR 2024, "Large Language Models Cannot Self-Correct Reasoning Yet"), which is not cited. CRITIC, ToolSword, InjecAgent, the knowledge-conflict line (Xie et al., ICLR 2024; ClashEval), and data-agent benchmarks such as DA-Code, DSBench, InfiAgent-DABench and DABstep are also missing.
Questions for authors
- In the full Haiku defense matrix, how often did the second route receive any corrupted observation? - Does Double-pass still beat Base when the second route is also exposed?
- If VR is restricted to first-route checks or to checks that touch the corrupted value, do two-route BCR values stay at zero?
- How often do the historical poisoned observations still contain a clean alias of the answer? How do the main TSR drops and RR change on alias-free, full-target deliveries?
- Under human labels or a structured answer field on a larger sample, does Verification-only still trail Double-pass?
3 comments
Response to Reviewer YfyS (2/2)
Q1. Second-route exposure and Double-pass versus Base: Historical second-route exposure is 2/118, 3/118, and 3/118; the new persistent comparison gives Base and Double-pass 33/40 each. W1 provides the full matrices, conditional denominators, and intervals. Double-pass records second-route exposure in 35/40 new runs, with both-route exposure in 34/40. The historical Haiku audit and new DeepSeek experiment are reported separately. Method comparisons use complete experimental arms, and exposure counts describe the evidence received within them.
Q2. Restricting which checks count toward VR: The check-scope audit gives explicit BCR values under each definition. Counting only first-route checks after exposure gives historical Verification-only 1/101. Target-related checks give a logical range of 0/101 to 1/101, accounting for unresolved cases. Historical Double-pass and Guard remain zero, consistent with their zero PAR.
In the new persistent matrix, restricting to first-route checks changes the original zero BCR counts to Double-pass 3/36, Verification-only 2/35, and Guard 1/36. PAR and ADR stay frozen. The target-related range is a logical bound based on unresolved checks. These diagnostics make the role of check scope explicit and distinguish check attempts from the evidence they supply.
Q3. Clean aliases and results on full-target deliveries: We confirm correctly bound clean-answer aliases in 8/694 events, find none in 452, and leave 234 unresolved. An alias is another occurrence of the correct answer with the correct binding. Confirmed full-target changes with no direct clean alias retain 437 events. Task success still falls in the resulting subsets:
| Adapter | Full-set TSR drop | Certified-subset TSR drop | Full-set RR | Certified-subset RR |
|---|---|---|---|---|
| AutoGen | 31/118 | 19/63 | 32/101 | 30/63 |
| LangGraph | 38/118 | 14/66 | 33/101 | 32/66 |
| smolagents | 46/118 | 27/61 | 13/87 | 10/61 |
TSR drop is the net decrease in correct answers divided by paired tasks. RR uses exposed runs. These estimates describe subsets with certified direct answer fields. The common defense subset contains 50/54 semantic tasks and scores near the ceiling. The matched experiment and human checks provide separate evidence for method comparisons.
Q4. Human or structured scoring on a larger sample: The available human reassessment answers this comparison on the existing 20 core poisoned tasks: Verification-only and Double-pass each score 20/20. W3 provides the paired automatic/human table and clean results. The current human comparison is this existing audit, while full-matrix Haiku results use automatic scoring. The new structured-answer experiments separately test information and cost effects on DeepSeek. Each comparison is presented with its model, scoring method, and task coverage.
Revision: The revised paper will clarify retry and scoring results in the experiments, report route and alias audits in the appendix, and expand Related Work with the suggested comparisons.
Thank you for the detailed response and additional analyses. The route-exposure audit and the new persistent-corruption comparison substantially address my concerns about missing controls. The alias audit also provides useful evidence that the task-success degradation persists in subsets with certified target changes and no direct clean-answer alias.
These results clarify the scope of the conclusions. The historical retry gains include opportunities to obtain unmodified evidence, while the new persistent comparison shows no observed advantage of Double-pass over Base, with substantial uncertainty. The existing human audit also places Verification-only and Double-pass at the same accuracy on the audited poisoned tasks.
I am increasing my overall assessment by one level because these additions strengthen the benchmark contribution and make the empirical findings more interpretable. My increased assessment assumes that these qualifications and the new results are incorporated into the revised paper.
Response to Reviewer YfyS (1/2)
Thank you for the detailed questions. We audited route exposure and aliases, added a matched persistent-corruption comparison, and reanalyzed existing human labels by method. W and Q follow the original review order.
W1. Route exposure and the persistent Base comparison: In the full historical Haiku matrix, second-route exposure is 2/118 for Double-pass and 3/118 for each verification method. All 944 retained trajectories match tool-event records. No run is exposed in both routes. Conditional on first-route exposure, the counts are 0/99, 0/98, and 0/101, respectively. Historical retry gains thus include later opportunities for unmodified evidence.
We add 480 natural-tool trajectories using DeepSeek-v4-pro-ga-260813, crossing four methods with three conditions on 40 tasks fixed before outcomes. This adds a Base control beyond the earlier ceiling subset, with frozen automatic scores and recorded exposure:
Base has one route. Double-pass records both-route exposure in 34/40 runs. Its persistent difference from Base is 0 points, with paired 95% interval [−12.5, +12.5] and source-table cluster interval [−10.9, +10.0]. It uses 8.60 requests and 7,189 reported tokens, versus 4.42 and 3,772 for Base. Rules certify target changes in 23 of its 35 second-route exposures; 12 remain unresolved. A separate diagnostic with an error warning, an explicitly faithful source, seeded observations, and structured answers gives both methods 40/40 under clean and persistent conditions.
Persistent poison-adoption counts are Base 6/40, Double-pass 6/40, Verification-only 4/40, and Guard 2/40. These require recorded exposure and use all 40 planned tasks as the denominator. The original PAR uses exposed runs as its denominator.
W2. Persistent and transient failure sources: Native pandas 1.3.3 and DuckDB 1.3.0 bugs reproduce across three identical calls and a new-process call. Later versions return correctly. RFC 5861 describes caches serving old content while updating, with later freshness depending on update timing. One-shot tests recovery after one wrong return; persistent poisoning tests errors recurring on eligible calls.
W3. ADR, BCR, and method-dependent scoring: Our existing human evaluation of 200 trajectories yields 96% task-success scoring agreement and confirms adoption of poisoned answers after checking.
Six of the eight answers accepted by humans but rejected by the automatic scorer are Verification-only outputs. On the 20 core poisoned tasks:
Both methods gain 4/20 correct answers over Base, and all four methods are human-correct on 20/20 clean tasks. The 118-task matrix and new-model free-form comparisons use automatic scores; Q4 describes the human audit's coverage.
BCR identifies 8/10 human positives, with two false negatives and zero false positives; ADR recall is 0/1. ADR measures explicit recognition in the final answer. When ADR is zero, BCR records poison adoption with VR also zero. Q2 shows how alternative check definitions affect these counts while keeping the scorer frozen.
W4. Related work and our contribution: We added the suggested papers. Huang et al. establish fair additional-attempt comparisons, and CRITIC uses external tool feedback for correction. ToxicBench tests this principle when feedback can be corrupted, adding actual-exposure audits and a persistent Base comparison.
ToolSword includes erroneous feedback, InjecAgent studies instructions in tool returns, and Xie et al. and ClashEval study conflicts between model priors and external evidence. ToxicBench changes returned values or bindings over fixed current source tables, tracing exposure, checking, adoption, and recovery against computed references.
DA-Code, DSBench, InfiAgent-DABench, and DABstep assess broader data-analysis skills and workloads. ToxicBench complements these benchmarks with paired evidence changes for the same task and source, supported by route, alias, and human-scoring audits.