acceptodds
Under review as a conference paper at ICLR 2027

Intervention Success Is Not Mechanism Identification in Tool-Using Agents

Abstract

Tool-using language model agents are often diagnosed by changing prompts, tools, or available information and then attributing the resulting performance change to a particular failure mechanism. We show that this inference can be misleading. We introduce **Causal Agent Bench (CAB)**, a framework for testing such claims with paired tasks, matched controls, model and tool-interface comparisons, and direct interventions. Across several repository-based evaluations, an instruction that contains no task-specific information can improve accuracy, reduce it, or have little measurable effect. This behavior is not tied to one sentence. Across six matched neutral and procedural prompt pairs, Qwen under one setting drops by 11.5 percentage points while every run still produces a valid final answer. A larger study with 30,240 runs finds substantial procedural-minus-neutral differences in ten of twelve settings, although many of the largest differences occur because the model fails to produce a valid tool action. We also show that a successful repair need not identify the original cause of failure. In one controlled experiment, directly providing the source needed after a tool failure removes the observed accuracy gap. These results show that intervention success is not mechanism identification and that controls must be tested in the model and tool interface in which they are used. Code: https://anonymous.4open.science/r/CAB_ICLR27-309E/README.md

Primary area:
interpretability and explainable AI
Keywords:
Causal Agent Evaluation, Tool-Using Agents, Mechanism Identification, Comparator Validity

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FJdXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper introduces Causal Agent Bench (CAB) to study when interventions on tool-using agents support causal conclusions about why the agents fail. Its thesis is that an intervention which improves task performance does not thereby identify the mechanism of the original failure. Using paired repository-based retrieval tasks under CLEAN, PRESERVE and DISRUPT, matched BASE/PLACEBO and NEUTRAL/PROCEDURAL prompts, SERIAL and PARALLEL interfaces, and direct source-provision interventions, the authors report four findings:

  • An information-free instruction can improve, reduce or leave accuracy unchanged depending on model, interface and task;
  • The effect is not reducible to one sentence or to parser failure (a six-pair study finds −11.5 for Qwen with all runs valid; a 30,240-run study finds ten of twelve comparisons negative, many associated with invalid tool actions);
  • Successful repairs do not necessarily identify failure mechanisms. Providing correct source identifiers improves accuracy, but similar benefits also appear on clean tasks, so the improvement does not establish a disruption-specific source-selection failure.
  • One targeted intervention provides stronger mechanistic evidence. In the Gemma/parallel-interface setting, directly providing the source needed after a tool failure eliminates the observed accuracy gap between the base and control conditions.
Critical strengths
  • The paper draws a distinction agent-debugging work routinely ignores, and the source-hint experiment makes it concrete.
  • The paper presents a reasonably systematic experimental design: The use of paired tasks, matched prompt conditions, alternative interfaces, repository-level resampling, and multiple-comparison corrections is a strength. The paper also examines action validity and tool-use trajectories rather than relying exclusively on final accuracy.
  • The authors investigate whether large accuracy differences arise before the agent successfully executes a tool call. The six-pair prompt experiment is particularly useful because its reported effect persists despite valid final answers and no parser rejections. This helps avoid incorrectly interpreting every drop in end-task accuracy as a deterioration in reasoning.
Critical weaknesses

W1. The central claim of this paper is that "a successful intervention does not necessarily identify the underlying causal mechanism" is sensible, but it is also a fairly general principle of causal inference. The paper needs to establish more clearly what is specifically new about applying this principle to tool-using agents. The empirical findings primarily show that prompt effects vary across models, interfaces, and task settings. Such heterogeneity is useful to document, but heterogeneity alone is not a deep explanatory insight. For example, the paper shows that a procedural prompt can help one model and hurt another, and that changing the tool interface can substantially alter the effect. However, what generalizable insight does this provide about why these effects differ? Does the paper identify conditions under which intervention effects can be predicted or interpreted mechanistically, or does the conclusion remain that each configuration must be evaluated independently?

W2. The paper systematically varies prompt wording, model, interface, and task condition. These are appropriate experiments for estimating intervention effects, but the analysis often stops after establishing that the effects differ across conditions. A more informative study would identify and test the underlying behavioral factors that explain this variation. For example, does procedural wording alter source-selection behavior, action generation, or recovery after tool failures? Under what conditions does parallel tool access amplify or mitigate these effects? Although the paper includes some trajectory analyses, it is unclear whether these observations yield a unified, testable explanation of the results. Without stronger analysis of competing explanations, the experiments risk remaining a collection of conditional effects rather than establishing a more general account of when and why interventions work.

W3. In the 30,240-run study, the authors report 2,289 JSON-decoding failures and 124 invalid read actions. For the longest procedural prompt, Gemma fails to produce a valid first tool action in all 360 runs in each reported setting. These are important system-level failures, but they complicate the interpretation of the observed accuracy differences. In particular, an intervention that disrupts action formatting may reduce task success without providing much evidence about the underlying task-reasoning failure that the paper seeks to explain. The authors should more clearly distinguish at least three outcomes: (1) action-format validity, (2) successful tool execution and information acquisition, and (3) final-answer correctness conditional on successful execution. This decomposition would help clarify how much of the reported effect reflects changes in task-solving behavior versus failures at the interface or execution level. It would also help establish whether the proposed conclusions about mechanism identification hold when valid tool use is maintained.

W4. The evaluation focuses on relatively small open-weight models (approximately 3B–14B parameters) and controlled repository-based retrieval tasks, so it remains unclear whether the findings generalize to larger or closed models, broader software-engineering tasks, or other tool-using-agent settings.

Questions for authors

Please see weaknesses.

Reviewer_ycg2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper's goal is to show that being able to fix an error does not necessarily lead to gaining understanding of how the errors occur in the first place. This paper studies this in tool-calling agents.

Critical strengths
  • The related work section is written relatively clearly
Critical weaknesses
  • The paper is very difficult to read and understand. The main reason is that many terms are introduced without being defined. For example, what are Procedural Prompts? What are Neutral Prompts? What are Base Prompts? Placebo Prompts? What are decoy cases?
  • Results are simply listed in a long list without explaining what they mean or how they support any hypothesis.
  • The main point that the authors try to make (see paper summary above) is very obvious, and almost no one would disagree. As a result, the paper is not particularly novel.
  • There are obvious mistakes. For example, in table 5, the difference is listed as +3.47 but other information in the table indicates it should be 5.
Questions for authors

Can authors clearly define all the terms introduced in the paper?

Reviewer_5dwLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAB asks whether an intervention that raises a tool-using agent's accuracy actually identifies why the original failure happened, or merely patches a surface symptom. Its core device is a matched comparison between informative and uninformative (placebo) instructions on self-built repository tasks: a content-free placebo moves accuracy by tens of points depending on model and tool interface, and supplying the missing source closes the gap only in one narrow Gemma–PARALLEL cell. The paper's point is that intervention success does not by itself establish mechanism identification.

Critical strengths
  1. The core statistics are sound: paired exact tests on binary outcomes, repository-level rather than trajectory bootstrap for CIs, and Holm correction within pre-specified families. The authors also disclose null results, a validator bug, and six failed public-benchmark attempts (Appendix J/K) rather than dropping them.

  2. The required-source experiment (Sec 4.6) is the most direct causal check in the paper. In Gemma–PARALLEL failures, BASE reads the needed source 23/24 times and is correct 23/24, while PLACEBO reads it 7/24 and is correct 7/24; giving an unrelated source leaves the gap (17 vs 5/24), and giving the required source raises both arms to 24/24.

Critical weaknesses
  1. The headline 180-repository study is dominated by output-format failures rather than reasoning differences. Base accuracy sits at 98–100% (ceiling), Gemma's longest procedural prompt scores 0/360 because runs terminate before the first valid tool action, and 2,289/2,413 recorded errors are JSON-decode failures — the authors state themselves that this does not isolate a reasoning loss. The abstract nonetheless leads with those runs.

  2. The direct-mechanism evidence is confined to one small setting. The required-source effect is significant only in Gemma–PARALLEL (adjusted p=.031, n=24/arm); the other seven model–interface combinations are not significant (Table 4/8). The restart/seed reruns (Appendix F.2) use a different 12-task CLEAN panel, so they do not test whether this cell holds up under other inference setups.

  3. Tasks are author-built and non-native. The 180-repository set uses small declared-source environments with precomputed fields rather than native repository exploration (Appendix B.4/E.2), and the 32/48-case sets are hand-constructed disruptions; there is no inter-rater or external check that these correspond to failure modes seen in deployed agents.

  4. DoVer already notes that multiple distinct interventions can independently repair a failed task, and Causal Agent Replay already applies do(·) interventions to agent steps with reported CIs. CAB's added value is the paired model×interface evidence and the explicit required-source control; a concurrent preprint (Modrzejewski & Radzikowski 2026) reports related interface-dependent wording effects and should be acknowledged as contemporaneous rather than treated as a citation omission.

Questions for authors

Refer to above weaknesses.

Comments on the paper

1 comment

In particular, our findings provide three concrete observations.

First, an instruction containing no task-specific information can substantially change performance, including when every tool action is valid. The original model–interface experiments further show that the same instruction can have different effects when only the tool interface changes.

Second, a successful repair need not be specific to the failure being investigated. In the original source-identifier experiment (Section 4.2, Table 1), correct identifiers improve performance by 77.1 points for Qwen3 and 62.5 points for Ministral under disruption. However, the corresponding improvements on clean tasks are 77.1 and 64.6 points. The large improvement therefore does not establish a disruption-specific source-selection failure.

Third, the new experiments directly test how the action interface contributes to the measured intervention effect. We find that constraining the action format removes nearly all of the original prompt-related accuracy difference across seven settings and three models. Separately, the required-source replications establish a reproducible reduction in unaided recovery performance, while demonstrating that directly providing evidence can bypass the very search process whose failure is being investigated.

Together, these results go beyond observing that intervention effects vary across models or tasks. They identify particular stages at which the differences arise and test competing explanations through additional controlled experiments. They also show why improvements in final accuracy, even when substantial and reproducible, must be interpreted alongside changes in tool execution and evidence retrieval.

We also acknowledge Modrzejewski and Radzikowski (2026) as concurrent work, which appeared after our submission. Their study examines how question wording affects model comparisons under different tool conditions. Our work instead examines whether the controls and repairs used to diagnose an agent's failure support the proposed explanation within a given agent configuration.

We believe the additional evidence addresses the reviewer's two principal technical concerns. The action-format experiments establish what accounts for many of the largest accuracy differences, while the recovery replications provide stronger evidence for an effect previously supported by only 24 tasks. At the same time, we retain the limits of the constructed tasks and the single-model recovery experiment rather than drawing conclusions that the available results cannot support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.