acceptodds
Under review as a conference paper at ICLR 2027

ProbeAgent: Principled Diagnosis of Agent Failures in Verifiable Environments

Abstract

Improving agents requires discovering where they fail and understanding why. An agent acts over multiple steps in a changing environment, so diagnosing its failures requires more than inspecting the final response of a large language model (LLM). The same unsuccessful outcome can arise from task difficulty or from an error the agent repeats across tasks. We introduce ProbeAgent, an automated framework that discovers agent failures, tests their proposed causes, and identifies shared failure modes. Starting from benchmark tasks, ProbeAgent builds executable tests and validates them using reference solutions, automated checks, and review of the task requirements. For each failed trajectory, it explains the error and constructs new tasks that preserve the suspected cause while changing the surrounding context. Running the agent on these tasks tests whether the same error recurs. ProbeAgent then groups failures with similar causes and generates descriptions that distinguish their errors. This process connects explanations of individual failures to executable tests and descriptions of shared failure modes. We evaluate ProbeAgent across 22 agent configurations using tasks from 10 benchmarks, collecting 1,244 failures and producing 536 failure-mode descriptions. These descriptions distinguish specific errors within broad outcomes such as incomplete work, including planning without acting and stopping before delivering a file. In human evaluation, annotators accept 90.0% of assigned descriptions and correctly reject 83.3% of similar alternatives, supporting the accuracy and specificity of the discovered failure modes.

Primary area:
generative models
Keywords:
LLM agents, agent evaluation, failure diagnosis, automated testing, failure mode discovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Jmf2OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PROBEAGENT, a framework for automatically discovering and diagnosing agent failure modes. Its goal is to distinguish two kinds of failures: those caused by the difficulty of a particular task, and errors the agent repeats across different tasks.

The framework has three stages:

  1. Task construction and diagnosis. Starting from benchmark seed tasks, an LLM rewrites them into new executable tasks (probes). Each new task must pass three checks: the reference solution passes, doing nothing fails, and an independent review finds no shortcuts or loopholes. When the target agent fails, an LLM reads the trajectory and writes a failure explanation consisting of the erroneous action, where it occurs, the triggering condition τ (the task condition under which the error arises), and a corrective action.
  2. Recurrence testing. The framework constructs several new tasks (siblings) that preserve τ while changing the domain, data, tools, or output format, and runs the same agent on them again. An LLM judges whether the agent makes the same error, and the error is then classified as stable agent behavior or as specific to the original task.
  3. Grouping. Failures are grouped by their causes, and a one-sentence description is generated for each group.

The experiments cover 22 model–harness configurations and 10 benchmarks, yielding 1,244 error records and 536 failure-mode descriptions. The automatic evaluation reports 88.5% agreement. In the human evaluation, annotators accept 90.0% of the descriptions and correctly reject 83.3% of similar distractor descriptions.

Critical strengths
  1. The problem is valuable and the idea is clear. The paper separates "the task itself is hard" from "the agent repeatedly makes the same error". Rather than stopping at post-hoc explanation, it treats each explanation as a hypothesis, constructs new tasks to test it, and forms a "diagnose → construct tasks → diagnose again" loop. This goes a step beyond purely post-hoc attribution.
  2. Quality control during task construction is careful. Across 1,451 construction rounds, the independent review rejects a further 638 candidates that had already passed the execution checks, and shortcuts that bypass the task requirements are found in 565 rounds. This shows that "it runs" does not mean "it tests the right thing", which is also useful for research on the validity of agent benchmarks.
  3. The experiments are broad and the appendix is candid. The experiments cover many models and harnesses. The appendix honestly reports the trade-off in grouping granularity, groups for which no description could be generated, timeout differences, and environment defects. The ablation of the grouping constraint (78.0% → 91.8%) is convincing.
Critical weaknesses

W1. The recurrence test, a main addition over ProbeLLM, is neither validated nor sufficient to identify the cause of a failure.

  1. The trigger τ, the siblings built from it, and the recurrence verdict are all produced by GPT-5.6-sol (Appendix A). The paper does not check whether τ is correct or stable across repeated analyses of the same failure, whether siblings actually preserve τ, or whether recurrence verdicts agree with humans, and sibling builds are not reported separately in the admission statistics. Generated environments are often flawed: the manual audit finds a wrong expected value or a format ambiguity in 6 of 20 environments (Table 9), and Appendix A.1 notes that executable controls alone miss semantic drift, such as a GAIA question turned into an unrelated code-editing task.
  2. Recurrence on siblings does not show that τ is the cause. No sibling holds τ fixed while controlling task length or budget, and no intervention, such as giving the agent the corrective action, tests whether the error disappears. Timeouts are a plausible confound: Terminus 2 with Sonnet 5 times out on 46% of rollouts versus 12–13% under the other three harnesses (Appendix D.2), and a missing deliverable caused by a timeout would also count as recurrence. In addition, a sibling counts as reproducing if any one of its rollouts is judged a recurrence (Eq. 4), and a task-specific verdict only means that the error did not recur in the siblings tested.
  3. Few failure modes are supported by the recurrence test. Only 119 of 239 explanations reached sibling testing, and 20 met the agent-behavior criterion (Figure 3). Of the 565 induced groups, 404 (71.5%) have no linked sibling test, and 361 (63.9%) contain only one failure (Table 6).

W2. The evaluation measures agreement within the pipeline, and there is no baseline.

  1. Held-out failures are assigned to a group by the induction judge and confirmed by the same judge against the group's first member, and no-match outputs and failed calls are excluded (Appendix A.2). Since every role uses GPT-5.6-sol, the 88.5% and 91.8% figures measure the self-consistency of one model rather than diagnostic accuracy.
  2. The human evaluation uses 2 annotators and 30 failures and reports no inter-annotator agreement (Appendix C.5). Annotators judge whether a description matches the failure as specified by the pipeline; no one checks whether the failure explanation itself, namely the erroneous action, its location, τ, and the corrective action, is correct.
  3. The only comparison is with a fixed eight-category taxonomy (Figure 2), which shows finer granularity rather than correctness. There is no comparison with ProbeLLM (arXiv:2602.12966), which the paper extends, or with a simple baseline that asks an LLM to summarize failure modes. Although Eq. 1 defines the objective in terms of comparing search policies, no alternative policy is evaluated, whereas ProbeLLM itself reports comparisons with prior probing methods.
Questions for authors
  1. Under the same budget, how does PROBEAGENT compare with ProbeLLM (arXiv:2602.12966), both in failure discovery (τ-preserving siblings versus ProbeLLM's local perturbation of failing seeds) and in grouping (ProbeLLM's induction versus density-constrained grouping on the same failures)? Since Eq. 1 is defined over search policies, could you also compare with at least one alternative policy, such as exploration without sibling tests?
  2. In the held-out evaluation, how many failures were assigned to a group, how many received no match, and how many calls failed? Could humans annotate a sample of failures without seeing the pipeline's diagnosis, so that you can report the accuracy of the failure explanations (erroneous action, location, τ, corrective action), the agreement of the pairwise and recurrence judgments with human labels, and the agreement between annotators?
  3. How do you verify that a sibling preserves τ? Could you report the admission results of sibling builds separately, manually audit a sample of siblings, and report whether repeated analyses of the same failure yield a consistent τ?
  4. Does the recurrence judge require the sibling rollout to fail? How do the verdicts in Figure 3 change when timed-out rollouts are excluded, when recurrence must occur in a majority of rollouts, or when siblings are matched to the source probe in length and budget, and does the error disappear when the corrective action is given to the agent as a hint?
  5. Since Table 5 lists 13 seed tasks from 8 suites, how are the 14 seed tasks and 10 benchmarks in Section 4.1 counted? Could you also release the prompts and code, and give one end-to-end case each for an agent-behavior and a task-specific verdict, covering the seed task, the probe, the siblings with their instructions and checkers, the trajectories, and the verdicts?
  6. As a suggestion for future work, would you consider testing whether the discovered failure modes can be used to improve agents, for example through fine-tuning or prompting, as ProbeLLM does for LLMs?
Reviewer_DaJyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProbeAgent, a framework for finding and analyzing agent failures. It generates executable tasks, analyzes failed trajectories, creates new tasks to check whether the same failures happen again, and groups similar failures into common failure modes. The experiments cover 22 agent configurations and 10 benchmarks.

Critical strengths
  1. The paper studies an important problem. Most agent benchmarks focus on whether a task is completed, but understanding why an agent fails is also important.
  2. The framework pays attention to the quality of generated tasks. It checks reference solutions, test cases, and task requirements to reduce errors in the evaluation process.
Critical weaknesses

1.Most parts of the framework are based on existing methods, and the overall approach is quite similar to ProbeLLM. The main difference is the use of executable environments and sibling tasks to check whether failures happen again. 2.The paper uses sibling tasks to check whether the same error appears in different settings. However, seeing the same error again does not necessarily mean that the underlying cause has been identified. The paper could be clearer about what this testing process can telll.

Questions for authors
  1. What are the main new contributions compared with ProbeLLM, beyond applying the framework to agent environments?
  2. Does generating sibling tasks based on failure explanations work better than simpler task generation methods?
  3. How should we interpret the large number of failure groups that contain only one example or have not been tested on sibling tasks?
Reviewer_TbDMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Understanding why multi-step language model agents fail requires more than looking at final benchmark scores or diagnosing isolated execution traces. This paper introduces ProbeAgent, an automated framework that extends active failure mode discovery to containerized agent environments. The method generates and validates new executable tasks from seed benchmarks, creates sibling tasks to test whether an observed failure mechanism recurs in new contexts, and clusters failure explanations into natural language failure mode descriptions.

Critical strengths
  1. The paper moves agent evaluation from static pass rates toward automated, testable root cause diagnosis. Generating sibling environments to verify whether a hypothesized failure trigger causes the same error in a new domain is a good way to separate recurring agent flaws from task-specific errors.
  2. The three-stage probe validation pipeline shows the value of combining execution checks with semantic review. Across 1,451 build rounds, the separate read-only reviewer catches 638 flawed task candidates that already passed reference and null execution controls, mostly by spotting reward shortcuts and unstated checker requirements.
  3. The human evaluation confirms that the induced failure descriptions are specific and accurate for the clusters they represent.
Critical weaknesses
  1. The complex pairwise LLM judging stage in the clustering pipeline is not justified by the ablation results. In Table 2, a simple single cosine similarity threshold requiring zero LLM judge calls achieves 92.1 percent held-out agreement and finds 500 groups, slightly outperforming the full method at 91.8 percent agreement despite the full method making 7,449 LLM judge calls.
  2. Most of the discovered failure modes are isolated singletons or lack sibling recurrence verification, which weakens the central claim of discovering shared, recurring behaviors. In Table 6, only 5.8% of groups are verified as recurring agent behavior, which is very low hit rate.
  3. The generated environments still suffer from a high rate of specification defects, and the empirical observations come mostly from a small number of seed tasks. The manual audit in Table 9 shows that 30 percent of sampled environments contain wrong expected values or unstated format ambiguities despite passing all validation gates.
Questions for authors
  1. Why do you need the calibrated ambiguous band with 7,449 pairwise LLM judge calls, when the single threshold baseline in Table 2 achieves higher held-out agreement with zero judge calls?
  2. Equation 1 formalizes probe search as an optimization problem over search policies, so why does the paper only evaluate one fixed heuristic schedule instead of comparing alternative search policies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.