acceptodds
Withdrawn from ICLR 2027

AgentEx-Bench: A Step-Level Benchmark for Evaluating Explanation Quality in Agentic Trajectories

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model agents increasingly expose intermediate plans, evidence, calculations, and tool decisions, creating an opportunity to evaluate how their observable behavior supports task outcomes. However, existing agent evaluation remains centered on task success, action correctness, or domain-specific process checks. These metrics do not determine whether an emitted trajectory provides a well-grounded, specific, and diagnostically valuable explanation of agent behavior. We introduce AGENTEX-BENCH, a benchmark and measurement framework for evaluating observable agent trajectories as auditable artifacts. AGENTEX-BENCH standardizes heterogeneous reasoning and tool-use trajectories and evaluates key steps along five complementary dimensions: validity, task-grounded faithfulness, specificity, diagnostic utility, and evidence alignment. The framework combines source-aware outcome verification, provenance-aware annotation, controlled perturbation tests, and uncertainty-aware comparison across four task families and 15 systems. Our analyses show that task success and trajectory explanation quality provide related but non-equivalent views of agent behavior. Dimension-level profiles reveal distinctions that aggregate scores can obscure, while matched human-reference analyses show substantial uncertainty in fine-grained model ordering. These findings support treating observable trajectory quality as a distinct evaluation target that complements task success and process correctness for agent auditing, diagnosis, and oversight.

Primary area:
datasets and benchmarks
Keywords:
Large Language Model Agents, Agent Evaluation, Observable Agent Trajectories, Explanation Quality, Benchmarking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XoaaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

AgentEx-Bench evaluates the quality of observable explanations separately from final-answer correctness. It collects 1,186 trajectories from 15 systems on 80 tasks drawn from BFCL, GSM8K, HotpotQA, and StrategyQA. Selected key steps are scored on validity, task-grounded faithfulness, specificity, diagnostic utility, and evidence alignment. The paper combines outcome checks, human annotation and adjudication, controlled perturbations, and uncertainty-aware comparisons. Its main claim is that trajectory explanation quality provides information that final-answer accuracy alone does not capture.

Critical strengths
  1. The paper addresses a useful evaluation problem. A correct final answer does not guarantee that its explanation is specific, supported, or useful for diagnosis. Separating these properties from task success can make evaluation more informative than a single accuracy score.

  2. The study includes human-reference analyses, agreement measurements, controlled perturbations, and uncertainty estimates. These are relevant components of a measurement framework. The authors also explicitly distinguish task-grounded faithfulness from causal faithfulness and acknowledge limitations in generation provenance and fine-grained model ranking.

Critical weaknesses
  1. The reported results are not internally consistent, so the final data version and aggregation pipeline are unclear. Table 8 and Figure 8 both report trajectory-balanced scores on a 0–100 scale, but several entries differ substantially. For Claude Opus 4.5, validity is 98.7 in Table 8 and 97.2 in Figure 8, while faithfulness is 98.9 and 96.9. Gemma 3 27B specificity is 58.0 and 54.5. These differences exceed rounding error. Figure 7 reports 18/20, 20/20, 20/20, and 20/20 correct answers for Opus, which gives 78/80 = 97.5%, not the 96.2% in Table 8. Its four source-level explanation scores average to 93.8, rather than the 95.0 in Table 8. These may reflect different annotation or processing versions, but no such distinction is documented. Until the figures and tables are reconciled, the empirical comparisons cannot be reliably interpreted.

  2. The validity of the generated data and the scoring procedure is not sufficiently established for the central evaluation claims. Section 3.3 and the appendix acknowledge that validation retries can reveal gold answers or expected function names. There are 138 trajectories with one recorded validation retry and 80 with missing retry metadata; these counts do not establish exactly which outputs received gold information. Such outputs can contain post-hoc explanations and may change both correctness and its association with explanation quality. The disclosure is useful, but a uniformly gold-blind comparison is still missing. Scoring reliability is also a concern: Table 5 reports kappa = 0.074 for specificity and 0.226 for trajectory explanation quality. Label imbalance may affect kappa, but the uncertainty should be examined rather than resolved solely by adjudication. Moreover, the matched-human and full-hybrid explanation rankings have Spearman rho = -0.046. As the authors acknowledge, this comparison mixes task composition and score provenance. It therefore leaves unresolved whether the hybrid scores support reliable model comparisons. A comparison on the same tasks, trajectories, and weights is needed to isolate this issue. Finally, the trajectories are largely one-shot structured explanations, and BFCL tool calls are not executed. The claimed diagnostic utility is not validated through actual error localization or debugging outcomes.

Overall, I recommend clear rejection in the current form. My main concern is the reliability and interpretability of the evidence supporting the benchmark, rather than the motivation. The numerical inconsistencies and unresolved generation and annotation effects require substantial clarification and validation. These observations do not, by themselves, establish data fabrication.

Questions for authors
  1. Clarification of conflicting results: Which data and annotation version was used for Table 8 and Figures 7–8? Please reconcile the Opus accuracy calculation (78/80 = 97.5% in Figure 7 versus 96.2% in Table 8), the five-dimensional score differences, and the source-level versus aggregate explanation scores. Can you provide the per-trajectory values, inclusion masks, and aggregation procedure that reproduce a single consistent set of results?

  2. Validation of generation provenance: How many retained trajectories actually received a gold answer or an expected function name during retry, and how are they distributed across models and task sources? How are the 80 trajectories with missing retry metadata handled? Do the main findings hold on uniformly gold-blind first-attempt outputs or a clean rerun? Simply removing failed-and-retried cases would introduce selection bias, so please distinguish that analysis from a gold-blind evaluation.

  3. Validation of the scoring instrument: On the same matched tasks and trajectories, with identical weights, how do the original hybrid labels compare with the adjudicated human labels? Please report dimension-level agreement, label distributions, and model-level score changes. In particular, how do the low agreement for specificity and trajectory explanation quality affect the conclusions? Can annotation uncertainty be incorporated into the reported comparisons, rather than only task-sampling uncertainty?

  4. Clarification of construct validation and scope: What numerical invariance margin and exact both-annotator decision rule were fixed before the held-out perturbation test? Please provide paired ratings to make the zero-width intervals and 16/16 pass result auditable, and identify the reported equal-weight and step-weighted sensitivity results. Given that tool calls are not executed, what existing evidence connects the diagnostic-utility scores to successful error localization or debugging? If no such evidence is available, please delimit the claims to ratings of emitted explanations.

Reviewer_yjqyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces AgentEx-Bench, a benchmark for measuring agent trajectory quality. It provides a unified representation of reasoning and tool-use outputs, covering 80 tasks from BFCL, GSM8K, HotpotQA, and StrategyQA and collecting 1,186 trajectories from 15 systems. The experimental results highlight the need to evaluate trajectory quality separately from final-answer correctness.

Critical strengths
  • The motivation is clear. Evaluating trajectory quality beyond final-answer correctness is valuable for understanding agent behavior and analyzing failures.
  • The experiments cover diverse tasks and models. The evaluation includes four task families and 15 systems, enabling comparisons of trajectory quality across different settings.
Critical weaknesses
  • While the five-dimensional rubric appears comprehensive, its complexity raises concerns about its applicability. The paper could further clarify how the framework can be extended to coding agents and search agents.
  • The reliability of the benchmark is also a concern. Its design appears to depend on too many uncontrolled factors, making it difficult to draw reliable diagnostic conclusions. The reported inter-annotator agreement is also relatively low.
  • Overall, the novelty appears limited, and the practical value of the benchmark remains unclear.
Questions for authors

see Weaknesses

Reviewer_3fxcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AgentEx-Bench, which is a benchmark for scoring observable post-hoc trajectories that LLMs emit alongside answers. Each key step is rated from 0 to 2 on five dimensions, which are combined into a weighted Explanation Score. The benchmark includes 80 tasks (20 each from BFCL, GSM8K, HotpotQA, and StrategyQA) and evaluates 15 hosted systems with over 1000 trajectories. Validation includes annotator agreement, a held-out perturbation study, and a 16-task matched panel. Among others, one of the paper’s main findings is that accuracy and explanation quality diverge.

Critical strengths

A major strength of the paper is the multi-dimensional aspect of evaluating steps in traces. The rubric allows the dimensions to disagree, and a step can be correct but unsupported, or incorrect but diagnostically useful because it exposes where the failure occurred. This is a useful distinction that a single correctness label would be unable to capture. The diagnostic-utility dimension feels like the most distinctive element relative to prior step-level evaluation.

The work covers a variety of tasks, by including BFCL into the more standard CoT benchmarks.

Critical weaknesses

Something that I consider a major weakness is that the paper seems unclear about the main artifact that it is measuring. From my understanding, the paper does not really evaluate an agentic workflow (e.g., BFCL tool calls are never executed) or even thoughts in a chain-of-thought generation. The object of measurement seems like a post-hoc, format-constrained explanation written alongside the answer. Since steps are produced on request in a constrained format, they say little about how agents behave across tool use and environment feedback. I think this should be more transparent.

I am not familiar with the latest literature in this area, but I found the review somewhat lacking. For instance, the paper seems to omit some important work on step-level evaluation of reasoning chains that targets nearly the same dimensions: ROSCOE (Golovneva et al., 2023), ReCEval (Prasad et al., 2023), and REVEAL (Jacovi et al., 2024). Some of these papers describe evaluation rubrics for steps at a substantial level of sophistication. I think the submission needs to do a better job of comparing against this line of work. The authors should clarify what the five-dimension rubric adds beyond these benchmarks.

I am not sure that the authors do enough in terms of evidence of usefulness for the benchmark. While the paper argues that trajectory quality aid auditing and diagnosis, there is no investigation on whether higher-scored trajectories help humans or tools localize errors.

There are some choices that seemed arbitrary to me, such as those in Section 4.3.

Questions for authors

Some comments and questions follow:

I could not fully appreciate how the work is different from some prior work, even though various attempts were made to explain this in various places. I see that this is also attempted in Appendix A. Maybe it is worthwhile to rethink how to distinguish this work from a long line of work in this general space.

This seems to be a trend, but a lot of content is currently in Appendices. I sympathize with this approach for a benchmark paper, in particular. However, this makes it tricky for reviewers with limited time – most reviewers these days – to assess the work efficiently.

I notice that there are no references to other work in the methodological sections of Section 3 and 4. As I mentioned previously, there are several other papers that assess the quality of steps in reasoning. I would expect the authors to provide some connection to prior work as they describe their metrics and broader approach. Please see my recommended references.

Some trajectories were regenerated after the model was shown the correct answer, so their steps may justify an answer the model was given rather than reflect how it solved the problem. Why not rerun generation without ever revealing gold answers? Until then, how can the explanation-quality scores and the model comparisons be trusted, particularly for the up to 218 affected trajectories?

Is there any evidence that high-scoring trajectories improve error localization by auditors?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.