acceptodds
Under review as a conference paper at ICLR 2027

Redundant or Necessary? A Benchmark for Detecting Redundant Steps in Agent Trajectories

Abstract

Computation cost is a critical challenge for LLM agent systems, as their long-horizon trajectories continuously accumulate computational overhead. While existing studies focus on improving the overall efficiency of agent systems, the efficiency of individual steps remains largely unexplored. In this paper, we propose and formulate a new research problem: redundant step detection, aiming to detect these steps that consume substantial computational resources while contributing little or nothing to task completion. To support this initiative, we introduce RedundancyBench, a new benchmark that contains 1,920 trajectories collected from 6 different LLMs with 3,613 step-level annotations. Using RedundancyBench, we develop and evaluate 6 frontier LLMs and Jev as redundant step detectors, summarizing their strengths and limitations. Our experiments reveal that the redundant step detection is still challenging even for SOTA models, such as Claude-Opus-5, particularly without prior information. Code and dataset in this paper are both available in https://anonymous.4open.science/r/RedundancyBench-E0CB.

Primary area:
datasets and benchmarks
Keywords:
Redundant Step Detection, Agent Trajectories, Task Efficiency, Agent Benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CyZROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces new problem named redundant step detection for detecting the steps not contributing to task completion. Redundancy is defined conceptually as a step whose change in task progress is lower than a threshold. It is divided into four types, abnormal (failures cause by external errors), duplicated (re-acquiring information that already obtained), incorrect (violating the task objective, policy or tool constraints), and exploratory (reasonable at that time, but not used). To measure the redundancy, the authors proposed RedundnacyBench, built from existing benchmark in three domains. Human experts and LLMs label the trajectories, and the final benchmark has 1920 trajectories and 3613 step-level annotations. The authors conducted experiments across 6 LLMs, additionally with Jev.

Critical strengths
  • The paper proposes a novel concept of research problem. Existing benchmarks only focus on token, latency, or failure attribution, but the authors introduce redundancy, focusing on step-level efficiency. It is clearly important aspect when evaluating the complicated long-horizon agentic tasks.
Critical weaknesses
  • Two proposed types are not proper. "Exploratory" is reasonable step at that moment, so the model cannot fix its action in advance. It can be detected after the task is completed. Additionally, "Abnormal" is an external error which agent cannot handle. The concept of redundancy is well motivated, but these two types seem to be inappropriate for redundancy check.
Questions for authors
  • Is there any possibility that the paper including LLM bias? The model generated the label and the model for tested are same family.

  • The authors only report the precision in StepDet, without the recall. Can false positive be computed in trajectories that all of them have at least one redundant step? Furthermore, there could be multiple valid evidence.

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors!(1/2)

Thank you for taking the time to review our work. We are pleased that you recognize the novelty of our proposed research problem and acknowledge its importance in evaluating complex, long-horizon agentic tasks. In the following we address your concerns and questions in the order they are asked.

W1: "Exploratory" is reasonable step at that moment, so the model cannot fix its action in advance. It can be detected after the task is completed. Additionally, "Abnormal" is an external error which agent cannot handle. The concept of redundancy is well motivated, but these two types seem to be inappropriate for redundancy check.

We would like to clarify that redundant-step detection is not intended to identify redundancy that model can fix it in advance, but design for post-hoc, offline diagnosis of agent efficiency bottlenecks. This diagnosis covers the entire agent execution pipeline, including LLM, agent harness and external environment. Therefore, we include both Abnormal Steps and Exploratory Steps as redundancy categories: the former represent execution overhead caused by external failures, while the latter capture unnecessary exploratory interactions arising from insufficient information by LLM, neither of which contributes to task completion.

For agent efficiency optimization, we believe that different redundancy categories call for different optimization strategies:

  • Abnormal Steps: Address failures in the external environment (e.g., unstable network connections) to prevent unnecessary retries.
  • Exploratory Steps: Provide relevant prior knowledge to narrow down the search space and eliminate unnecessary exploration.
  • Duplicated and Incorrect Steps: Directly intervene in agent execution to prevent repeated or erroneous actions. Thus, although some redundant steps (e.g., Exploratory Steps) may be reasonable given the information available at the time of execution, identifying them through post-hoc diagnosis can still reveal opportunities for improving future agent efficiency.

Q1: Is there any possibility that the paper including LLM bias? The model generated the label and the model for tested are same family.

We thank the reviewer for raising this concern. A straightforward way to assess potential LLM bias is to examine whether the performance rankings remain consistent when evaluated on a human-annotated dataset, where the gold labels are provided by human annotators. To investigate this, we compare performance rankings of 6 LLMs on the human-annotated and auto-annotated dataset. Specifically, for each evaluated LLM, we compute the average Prec@K across K={1,5,10}K=\{1, 5, 10\} on StepDet task and rank the models accordingly. We observe strong consistency between the two rankings. The only differences are minor rank swaps among the top four models, while Qwen3.8-27B and GPT-5.4-mini retain exactly the same positions. These results suggest that potential biases introduced by LLM-based annotation have a limited impact on the relative performance rankings of the evaluated models. More details about this experiment can be found in our response to Question 3.3 from Reviewer tuQb.

AuthorsOpenReview ↗

Rebuttal by Authors!(2/2)

Q2.1: The authors only report the precision in StepDet, without the recall. Can false positive be computed in trajectories that all of them have at least one redundant step?

We only report Precision@K for StepDet task as it measures reliability of detectors by penalizing false positives. Specifically, Precision@K and Recall@K are computed as follows:

\mathrm{Recall@K}=\frac{1}{N}\sum_{i=1}^N\frac{P_i\cap G_i}{|G_i|},$$ where $N$ is the number of trajectories, $P_i$ and $G_i$ denote the set of predicted and ground-truth triplets (step location, category, and supporting evidence) for $i$-th trajectory. A **high Precision@K** indicates a **low false-positive rate** (i.e., necessary steps incorrectly identified as redundant), whereas a high Recall@K indicates a low false-negative rate (i.e., fewer ground-truth redundant steps are missed). For efficiency optimization, false positive predictions are catastrophic than missed redundant steps: removing a necessary step that is incorrectly identified as redundant may lead to task failure. Here we also report Recall@K for StepDet task, as following: | Model | R@1 | R@5 | R@10 | Mean | |---|---:|---:|---:|---:| | GPT-5.4-mini | 2.98 | 5.29 | 5.40 | 4.56 | | Qwen3.8-27B | 14.87 | 20.26 | 20.52 | 18.55 | | GLM-5.3 | 29.93 | 40.54 | 40.66 | 37.04 | | DeepSeek-V4.1-flash | 52.57 | 63.03 | 63.36 | 59.65 | | GPT-5.4 | 44.54 | 60.85 | 61.32 | 55.57 | | Claude-Opus-5 | 55.22 | 76.25 | 76.79 | 69.42 | As shown in the table, most LLMs struggle to achieve high recall, even with a relatively large $K$ (e.g., $K=10$), indicating that they still miss a substantial proportion of ground-truth redundant triplets. **Q2.2: Furthermore, there could be multiple valid evidence.** Yes, a redundant step may be supported by multiple valid evidence. To address this ambiguity, we explicitly define evidence in our annotation guidelines as **the smallest sufficient event or combination of events needed to justify a redundancy label** (see detail in Appendix A). We also provide category-specific explanations to guide annotators in identifying appropriate evidence. In practice, this definition has proven sufficient for annotators to reach a consensus during the annotation process.
Reviewer_bWKsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RedundancyBench, a benchmark for detecting redundant steps in LLM agent trajectories. The authors define four types of redundant steps: abnormal, duplicated, incorrect, and exploratory. They collect trajectories from existing agent benchmarks and use human experts and LLMs to annotate redundant steps.

Critical strengths

The paper focuses on an important and common problem in real agent tasks. Agents often perform unnecessary tool calls or repeat actions, which can increase execution cost. I think identifying these steps is useful for understanding and improving agent efficiency.

Critical weaknesses

In real agent tasks, whether a step is redundant often depends on the task and context. I think the paper does not clearly justify why these four categories are used or how they are distinguished. Some categories may overlap, and some steps may actually be necessary. This makes me question how well the benchmark reflects real agent behavior and whether its findings can be applied to real-world tasks.

The paper uses multiple LLMs for annotation and only keeps trajectories when the models agree. I think this may exclude some difficult cases, especially when redundancy is not clear. The paper also shows different distributions of redundancy types between human and automated annotations. More analysis is needed to understand how this filtering affects the final dataset and the evaluation results.

The paper mainly uses precision to evaluate redundant step detection, but all trajectories in the benchmark contain redundant steps. I think this setting is limited because real agent trajectories may not always contain redundant steps. The paper does not show how the models perform in these cases, especially when necessary steps are incorrectly identified as redundant.

I also question whether the efficiency experiment really shows the benefit of redundant step detection. The paper removes steps from previous trajectories and uses them to guide future executions. This is different from identifying and avoiding unnecessary steps during the original execution. The lower token consumption may come from reusing previous experience rather than from the redundancy detection itself. It is unclear how much this result supports improving efficiency in real agent tasks.

Questions for authors
  • Could the authors explain why these four categories are used to define redundancy? Some categories may overlap, and some steps may be necessary depending on the task. How are these cases handled in the benchmark?
  • In real agent tasks, exploration may be necessary to find the correct solution. How does the paper distinguish between unnecessary steps and reasonable exploration? Does the current definition consider the information available to the agent at that time?
  • The paper only keeps trajectories when multiple LLM annotators agree. I wonder how this filtering affects the final dataset. Could the authors provide more analysis of the excluded cases and the differences between human and automated annotations?
  • The benchmark only includes trajectories with redundant steps. How would the models perform on trajectories without redundant steps? Could the authors explain how the current evaluation reflects the risk of incorrectly identifying necessary steps?
  • In Table 2, removing domain policies improves StepCls accuracy but reduces StepDet precision. What causes this difference? Could the authors provide more analysis of how domain policies affect these two tasks?
  • The efficiency experiment uses simplified historical trajectories to guide future executions. I wonder how much of the token reduction comes from redundancy detection rather than reusing previous experience. How would these findings apply to improving efficiency during actual agent execution?
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors!(1/4)

Thank you for your careful review and constructive feedback. We appreciate your recognition that unnecessary tool calls and repeated actions are a common issue in agent tasks, and that identifying them is useful for understanding and improving agent efficiency. In the following, we address your questions and concerns in the order of being proposed.

W1.1&Q1.1: I think the paper does not clearly justify why these four categories are used or how they are distinguished.Could the authors explain why these four categories are used to define redundancy?

We derive these four categories from manual analysis of agent trajectories. By examining task objectives and agents' execution paths, we identify four recurring inefficient patterns that make no or negligible progress toward task completion (see our response to Reviewer 1's W1&Q1 for detailed definitions).

These four categories represent different reasons why an agent step fails to make meaningful progress: execution failure, repeated information, constraint violations, and exploration without meaningful task progress. They also serve as observable manifestations of the formal progress-gain criterion: Abnormal and Duplicated correspond to Δ=0\Delta = 0, Incorrect to Δ≤0\Delta \leq 0, and Exploratory to Δ≈0\Delta \approx 0. Thus, the four categories are empirically grounded in agent trajectories while operationalizing the abstract progress-gain criterion into concrete, observable patterns for annotation. We will add an explicit discussion of this mapping in the revised manuscript.

W1.2&Q1.2: Some categories may overlap, and some steps may actually be necessary. This makes me question how well the benchmark reflects real agent behavior and whether its findings can be applied to real-world tasks.How are these cases handled in the benchmark?

We handle these cases through explicit annotation guidelines:

  • Overlapping categories. Each redundant step is assigned exactly one category. If a step exhibits characteristics of multiple categories, annotators follow the priority order specified in the annotation guidelines: Abnormal → Duplicated → Incorrect → Exploratory. This order follows increasing annotation difficulty.

  • Necessary steps. We determine whether a step is necessary by assessing whether it makes an actual contribution to the local objective. Annotators are provided with the task objective and the complete trajectory, and must consider the information available to the agent at the time of execution. We also require annotators to identify supporting evidence steps for each redundant exploratory step. These evidence steps correspond to subsequent events that demonstrate the step's lack of necessary contribution to task completion, such as identifying the target, locating the fault, establishing a feasible solution, or returning to the necessary task path. If no supporting evidence can be identified, the step is not labeled as redundant.

Therefore, it considers the task objective, the information available to the agent, and the supporting evidence in the trajectory when determining whether a step is redundant.

W2.1&Q3.1: The paper uses multiple LLMs for annotation and only keeps trajectories when the models agree. I think this may exclude some difficult cases, especially when redundancy is not clear.Could the authors provide more analysis of the excluded cases?

Yes, the automatic annotation pipeline applies consensus filtering among 3 LLM annotators, which may discard ambiguous but potentially hard cases. A possible solution is to further refine these ambiguous cases by human experts. Specifically, these ambiguous cases can be independently annotated by three human experts, followed by cross-verification. However, such human refinement involves substantial annotation effort, as the consensus filter discards 2,693 trajectories containing 12,200 ambiguous annotations. Therefore, we leave the systematic refinement of these ambiguous cases as future work. We further analyze the differences between retained and excluded cases from three perspectives: redundant-step number, trajectory complexity, and redundancy-type distribution. A detailed analysis is provided in our response to Reviewer 1's Q4.2.

AuthorsOpenReview ↗

Rebuttal by Authors!(2/4)

W2.2&Q3.2: The paper also shows different distributions of redundancy types between human and automated annotations. Could the authors provide more analysis of the differences between human and automated annotations?

In Section 3.3, we provide a statistical comparison between the human-annotated and automatically annotated datasets. Here, we further analyze their differences by comparing the performance rankings of six LLMs on the StepDet task using the two datasets. Specifically, for each LLM, we compute the average Prec@K across K={1,5,10}K=\{1, 5, 10\} and rank the models accordingly.

  • On the human-annotated subset (189 trajectories), the ranking is: DeepSeek-V4.1-flash > Claude-Opus-5 > GLM-5.3 > GPT-5.4 > Qwen3.8-27B > GPT-5.4-mini.
  • On the auto-annotated subset (1,731 trajectories), the ranking is: Claude-Opus-5 > DeepSeek-V4.1-flash > GPT-5.4 > GLM-5.3 > Qwen3.8-27B > GPT-5.4-mini.

Overall, the two rankings exhibit strong consistency. The only differences are minor rank swaps among the top four models, while Qwen3.8-27B and GPT-5.4-mini retain exactly the same positions. These results suggest that, despite differences in redundancy-type distributions, the automatically annotated dataset largely preserves the relative performance trends observed on the human-annotated dataset, supporting its reliability for comparative model evaluation.

W3&Q4.1: The paper mainly uses precision to evaluate redundant step detection, but all trajectories in the benchmark contain redundant steps. The paper does not show how the models perform in these cases, especially when necessary steps are incorrectly identified as redundant.How would the models perform on trajectories without redundant steps?

Yes, all trajectories in the benchmark contain redundant steps, as our task settings focus on identifying step-level redundacy. Specifically, both StepCls and EvidenceRet assume that the locations of redundant steps are provided as input when predicting redundancy categories or retrieving supporting evidence. Therefore, trajectories without redundant steps fall outside the scope of the current task settings.

To further investigate false-positive risks, we conduct an additional trajectory-level redundancy detection experiment using 60 trajectories across three domains, including 30 trajectories without redundant steps. The results suggest that LLMs can identify redundancy at the trajectory level with promising performance, even without prior knowledge of redundant step locations. Detailed experiment results and analysis can be found in the response to Reviewer 1's W3.1.

W4.1: The paper removes steps from previous trajectories and uses them to guide future executions. This is different from identifying and avoiding unnecessary steps during the original execution.

We would like to clarify that redundant-step detection is not intended for online identification and intervention of redundancy during execution, but rather for post-hoc, offline diagnosis of agent efficiency bottlenecks. This diagnosis covers the entire agent execution pipeline, including LLM, agent harness and external environment. Accordingly, we consider 4 categories of redundant steps to capture different source: Abnormal Step (failure due to external environment), Exploratory Step (unnecessary exploration arising from insufficient information by LLM), Duplicated Step (duplicated information acquisition for LLM) and Incorrect Step (incorrect tool calls).

AuthorsOpenReview ↗

Rebuttal by Authors!(3/4)

W4.2&Q6.1: The lower token consumption may come from reusing previous experience rather than from the redundancy detection itself. I wonder how much of the token reduction comes from redundancy detection rather than reusing previous experience.

We would like to clarify the experiment setting and results in Section 4.3, which investigates whether removing redundant steps from historical trajectories can improve agent efficiency in subsequent task executions.

  • Experiment settings. (1) No Experience, without previous experience; (2) Original, using previous experience through the original trajectory; (3) LLM, using the original trajectory with redundant steps identified by an LLM detector removed; and (4) Gold, using the original trajectory with redundant steps identified by ground-truth annotations removed. For the last three settings, the resulting trajectories are standardized into a uniform format and then converted into execution workflows, which are provided to the agent as guidance. We evaluate 10 tasks per domain, each repeated 3 times to account for the stochasticity of LLM outputs, measuring task success and token consumption.
  • Experiment results. To isolate the additional benefits of redundancy removal beyond experience reuse, we compare the two pruning settings (LLM and Gold) against Original. Compared with Original, Gold further reduces token consumption by 10.46%, 1.64%, and 8.80% across the three domains, respectively. Meanwhile, task success is maintained or improved: Original achieves 26/30, 28/30, and 28/30, while Gold achieves 26/30, 29/30, and 29/30. These results demonstrate that redundancy removal provides additional token reduction beyond experience reuse, while largely preserving task performance.

Q2: How does the paper distinguish between unnecessary steps and reasonable exploration? Does the current definition consider the information available to the agent at that time?

The key distinction lies in whether an exploratory step makes a meaningful contribution to the local objective. An unnecessary step may eliminate a candidate or possible cause, narrow the search space, or explore a particular branch, but it does not provide necessary support for achieving the objective. During human annotation phase, annotators are provided with the task purpose alongside the trajectory to help them understand the underlying task objective. They are also required to consider the information available to the agent at that time and provide supporting evidence steps for each redundant exploration. These evidence steps correspond to key subsequent event that identify the target, locate the fault, establish a viable solution, or return to the necessary task path. If the supporting evidence cannot be clearly identified, the step is not labeled as redundant.

Q4.2: Could the authors explain how the current evaluation reflects the risk of incorrectly identifying necessary steps?

We only report Precision@K for StepDet task as it measures reliability of detectors by penalizing false positives. Specifically, Precision@K is computed as follows: Prec@K=1N∑i=1NPi∩Gimin⁡(K,∣Pi∣),\mathrm{Prec@K}=\frac{1}{N}\sum_{i=1}^N\frac{P_i\cap G_i}{\min(K, |P_i|)}, where NN is the number of trajectories, PiP_i and GiG_i denote the set of predicted and ground-truth triplets (step location, category, and supporting evidence) for ii-th trajectory. A high Precision@K indicates a low false-positive rate (i.e., necessary steps incorrectly identified as redundant). For efficiency optimization, false positive predictions are catastrophic than missed redundant steps: removing a necessary step that is incorrectly identified as redundant may lead to task failure.

To offer a complete performance picture, here we also report Recall@K for StepDet task in our response to Reviewer 1's W3.2

AuthorsOpenReview ↗

Rebuttal by Authors!(4/4)

Q5: In Table 2, removing domain policies improves StepCls accuracy but reduces StepDet precision. What causes this difference? Could the authors provide more analysis of how domain policies affect these two tasks?

We thank the reviewer for pointing out this detail. The difference primarily stems from the distinct requirements of the two tasks. StepCls classifies each redundant step into one of four categories, whereas StepDet requires simultaneously localizing all redundant steps, classifying their categories, and retrieving the corresponding evidence steps. Our further analysis reveals that domain policies provide additional prior knowledge that benefits redundant-step localization and evidence retrieval. However, their impact on redundancy-type classification is mixed:

  • Positive effect: Domain policies provide information about policy constraints, helping LLM detectors identify whether an action violates these constraints and thus improving the classification of Incorrect Steps.
  • Negative effect: Domain policies also introduce event-specific information and prescribed action workflows for handling particular events. This additional information can bias the detector judgment of the local task objective, leading to the misclassification of Exploratory Steps as Incorrect or Duplicated Steps.

Since Exploratory Steps account for 63.5% of the dataset, such misclassifications have a substantial impact on overall StepCls accuracy. Consequently, removing domain policies improves StepCls accuracy by reducing this source of classification bias, while simultaneously weakening the localization and evidence retrieval capabilities required by StepDet.

Q6.2 How would these findings from the efficiency optimization experiment be applied to improving efficiency during actual agent execution?

In Section 4.3, we investigate removing redundant steps from historical trajectories as a preliminary strategy for improving efficiency in subsequent agent executions. Beyond this approach, we believe that different redundancy categories call for different optimization strategies:

  • Abnormal Steps: Improve the reliability of external environments (e.g., by addressing unstable network connections) to prevent unnecessary retries.
  • Exploratory Steps: Provide relevant prior knowledge to narrow down the search space and reduce unnecessary exploration.
  • Duplicated and Incorrect Steps: Directly intervene in agent execution to prevent repeated or erroneous actions.

Developing effective optimization methods based on the above strategies and integrating them into actual agent execution is an interesting direction that we leave for future work.

Reviewer_tuQbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper formulates the redundant step detection task for LLM‑agent trajectories, i.e., given a successfully completed agent trajectory, identify tool‑call steps that consume computation but bring little‑to‑no contribution toward task completion, classify them into four redundancy categories, and retrieve supporting evidence steps. To support this research, the authors construct RedundancyBench, which is built through a two‑phase pipeline. Three progressive evaluation settings are designed, gradually removing ground‑truth prior information. The authors benchmark six LLMs plus Jev, conduct diagnostic analyses across redundancy types, domains and trajectory lengths, and carry out preliminary downstream trials that show pruning annotated redundant steps can reduce token consumption in subsequent agent runs.

Critical strengths
  1. Step‑level efficiency diagnosis for agent trajectories fills a gap that existing agent evaluation mostly focuses on coarse‑grained success rate, latency or total token cost, while fine‑grained assessment of individual step utility is largely missing.
  2. Annotations record redundant step location, fine‑grained category, supporting evidence and rationale. The three task settings that progressively strip prior information help localize model failure modes (classification error, evidence retrieval failure, step localization failure).
  3. This work combines human expert labeling and multi‑LLM consensus scaling, and provides detailed quality validation against human gold labels in Appendix E. Preliminary downstream experiments demonstrate the potential practical value of redundancy detection for agent efficiency optimization.
Critical weaknesses
  1. The paper introduces a formal interaction framework, task‑progress function P and threshold‑based definition of redundant steps (Δ<δ\Delta < \delta), yet this formalism is never used in annotation or evaluation. All practical labeling relies on heuristic categorical rules instead of the formal progress‑gain criterion.
  2. Low performance under exact‑match metrics may stem from legitimate alternative judgments instead of genuine detection failure. It would be stronger to complement annotation‑based metrics with downstream outcome metrics: e.g., prune steps predicted redundant by detectors, then compare task success rate and token overhead of revised trajectories against original ones.
  3. Benchmark representativeness and ground‑truth reliability concerns.
    • The dataset excludes all redundancy‑free trajectories, disabling proper evaluation of false‑positive risks — a critical hazard for redundancy pruning, where mis‑marking necessary steps as redundant can break agent execution. StepDet only reports Precision@K without recall / F1, offering incomplete performance picture.
    • Most annotations come from consensus of three LLMs, while the benchmark evaluates stronger frontier models. Aggressive consensus filtering discards a large fraction of raw trajectories; end‑to‑end recall against human gold is only 26.2%. Stronger models may discover valid redundant steps or alternative evidence missed by the LLM annotators, but exact‑match evaluation will penalize such correct predictions. The benchmark may be biased toward easy cases where annotators agree, filtering ambiguous hard instances.
  4. The paper mainly delivers benchmark and model ranking results. There is no proposed algorithm for improving redundant‑step detection or agent efficiency. Section 4.3 is only a small preliminary proof‑of‑concept trial, with limited actionable insights.
Questions for authors
  1. How do the four empirical redundancy categories map back to the formal task‑progress‑gain definition with threshold δ\delta? Why is the formal criterion not operationalized during annotation?
  2. Who are the target users of this redundancy detector, and at which stage of an agent pipeline would this tool be deployed?
  3. What motivates the selection of the three LLM annotators, given that evaluated models include stronger systems such as Claude‑Opus‑5? Can you perform human adjudication on cases where high‑performing detectors disagree with automatic labels, and compare model rankings between human‑annotated subset vs auto‑annotated subset?
  4. What are primary sources of annotator disagreement (annotator capability limits, guideline ambiguity, intrinsically ambiguous cases)? Could you analyze whether discarded trajectories systematically differ from retained ones in task difficulty, trajectory complexity, or redundancy‑type distribution?
  5. Exploratory is the dominant redundancy class in the benchmark. How do annotators reliably separate legitimate trial‑and‑error exploration (required under incomplete agent knowledge) from genuinely redundant exploratory steps?
5 comments
AuthorsOpenReview ↗

Rebuttal by Authors!(1/5)

Thank you for the detailed and constructive comments. We are pleased that you recognize our work as filling an important gap in the fine-grained assessment of individual steps. In the following, we address your questions and concerns in the order of being proposed.

W1&Q1: All practical labeling relies on heuristic categorical rules instead of the formal progress‑gain criterion. Why is the formal criterion not operationalized during annotation?

Thank you for raising this concern. We would like to clarify this point from two aspects:

  • Limitation of progress‑gain criterion. The formal criterion is the idealized definition of redundancy: a step is redundant if its progress gain Δ=P(scur,sgoal)−P(sbe,sgoal)Δ = P(s_{cur}, s_{goal}) − P(s_{be}, s_{goal}) falls below a small threshold δδ. However, directly computing P in realistic environments is generally intractable as the input state ss lacks computable representations, making the progress gain difficult to quantify explicitly.

  • Connection between the annotation and criterion. Although P cannot be directly computed, we operationalize the progress-gain criterion through 4 redundancy categories, each corresponding to an observable pattern of progress gain:

    • Abnormal: Δ = 0 (execution failure; the progress is only made by a later retry step).
    • Duplicated: Δ = 0 (zero information gain relative to already-obtained information).
    • Incorrect: Δ ≤ 0 (violates task objective, policy, or tool constraints and may require rollback).
    • Exploratory: Δ ≈ 0 (eliminates a candidate but is not used in the successful path).

    These 4 categories translate the abstract progress-gain criterion into concrete, observable features, enabling annotators to assess redundancy without intractable progress function P.

We will add an explicit discussion of this mapping in the revised manuscript.

W2: It would be stronger to complement annotation‑based metrics with downstream outcome metrics: e.g., prune steps predicted redundant by detectors, then compare task success rate and token overhead of revised trajectories against original ones.

This is a very interesting insight. Intuitively, a step is redundant if removing it reduces computational overhead without compromising task success. However, our early exploration reveals two challenges in operationalizing this outcome-based criterion:

  • Misclassification. Simply removing a step from a completed trajectory will lead to serve misclassification. For example, a necessary step may retrieve information from the environment state that is used as arguments for subsequently tool calls. Removing this step alone does not affect task success because the subsequent tool calls already incorporate the retrieved information. Consequently, this step removal may incorrectly classify necessary steps as redundant, as it does not account for information already acquired and propagated throughout the trajectory.
  • Computation Cost. A possible alternative is counterfactual re-execution, in which the target step and all subsequent steps are removed, and the remaining execution is regenerated from the preceding point. This approach can mitigate the misclassification issue by re-execution. However, due to the stochasticity of LLMs, multiple rollouts are required to reliably assess the impact of removing each step. For example, evaluating a 20-step trajectory may require hundreds of executions, resulting in substantial computational costs.

In this work, we adopt the progress-gain criterion, which avoids the above limitations for direct step removal and counterfactual re-execution. We consider this a more practical and scalable approach for benchmark construction, while outcome-based evaluation remains a valuable direction for future work.

AuthorsOpenReview ↗

Rebuttal by Authors!(2/5)

W3.1: The dataset excludes all redundancy‑free trajectories, disabling proper evaluation of false‑positive risks.

Yes, our origin dataset includes only trajectories containing at least one redundant step, as our task settings focus on identifying step-level redundacy. Specifically, both StepCls and EvidenceRet assume that the locations of redundant steps are provided as input when predicting redundancy categories or retrieving supporting evidence. Therefore, trajectories without redundant steps fall outside the scope of the current task settings.

To further investigate false-positive risks, we conduct an additional trajectory-level redundancy detection experiment using 60 trajectories across three domains. These trajectories are evenly divided into two categories: 30 trajectories containing at least one redundant step are labeled as positive (label 1), while the remaining 30 trajectories containing no redundant steps are labeled as negative (label 0). Unlike the original step-level tasks, in this setting, detectors are provided with no prior information (e.g., redundant step locations or supporting evidence) and are required to classify each trajectory as either positive or negative. The results are summarized below:

ModelAccuracy (%)
GPT-5.4-mini71.67
Qwen3.8-27B55.00
GLM-5.368.33
DeepSeek-V4.1-flash86.67
GPT-5.491.67
Claude-Opus-593.33

The results show that most LLMs achieve high accuracy, with GPT-5.4 and Claude-Opus-5 achieving over 90%. These findings suggest that LLMs can identify redundancy at the trajectory level with promising performance, even without prior knowledge of redundant step locations.

W3.2: StepDet only reports Precision@K without recall / F1, offering incomplete performance picture.

We only report Precision@K for StepDet task as it measures reliability of detectors by penalizing false positives. Specifically, Precision@K and Recall@K are computed as follows:

\mathrm{Recall@K}=\frac{1}{N}\sum_{i=1}^N\frac{P_i\cap G_i}{|G_i|},$$ where $N$ is the number of trajectories, $P_i$ and $G_i$ denote the set of predicted and ground-truth triplets (step location, category, and supporting evidence) for $i$-th trajectory. A high Precision@K indicates a low false-positive rate (i.e., necessary steps incorrectly identified as redundant), whereas a high Recall@K indicates a low false-negative rate (i.e., fewer ground-truth redundant steps are missed). For efficiency optimization, false positive predictions are catastrophic than missed redundant steps: removing a necessary step that is incorrectly identified as redundant may lead to task failure. To offer a complete performance picture, here we also report Recall@K for StepDet task, as following: | Model | R@1 | R@5 | R@10 | Mean | |---|---:|---:|---:|---:| | GPT-5.4-mini | 2.98 | 5.29 | 5.40 | 4.56 | | Qwen3.8-27B | 14.87 | 20.26 | 20.52 | 18.55 | | GLM-5.3 | 29.93 | 40.54 | 40.66 | 37.04 | | DeepSeek-V4.1-flash | 52.57 | 63.03 | 63.36 | 59.65 | | GPT-5.4 | 44.54 | 60.85 | 61.32 | 55.57 | | Claude-Opus-5 | 55.22 | 76.25 | 76.79 | 69.42 | As shown in the table, most LLMs struggle to achieve high recall, even with a relatively large $K$ (e.g., $K=10$), indicating that they still miss a substantial proportion of ground-truth redundant triplets. **W3.3: Most annotations come from consensus of three LLMs, which may be biased toward easy cases where annotators agree, filtering ambiguous hard instances.** Yes, the automatic annotation pipeline applies consensus filtering among 3 LLM annotators, which may discard ambiguous but potentially hard cases. A possible solution is to further refine these ambiguous cases by human experts. Specifically, these ambiguous cases can be independently annotated by three human experts, followed by cross-verification. However, such human refinement involves substantial annotation effort, as the consensus filter discards 2,693 trajectories containing 12,200 ambiguous annotations. Therefore, we leave the systematic refinement of these ambiguous cases as future work.
AuthorsOpenReview ↗

Rebuttal by Authors!(3/5)

W3.4: In the Appendix E, the end‑to‑end recall of consensus filtering against human gold is only 26.2%.

We would like to clarify that the 26.2% end-to-end recall does not equal to poor annotation quality. In Appendix E, we evaluate the reliability of automatic annotation pipeline by applying it to a human-annotated dataset. This evaluation considers two different scopes:

  • Conditional evaluation: Precision (P), Recall (R), R(id), and R(id+type) are computed only on the retained trajectories, measuring the alignment between automatic and human annotations within the retained subset.
  • End-to-end evaluation: End-to-end recall (E2E R) is computed over all trajectories, including discarded and retained trajectories. All redundant steps in discarded trajectories are counted as missed annotations.

As shown in Table 8, the LLM annotators combined with consensus filtering achieve 98.9% precision and 83.2% recall on the retained trajectories, demonstrating strong alignment with human annotations. In contrast, the end-to-end recall is only 26.2%, primarily because redundant steps in discarded trajectories are missed. This difference reflects the conservative nature of consensus filtering rather than poor annotation quality. The strategy prioritizes annotation reliability by discarding uncertain cases, thereby achieving high precision and recall on retained trajectories at the expense of overall annotation coverage.

W4: There is no proposed algorithm for improving redundant‑step detection or agent efficiency.

We would like to clarify that our primary contribution is not to propose a new algorithm, but rather to introduce a new research problem and establish a systematic benchmark:

  • A new research problem. We introduce the problem of redundant-step detection, which aims to identify low-value steps in agent trajectories.
  • A new benchmark. To support this initiative, we introduce RedundancyBench, a new benchmark comprising 1,920 trajectories collected from 6 different LLMs, with 3,613 step-level annotations.
  • Comprehensive analysis. We evaluate six frontier LLMs and Jev on 3 different task settings, providing insights into their strengths and limitations.

Developing more effective detection algorithm or efficiency optimization strategy is an interesting problem, but which lies beyond the scope of this paper.

Q2: Who are the target users of this redundancy detector, and at which stage of an agent pipeline would this tool be deployed?

The target users are agent evaluation engineers and agent developers. For these users, the redundancy detector is primarily intended for post-hoc, offline diagnosis, where completed trajectories are analyzed to identify redundant steps and characterize patterns of inefficiency. These diagnostic insights can then inform downstream optimization efforts, helping developers refine agent behaviors, improve execution efficiency, and reduce unnecessary computational overhead.

Q3.1: What motivates the selection of the three LLM annotators?

In the automation scaling phase, we select DeepSeek-V4-Flash, GLM-5.3-Flash, and GPT-5.6-Luna as annotators to facilitate large-scale annotation. The primary consideration is the annotation cost. To improve human alignment, we conduct extensive explorations with different annotation configurations, including few-shot example selection, useless information filtering, annotation guideline refinement, and output format design. Exploring these configurations incurs substantial annotation costs. Based on these explorations, we identify an effective configuration for automatic annotation, consisting of 9 few-shot examples, triplet-formatted outputs, and a consensus-based filtering strategy. This configuration achieves 98.9% precision and 83.2% recall against human annotations (see Appendix E for details).

Q3.2: Can you perform human adjudication on cases where high‑performing detectors disagree with automatic labels?

We perform human adjudication on 30 cases, which are random sampled from the error set of Claude-Opus-5 on the StepDet task. We observe that the failure focuses on two sources: redundancy-type misclassification and evidence-step retrieval errors.

  • Category Misclassification. Claude-Opus-5 struggles to distinguish between Incorrect Steps and Exploratory Steps in some cases. Human adjudication reveals that these cases are inherently borderline, making them challenging even for human experts.
  • Evidence-Step Retrieval Errors. In other cases, Claude-Opus-5 correctly identifies redundant steps and assigns their categories but retrieves incorrect evidence steps. Human adjudication shows that these errors can be readily resolved by human experts following the annotation guidelines. Specifically, Claude-Opus-5 tends to violate the minimal evidence principle by retrieving more steps than necessary as supporting evidence or failing to follow the prescribed format for evidence steps.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response!

We sincerely thank you for your time and expertise, and we are grateful to all reviewers for their valuable feedback.We are encouraged that all three reviewers recognize the importance of step-level redundancy analysis in long-horizon agent tasks (reviewer tuQb bWKs CyZR); that step-level redundancy detection addresses an underexplored gap in existing agent evaluation(Reviewers tuQb CyZR); and that RedundancyBench provides fine-grained annotations, systematic human–LLM validation, and preliminary evidence of downstream utility (Reviewer tuQb).

We appreciate the valuable feedback, and take this opportunity to highlight a few important enhancements we have made during the rebuttal period. Details and additional experiments can be found under individual comments to the reviewers.

Redundant Step Definition

  • Operationalization: Reviewers tuQb and bWKs request how the progress-gain criterion is operationalized during annotation. We response with the limitations of directly computing progress gain and explain how the 4 redundancy categories map the formal criterion to observable patterns.
  • Overlap: Reviewer bWKs asks how to handle overlapping categories. We response with the priority order specified in the annotation guidelines.
  • Separation: Reviewer tuQb requests how to distinguish legitimate trial-and-error exploration from redundant exploratory steps. We clarify the distinction based on whether an exploratory step makes a meaningful contribution to the local objective, and explain the annotation principles for labeling a exploratory step.
  • Scope: Reviewer CyZR questions whether Exploratory and Abnormal Steps should be included in redundancy detection. We clarify that our benchmark targets post-hoc, offline diagnosis rather than real-time prevention, covering the entire agent execution pipeline.

Automatic Annotation Pipeline

  • LLM annotator:
    1. Reviewer tuQb asks why we selected three LLM annotators. We clarify that this choice was motivated by annotation costs, given the extensive exploration required across different model configurations.
    2. Reviewer tuQb asks about the primary sources of disagreement among LLM annotators. We provide a further analysis of disagreement cases and find that disagreements mainly stem from model-specific preferences, borderline cases, and limitations in model capabilities.
  • Consensus filter:
    1. Reviewer tuQb and bWKs raises concerns that consensus filtering may discard ambiguous but challenging cases. We acknowledge this limitation and highlight the substantial human annotation costs of refining discarded cases, leaving systematic human adjudication for future work.
    2. Reviewer tuQb questions the low end-to-end recall (26.2%). We clarify the distinction between conditional and end-to-end evaluation, attributing the low recall primarily to conservative filtering rather than poor annotation quality.
    3. Reviewers tuQb and bWKs ask how retained and discarded trajectories differ. we response with a futher analysis in these trajectories. We find that discarded trajectories generally contain more redundant steps and tool calls, with substantial redundancy-type distribution differences.
  • Bias: Reviewer CyZR request the possible LLM bias, as the model generated the label and the model for tested are same family partly. We response with an experiment by comparing performance ranking of 6 LLMs on the human-annotated and auto-annotated dataset. The results suggest that potential biases introduced by LLM-based annotation have a limited impact on the relative performance rankings of the evaluated models.

Evaluation

  • False positive risk: Reviewer tuQb, bWKs and CyZR ask how false-positive risks are evaluated. We response with introduction of Prec@K metric and clarify how it measures detector reliability by penalizing false positives.
  • Recall@K: Reviewer tuQb, bWKs and CyZR request a more complete performance evaluation. We response with Recall@K results for 6 LLMs on StepDet task. Results show that most LLMs struggle to achieve high recall (a low false-negative rate), even with a large KK value.
  • Task-level redundancy detection: Reviewers tuQb and bWKs raise concerns about the lack of redundancy-free trajectories. We response with clarification that such trajectories fall outside the scope of the current step-level detection settings. Additionally, we conduct a trajectory-level detection experiment, classifying a trajectory as postive or negative (with or without redundant steps). Results demonstrate promising performance of LLMs in identifying redundancy at the trajectory level.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.