acceptodds
Under review as a conference paper at ICLR 2027

TraceDance: An Automated System for Building Agent Behavior Benchmarks from Real-World Agent Deployment Traces

Abstract

An agent can complete a task while exhibiting undesirable behavior during execution. Developers need tests for the specific behaviors encountered in deployment, beyond fixed benchmark suites. We present TraceDance, an agent system that constructs targeted benchmarks from deployment traces for user-specified undesirable behaviors. For efficient construction, Anchor-and-Confirm combines programmable retrieval with candidate-level confirmation by a Flash large language model (LLM), while the Anchor Synthesis Loop generates and revises specifications for custom behaviors. The benchmarks use decision-point continuation to evaluate an LLM's next turn at a recorded decision point with a behavior-specific rubric, without a reference answer or environment replay. Experiments in coding and general tool use draw on 252,557 sessions and produce 107 benchmarks with 4,125 instances, fulfilling 95.3% of build-target requests. Both human annotators confirm the requested behavior in 84% of sampled instances, and the automated grader's agreement with human pass/fail judgments is comparable to that between the annotators. Nine frontier LLMs achieve a mean pass rate of only 26.7%, showing that they still struggle to respond appropriately at the evaluated decision points. Analysis across behavior-specific benchmarks further reveals weaknesses in how current LLMs behave as agents. By turning deployment problems into targeted benchmarks, TraceDance could serve as a key component of the recursive self-improvement (RSI) loop.

Primary area:
datasets and benchmarks
Keywords:
Agent Evaluation, Automated Benchmark Construction, LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qH4AOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

TraceDance constructs behavior-specific agent benchmarks from real deployment traces. Given a natural-language behavior query, it selects or synthesizes an executable retrieval anchor, a confirmation criterion, a decision-point cut, and a grading rubric. Anchor-and-Confirm combines programmatic retrieval with LLM-based validation, while an Anchor Synthesis Loop revises specifications for new behaviors. Evaluated models generate one assistant turn from the recorded prefix without environment execution. Experiments use 252,557 sessions from two agent harnesses, producing 107 benchmarks with 4,125 instances and evaluating nine models.

Critical strengths
  1. Relevant problem and practical design. Task completion can conceal undesirable execution behavior. Reusing deployment prefixes enables targeted evaluation when private repositories, tools, or accounts make environment reconstruction impractical.
  2. Coherent construction framework. The separation of retrieval, confirmation, context construction, and grading is sensible. The action, failure, and claim frames clarify the decision being evaluated, and rubrics accommodate multiple appropriate responses.
Critical weaknesses

W1. Strict rubric failure is not equivalent to exhibiting the target behavior

The central measurement concern is that a response can avoid the undesirable behavior and still fail the rubric, as explicitly acknowledged in Appendix C.2 and Appendix F. Consequently, the 26.7% mean pass rate should not be interpreted as a corresponding rate of undesirable behavior.

W2. The construction method lacks a controlled comparison

The experiments establish feasibility but do not isolate the benefits of Anchor-and-Confirm or iterative specification synthesis. Table 23 reports averages of 552 LLM calls and 25.14 million input/output tokens per successful query, without a matched cost-quality baseline. These counts alone establish neither efficiency nor inefficiency. Confirmation of retrieved candidates also leaves retrieval coverage unmeasured.

Questions for authors
  1. Among failed responses, what fraction actually exhibits the target behavior, versus avoiding it without satisfying the preferred next-action rubric?
  2. What quality-cost advantage does the full construction pipeline provide over simpler alternatives under the same budget?
Reviewer_pPfhOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents TraceDance, a benchmark generator that scores LLMs on their ability to pass a step in a task where that step could lead to undesirable behavior. Users query TraceDance for the behavior to evaluate, and TraceDance constructs a series of test instances to measure performance on that behavior. TraceDance derives test instances from real data from Claude Code and OpenClaw. The paper describes how TraceDance turns are user query into testable instances, and uses TraceData to evaluate 8 frontier models in different scenarios.

Critical strengths
  • Generating single-step use cases to test for bad behavior is critical as outcome based benchmarks do not distinguish between good and bad agent trajectories
  • TraceDance provides practitioners with a framework to more easily generate cases created from real data (and not synthesized test case)
  • Evaluation of 8 frontier models provides good snapshot of current state-of-the-art for agent misalignment (though limited)
  • Paper is well-written and accessible to a broad audience
Critical weaknesses

Single-step evaluation is useful but limited as it may take multiple steps for misbehavior to manifest (as acknowledged by paper)

Questions for authors

While deriving cases based on real data is a strength of the paper, there may be some value in having additional and varied test cases. Have you investigated synthesizing tests from the real data to increase the number of cases per query?

Reviewer_PAtROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TraceDance proposes an automated pipeline for constructing behavior/process-level evaluation benchmarks for LLM agents by reusing real agent traces. Its Anchor-and-Confirm procedure first uses programmable rules to retrieve candidate trace locations and then uses an LLM to confirm whether a requested behavior actually occurred. The resulting benchmark evaluates LLMs by giving a partial agent trajectory prefix before a behavior-critical decision point and grading the next-turn action using a behavior-specific rubric.

Critical strengths
  1. The paper tackles an important and difficult evaluation problem with a reasonable single-turn framing.

  2. The empirical effort is substantial, and the authors perform several useful robustness checks. The work uses a large collection of deployment sessions and examines judge self-preference, alternative passing thresholds, source-model effects, and ranking uncertainty rather than reporting only a single aggregate number.

Critical weaknesses
  1. Unlike data synthesis for training and development, benchmark data requires nearly perfect data quality, especially when it comes to synthetic benchmarks. The proposed pipeline and resulting benchmarks are not of evaluation-level rigor yet. Both human annotators confirm the requested behavior (to be evaluated) in only 84 of the 100 sampled instances. Query interpretation is also judged fully accurate on only 78% of the annotated queries. These numbers appears to be high, but not satisfying the standard of rigorous evaluation yet.

  2. The claimed generalization to previously unspecified behaviors of the proposed pipeline in this paper feels unsupported. Among the 107 queries used to build benchmarks, 98 are predefined-behavior queries and only nine are genuinely custom. This makes the headline 95.3% construction success rate less convincing, if applying the pipeline on truly OOD queries. Also, are there any potential selection biases in the Anchor-and-Confirm procedure? Maybe measure anchor recall and retrieval-selection bias on a subset of the trace corpus.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.