acceptodds
Under review as a conference paper at ICLR 2027

DENSE: Distilling Agent Trajectories into Evidence-Grounded Shortcut Trees for Self-Refinement

Abstract

Online agent deployments accumulate execution trajectories at massive scale and behavioral diversity, for which predefined annotation criteria hardly exist. Extracting useful evidence therefore demands costly manual annotation or verifier signals that fail to scale, leaving valuable evidence buried among redundant, incomplete, and failed executions. This raises a question: without post-execution rewards or correctness labels, how can reusable experience be distilled from the trajectories themselves? To address this challenge, we introduce DENSE (Distilling Evidence from Nested Subtask Executions), which organizes trajectory-derived evidence into nested shortcut trees. By consolidating redundant attempts, identifying resolved subtasks, and retaining useful steps alongside outstanding requirements, DENSE transforms noisy execution traces into structured and reusable task-solving feedback. To evaluate whether such feedback helps agents retry the same task, we design REFIT, which measures success-rate changes between the initial attempt and feedback-guided retries. Among feedback methods without external outcome supervision, DENSE achieves the highest strict pass rate across four agent models on Terminal-Bench 2.1, improving over initial attempts by 7.12–21.81 percentage points with 19.0–43.6% fewer agent tokens on retries. In addition, on hard tasks DENSE consistently outperforms self-reflection in cumulative pass rate across multiple feedback iterations on all four models, demonstrating its strong potential for continual agent self-improvement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent self-refinement, trajectory feedback, agent trajectories, trajectory distillation, agent evaluation, long-horizon agents, experience reuse

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2VW7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DENSE, a method for extracting reusable feedback from agent trajectories without access to post-execution rewards or hidden verifier outputs. DENSE constructs a nested subtask hierarchy, compresses repeated attempts into shortcuts, reconciles outstanding issues using subsequent observations, and renders feedback that emphasizes unfinished requirements while retaining selected executable details.

The paper also introduces REFIT, an evaluation protocol in which feedback methods share initial trajectories and guide separate attempts in reset environments with fresh model contexts. Experiments on Terminal-Bench 2.1 show the highest strict pass rates among the tested non-privileged feedback methods across four recipient models. GPT ablations examine hierarchy and shortcut/reconciliation components, and an exploratory extension evaluates repeated feedback on hard tasks.

Critical strengths

A useful approach to extracting partial progress without external outcome labels. DENSE distinguishes successful local discoveries from overall task completion and connects reusable evidence to outstanding requirements. This addresses a practical failure mode in long-horizon agents: useful discoveries can be lost among unsuccessful exploration or fail to translate into the required deliverable.

A reasonably controlled evaluation with consistent cross-model results. Sharing initial trajectories and resetting both environments and contexts makes comparisons between feedback methods interpretable. DENSE improves strict pass rates across all four recipients. The detailed prompts, algorithms, and outcome-transition analysis strengthen the empirical presentation, particularly by disclosing regressions and accounting limitations.

Critical weaknesses

The experiments do not sufficiently isolate the contribution of the proposed structure from actionable summarization and additional analysis computation. All-at-once performs action-level scoring, while Step-by-step evaluates decision-time justification without observing the target action’s outcome. Neither directly matches DENSE’s objective of synthesizing reusable paths and remaining requirements. The single-root summary ablation partially addresses this concern, but is evaluated only on GPT and differs from full DENSE in analysis computation and evidence organization. Consequently, the results establish the effectiveness of the complete pipeline more clearly than the necessity of its hierarchy and reconciliation mechanisms.

This concern is particularly consequential for MiniMax. Appendix D.4 reports that 32 of VF’s 37 regressions and 27 of Self-reflection’s 36 regressions involve no tool calls, whereas DENSE has no zero-tool executions. VF also repairs more initially unsuccessful executions than DENSE. Although the shared fresh-environment wrapper is a useful control, these results leave open how much of DENSE’s advantage comes from preventing premature termination rather than improving substantive recovery.

The evidence supporting the magnitude and broader interpretation of the gains remains incomplete. The study lacks an independent no-feedback retry control, limiting attribution of improvements over run0 to feedback rather than repeated attempts. This does not invalidate comparisons between feedback methods, and the appendix appropriately acknowledges the limitation for cumulative success. However, confidence intervals are also absent, despite margins of approximately 2.6–3.0 percentage points over the strongest non-privileged baselines on GPT and Kimi.

Efficiency claims require similarly careful qualification: the reported token reductions concern recipient execution only, while Figure 12 shows higher combined generation-plus-execution token use than run0 for all four recipients. The GPT dollar-cost analysis is informative, but does not establish a general reduction in total computation. Together, these limitations make the evidence insufficient for a strong claim about broadly efficient continual self-improvement.

Questions for authors

Can you provide a direct full-trajectory summarization baseline using the same generator, with matched feedback length and analysis budget, that explicitly extracts useful paths, failed approaches, supporting evidence, remaining requirements, and a rerun plan? The existing single-root ablation is relevant; extending this comparison across recipients would materially affect my assessment.

4 comments
AuthorsOpenReview ↗

Response to W1 & Q1 (Tight Coupling Between the Subtask Tree and Shortcut/Reconciliation,and Cross-Recipient `Flat` Summarization Baseline)

We sincerely thank you for your thoughtful review and constructive feedback, and for recognizing DENSE's ability to extract partial progress without external outcome labels, our controlled cross-model evaluation, and our transparent disclosure of regressions and accounting limitations.

  1. Tight Coupling & Why Removing Hierarchy Degenerates to a Flat Call (Section 2.3–2.4, Section 4.5 & Appendix F): We have already conducted controlled ablations in Section 4.5 (Table 4) and Appendix A.2 (w/o S&R and w/o hierarchy) to examine DENSE's structural benefit as far as our workflow permits. In DENSE, Shortcut & Reconcile is tightly coupled to—and attaches directly onto—the subtask tree: as formalized in Section 2.3–2.4 (Equation 1) and Appendix F (Algorithms 1 & 3), it processes trajectories layer by layer from child subtasks to parents, compressing repeated local attempts within each subtask scope and reconciling inherited child issues upward via later recovery evidence. Consequently, once the hierarchical structure is removed, this layer-wise workflow naturally degenerates into a flat whole-trajectory call (w/o hierarchy), which is why our submitted ablation took a single-root form while still isolating structural organization as cleanly as possible.

  2. Cross-Recipient Flat Baseline (Answering W1 & Q1): To directly address your request in Q1, we implemented Flat and evaluated it across all four recipients (3 paired repetitions per task) under the shared REFIT protocol. Operating on the full trajectory under the exact same generator (Gemini 3 Flash Preview), shared run0 traces, per-call output cap (8,192 tokens), feedback length budget (180/200 KiB), and outcome-blind boundary, Flat performs two-stage synthesis and evidence review over five structured fields:

FieldDescription
useful_pathsReusable approaches and findings. Summarizes what helped make progress in the previous attempt, such as a useful technique, discovery, or partial solution. Each entry cites the supporting actions or observations.
failed_approachesUnsuccessful approaches and pitfalls. Records what was tried, what went wrong, and which actions or observations support that assessment.
issuesIssue states and remaining requirements. Records each issue or outstanding requirement, its status (resolved, unresolved, or unknown), and the relevant evidence. An issue is marked resolved only when a concrete observation shows recovery. That recovery concerns the specific issue.
key_action_indicesReferences to critical historical actions. Identifies actions whose exact contents matter for the next attempt. The referenced actions are reproduced verbatim in the feedback so the agent can recreate them as needed.
rerun_planOrdered plan for a fresh attempt. Turns the evidence into a sequence of next steps, including what to recreate in the fresh environment and which remaining requirements to address.
MethodMiniMax-M2.7DeepSeek V4 ProGPT-5.5Kimi K2.6
Flat49.44%65.43%70.04%46.07%
DENSE52.43% (+3.00)73.25% (+7.82)75.66% (+5.62)50.94% (+4.87)

Table: Strict pass rate (%) across all four recipients (parentheses show DENSE −- Flat in pp).

DENSE consistently outperforms Flat across all four recipients (+3.00 to +7.82 pp). Inspecting paired feedback and trajectories confirms two core capabilities of DENSE that Flat lacks: (i) scope-aware separation of local recovery from task-level completion—without nested subtask boundaries, Flat frequently marks repaired intermediate sub-steps as [resolved] while overlooking unwritten or unverified final deliverables that DENSE keeps explicitly expanded at higher levels; and (ii) state-guided evidence folding without speculative over-constraint—whereas DENSE folds observation-verified local progress and highlights only open obligations, Flat's unverified global rerun_plan can omit subtle failure modes or steer recipients into redundant re-exploration.

AuthorsOpenReview ↗

Response to W2 (MiniMax-Specific Premature Termination vs. Substantive Recovery Gains)

  1. A Model-Specific Phenomenon Unique to MiniMax-M2.7: The premature-termination factor you identified does exist on MiniMax-M2.7, and our cross-recipient audit confirms it is a phenomenon unique to MiniMax-M2.7 . Specifically, on MiniMax-M2.7, all four baselines that present linear step-by-step action–observation tables (Self-reflection, VF, Step-by-step, and All-at-once) incur 23–37 zero-tool runs—<think> traces show MiniMax mistaking linear historical steps or "all tests passed" verifier logs for actions already completed in the current session—whereas DENSE incurs 0 zero-tool runs. This also clarifies the comparison with privileged VF: having direct access to ground-truth test outputs, VF repairs more initial failures overall (38 vs. 30), while DENSE's net lead (140 vs. 120) stems primarily from higher solution retention (110/119 vs. 82/119) and avoiding VF's 32 zero-tool regressions.

Notably, the structured Flat baseline designed in Q1 above likewise exhibits this exact non-linear-table behavior (0 zero-tool runs on MiniMax-M2.7), yet DENSE still maintains a clear +3.00 pp lead (52.43% vs. 49.44%).

  1. DENSE Still Outperforms All Non-Privileged Baselines After Excluding Zero-Tool Runs: Furthermore, auditing all six main feedback methods (n=267n=267: 119 initial passes, 148 initial failures) and evaluating Both-Active (tools>0\text{tools}>0) paired subsets—strictly excluding every run where either method makes zero tool calls—confirms that DENSE consistently outperforms every non-privileged baseline in both active pass rate and substantive failure repairs (and even among the 146 active failed-run0 pairs against privileged VF, DENSE repairs 13 tasks where VF actively invokes tools yet fails):

    Comparator on MiniMax-M2.7Full-Sample Pass (/267)Total 0-Tool Runs (/267)Both-Active Pairs (tools>0\text{tools}>0)Active Pass (DENSE vs. Comp)
    Advisor (no trace)110 (41.20%)0267 / 267140 vs. 110 (+11.24 pp)
    Self-reflection103 (38.58%)37 (27 reg.)230 / 267113 vs. 102 (+4.78 pp)
    Step-by-step114 (42.70%)30 (19 reg.)237 / 267120 vs. 114 (+2.53 pp)
    All-at-once119 (44.57%)23 (14 reg.)244 / 267123 vs. 118 (+2.05 pp)
    VF$^\dagger$ (privileged)120 (44.94%)34 (32 reg.)233 / 267108 vs. 120 (-5.15 pp)

    Table: MiniMax-M2.7 zero-tool counts and Both-Active (tools>0\text{tools}>0) paired comparisons against DENSE (full-sample DENSE: 140/267 = 52.43%, 0 zero-tool runs, 110/119 retained, 30/148 repaired).

    In the revised Appendix, we will reorganize Appendix D.4 into two separate subsections to clearly distinguish and discuss (i) this MiniMax-M2.7-specific premature-termination behavior (and its boundary relative to privileged VF) and (ii) DENSE's consistent substantive recovery gains after excluding these zero-tool exceptions.

AuthorsOpenReview ↗

Response to W3 (Independent No-Feedback Retry Control and Paired 95% Confidence Intervals):

1. Independent No-Feedback Retry Control on GPT-5.5 : Because our main evaluation (Table 2) measures single-run pass rates across all tasks in reset environments averaged over 3 repetitions (3-rep)—rather than selectively retrying failed runs or taking cumulative maxima—an unguided retry has zero expected gain over run0. To verify this directly, we ran an Independent No-feedback Retry control on GPT-5.5 across all 89 tasks ×\times 3 repetitions (n=267n=267) under identical reset environments, fresh contexts, tools, and budgets:

Recipient (89×3=26789 \times 3 = 267 runs)Baseline (run0)No-feedback RetryDENSE (ours)
GPT-5.5 Strict Pass (%)68.54% (183/267)68.16% (182/267, -0.37 pp)75.66% (202/267, +7.12 pp)

No-feedback Retry matches run0 (-0.37 pp), whereas DENSE improves over run0 by +7.12 pp and over No-feedback Retry by +7.49 pp, confirming that the single-round gain comes from trajectory feedback rather than repeated attempts.

2. Complete Paired 95% Confidence Intervals: From the exact paired main-experiment records (7,308 runs), we computed task-cluster paired 95% bootstrap confidence intervals (10,00010,000 resamples over tasks, preserving all 3 matched repetitions and shared run0 pairing within each task) for ΔStrict Pass=P(DENSE)−P(Comparator)\Delta\text{Strict Pass} = P(\text{DENSE}) - P(\text{Comparator}) (pp):

Comparator (DENSE −- Comparator)MiniMax-M2.7DeepSeek V4 ProGPT-5.5Kimi K2.6
Baseline (run0)+7.87 [+1.87, +14.23]+21.81 [+13.99, +29.63]+7.12 [+2.25, +12.36]+11.61 [+7.87, +15.73]
Advisor+11.24 [+4.87, +17.60]+19.34 [+11.11, +27.57]+6.37 [+0.75, +11.99]+7.12 [0.00, +14.23]
Self-reflection+13.86 [+7.87, +19.85]+9.47 [+2.88, +16.05]+3.37 [-1.50, +8.61]+4.87 [+0.37, +9.36]
All-at-once+7.87 [+3.00, +13.11]+11.11 [+4.53, +17.70]+2.62 [-2.25, +7.49]+3.00 [-1.87, +8.24]
Step-by-step+9.74 [+3.37, +16.10]+7.82 [+1.65, +14.40]+2.62 [-1.50, +7.12]+4.12 [0.00, +8.24]
Verifier Feedback†^\dagger+7.49 [0.00, +14.61]+6.58 [-0.82, +13.99]-8.24 [-15.73, -1.12]-3.00 [-9.36, +3.37]

Moreover, DENSE's gains manifest jointly across both task performance and retry execution cost/efficiency (Table 3 & Figure 4). Taking the exact strongest non-privileged baselines on GPT-5.5 and Kimi K2.6 mentioned in your comment (+2.62 to +3.00 pp pass-rate margins): on GPT-5.5, DENSE (179.9K tokens) simultaneously cuts recipient execution cost by 14.9% relative to All-at-once (211.4K) and 39.0% relative to Step-by-step (294.8K) (-43.6% vs. run0 319.1K); on Kimi K2.6, DENSE (416.1K) cuts recipient execution cost by 23.9% relative to All-at-once (546.9K) and 29.2% relative to Step-by-step (587.8K) (-23.7% vs. run0 545.3K). We will include both the No-feedback Retry control and the paired 95% CI table in Appendix.

AuthorsOpenReview ↗

Response to W4 (Qualifying Efficiency Claims and Cost Accounting Metrics)

Thank you for your careful reading and for pointing this out. Separating recipient retry execution cost from end-to-end pipeline computation helps place our efficiency results in clear context: reductions in recipient execution tokens (Table 3) and USD savings under a specific generator–recipient price ratio (Table 5) characterize retry-stage efficiency and analyzer–executor pricing tradeoffs rather than a universal reduction in total computation.

Specifically, our four reported metrics capture complementary aspects of cost:

  • Recipient execution tokens (run1 vs. run0; Table 3 & Figure 4): Reduced by 19.0%–43.6%, counting only the recipient's retry execution (excluding feedback generation).
  • Combined generation + execution tokens (Gen + run1 vs. run0; Figure 12): Sums Gemini distillation tokens and recipient run1 tokens; this total is higher than run0 across all four recipients.
  • Single-retry estimated USD (Gen + run1 vs. run0; Appendix B.1, Table 5): Under standard uncached base rates, Gemini feedback ($0.1396, $0.14) + GPT-5.5 run1 ($1.0075, $1.01) totals $1.1470 ($1.15) vs. $1.8085 ($1.81) for run0 (-36.6%), reflecting a price tradeoff between a cheaper analyzer and a pricier executor rather than a universal saving across all model pairs.
  • Complete workflow USD (run0 + Gen + run1; Appendix B.1): Totals $2.96 on GPT-5.5; thus, -36.6% compares a guided retry against run0, not the full two-round workflow from scratch against a single attempt.

In the revised manuscript, we will keep efficiency descriptions explicitly scoped to recipient execution tokens on retries with clear cross-references to the full accounting breakdown (Figure 12; Appendices A.3–A.4 & B.1), and ensure our discussion of iterative refinement remains aligned with this scope.

Thank you again for your constructive guidance—we hope the new cross-recipient Flat results (Q1) and supporting analyses fully resolve your concerns and favorably inform your assessment, and we welcome any further discussion.

Reviewer_gsaDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines how feedback extracted from agent execution trajectories can improve subsequent attempts at the same task without using post-execution rewards or verifier outputs. The authors propose DENSE to organize prior actions into nested subtasks, consolidate repeated attempts, and update issue states based on subsequent evidence. The method achieves meaningful improvements in the same-task retry experiments conducted in the paper, which also includes ablation studies, case studies, and experiments involving multiple rounds of retries on the same task.

Critical strengths
  • The paper proposes an approach to addressing the challenges of trajectory reuse by connecting local progress, resolved issues, and remaining task requirements to provide guidance for the next run.
  • The complete pipeline yields gains across multiple recipient models. Under REFIT, the methods use shared initial trajectories and begin a new run with the environment and context reset, providing relatively clear experimental controls for before-and-after comparisons.
Critical weaknesses
  • The incremental value of the nested shortcut tree over simpler forms of organization has not been sufficiently validated. The main experiments demonstrate the effectiveness of the complete DENSE pipeline, but they do not isolate the contribution of the tree structure. The ablation also replaces multistage nested analysis with a single holistic analysis, changing the structure, analysis effort, and content generation process simultaneously. The results therefore do not establish how much better the tree performs than a clear flat or sequential organization that provides the same feedback functions with comparable analysis effort.
  • The experiments validate only same-task retries and do not substantiate the research motivation of accumulating experience from past trajectories with establishing the method's practical value. The paper is motivated by the growing volume of trajectories accumulated in real deployments and the experience that can be extracted from them. However, the main experiments are limited to having an agent retry a given task using feedback generated from its previous execution trajectory, the experiments with multiple rounds of retries also remain focused on the same task. The results therefore demonstrate only the benefit of retrying the original task, do not show the additional benefits of extracting and accumulating experience from multiple trajectories and applying that experience to other tasks.
Questions for authors
  • The paper uses the nested shortcut tree structure to organize local and overall level actions, update issue states, and organize feedback for the next run according to branch status. What improvements come from the structured analysis and organization, and what improvements come from the feedback?
  • What additional role does the nested tree structure play compared with a flat or sequential representation that also records task evidence, task status, and unfinished requirements? A flat or sequential variant should be constructed under the existing experimental setting, with the same functions for recording task evidence, task status, and unfinished requirements, as well as retaining key actions. Then it should be evaluated using the same source trajectories and model to assess the contribution of the nested tree organization.
  • Which practically relevant deployment workflows do the paper's same-task retry experiments represent, and what practical need is there to use past trajectories to retry the same task after resetting the environment? How do the results differ between retaining feedback from multiple attempts and using only the most recent feedback? This comparison would help identify the additional value of accumulated experience.
9 comments
AuthorsOpenReview ↗

Response to Q1, W1 & Q2 (Gains from Feedback vs. Structured Organization, Structural Ablations, and Tight Coupling Between Hierarchy and Shortcut/Reconciliation):

  1. Gains from Feedback Alone vs. Gains from Structured Analysis and Organization (Answering Q1 — Tables 2 & 3):

    We have already implemented the raw trajectory feedback baseline as Self-reflection

    • Improvements from feedback alone: Supplying trajectory feedback without structural distillation (Self-reflection) yields baseline gains on capable recipients (+3.75 pp on GPT-5.5, 68.54% -> 72.28%; +12.35 pp on DeepSeek V4 Pro, 51.44% -> 63.79%; +6.74 pp on Kimi K2.6, 39.33% -> 46.07%), though raw linear history can even hurt performance when inducing cross-session confusion (-5.99 pp on MiniMax-M2.7, 44.57% -> 38.58%).
    • Additional improvements from structured analysis and organization: Beyond raw trajectory feedback (Self-reflection), DENSE delivers consistent further gains across all four recipients—adding +13.86 pp on MiniMax-M2.7 (52.43%), +9.47 pp on DeepSeek V4 Pro (73.25%), +4.87 pp on Kimi K2.6 (50.94%), and +3.37 pp on GPT-5.5 (75.66%)—while simultaneously cutting retry execution tokens further (e.g., -22.0% on GPT-5.5 from 230.7K to 179.9K, and -25.7% on Kimi K2.6 from 559.7K to 416.1K).
  2. Submitted Structural Ablations & Additional Role of the Nested Tree (Answering W1 & Q2, Part 1 — Section 4.5, Section 5.3 & Appendix A.2):

    We have conducted our ablation study at Section 4.5

    • Joint accuracy and efficiency gains in ablations: On GPT-5.5 (n=267n=267 paired runs, Table 4), retaining the tree without Shortcut & Reconcile (w/o S&R: 72.28%, 228.6K tokens) and removing the hierarchy to extract completion status, supporting evidence, lessons, and next steps from the same run0 traces (w/o hierarchy: 72.66%, 199.2K tokens) both remain near raw feedback levels. Combining both in full DENSE simultaneously achieves higher strict pass (75.66%, +3.37 and +3.00 pp; +3.41 and +2.65 pp on n=264n=264 in Appendix A.2) and lower recipient token cost (179.9K, saving 21.3% and 9.7%, and -43.6% vs. run0).
    • Additional role of the nested tree over flat representations: Even when a flat or sequential representation records task evidence, status, unfinished requirements, and key actions, it places intermediate sub-step repairs and final deliverable checks inside a single undifferentiated scope—making it prone to conflating locally resolved sub-errors with overall task completion (Section 5.3 & Appendix D). The nested tree provides explicit parent–child scoping: it folds observation-verified local detours inside completed child branches while keeping unverified or unwritten task-level requirements explicitly expanded at parent/root scopes.
  3. Tight Coupling & Why Removing Hierarchy Degenerates to a Flat Call (Clarifying W1 & Q2, Part 2 — Section 2.3–2.4 & Appendix F): Crucially, DENSE is not a detachable tree formatting wrapper around standalone summarization functions; Shortcut & Reconcile attaches directly to the subtask tree and operates layer by layer from child subtasks to parents (Equation 1; Appendix F, Algorithms 1 & 3), compressing repeated local attempts within each child scope and reconciling inherited child issues upward via recovery evidence (with sibling branches processed concurrently, and verbatim key-action retention plus branch folding/expansion executed deterministically without extra LLM calls). Once hierarchical parent–child scopes are removed, this layer-wise workflow has no structural boundary to reconcile across and naturally degenerates into a flat whole-trajectory call (w/o hierarchy). Thus, our submitted ablations already isolate structural organization as far as DENSE's coupled mechanism permits; engineering a separate multistage flat reconciliation pipeline amounts to designing a different system outside the scope of ablating our core method.

AuthorsOpenReview ↗

Response to W2 & Q3 (Research Scope, Controlled Same-Task Reset & Practical Workflows, and Single-Trajectory vs. Multi-Attempt Memory):

This paper focuses strictly on within-trajectory, outcome-blind evidence distillation. Building multi-trajectory memory banks for cross-task experience transfer (explicitly noted as future work in Section 5.2) and optimizing cross-attempt memory retention windows (e.g., ablating K=3K=3 vs. K=1K=1) is outside the scope of our claimed contributions.

Within this scope, we clarify each part of W2 and Q3 below:

  1. Foundational Prerequisite Before Cross-Task Reuse & Manuscript Alignment (Answering W2): Our introduction references growing deployment trajectories to highlight a fundamental supervision bottleneck: the absence of post-execution rewards, hidden verifier outputs, or correctness labels, upon which prior experience-reuse methods (e.g., Reflexion, ExpeL, DoVer) rely. Before scaling to cross-task retrieval and transfer, a prerequisite question must be answered: without any outcome labels, what reusable evidence exists inside an individual noisy trajectory, and how can validated local progress and resolved detours be separated from unfinished requirements? Answering this foundational question is the core contribution of DENSE. In the revised Abstract, Introduction, and Conclusion.

  2. Why Same-Task Reset in REFIT & Corresponding Practical Workflows (Answering Q3, Part 1):

    • Primary scientific purpose (controlled causal evaluation): Long-horizon terminal trajectories directly mutate or corrupt filesystems, packages, and processes. REFIT (Section 3) shares run0 and resets to the exact initial environment s0s_0 with a fresh context so that the distilled feedback is the sole channel of information transfer, excluding residual state or retrieval noise and making ΔP\Delta P a clean, fair measure of extracted trajectory value .

    • Practical workflow and need (efficient sandboxed self-generated trajectory collection): Practically, resetting to a clean sandbox s0s_0 and retrying the task directly supports efficient collection of high-quality self-generated agent trajectories in sandboxed task environments. Collecting clean, end-to-end completion traces inherently requires starting from the fresh initial state s0s_0; compared with brute-force mass unguided rollouts followed by post-hoc filtering, or relying on imitation of human experts or stronger teacher models, guiding the agent with outcome-blind feedback distilled from its prior attempt substantially improves the quality and compactness of self-generated trajectories.

  3. Multi-Attempt vs. Most-Recent Feedback Retention Is Outside Our Scope (Clarifying Q3, Part 2): DENSE operates strictly within a single execution trajectory. Even in our exploratory hard-task extension (Section 4.6), Appendix A.5 specifies that each new feedback item is distilled independently from the immediately preceding trajectory .We did not directly compare retaining feedback from multiple attempts with retaining only the most recent feedback.

Thank you for the further explanation and for clarifying that the paper focuses on distilling evidence from a single trajectory. These clarifications are helpful, but I still have two main concerns.

1. The incremental contribution of the nested tree over a multistage flat feedback pipeline remains unclear.

I acknowledge the gains achieved by the full DENSE pipeline. To assess the incremental contribution of the nested tree, however, we still need a comparison with a multistage, non-nested pipeline that provides similar feedback and uses a comparable amount of analysis. For example, Trajectory-Informed Memory Generation for Self-Improving Agent Systems, which the paper cites, uses a multistage process to extract lessons separately for each subtask. This approach could serve as a flat feedback baseline.

I noticed that your response to Reviewer 2VW7 reports a two-stage Flat baseline that is closely related to the comparison I requested in Q2. Your response to me, however, refers to the existing single-root ablation and describes a multistage flat pipeline as outside the scope of the paper. Could you clarify how the new Flat baseline corresponds to the comparison I requested and provide the prompts for both stages, the steps each stage performs, and the total analysis budget? In particular, on GPT-5.5, the new Flat baseline achieves a pass rate of 70.04%, compared with 72.66% for the original single-root summary. Could you explain this difference? It would help me assess whether the new baseline represents a reasonably strong flat method. Adapting the staged extraction procedure of Fang et al. would also strengthen this comparison.

2. The practical use of resetting and retrying the same task, and its advantages over continuing from the current state, still need more concrete justification.

I understand that the paper focuses on same-task retries, but I still find it difficult to connect this setting to a practical need. For example, when a coding agent encounters an error, it can retain its workspace and continue fixing the problem. Which specific workflows require the agent to discard its current state and start the same task again from the initial environment, carrying over only feedback from the previous attempt? In those workflows, what advantage does restarting with DENSE offer over continuing the repair in the existing workspace?

You mention collecting high-quality, self-generated trajectories as one application. How would the resulting trajectories be used? If their sole purpose is to guide another attempt at the same task, my first concern still applies. What additional benefit does the nested tree provide over a flat feedback pipeline for that retry?

Reviewer_LmovOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces DENSE, a method to distill reusable agent experience from unlabeled execution trajectories without post‑hoc rewards or verifier signals. It builds evidence‑grounded nested shortcut trees to organize subtask progress, redundant attempts and unresolved requirements. A paired evaluation protocol REFIT is proposed to isolate feedback effects with reset environments and fresh model contexts. Evaluated on Terminal‑Bench 2.1 over four agents, DENSE yields 7.12‑21.81 pp strict pass improvement and cuts rerun tokens by 19.0‑43.6%. Multi‑iteration experiments on hard tasks show better cumulative performance than self‑reflection for agent self‑refinement.

Critical strengths

The paper targets an interesting and practically‑relevant research problem of reusing unannotated agent trajectories without external supervision. Well‑organized writing and clear overall paper structure.

Critical weaknesses

The paper employs multiple overlapping notation systems without clear reconciliation. Formal mathematical symbols appear alongside XML-style tags in the algorithm descriptions, creating ambiguity about their semantic relationships.

Questions for authors

1.How much genuine synergistic gain does the combination of evidence grounding and nested trees provide beyond simpler action-level scoring baseline ? The ablation study shows that removing shortcuts or the hierarchical structure yields only marginal performance drops.

2.The paper mixes formal mathematical notation with XML-style tags, and key terms such as “shortcut” are defined differently in the main text (Section 2.3) versus the appendix pseudocode (Appendix F). These inconsistencies hinder readability and reproducibility. The authors should unify all notation throughout the manuscript.

3.The method relies on six carefully designed prompts , yet no prompt perturbation or sensitivity analysis is provided. There is a risk of prompt overfitting to Terminal-Bench 2.1’s task style. How does DENSE generalize to other task domains and agent behaviors?

  1. DENSE fixes Gemini 3 Flash Preview as the feedback generation model without any sensitivity analysis. Given the pipeline’s dependence on six LLM calls, it is unclear whether the observed gains stem from the methodological framework or from the specific capabilities of this model. If a stronger analyzer were used, would DENSE’s advantages persist?
4 comments
AuthorsOpenReview ↗

Response to Q1 (Joint Gains in Both Execution Performance and Token Savings):

Thank you for recognizing the practical relevance of reusing unannotated trajectories without external supervision and the clarity of our paper's structure, as well as for raising this thoughtful question.

We have already conducted controlled ablation studies in Section 4.5 (Table 4) and Appendix A.2 (w/o S&R and w/o hierarchy) on GPT-5.5 across all 89 tasks ×\times 3 paired repetitions (n=267n=267) under identical run0 trajectories, generator (Gemini 3 Flash Preview), and outcome-blind boundary. Looking at pass rate alone understates the genuine gain: nested subtask scoping and evidence-grounded Shortcut & Reconcile work as one cohesive method that executes tasks more accurately while simultaneously saving substantial execution tokens:

  • Better task execution: Full DENSE lifts strict pass to 75.66%, outperforming w/o S&R (72.28%, +3.37 pp), w/o hierarchy (72.66%, +3.00 pp), and run0 (68.54%, +7.12 pp; excluding the port-conflict task in Appendix A.2 similarly preserves +3.41 and +2.65 pp margins on n=264n=264).
  • Greater token savings: At the same time, DENSE cuts recipient execution tokens to 179.9K—saving 21.3% relative to w/o S&R (228.6K), 9.7% relative to w/o hierarchy (199.2K), and 43.6% relative to run0 (319.1K).

Plotting strict pass rate against recipient execution tokens makes this joint advantage clear: DENSE sits cleanly in the upper-left Pareto-optimal corner, achieving both the highest pass rate and the lowest token cost.

GPT-5.5 ablation plot (PNG)

AuthorsOpenReview ↗

Response to W1 & Q2 (Unifying Mathematical Notation, Pseudocode Variables, XML/JSON Schemas, and Term Definitions):

Thank you for your careful reading and constructive suggestions on notation clarity. Explicitly bridging the high-level mathematical formulation (Section 2), the LLM JSON interfaces and validation pseudocode (Appendices E–F), and the rendered XML feedback artifact (Appendix C) will make the cross-layer correspondence much easier to follow.

The underlying implementation is unified across the manuscript; these three representation layers were previously introduced in separate sections without a single cross-reference table, and Section 2.3 introduced shortcut intuitively while Appendix F specified its exact partition boundary conditions. We will clarify this correspondence along four concrete directions:

  1. Unified Cross-Layer Notation Table (Bridging Math ↔\leftrightarrow Pseudocode ↔\leftrightarrow JSON/XML): We will add a master notation table in the Appendix (and cross-reference it in the main text) that explicitly maps every mathematical symbol to its pseudocode variable, LLM JSON field (snake_case), and recipient-visible XML tag (kebab-case):
ConceptMath Notation (Sec. 2 & App. F)Pseudocode Variable (Algs. 1–3)Prompt JSON Field (App. E, P5–P10)Rendered XML Tag (App. C, Table 6)Scope & Semantics
Action–Observation Leafℓi=(ai,oi)∈τ\ell_i = (a_i, o_i) \in \tauInitial queue QQ elementsGlobal action_index (P3, P4)a1, a2, ...; <key-action source="a_i">Single trajectory step and its immediate observation
Queue & Subtask PrefixQueue QkQ_k, prefix G=(u1,…,ub)G=(u_1,\ldots,u_b), parent vvQueue QQ, window WW, next level HH; head hh, cut ordinal ddP5 action_index (queue ordinal h:th{:}t); P8 can_mount_allNode IDs n0, n1, ...; <task> blockStage 1 bottom-up hierarchy construction (T\mathcal T)
Sibling Partition & ShortcutCompressed local subtree T~v\widetilde{\mathcal T}_vOrdered children C→PC′C \xrightarrow{\mathcal P} C'P9 groups: mode (keep if ∥S∥=1\|S\|=1; shortcut if ∥S∥≥2\|S\|\ge 2), source_node_ids, dead_end, working_path, outcome<shortcut outcome="..." source="...">, <dead-end>, <working-path>Stage 2A order-preserving sibling compression under parent vv
Issue Reconciliation & State(Tv⋆,σv,Iv)(\mathcal T_v^\star, \sigma_v, \mathcal I_v) (Eq. 1); state⁡(c,p,O,F)\operatorname{state}(c,p,O,F) (Eq. 6)Node uu, validated critic verdict zzP10 coherence (cc), completion (pp), summary (σv\sigma_v), open_issues (OO), fatal_issues (FF), resolved_issue_ids<shortcut-tree state="...">, <task state="...">, <open-issue evidence="...">Stage 2B cross-level issue resolution and deterministic state (Iv=O∪F\mathcal I_v = O \cup F)
  1. Unifying the Definition of shortcut Between Section 2.3 and Appendix F:

    • Clarification: In Appendix F.3 (Algorithm 3, Table 15) and Prompt P9, the cleaner outputs a complete, order-preserving contiguous partition of a parent's direct children C=(c1,…,cm)C=(c_1,\ldots,c_m): a multi-node group (∣S∣≥2|S|\ge 2 consecutive siblings) is fused into a shortcut (mode="shortcut"), whereas a standalone single child (∣S∣=1|S|=1) requires no fusion and is retained as-is (mode="keep"). Section 2.3 previously focused on the multi-node fusion intuition and omitted the singleton keep case.
    • Revision: We will update Section 2.3 to explicitly state this contiguous partition rule (keep for singletons, shortcut for ≥2\ge 2 consecutive sibling attempts) and distinguish the <shortcut> tree element from the codebase name Shortcut-Tree (Appendix E) and the summary instruction label [Shortcut rule] (Prompt P7).
  2. Eliminating Symbol Overloading and Aligning Equations (1) and (6):

    • Removing overloading of GG: In Algorithm 3 (lines 11–13), we will rename the cleaner's output partition from GG to P\mathcal P, so that GG exclusively denotes the queue prefix G=(u1,…,ub)G=(u_1,\ldots,u_b) from Section 2.2 and Algorithm 2.
    • Connecting Eq. (1) to Algorithm 3 and Eq. (6): We will add explicit variable annotations in Section 2.4 and Appendix F.3 showing that the compiled child list C′C' in Algorithm 3 forms T~v\widetilde{\mathcal T}_v in Eq. (1), and that the validated critic output z=(c,p,σv,O,F)z=(c,p,\sigma_v,O,F) yields Iv=O∪F\mathcal I_v = O \cup F and deterministically computes the XML state attribute of Tv⋆\mathcal T_v^\star via Eq. (6).
  3. Explicitly Disambiguating the Three Index Systems: We will highlight in Section 2.2 and Appendix F.1 the strict separation among: (i) global trajectory action indices (a1, a2, ...), (ii) tree node IDs (n0, n1, ...), and (iii) temporary level-queue ordinals (h:th{:}t, returned as action_index in Prompt P5).

AuthorsOpenReview ↗

Response to Q3 (Prompt Design and Cross-Benchmark Generalization to SkillsBench):

We address this concern from both prompt inspection and cross-benchmark evaluation:

  1. Domain-Agnostic Prompts (Appendix E & F): All six prompts are provided verbatim in Appendix E (Table 5, Prompts P5–P10). Direct inspection shows that they contain no task- or benchmark-specific descriptions tailored to Terminal-Bench 2.1. Instead, they only define generic structural operations over observation–action sequences (e.g., subtask segmentation, shortcut compression, and evidence-grounded issue reconciliation) under strict JSON schemas, which are then validated and rendered deterministically (Appendix F).

  2. Zero-Shot Transfer to SkillsBench (94 Tasks ×\times 3 Repetitions): To directly test generalization across task domains and agent behaviors, we applied the exact same DENSE pipeline and unmodified prompts to SkillsBench across all 94 tasks ×\times 3 paired repetitions on DeepSeek V4 Pro and GPT-5.5. SkillsBench covers diverse specialized domains (e.g., CAD repair, seismic analysis, Lean 4 proving, power-grid optimization, and multimodal workflows) where agents coordinate native tools with external skill specifications.

Recipient ModelBaselineAdvisorSelf-reflectionAll-at-onceStep-by-stepDENSE (ours)Verifier Feedback†^\dagger
DeepSeek V4 Pro37.2341.49 (+4.26)40.78 (+3.55)39.72 (+2.48)39.72 (+2.48)43.97 (+6.74)67.38 (+30.14)
GPT-5.537.9436.52 (-1.42)39.01 (+1.06)39.36 (+1.42)39.36 (+1.42)41.49 (+3.55)72.34 (+34.40)

Table: SkillsBench official pass rate (%) and change from Baseline (pp; †^\daggerprivileged reference).

Without any prompt modification, DENSE achieves the highest official pass rate among all non-privileged methods on both models, improving over Baseline by +6.74 pp on DeepSeek V4 Pro and +3.55 pp on GPT-5.5 (2.5$\times$ the gain of the strongest trajectory-scoring baselines, +1.42 pp). We will include these cross-benchmark results in the revised manuscript.

AuthorsOpenReview ↗

Response to Q4 (Sensitivity Analysis with a Stronger Feedback Generator API):

To directly test generator API sensitivity and verify whether DENSE's advantages persist with a stronger analyzer, we re-evaluated the trajectory-annotation methods (All-at-once, Step-by-step, and DENSE) by upgrading the feedback generator from Gemini 3.0 Flash (google/gemini-3-flash-preview, Table 2) to the stronger Gemini 3.8 Flash (google/gemini-3.8-flash) on DeepSeek V4 Proand GPT-5.5 , holding the shared run0 trajectories, prompts, per-call token limit (8,192), and feedback budget identical:

DeepSeek V4 Pro (81 Tasks ×\times 3 Repetitions, n=243n=243 per method)

Generator APIBaseline (run0)All-at-onceStep-by-stepDENSE (ours)
Gemini 3.0 Flash (Table 2)51.4462.14 (+10.70)65.43 (+13.99)73.25 (+21.81)
Gemini 3.8 Flash (Stronger API)51.4462.14 (+10.70)65.84 (+14.40)72.02 (+20.58)

GPT-5.5 (89 Tasks ×\times 3 Repetitions, n=267n=267 per method)

Generator APIBaseline (run0)All-at-onceStep-by-stepDENSE (ours)
Gemini 3.0 Flash (Table 2)68.5473.03 (+4.49)73.03 (+4.49)75.66 (+7.12)
Gemini 3.8 Flash (Stronger API)68.5473.03 (+4.49)74.53 (+5.99)76.03 (+7.49)

Tables: Strict pass rate (%) and change from shared run0 Baseline (pp) on Terminal-Bench 2.1.

Under the stronger Gemini 3.8 API, DENSE continues to exhibit a clear advantage on both models (72.02% / +20.58 pp on DeepSeek V4 Pro and 76.03% / +7.49 pp on GPT-5.5). Because each analysis procedure explicitly specifies and constrains the analyzer's behavior over concrete execution observations, DENSE relies only on the API's general semantic processing capability—such as organizing local observation–action evidence and checking whether earlier issues are resolved by later outputs—rather than depending on whether the analyzer itself possesses strong domain or task-solving priors.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on our rebuttal and revisions :)

Dear Reviewers and Area Chair,

Thank you very much for your insightful comments and constructive feedback, which have helped us significantly improve our paper.

We have posted our detailed responses along with new experimental results and updated the manuscript accordingly. As the discussion period is drawing to a close, please let us know if our revisions have adequately addressed your concerns, or if there are any remaining questions we could clarify before the deadline.

We truly appreciate your time and effort in reviewing our work :)

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.