acceptodds
Withdrawn from ICLR 2027

From Trace to Fix: Residual Task State in Long-Horizon Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recovering from failures in long-horizon, stateful language agents is challenging because the action introducing a defect, the symptom exposing it, and the persistent state requiring repair frequently diverge. Existing methods largely focus on retrospective failure attribution, leaving sequential diagnosis and state repair poorly integrated under constrained budgets. We introduce TRACE–FIX, a unified recovery framework centered on versioned residual task state. Trace transforms execution logs into a structured dependency-linked ledger, prioritizing unresolved task requirements rather than salient surface errors. Fix formulates recovery as budgeted planning over admissible actions, incorporating outcome forecasting and explicit repair-reserve constraints to prevent premature budget exhaustion. To rigorously decouple localization accuracy from downstream repair capability, we present LongH-Debug, evaluating root-cause attribution, verified-fault recovery, and end-to-end closed-loop execution independently. Our empirical analysis demonstrates that structured residual tracking substantially improves causal localization over raw-trace baselines, while budget-aware sequential planning converts verified fault signals into more reliable task recovery without destructive trial-and-error.

Primary area:
causal reasoning
Keywords:
LLM Agent, Error Tracing, Long Horizon

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_wSh1OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies failure recovery for long-horizon agents that modify persistent environment state. TRACE organizes execution history around unresolved task requirements, while FIX selects checks and repairs under a shared budget. The paper also introduces LongH-Debug to evaluate fault localization, recovery from verified faults, and end-to-end task completion separately.

Critical strengths

1.The problem is well motivated. Finding the original mistake is not always enough to recover a task. The paper usefully shifts attention to what still needs repair in the current state.

2.The proposed evaluation design includes useful controls. Giving recovery methods the same verified fault information helps separate diagnosis quality from repair quality. Distinguishing local repair from final task success is also appropriate.

3.I appreciate the inclusion of unsuccessful case studies. Appendix F discusses cases where TRACE misses the relevant event or a baseline performs better. These examples help explain the method’s limitations beyond aggregate scores.

Critical weaknesses

1.The method needs a more concrete problem formulation. Section 2 introduces task obligations, evidence, and resource versions mainly through prose. Appendix B gives an abstract state, but its connection to the actual task remains difficult to follow. Please carry one example through the method section. For the database case, show the relevant files, required predicates, and state before and after log deletion. Then specify how these objects determine the residual state, actions, and subsequent state updates. Explicit definitions of the state construction and planning utility would be more useful than further descriptive text.

2.The interaction between the representation, TRACE, and FIX needs a clearer explanation. The broad execution sequence is described, but the interfaces are scattered across the text. Please show how the three views are linked, what information each component consumes, and how an executed action updates them. This should also explain why each view is needed.

3.The paper does not clearly establish which evaluations are complete or which configurations produced the reported results. For example, Figure 5 includes placeholder points for an unfinished budget sweep. Section 4.5 excludes outcome-model adaptation. These inconsistencies make the empirical conclusions difficult to assess. I suggest clearly distinguishing measured results from planned evaluations, and clarifying the specific configuration setup for each benchmark comparison.

4.Please use specific appendix references. For example, the introduction mentions "The appendix provides the command-level reconstruction and retained receipts for this case." without naming its location.

5.Several figures are difficult to read at normal page size, especially Figure 2. Please enlarge the labels and typeset mathematical notation properly. For example, V_h(z,B) appears as literal text rather than with a properly rendered subscript.

Questions for authors

What are the unique challenges posed by "long-horizon" tasks for LLMs, especially when compared to other types of tasks or different model architectures? Furthermore, what are the key innovations introduced in this paper to address these specific characteristics?

Reviewer_r7dtOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies fault attribution and task recovery in long-horizon agents. It proposes TRACE-FIX, which connects localization and budgeted recovery planning through a versioned residual task state. Furthermore, it introduces LongH-Debug to evaluate localization, recovery, and end-to-end performance separately.

Critical strengths
  1. The problem is practically relevant, with a clear distinction between fault localization, local repair, and task completion.
  2. Binding evidence to resource versions helps prevent reliance on stale check results after state changes.
  3. Comparing recovery methods under the same fault information helps separate localization quality from recovery quality.
Critical weaknesses
  1. The paper does not explain how failure cases were collected and selected, or how reference root-cause labels were assigned and checked. Appendix F describes its case-study labels as provisional model judgments. However, it is not clear that whether the labels for the 200-case evaluation produced the same way, and how many were independently verified.

  2. Figure 4(a) marks the daggered intermediate ablations as unexecuted planning targets, Figure 5(a) includes placeholder budget points, and Section 4.4 refers to a target matrix. Section 3 also states: "A final characterization to run before submission is a per-case causal-complexity audit." I could not find the promised distributions and rank correlations. These passages read as work still to be done, but the conclusion presents the improvements as established results.

Questions for authors

For W1: Are the labels for the 200-case in Appendix F evaluation produced the same way, and how many are independently verified?

For W2: Which experiments have actually been completed?

Q3: The paper gives detailed logging and auditing rules, but says little about how task graphs, resource dependencies, and check/repair catalogues are built. Which parts are manual, and which are automated? For VERIFIEDROOT@B, Table 1 does not specify B, which candidates are checked, or what evidence is sufficient to verify a historical cause.

Q4: Section 4.5 says experience adaptation is excluded from the primary experiments, but Appendix E, Table 6 enables it for Full. Which configuration is used for the main results?

Q5: FIX reports 67.5% task recovery, compared with 61.7% for information gain with a repair reserve. Without confidence intervals or paired case outcomes, it is hard to judge whether this 5.8-point difference is reliable.

Reviewer_QsxMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes TRACE–FIX, a framework that tracks unresolved task requirements and plans checks and repairs within a limited budget to help long-horizon LLM agents recover from failures.

Critical strengths
  • The proposed problem seems interesting.
  • The writing is generally standard.
Critical weaknesses
  1. Unclear status of experimental results. On page 7, the caption of Figure 4 describes some ablation results as “unexecuted planning targets.” On page 8, the caption of Figure 5 states: “Interior points are planning placeholders until the full budget sweep is executed.” Section 4.4 also calls the closed-loop results a “target matrix,” leaving their status unclear. These remaining planning notes give the impression of an unfinished automated research workflow, although the manuscript alone cannot confirm their origin. The paper should clearly distinguish completed measurements from planned results and adjust its conclusions accordingly.

  2. Figure readability. Several figures, especially Figures 4 and 5, contain small labels and text that are difficult to read at normal page size. Enlarging the figures or simplifying their layout would make the results easier to assess.

  3. Overall completeness. The framework is promising, but the current manuscript still reads as a work in progress. A finalized experimental record, with clear links between the reported results and completed evaluations, would help readers assess the contribution with greater confidence.

Questions for authors

See the Weaknesses.

1 comment

As the authors did not submit a rebuttal addressing my concerns, and the manuscript contains numerous apparent remnants of incomplete or insufficiently reviewed AI-assisted writing, I have decided to maintain my original score. Nevertheless, I greatly appreciate the core idea behind TRACE–FIX and believe it has strong potential. I sincerely encourage the authors to further develop and polish the work and consider submitting a substantially improved version to a future venue such as ICML or NeurIPS.

Reviewer_uTjaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TRACE–FIX, a recovery framework that represents unresolved task requirements through versioned execution evidence and selects checks, repairs, and continuation actions under a shared budget. LongH-Debug separates localization, verified-fault recovery, and end-to-end evaluation. The problem formulation is worthwhile, but I recommend clear rejection in the current form. The manuscript presents substantial performance improvements while describing several corresponding results as targets or unexecuted placeholders, leaving the central empirical claims unsubstantiated.

Critical strengths

The distinction between historical cause, current repair target, and final task recovery is useful. Version-bound evidence and residual obligations provide a sensible way to avoid acting on failures that have already been resolved or on evidence invalidated by later state changes.

The proposed evaluation design addresses important confounds. Giving fixers identical verified-fault information and crossing tracing and fixing components would help distinguish better diagnosis from better recovery. Separately recording local resolution, harmful interventions, and official task outcomes is also appropriate.

Critical weaknesses

The experimental status is inconsistent across the manuscript. Figure 4 describes unexecuted planning targets, Figure 5 includes forecast placeholders, and Section 4.4 calls the factorial results a “target matrix.” Appendices C–E also describe outstanding launch conditions. Nevertheless, the abstract and conclusion present improvements as established findings. The authors need to clearly identify completed experiments, remove projected values from result figures, and provide case-level evidence supporting the headline comparisons.

The completed evidence described does not establish an advantage from the proposed planning mechanism. In the 24-case controlled study, dynamic planning and information gain with repair-budget reservation both resolve 16 cases. The natural case studies illustrate implementation behavior but explicitly do not establish comparative superiority. A completed, matched-budget comparison against the reserve-aware baseline is needed before attributing improved recovery to lookahead and residual-state planning.

Questions for authors

Which entries in Tables 1–3 and Figures 4–6 come from completed executions? Can you provide the corresponding case identities, execution receipts, verifier outputs, and uncertainty estimates?

Among completed experiments, does FIX outperform information gain with repair reservation when both use the same forecasts, legal actions, and fully charged cost ledger?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.