- Rating
- 2(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies failure recovery for long-horizon agents that modify persistent environment state. TRACE organizes execution history around unresolved task requirements, while FIX selects checks and repairs under a shared budget. The paper also introduces LongH-Debug to evaluate fault localization, recovery from verified faults, and end-to-end task completion separately.
Critical strengths
1.The problem is well motivated. Finding the original mistake is not always enough to recover a task. The paper usefully shifts attention to what still needs repair in the current state.
2.The proposed evaluation design includes useful controls. Giving recovery methods the same verified fault information helps separate diagnosis quality from repair quality. Distinguishing local repair from final task success is also appropriate.
3.I appreciate the inclusion of unsuccessful case studies. Appendix F discusses cases where TRACE misses the relevant event or a baseline performs better. These examples help explain the method’s limitations beyond aggregate scores.
Critical weaknesses
1.The method needs a more concrete problem formulation. Section 2 introduces task obligations, evidence, and resource versions mainly through prose. Appendix B gives an abstract state, but its connection to the actual task remains difficult to follow. Please carry one example through the method section. For the database case, show the relevant files, required predicates, and state before and after log deletion. Then specify how these objects determine the residual state, actions, and subsequent state updates. Explicit definitions of the state construction and planning utility would be more useful than further descriptive text.
2.The interaction between the representation, TRACE, and FIX needs a clearer explanation. The broad execution sequence is described, but the interfaces are scattered across the text. Please show how the three views are linked, what information each component consumes, and how an executed action updates them. This should also explain why each view is needed.
3.The paper does not clearly establish which evaluations are complete or which configurations produced the reported results. For example, Figure 5 includes placeholder points for an unfinished budget sweep. Section 4.5 excludes outcome-model adaptation. These inconsistencies make the empirical conclusions difficult to assess. I suggest clearly distinguishing measured results from planned evaluations, and clarifying the specific configuration setup for each benchmark comparison.
4.Please use specific appendix references. For example, the introduction mentions "The appendix provides the command-level reconstruction and retained receipts for this case." without naming its location.
5.Several figures are difficult to read at normal page size, especially Figure 2. Please enlarge the labels and typeset mathematical notation properly. For example, V_h(z,B) appears as literal text rather than with a properly rendered subscript.
Questions for authors
What are the unique challenges posed by "long-horizon" tasks for LLMs, especially when compared to other types of tasks or different model architectures? Furthermore, what are the key innovations introduced in this paper to address these specific characteristics?
As the authors did not submit a rebuttal addressing my concerns, and the manuscript contains numerous apparent remnants of incomplete or insufficiently reviewed AI-assisted writing, I have decided to maintain my original score. Nevertheless, I greatly appreciate the core idea behind TRACE–FIX and believe it has strong potential. I sincerely encourage the authors to further develop and polish the work and consider submitting a substantially improved version to a future venue such as ICML or NeurIPS.