- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how long-horizon language model agents can link historical information with current task progress and execution requirements. The authors propose TaskGraph, a four-layer graph composed of goals, phases, steps, and atomic records, which unifies the organization of tasks and memory through attribution, backbone, and dependency relations. During execution, the system expands phases on demand, filters historical dependencies for the current step, and writes execution results back into the graph to support subsequent actions. The paper evaluates the approach across five settings: tool use, historical interaction QA, dialogue summarization, multi-document QA, and long-form generation. Furthermore, it analyzes the contribution of specific design components through ablation studies on the tool use task.
Critical strengths
The paper addresses the meaningful problem that "historical information may be semantically relevant but not necessarily meet the actual needs of the current step." It provides a coherent solution through a unified task-and-memory graph, step-oriented dependency filtering, and result back-writing, demonstrating clear value in system design. The experiments also provide substantial support: on StableToolBench, compared to DFS with the same backbone, TaskGraph improves the final answer accuracy by 14.97 and 8.18 percentage points for Qwen and DeepSeek, respectively. Four component ablation studies further support the effectiveness of the main design. Additionally, the long-form generation experiments, which utilize human evaluation, showcase the potential of leveraging distant historical information to advance plot development. These results make the method worthy of further research and application.
Critical weaknesses
- The alignment and modification rules for task paths are unclear. Appendix A.2 does not explain how new inputs are mapped to existing paths, nor when the system should continue, modify, or create a path; Appendix A.3 further stipulates that already-expanded phases usually retain their original decomposition. As a result, it remains unclear how the system adjusts or creates branches when execution results invalidate the original plan, limiting the assessment of its dynamic execution capability.
- Performance on dynamic state and temporal reasoning is weak. TaskGraph achieves 20.9% accuracy on the LongMemEval Dynamic category, lower than Atomic RAG’s 50%; on the MultiHop-RAG Temporal category, it scores 58.82%, below all three baselines. This suggests that maintaining historical order and state records is not yet sufficient to ensure information applicability, and the paper lacks further analysis distinguishing errors from state extraction, outdated-value handling, retrieval, and final reasoning.
- The three recall channels lack independent ablation. The existing Hard-Dependency Only ablation changes the dependency construction method as a whole, making it impossible to identify the individual contributions of the structural, semantic, and activity channels. The candidate-count statistics in Figure 8 also do not establish their complementarity; channel-by-channel comparisons with controlled candidate counts are still needed.
- Attribution and generalization evidence for long-horizon capability is limited. In M5, each method generates only one novel, with long-distance dependency evaluation based on 10 foreshadowing items; local coherence covers only a subset of chapters. The paper also does not analyze performance by task length or dependency distance, so it is difficult to judge how the advantage changes as tasks become longer and whether it generalizes stably.
- Key details of context packing are missing. Figures 1 and 2 use Pack(B), but B is not clearly defined, and the paper does not explain how dependencies are preserved and evidence is truncated when content exceeds the limit. This directly affects the actual execution input and makes it difficult to determine whether the budget and truncation rules are consistent in the Flat-Context ablation.
Questions for authors
- How are new inputs aligned with existing task paths and nodes? When execution results conflict with the original plan, what rules determine whether the system continues along the current path, modifies it, or creates a new branch? Please provide a complete worked example.
- At which stage do most Dynamic and Temporal errors occur? For multiple historical values of the same variable, how does the system select the applicable state based on the temporal conditions in the question?
- Are there results that remove the structural, semantic, and activity channels one at a time? When the number of candidates is controlled, does each channel still contribute independently?
- Can existing logs be used to report performance by task length or dependency distance? Are there results from other independent long-horizon tasks that support the generalizability of the current findings?
- How is B in Pack(B) defined? When the limit is exceeded, how does the system retain necessary dependencies while truncating other content? Does Flat-Context use the same budget and truncation rules?
2 comments
Thank you for the follow-up and the concrete examples from the execution logs. The traces clarify how failed prerequisites affect scheduling, and the historical-state examples demonstrate how graph expansion can recover relevant earlier evidence. Together with the additional ablations and common-budget comparisons, these results strengthen my assessment of the method’s practical value. I will raise my rating to 3.
Thank you for the detailed response and additional experiments. The channel-wise ablations, execution-length analysis, and repeated long-form generations provide useful evidence, and the context-packing description clarifies the implementation. My main concerns remain the handling of tool observations that invalidate an existing plan and the selection of applicable historical states for dynamic and temporal questions. The added explanations and diagnostics clarify the current behavior, while reliable handling of these cases remains insufficiently demonstrated. I therefore retain my original rating.