- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Agentic Attribution, a framework that explains why LLM agents take specific actions beyond traditional failure analysis. It uses temporal likelihood dynamics to identify influential trajectory components and perturbation-based analysis to locate key sentences. The framework is evaluated on Llama-3.1-70B-Instruct across agent scenarios and benchmarks (GAIA, WebShop, ALFWorld), showing reliable identification of decision drivers and improving agent transparency and accountability.
Critical strengths
- This paper identifies an important problem: Agentic Attribution.
- The authors manually annotate and analyze attribution data from 8+53 trajectories.
Critical weaknesses
- The proposed Agentic Attribution Framework seems like a straightforward application of previous attribution methods to agent trajectories, such as ContextCite [1] TracLLM [2], TRACE [3], etc. It also adopts similar ideas, including counterfactual / perturbation-based methods and coarse-to-fine attribution. Therefore, I remain skeptical about the novelty and contribution of this work.
- The experimental design makes me unconvinced about the effectiveness of the proposed method:
- The proposed method is only evaluated based on the analysis of 8+53 trajectories, which is too limited in scale.
- The paper does not compare against previous works, especially given the strong similarity between existing attribution methods and the proposed framework. I am curious whether previous attribution methods can be directly applied to agent trajectories.
- The paper only uses Llama-3.1-70B-Instruct, which is too limited and somewhat outdated. I am curious whether the proposed framework can be applied with more advanced open-source models and achieve better attribution performance.
[1] ContextCite: Attributing Model Generation to Context (NeurIPS 2024)
[2] TracLLM: A Generic Framework for Attributing Long Context LLMs (SEC 2025)
[3] ContextCite: Attributing Model Generation to Context (Arxiv 2026)
Questions for authors
- How does the proposed Agentic Attribution Framework differ fundamentally from existing attribution methods such as ContextCite, TracLLM, and TRACE, beyond adapting them to agent trajectories?
- Have the authors evaluated whether existing attribution methods can be directly applied to agent trajectories as baselines? If not, why?
- Does the proposed framework generalize to more advanced open-source models, and how does the choice of the underlying LLM affect attribution performance?