- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents TRACE-MAS, a monitoring framework for detecting, diagnosing, and repairing failures in multi-agent systems. It models agent states, message exchanges, and task dependencies through a structured execution graph, learns normal collaboration patterns from benign trajectories, and uses counterfactual restoration to identify failure sources and select corrective interventions. Evaluations show promising improvements in failure detection, root-cause localization, and task recovery.
Critical strengths
- Detecting and correcting failures in multi-agent systems is an important problem for practical deployments.
- The paper goes beyond agent-level failure detection by considering message exchanges and task dependencies and providing corrective interventions.
- The evaluation demonstrates encouraging improvements over existing approaches.
Critical weaknesses
- Incremental novelty. The proposed solution largely combines existing techniques, such as graph-based representation learning, anomaly detection, counterfactual attribution, and training.
- The prevalence and practical significance of the targeted failure types are unclear. Existing work, such as Who&When (ICML 2025) and CORRECT (ICML 2026), has studied agent-level failures, but it is unclear how frequently message-level and dependency-level corruptions occur in realistic multi-agent workloads.
- The evaluation relies on a single backbone model, limiting the generalizability of the results. Additional experiments and a more comprehensive cost analysis would strengthen the paper.
Questions for authors
Thank you for submitting your work. I enjoyed reading the paper and agree that diagnosing and repairing collaboration failures is an important research problem. The event-level perspective is interesting, and the empirical improvements are encouraging. However, I have concerns about the methodological novelty, workload representativeness, and practical applicability of the proposed framework.
-
The individual designs largely build on existing techniques. What are the key technical innovations beyond combining these methods?
-
The paper motivates its approach through message-level and dependency-level attacks. However, how frequently do these failures occur in real-world multi-agent systems compared with agent-level failures?
-
How would TRACE-MAS perform with other backbone models, especially more recent and capable LLMs?
-
What is the end-to-end overhead, including training costs, generation latency, token consumption, and repair execution?