- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Cirdan, a framework for diagnosing failures in distributed systems. It organizes competing explanations and their supporting evidence in a tree structure and performs diagnosis through three stages. Experiments show promising results across multiple models and in comparisons with several agent frameworks.
Critical strengths
- This paper is well motivated: investigating upstream root causes helps prevent models from progressing further down incorrect investigative paths.
- This paper provides extensive experimental evaluation and reports both the method’s applicability limits and the settings in which it underperforms.
Critical weaknesses
-
One claimed innovation of this paper is deep-branch backtracking. However, all ten structured trajectories in Appendix H already contain the main failure mechanism during the initial survey, with no subsequent deep branch switches, revisits, or second-level expansions. This does not demonstrate the mechanism’s effectiveness.
-
In my opinion, one innovation of this paper appears to be tree-structured exploration of competing explanations and backtracking. However, similar mechanisms already exist in methods such as SWE-Search, leaving the incremental novelty unclear.
Questions for authors
- Can the authors provide direct evidence that deep branching and backtracking improve diagnosis?
- What specific advances distinguish Cirdan from SWE-Search, rather than merely a difference in application domains, because SWE-Search is mainly used to output solutions for software problem repair scenarios, which in my view covers the fault localization scenario in this paper.
- Which design choices specifically target distributed-system diagnosis, and can Cirdan generalize to other software diagnosis settings?