- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduces Mechanova, an agent-based framework designed to study how immune-cold tumors can be converted into immune-hot tumors. It arranges evidence drawn from the literature into competing causal graphs, chooses informative experiments, updates causal relationships according to observed results, and produces intervention plans. The framework also incorporates statistical predictors for biological responses and research-action outcomes, along with evidence management and hierarchical agent execution
Critical strengths
The paper addresses an interesting problem of evidence-grounded scientific reasoning. The notable strength is the controlled evaluation of structural revision, where the authors use matched experimental settings and graph-substitution tests to assess whether revised mechanisms improve downstream predictions. The reported improvement in Brier score provides meaningful evidence for the value of structural editing. Also, the Mechanism Evidence Chain offers a systematic way to track experimental evidence and constrain causal claims, improving the interpretability of the proposed reasoning process.
Critical weaknesses
The paper's methodological positioning is unclear, and its evaluation does not convincingly support its central claims. Although Mechanova is presented as a unified autonomous scientific discovery framework, most components are evaluated independently on different tasks, without demonstrating their necessity or collective contribution to scientific discovery. The main benchmark shows only modest improvements over a simple title-only baseline (5.6% points), while the structural-revision experiments primarily demonstrate improved probabilistic prediction rather than causal correctness. In the archival intervention evaluation, Mechanova reports 10/10 successful intervention programs but 9 cases remain structurally unchanged, and success is assessed against curated historical outcomes through intervention-class matching. The results provide limited evidence that the proposed mechanism-revision approach actually contributes to improved intervention design.
Besides, the roles of the Biological and Operational World Models remain insufficiently justified. The BWM is not evaluated in the structural-revision pipeline, and the OWM is mainly evaluated on operational efficiency but not how it improves mechanism discovery. The supplementary Replogle experiments compare existing perturbation-prediction model configurations but have no clearly demonstrated connection to the central tumor immune-cold-to-hot task or the proposed causal reasoning framework. These experiments appear to be primarily comparisons of single-cell representations (HVG-based versus STATE embeddings), but not the method's contribution to scientific discovery.
Questions for authors
How are the Biological and Operational World Models integrated into the core mechanism-revision pipeline, and how their predictions influence causal revision or downstream intervention selection?
What is the intended contribution of the Replogle experiments?
How structural revision contributes to more accurate causal mechanisms or better intervention decisions?
Could the authors better explain the contribution of evidence-grounded reasoning to mechanism identification?