- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents CHORUS, an LLM-based multi-agent system for spatial biology that treats spatial transcriptomics (ST) and H&E histology as independent sources of evidence. A Coordinator Agent routes each question to an ST Agent, an H&E Agent, or both. In joint tasks, each specialist first analyzes only its own modality and records its tool outputs as a provenance-grounded evidence logic tree that separates evidence from inferences and conclusions.
Cross-modal interaction is constrained in four ways: (i) a scientific-state gate compares claims only after specimen, coordinate, registration, and ROI correspondence are verified, and otherwise labels them incomparable rather than conflicting; (ii) during discussion, evidence nodes are immutable, and specialists can only keep or revise their own inferences/conclusions or request new evidence (formalized in Theorem 1 as evidence preservation and order independence); (iii) a Judgment Agent turns unresolved disagreements into targeted evidence requests instead of forcing consensus; and (iv) a state-conditioned experience memory reuses past tool trajectories only under compatible scientific states, so behavior adapts without model updates.
Experiments mainly use 12 Visium oral squamous cell carcinoma sections (GSE208253), plus several Xenium sections and case studies, with Qwen3-8B and DeepSeek-flash as backbones. Reported results:
- Higher agreement with published pathologist region annotations than Leiden, Louvain, SpaGCN, and an spEMO-CONCH adaptation (e.g., ARI 0.332 vs. 0.223 for Leiden).
- Exposing the correspondence matrix to the judgment step makes it visible in 16/22 runs (vs. 0/22) but leaves partition agreement essentially unchanged.
- Removing the gate, evidence acquisition, or discussion barely affects workflow completion, whereas the backbone matters much more (27/28 vs. 8/28 ST-verified dual-branch executions with DeepSeek-flash vs. Qwen3-8B).
- In a five-task user study, CHORUS completes tasks faster than human experts (13.4 vs. 22.0 min on average) and achieves a higher success rate than a DeepSeek baseline (86.4% vs. 30.0%).
- In an appendix experiment, state-conditioned memory raises accuracy from 0.40 to 1.00, compared with 0.22 for state-shuffled memory.
Critical strengths
S1. A principled design for evidence-grounded cross-modal collaboration. Tool-grounded evidence is separated from inferences and conclusions in provenance-tracked logic trees; evidence stays immutable during discussion, and specialists may revise only their own inferences or conclusions. A scientific-state gate returns "incomparable" rather than "conflict" when specimen, coordinate, registration, or ROI correspondence is unverified. These choices target real failure modes of free-form multi-agent debate (persuasion overriding observations, forced consensus) and of naive multimodal pipelines (comparing unregistered data), and they transfer to other agentic-science settings. This is the main consideration in favor of acceptance.
S2. Evaluation that separates process completion from verified execution. Table 4 reports process completion, ST-verified dual-branch execution, and conclusion production separately, and shows that they can diverge sharply (e.g., Qwen3-8B completes 26/28 runs but has verified dual-branch execution in only 8/28). This is a useful methodological point for auditing agentic scientific workflows.
Critical weaknesses
W1. The central claims are not supported by the reported evidence. The abstract and conclusion state that preserving evidence and enforcing state compatibility improve cross-modal decisions and that state-conditioned memory improves accuracy, and the contribution list says the experiments "demonstrate the superiority of our agent." However:
- In the component ablations (Table 4), removing the alignment gate, evidence acquisition, or discussion leaves DeepSeek-flash essentially unchanged (26–27/28 completed and ST-verified), and for Qwen3-8B every removal increases ST-verified dual-branch execution (8/28 → 10–13/28). DeepSeek-flash produces conclusion text in 0/28 runs under every setting.
- Exposing the executed correspondence matrix to the judgment step (Table 2) leaves partition agreement unchanged.
- The memory result (Fig. 4) appears only in the appendix, without a description of its 90 probes, and the appendix ("Cost and interpretation") itself states that the results do not establish a general accuracy benefit from memory or universal superiority.
This matters because the contribution rests on these mechanisms making cross-modal agent reasoning more reliable, and the reported evidence does not show that they do.
W2. The evaluation is incomplete, and its main favorable comparison is hard to interpret.
- Table 1 and the appendix describe the multi-step quantitative ST tasks (T2), the flat vs. tree evidence-format comparison (T3), and the adjudication replay and constraint checks (T4), but report no results for them beyond inference cost (Table 5). These are the experiments best suited to test numerical correctness, evidence organization, and the interaction constraints.
- Table 3 compares "one previously selected CHORUS output" per section with five runs per comparator (Appendix, "Partition agreement and comparisons"), although the table lists 60/12 runs for CHORUS, and it includes section s9, which was reserved for development. The joint-workflow runs in Table 2 reach only ARI ≈ 0.14, below Leiden's 0.223 in Table 3, and this gap is not explained.
- There is no comparison with the spatial-biology agents discussed in the related work (e.g., STAgent, SpatialAgent, ChatSpatial, STAT). The user study (Fig. 2) compares only with a DeepSeek baseline whose configuration is not described and gives no protocol details. The main quantitative evaluation uses 12 sections from a single OSCC dataset without established patient-level independence.
This matters because the only quantitative evidence favoring CHORUS over baselines may reflect output selection, and the experiments meant to validate the core mechanisms have no reported outcomes.
Questions for authors
W1. The central claims are not supported by the reported evidence. The abstract and conclusion state that preserving evidence and enforcing state compatibility improve cross-modal decisions and that state-conditioned memory improves accuracy, and the contribution list says the experiments "demonstrate the superiority of our agent." However:
- In the component ablations (Table 4), removing the alignment gate, evidence acquisition, or discussion leaves DeepSeek-flash essentially unchanged (26–27/28 completed and ST-verified), and for Qwen3-8B every removal increases ST-verified dual-branch execution (8/28 → 10–13/28). DeepSeek-flash produces conclusion text in 0/28 runs under every setting.
- Exposing the executed correspondence matrix to the judgment step (Table 2) leaves partition agreement unchanged.
- The memory result (Fig. 4) appears only in the appendix, without a description of its 90 probes, and the appendix ("Cost and interpretation") itself states that the results do not establish a general accuracy benefit from memory or universal superiority.
This matters because the contribution rests on these mechanisms making cross-modal agent reasoning more reliable, and the reported evidence does not show that they do.
W2. The evaluation is incomplete, and its main favorable comparison is hard to interpret.
- Table 1 and the appendix describe the multi-step quantitative ST tasks (T2), the flat vs. tree evidence-format comparison (T3), and the adjudication replay and constraint checks (T4), but report no results for them beyond inference cost (Table 5). These are the experiments best suited to test numerical correctness, evidence organization, and the interaction constraints.
- Table 3 compares "one previously selected CHORUS output" per section with five runs per comparator (Appendix, "Partition agreement and comparisons"), although the table lists 60/12 runs for CHORUS, and it includes section s9, which was reserved for development. The joint-workflow runs in Table 2 reach only ARI ≈ 0.14, below Leiden's 0.223 in Table 3, and this gap is not explained.
- There is no comparison with the spatial-biology agents discussed in the related work (e.g., STAgent, SpatialAgent, ChatSpatial, STAT). The user study (Fig. 2) compares only with a DeepSeek baseline whose configuration is not described and gives no protocol details. The main quantitative evaluation uses 12 sections from a single OSCC dataset without established patient-level independence.
This matters because the only quantitative evidence favoring CHORUS over baselines may reflect output selection, and the experiments meant to validate the core mechanisms have no reported outcomes.
Dear reviewers,
Thank you very much for your time and effort in reviewing this submission. The authors have now submitted their rebuttal, and the discussion period is nearing its end. We would greatly appreciate it if you could take some time to carefully review the authors’ responses and share your assessment of whether your concerns have been adequately addressed. Thank you again for your valuable time and contributions to the review process.