- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ScoutCrew is a coordinated multi-agent framework for spatial reasoning from limited input views. A Coordinator decomposes the question into observation subtasks and jointly constructs an Investigator Relation Graph (IRG), whose Complement edges link tasks supplying different premises of a joint inference and CrossCheck edges link tasks examining the same spatial proposition. Each Investigator executes its planned camera trajectory through Stable Virtual Camera (SVC) generative view synthesis and produces a report (tentative local answer, spatial findings, evidence gaps); Investigators exchange reports and observation context along the IRG and revise their findings; an Analyst assembles the revised reports into a Chain of Evidence (CoE) connecting observations and intermediate conclusions to the final answer, with unresolved gaps triggering replanning. On SAT-Real and MMSI-Bench, ScoutCrew improves overall accuracy across all evaluated backbones (mean +7.0 and +5.2 points; up to +14.0 and +9.5), with detailed ablations and an inference-cost analysis.
Critical strengths
A clearly articulated and operationalized principle: plan what to observe jointly with how observations will be combined and checked. The Complement/CrossCheck distinction is intuitive, and the ablations convincingly show that uncoordinated extra observations can hurt (fixed tasks: 60.7% vs. 64.7% direct), isolating coordination — not view synthesis per se — as the source of the gains.
Thorough process-level evaluation. Consistent gains across seven SAT-Real and six MMSI-Bench backbones, an evidence-integration study characterizing fixes vs. breaks (71.4% override precision vs. 63.3% for majority voting), and an inference-cost comparison showing higher accuracy with far fewer tokens/views than MindJourney (18.4K vs. 77.6K tokens; 2.03 vs. 10.92 views).
Critical weaknesses
-
Limited novelty relative to the strength of evidence. The framework is an orchestration-level combination of existing components (multi-agent decomposition, graph-structured communication, generative view synthesis, chain-of-evidence synthesis) with no new learning mechanism or benchmark; the method is essentially prompt-level control over black-box backbones. Notably, the core "active viewpoint reasoning" idea already exists in Chain-of-View (CoV) prompting (https://aclanthology.org/2026.findings-acl.1623/) , a training-free coarse-to-fine view-exploration framework with a View Selection agent, which the paper neither cites nor positions against; ScoutCrew's actual contribution is the coordinated multi-agent/IRG layer, and the paper should demonstrate what that layer adds over such single-agent viewpoint planning. The decisive weakness for acceptance is that gains concentrate on weaker backbones (GPT-4o +14.0, GPT-4.1 +12.0) and shrink to +1.4 (Gemini 3.8 Flash) and +2.6 (GPT-5.6 Sol) on the strongest — the value proposition erodes exactly as base models improve, and no evidence is given that the framework scales its advantage to state-of-the-art backbones.
-
The evaluation rests on small benchmarks. Several per-category cells reach 100%, indicating small sample sizes, so the headline percentage-point gains may be noisy; there is no human evaluation of the quality of the synthesized evidence chains, and failure cases (beyond the breaks analysis) are not characterized. As it stands, the paper demonstrates a plausible, well-engineered pipeline with suggestive but not conclusive evidence.
Questions for authors
What are the per-category sample sizes on SAT-Real and MMSI-Bench? Given the small gains on the strongest backbones, can the authors clarify how reliably the framework helps. i.e., whether the reported gains hold consistently across questions or reflect benchmark noise?
Why do gains collapse on the strongest backbones (Gemini 3.8 Flash +1.4, GPT-5.6 Sol +2.6)? Is the coordination mechanism redundant when the base model already reasons spatially, or is the bottleneck elsewhere (e.g., SVC view fidelity)?
When does the crew harm performance beyond the breaks analysis? e.g., for questions the input view already suffices to answer, is there a mechanism to detect and skip exploration?
The framework depends on SVC's generative view-synthesis quality. how do gains vary with the fidelity of synthesized views, and would alternative observation mechanisms (e.g., explicit 3D reconstruction / generalizable 3DGS) change the results?
The spatial-reasoning related work should engage Chain-of-View (CoV) prompting. Please clarify the differences in planning mechanism (single-agent coarse-to-fine exploration vs. coordinated multi-agent IRG) and in which settings ScoutCrew's coordination is preferable to CoV-style view selection.