acceptodds
Under review as a conference paper at ICLR 2027

ScoutCrew: Coordinating Multi-View Exploration and Evidence Synthesis for Spatial Reasoning

16 views

Abstract

Spatial reasoning often requires information that is not fully visible in the supplied images. Additional viewpoints can reveal useful cues, but their value depends on what they show and how their findings are combined. We present SCOUTCREW, a coordinated multi-agent framework that plans observations together with their roles in the inference. A Coordinator assigns observation subtasks and constructs an Investigator Relation Graph (IRG). Complement relations connect tasks that supply different premises, while CrossCheck relations connect tasks that examine the same spatial proposition. Investigators execute their plans through generative view synthesis. They then exchange reports and observation context along the IRG to resolve missing information and revise their findings. An Analyst assembles the revised reports into a chain of evidence (CoE), connecting observations and intermediate conclusions to the final answer. ScoutCrew improves overall accuracy across all evaluated backbones on SAT-Real and multi-image MMSI-Bench. Gains over direct inference reach up to 14.0 and 9.5 percentage points, respectively. These results support coordinating observation acquisition and evidence composition for spatial reasoning from limited views.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Spatial Reasoning, Multi-Modality, Multi-View

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HQmWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ScoutCrew is a coordinated multi-agent framework for spatial reasoning from limited input views. A Coordinator decomposes the question into observation subtasks and jointly constructs an Investigator Relation Graph (IRG), whose Complement edges link tasks supplying different premises of a joint inference and CrossCheck edges link tasks examining the same spatial proposition. Each Investigator executes its planned camera trajectory through Stable Virtual Camera (SVC) generative view synthesis and produces a report (tentative local answer, spatial findings, evidence gaps); Investigators exchange reports and observation context along the IRG and revise their findings; an Analyst assembles the revised reports into a Chain of Evidence (CoE) connecting observations and intermediate conclusions to the final answer, with unresolved gaps triggering replanning. On SAT-Real and MMSI-Bench, ScoutCrew improves overall accuracy across all evaluated backbones (mean +7.0 and +5.2 points; up to +14.0 and +9.5), with detailed ablations and an inference-cost analysis.

Critical strengths

A clearly articulated and operationalized principle: plan what to observe jointly with how observations will be combined and checked. The Complement/CrossCheck distinction is intuitive, and the ablations convincingly show that uncoordinated extra observations can hurt (fixed tasks: 60.7% vs. 64.7% direct), isolating coordination — not view synthesis per se — as the source of the gains.

Thorough process-level evaluation. Consistent gains across seven SAT-Real and six MMSI-Bench backbones, an evidence-integration study characterizing fixes vs. breaks (71.4% override precision vs. 63.3% for majority voting), and an inference-cost comparison showing higher accuracy with far fewer tokens/views than MindJourney (18.4K vs. 77.6K tokens; 2.03 vs. 10.92 views).

Critical weaknesses
  1. Limited novelty relative to the strength of evidence. The framework is an orchestration-level combination of existing components (multi-agent decomposition, graph-structured communication, generative view synthesis, chain-of-evidence synthesis) with no new learning mechanism or benchmark; the method is essentially prompt-level control over black-box backbones. Notably, the core "active viewpoint reasoning" idea already exists in Chain-of-View (CoV) prompting (https://aclanthology.org/2026.findings-acl.1623/) , a training-free coarse-to-fine view-exploration framework with a View Selection agent, which the paper neither cites nor positions against; ScoutCrew's actual contribution is the coordinated multi-agent/IRG layer, and the paper should demonstrate what that layer adds over such single-agent viewpoint planning. The decisive weakness for acceptance is that gains concentrate on weaker backbones (GPT-4o +14.0, GPT-4.1 +12.0) and shrink to +1.4 (Gemini 3.8 Flash) and +2.6 (GPT-5.6 Sol) on the strongest — the value proposition erodes exactly as base models improve, and no evidence is given that the framework scales its advantage to state-of-the-art backbones.

  2. The evaluation rests on small benchmarks. Several per-category cells reach 100%, indicating small sample sizes, so the headline percentage-point gains may be noisy; there is no human evaluation of the quality of the synthesized evidence chains, and failure cases (beyond the breaks analysis) are not characterized. As it stands, the paper demonstrates a plausible, well-engineered pipeline with suggestive but not conclusive evidence.

Questions for authors

What are the per-category sample sizes on SAT-Real and MMSI-Bench? Given the small gains on the strongest backbones, can the authors clarify how reliably the framework helps. i.e., whether the reported gains hold consistently across questions or reflect benchmark noise?

Why do gains collapse on the strongest backbones (Gemini 3.8 Flash +1.4, GPT-5.6 Sol +2.6)? Is the coordination mechanism redundant when the base model already reasons spatially, or is the bottleneck elsewhere (e.g., SVC view fidelity)?

When does the crew harm performance beyond the breaks analysis? e.g., for questions the input view already suffices to answer, is there a mechanism to detect and skip exploration?

The framework depends on SVC's generative view-synthesis quality. how do gains vary with the fidelity of synthesized views, and would alternative observation mechanisms (e.g., explicit 3D reconstruction / generalizable 3DGS) change the results?

The spatial-reasoning related work should engage Chain-of-View (CoV) prompting. Please clarify the differences in planning mechanism (single-agent coarse-to-fine exploration vs. coordinated multi-agent IRG) and in which settings ScoutCrew's coordination is preferable to CoV-style view selection.

Reviewer_3kKWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ScoutCrew, a coordinated multi-agent framework for spatial question answering from a limited number of input views. The Coordinator jointly plans observation subtasks and constructs an Investigator Relation Graph (IRG), distinguishing Complement relations, which supply different premises of a joint inference, from CrossCheck relations, which examine the same spatial proposition.

Each Investigator uses pretrained Stable Virtual Camera (SVC) to synthesize a planned view and examines it together with the original inputs. It produces a report containing a tentative local answer, spatial findings, and evidence gaps. Investigators then exchange reports and observation context along the IRG to revise their findings. The Analyst organizes supported findings into a chain of evidence (CoE) with source references and dependencies. If evidence remains insufficient, the system requests further investigation when the acquisition budget permits; otherwise, it returns the cached input-based answer with an incomplete-evidence status.

Compared with direct inference using the same answering backbone, ScoutCrew improves overall accuracy across all evaluated backbones on SAT-Real and multi-image MMSI-Bench.

Critical strengths
  1. The framework jointly organizes observation planning and the evidential roles of the resulting findings, with a clear task motivation. Complement relations supply missing premises, while CrossCheck relations examine the same spatial proposition. Messages retain the initial image, camera trajectory, generated view, and report, helping recipients account for object identities and reference frames. CoE synthesis records dependencies between findings and conclusions while preserving unresolved evidence gaps.

  2. Overall gains are observed across both closed- and open-weight backbones, and the supplement transparently reports harmful revisions and statistical limitations. On SAT-Real with GPT-4o, accuracy improves from 97/150 to 118/150 correct answers, with 35 fixes and 14 breaks. The supplement provides paired bootstrap intervals and an exact McNemar test, retains failed items, and distinguishes question resampling from model reruns. It also reports limitations: Perspective accuracy decreases from 22/33 to 20/33, and Appendix H.2 documents a case where report exchange overturns a correct minority report and produces an incorrect consensus.

Critical weaknesses
  1. The evidence-integration comparison does not fully support attribution to individual checks. Section 4.4 attributes the large drops without Source Correction or Alignment Check to the reliability of evidence integration. However, Appendix G.2 explicitly describes the alternatives as frozen-report integration strategies, with existing full-system predictions serving as a fixed reference, rather than a matched end-to-end component ablation. Table S10 further describes a checker protocol with visibility-based selection, forward examination, optional backward alignment, and report-admission rules, whereas full IRG exchange retains complete messages. The differences between 61.3%/62.7% and 78.7% therefore establish poorer performance of these configurations, but do not isolate the contribution of either individual check.

  2. The cost comparison does not establish a complete or consistently scoped end-to-end resource advantage. ScoutCrew reports 18.4K tokens and 2.03 selected additional views per question, compared with 77.6K tokens and 10.92 views for MindJourney. However, selected observations exclude some generation attempts, and SVC rendering incurs costs separate from VLM token usage. The documented renderer default generates eight target frames per invocation, which is distinct from the selected-view count. Moreover, Appendix G.5 states that cached-view baseline token counts exclude acquisition, whereas ScoutCrew and MindJourney retain run-level token counts. These disclosures are useful, but the broad resource-efficiency conclusion requires either consistent accounting or narrower wording.

Questions for authors
  1. Could you map the checker protocol in Table S10 to the full IRG exchange and report-revision process, identifying which operations differ beyond the named check removal? In particular, is there a configuration with both source correction and alignment checking enabled under the same frozen-report protocol? Such a matched control would help determine whether the 61.3%/62.7% versus 78.7% differences can be attributed to individual checks.

  2. The full system and the w/o IRG checks variant answer 118/150 and 113/150 questions correctly, respectively. Could you provide their per-question paired correctness outcomes, fixes and breaks, and a paired uncertainty estimate? Please also clarify which Complement exchanges, CrossCheck operations, report revisions, and CoE synthesis steps are retained or disabled. An expensive full rerun is not necessary, but the interpretation should distinguish a single-component effect from changes spanning multiple stages.

  3. Could you report costs with a consistent accounting scope, separating VLM tokens, generation attempts, internally generated frames, selected observations, retries/failures, and total latency, while marking unmeasured quantities? For cached-view methods, could acquisition costs be accounted for consistently or inference-stage comparisons be presented separately? If complete logs are unavailable, please restrict the conclusion to fewer reported VLM tokens and fewer selected additional views under the evaluated configurations, and clarify whether any comparison uses matched budgets.

Reviewer_QSGWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ScoutCrew, which jointly designs additional-view planning and evidence integration for spatial reasoning from limited images. A Coordinator specifies observation tasks and constructs an Investigator Relation Graph (IRG). Investigators interpret views generated by a Stable Virtual Camera (SVC) alongside the original images, then exchange and revise reports through the IRG’s Complement and CrossCheck relations. An Analyst integrates the evidence into a Chain of Evidence to derive the answer. Experiments across multiple benchmarks demonstrate improvements in accuracy and inference efficiency.

Critical strengths

The experiments provide evidence that evidence alignment and revision help reduce spatial reasoning errors.

Critical weaknesses

W1. The contributions of the IRG and role specialization are insufficiently disentangled from the benefits of additional inference compute. Although Tables 3 and 4 show component ablations, the comparison in §4.5, which matches the average number of additional views, shows that ScoutCrew uses approximately 7.5 times as many tokens as simple image concatenation to improve accuracy from 71.3% to 78.7%. It therefore remains unclear how much the proposed architecture contributes beyond additional reasoning opportunities, relative to alternatives using the same observations and alignment and revision procedures.

W2. The propagation of errors in generated views into subsequent reasoning is insufficiently evaluated. Sections 3.3 and 4.4 describe safeguards including cross-checking, source-dependency correction, and exclusion of unresolved evidence. However, the fixes and breaks in Table 4 do not distinguish errors arising from view generation, image interpretation, or evidence integration. Correlated errors across generated views could cause incorrect claims to be accepted as mutually supporting evidence, leaving the effectiveness of these safeguards unclear.

Questions for authors
  1. [Clarification] How do the Single Investigator, Image-aware Single Analyst, and Full conditions in Tables 3–4 compare in terms of observation images, provenance information, alignment and source-correction rules, model-call counts, and token budgets? If existing comparisons control for these factors, please provide their settings and results.

  2. [Clarification] How do you verify that the spatial relations and realized viewpoints in SVC-generated images are consistent with the original scene and requested camera movements?

  3. [Clarification] Can existing logs or previously examined cases distinguish view-generation errors from interpretation or integration errors among the 14 breaks under the Full condition in Table 4? Please provide examples, if available, of generation errors corrected through cross-checking and errors that remained undetected and affected the final answer.

(If you address all of my concerns, I am willing to update the score.)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.