acceptodds
Withdrawn from ICLR 2027

MosaicDrive: Task-Aware Multi-Camera Reasoning for Autonomous Driving

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Tool-augmented driving agents typically acquire additional camera views when they are uncertain about what they have already seen. We identify a structural limitation of this design: the confidence that triggers retrieval is computed from evidence the agent already holds, and so carries no information about regions it has never observed. Acquiring more evidence also introduces a second difficulty: a newly retrieved observation can contradict or override a correct prior judgment, so broader acquisition is not monotonically beneficial. We therefore treat acquisition and verification as parts of a single decision. We present MosaicDrive, a driving agent framework that derives camera and tool requirements from the question, requires the selected evidence before producing an answer, and resolves the resulting observations through Arbiter, a source-aware verifier that checks claims against available evidence, issues targeted re-queries, and preserves conflicts that cannot be settled. We evaluate on MOSICQA, a multi-camera driving question-answering resource with executed tool traces and scene-disjoint splits, under a protocol that separates coverage from selectivity, correction from abstention, and answer quality from acquisition cost. Controlled camera-policy comparisons and evidence-corruption tests isolate the contribution of task-conditioned acquisition and of explicit verification.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous driving, vision-language models, active perception, multi-camera reasoning, tool-using agents

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pFPhOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

MosaicDrive combines question-conditioned camera/tool selection, a source-aware evidence store, and an Arbiter that checks conflicting observations and requests additional evidence. The paper introduces MOSICQA and evaluates acquisition policies, answer-model training, and conflict handling using automatic answer grading, an annotation-based acquisition benchmark, and synthetic and real disagreement studies.

Critical strengths
  1. The evaluation includes useful diagnostic controls: matched-budget random selection, image removal/replacement, and scene-level confidence intervals. The transparent reporting of negative results helps distinguish evidence acquisition from actual evidence use.

  2. The separate annotation-based acquisition benchmark provides a more credible assessment of camera selection than agreement with generated answers, using disjoint scenes and explicit visibility controls.

Critical weaknesses
  1. Limited methodological novelty. Question-conditioned visual selection, progressive tool-use training, and conflict-aware evidence coordination are already explored in Frame-Voyager, DriveAgent-R1, and InfoCoordiBridge, respectively. MosaicDrive primarily combines fixed acquisition mappings with threshold checks and re-query heuristics (section 3.2 and 3.3). The substantive methodological advance beyond this integration remains unclear.

  2. The introduction claims that unseen regions produce no uncertainty and that adjusting retrieval thresholds cannot help. However, an agent can recognize missing evidence from the question and its observation history. The paper neither defines the relevant confidence signal precisely nor evaluates a question- and coverage-aware uncertainty baseline.

  3. The fixed acquisition prior lacks convincing effectiveness. On the annotation-based benchmark, category-mapped selection achieves 59.5% accuracy, essentially matching random selection at 59.4% with the same camera budget (Table 3). Thus, the high agreement with legacy camera annotations in Table 4 does not establish that the prior selects more useful evidence.

  4. Successful offline acquisition uses one training-derived camera-subset label per named region (section 4.3; Appendix F). A direct lookup of those same labels is not evaluated. Consequently, the reported gains do not establish what policy learning contributes beyond reproducing a region-to-camera rule.

  5. MOSICQA uses generated, unreviewed references (section 4.1). More concerningly, removing all images improves the SFT model’s grade by 2.4 points, while substituting another scene’s images has no significant effect (section 4.6). The full system also scores below front-only input, 24.8 versus 28.0 (Table 2).

Questions for authors
  1. How does the learned acquisition policy compare with directly executing its training-derived region-to-camera lookup table across the reported acquisition costs?

  2. Which evaluated setting demonstrates improved factual answer correctness from executable conflict-triggered re-query, rather than simulated returns or abstention? Can this benefit be established when either source may be erroneous, with answer coverage and query budget controlled

  3. Tier 2 lacks corresponding validation evidence. Table 1 marks semantic verification as evaluated on conflict subsets, but Appendix D states that these studies contain only structured distance claims and exercise Tier 1 and re-query. The semantic tier’s evaluation status needs clarification.

  4. The “matched-budget” baseline does not match executed calls. Tables 2 and 13 report 8.4 calls for random selection versus 7.7 for the proposed mapping. Please clarify whether matching concerns tool types, requests, or actual executed acquisition cost.

Reviewer_Ww47OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that confidence-triggered view acquisition in driving agents is limited in a structural way: confidence is computed from evidence the agent already has, so it carries no signal about regions it never observed. The paper also notes that newly acquired views can override correct judgments. MosaicDrive addresses this in two ways. First, it selects cameras and tools from the question through a keyword classifier and a fixed category-to-camera/tool mapping. Second, a verifier called Arbiter checks claims across sources, re-queries on disagreement, and flags conflicts it cannot resolve. The answer model is Qwen3.5-4B with LoRA, trained with SFT, a GRPO-style stage, and acquisition RL. The authors introduce MOSICQA, 5.6k GPT-4o-generated QA pairs on nuScenes. Evaluation covers four settings: camera policies on 1,719 held-out questions graded by an LLM judge, a new 1,800-question yes/no benchmark with answers from nuScenes 3D annotations, a synthetic corruption study for Arbiter, and 403 real cross-camera object pairs.

Critical strengths

The reporting is transparent. The paper includes an implementation-status table, scene-level confidence intervals, and sensible controls such as a no-image condition, a length-controlled regression, and a second judge, and it openly reports negative results. This makes the work easy to audit.

The exact-match acquisition benchmark is also a useful contribution. Its answers come from 3D annotations and its splits are scene-disjoint, and it shows clearly that acquiring the relevant camera matters (recall of 33.0% vs. 7.3%).

Critical weaknesses

The central acquisition claim is not supported by the results. For the task-trained models, the category prior performs worse than front-only for SFT-v2 (−3.2) and no better for the RL model, and on both benchmarks it is indistinguishable from random cameras at the same budget. Six raw images without tools outperform the full pipeline by 6.8 points. On the exact-match benchmark, letting the 72B reader choose cameras works best (67.3%), ahead of both the fixed prior and the learned policy, which the authors themselves describe as a region-to-camera rule. Because that benchmark states the region in the question, it also cannot test whether confidence-gated acquisition has the limitation claimed in contribution 1, and I found no other experiment that does.

Arbiter is not shown to work in practice. The deployed pipeline detected no conflicts in 67k runs. The 79.6% resolution rate follows from a simulated re-query with a fixed 80% success rate, and the real-pair study relies on oracle identity with a re-query that has no effect. The headline real-conflict number is also misreported. The paper says Arbiter is “wrong only 1.2% of the time it commits,” but Appendix D shows 2 errors out of 17 commitments (about 12%). The 1.2% is computed over all 171 cases, most of which are abstentions. More broadly, MOSICQA references are unreviewed GPT-4o outputs, the judge marks 72% of answers incorrect, and the RL prototype is trained and evaluated with the same judge. Together these limit what the main benchmark can say about correctness.

Questions for authors
  1. Is there direct evidence for contribution 1, for example a comparison against a confidence-triggered agent on questions whose evidence lies outside the front view and which do not name the region?
  2. Can Arbiter be evaluated without oracle identity and with a re-query that can return different evidence? How often does it then link the wrong objects?
  3. Please clarify how the “1.2% when it commits” figure was computed.
  4. Were the GPT-4o camera annotations used in Table 4 produced with the same category-to-camera mapping? If so, the reported F1 is partly circular.
  5. Is there a human-verified subset of MOSICQA on which the main conclusions still hold?
Reviewer_Y8inOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MosaicDrive combines task-conditioned camera/tool acquisition with source-aware evidence verification for static multi-camera driving question answering. A rule-based classifier maps questions to predefined acquisition sets, while Arbiter checks claims about aligned referents, requests additional evidence, or leaves disagreements unresolved. The paper also introduces MOSICQA, adapts an answer model, and evaluates offline learned acquisition on a separate annotation-grounded benchmark.

The offline acquisition policy achieves 64.8% accuracy using approximately 1.5 views, compared with 57.2% for front-only input and 61.4% for all six cameras. However, the fixed category mapping performs similarly to matched-budget random selection, and the complete pipeline does not demonstrate a consistent answer-quality improvement on MOSICQA.

Critical strengths
  1. The paper identifies a meaningful acquisition-and-verification problem. Confidence in currently available observations does not ensure coverage of task-relevant regions, and additional observations can introduce errors rather than improve an answer. Studying evidence selection together with source-aware verification is well motivated.

  2. The evaluation provides valuable diagnostics and transparently reports negative results. Scene-disjoint evaluation, matched-budget controls, counterfactual camera removal, image-removal tests, and separate annotation-grounded acquisition experiments help expose weaknesses that aggregate answer scores would conceal. The explicit distinction between implemented, trained, and evaluated components is also useful.

Critical weaknesses
  1. The evidence does not establish the effectiveness of the central acquisition mechanism or the complete framework. On the annotation-grounded benchmark, the fixed category mapping achieves 59.5% accuracy versus 59.4% for matched-budget random selection. The offline learned policy is promising, but learns a region-to-camera rule from questions that explicitly identify the region, rather than a visually grounded acquisition policy; live RL does not learn tool use. On MOSICQA, the SFT-v2 complete pipeline scores below front-only input, and removing images improves its grade. These findings substantially limit the benchmark’s ability to validate the proposed visual acquisition-and-verification framework. The authors acknowledge these limitations, but the resulting positive evidence remains narrower than the method’s intended contribution.

  2. Real-world conflict resolution remains insufficiently demonstrated, and one reliability statement uses an incorrect denominator. The end-to-end pipeline detects no natural conflicts because referents are camera-scoped. The separate real-conflict study uses oracle identity, while deterministic re-querying provides no new evidence. The 79.6% synthetic resolution rate depends on an explicitly simulated re-query reliability assumption. On real disagreements, Arbiter leaves 154 of 171 cases unresolved and commits on 17, making two errors. Thus, 1.2% is the error rate over all disagreement cases; the error rate conditional on committing is 2/17, approximately 11.8%. Describing 1.2% as the rate “of the time it commits” overstates selective reliability. Coverage and conditional error should be reported together.

Questions for authors
  1. Can task-conditioned acquisition outperform matched-budget random and question-conditioned adaptive selection on a human-verified benchmark whose answers genuinely require the acquired images? Please distinguish benefits of the fixed mapping, offline region rule, and complete pipeline.

  2. Can Arbiter be evaluated with non-oracle referent association and an executable re-query mechanism that obtains additional information? Please report correct resolution, unresolved coverage, and error conditional on committing, and correct the denominator used for the 1.2% statement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.