- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a multimodal deep-research agent for report generation at a requested level of detail (MIRA), together with an execution harness (MIRA-Harness) and a trajectory-construction pipeline (MIRA-Data). MIRA-Harness uses generation contracts, namely sets of hard acceptance criteria covering report granularity, multimodal composition, and evidence grounding; a report gate later checks the report draft against those contracts and provides feedback for iterative revision. It combines MIRA-Data report generation trajectories with general search data to conduct SFT on Qwen3.6-35B-A3B. In experiments, MIRA-SFT improves over MIRA-Base on all eight benchmarks, but the gain is small on the report-generation benchmark and larger on the seven multimodal search and VQA benchmarks.
Critical strengths
- The same-backbone comparisons provide evidence for the effectiveness of MIRA-Harness across the evaluated benchmarks, and SFT yields more pronounced gains on the seven multimodal search and VQA benchmarks.
Critical weaknesses
- The controllability of generation contracts is insufficiently validated. Their requirements are not adaptive, only instantiated through two fixed presets (concise or detail). And the evidence-grounding component checks source counts and citation provenance, but not whether cited sources support the report’s claims. The paper also doesn’t report MMDR’s TRACE/MOSAIC metrics, which can diagnose these failures.
- The contribution of MIRA-Data is not isolated. MIRA-SFT mixes 4.6K Harness-Native trajectories with 33K general search trajectories, so the gains cannot be directly attributed to the proposed data construction. This is particularly relevant because report generation is the paper’s core target, but MIRA-SFT improves over MIRA-Base by only 0.71 points on MMDeepResearch-Bench.
Questions for authors
- The evaluation includes only one multimodal deep-research report-generation benchmark with limited gains. Could the authors provide more results on additional report-generation benchmarks?
- The paper claims it yields competitive performance at the frontier scale. Could the authors report efficiency metrics (such as token usage, API cost, wall-clock time, and tool calls) for a comprehensive comparison with larger frontier models?
7 comments
Response to Reviewer 7MoM (Part 2/5)
(W1b): The evidence-grounding check counts sources and verifies citation provenance but not whether the cited sources support the report's claims, and the paper does not report MMDR's TRACE/MOSAIC metrics, which could diagnose such failures.
On the same backbone, MIRA-Harness outperforms the Vanilla Harness (equipped with a basic search toolset) on TRACE, which measures whether cited sources support the report's claims, and on MOSAIC, which measures text–image consistency.
We break down the MMDR score of the Qwen3.6-35B-A3B backbone in three settings: without a harness, with the Vanilla Harness, and with MIRA-Harness (MIRA-Base). FLAE scores report quality, TRACE scores whether citations support and align with the claims (including visual-evidence fidelity), and MOSAIC scores text–image consistency:
| Setting | FLAE | TRACE | MOSAIC | Overall |
|---|---|---|---|---|
| Qwen3.6 w/o Harness | 40.16 | 23.52 | 23.30 | 26.78 |
| Qwen3.6 w/ Vanilla Harness (basic search toolset) | 43.45 | 39.33 | 32.00 | 37.95 |
| MIRA-Base (MIRA-Harness) | 46.46 | 44.03 | 35.32 | 41.90 |
| Δ (MIRA-Base − Vanilla) | +3.01 | +4.70 | +3.32 | +3.95 |
With the backbone fixed, MIRA-Harness improves on all three metrics, and the setting without a harness is lowest on every metric. The report gate requires every citation to point to a page the agent opened and every image to keep its source-page provenance (Appendix A.2), so each claim and figure is tied to evidence the agent actually inspected.
Whether cited sources support the claims is therefore measured directly by TRACE, and MIRA-Harness improves it.
Response to Reviewer 7MoM (Part 3/5)
(W2): The contribution of MIRA-Data is not isolated: MIRA-SFT mixes 4.6K Harness-Native and 33K general search trajectories, so the gains cannot be attributed directly to the proposed data construction. This matters because report generation is the core target, yet MIRA-SFT improves over MIRA-Base by only 0.71 points on MMDR.
Each data source alone raises the average over MIRA-Base by about 3 points (59.36 and 58.61), and training on both gives the best average (61.34). We trained the same backbone on each source alone and on both.
| Setting | Post-training data | MMSearch | MMSearch-Plus | BC-VL | SimpleVQA | LiveVQA | FVQA | VDR-Bench | MMDR | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|
| (a) MIRA-Base | none | 73.10 | 33.44 | 56.64 | 74.33 | 78.33 | 70.00 | 19.00 | 41.90 | 55.84 |
| (b) Harness-Native only | 4.6K Harness-Native | 74.85 | 37.98 | 57.62 | 76.67 | 82.00 | 70.33 | 30.80 | 44.60 | 59.36 |
| (c) General only | 33K general search | 78.95 | 35.37 | 52.38 | 80.00 | 82.67 | 76.33 | 23.40 | 39.75 | 58.61 |
| (d) MIRA-SFT | both (MIRA-Data) | 80.70 | 38.91 | 58.40 | 79.00 | 82.00 | 79.33 | 29.80 | 42.61 | 61.34 |
Training on either source alone is stronger on the tasks that match it: general data on short, VQA-style search, and Harness-Native data on report writing within the harness. Training on both gives the best average.
This follows the complementary design of Section 3 (Capability-Balanced Data Mixture). The gains of MIRA-SFT therefore do not come from general search supervision alone; the Harness-Native trajectories are an essential part of MIRA-Data.
Response to Reviewer
7MoM(Part 1/5)We thank Reviewer
7MoMfor the recognition that the same-backbone comparisons support the effectiveness of MIRA-Harness and the constructive questions.Concise and Detail are ranges rather than fixed presets, and MIRA-SFT also strictly meets user-defined values, including requirements that cross the two presets, with high report quality. Concise covers 300–2,000 characters and Detail 5,000–10,000 (Table 5), so each preset already spans a range. Beyond the two presets, the user can specify custom values with each request, which are written into the system prompt (
Requirements: {L_min}–{L_max} characters, at least {N_img} images and {N_src} sources). We test four requirement sets on two queries drawn from MMDR: one inside Concise (C1), one inside Detail (C2), and two that cross Concise and Detail (C3, C4).The generation contract is therefore a set of user-set values, not a choice between two fixed presets.