acceptodds
Under review as a conference paper at ICLR 2027

MIRA: A Multimodal Illustrated Deep Research Agent for Controllable Reports

Abstract

Deep research agents are increasingly capable of searching the open web and synthesizing evidence into comprehensive answers, yet existing agents remain poorly suited to controllable report generation in multimodal deep research, a challenging task in which users provide interleaved textual and visual inputs, agents must gather multimodal evidence through multi-step web search, and users expect a comprehensive, illustrated report rather than a short answer. Two limitations are particularly prominent. During reasoning, existing search agents often underutilize visual information and lack effective mechanisms for interacting with images discovered throughout the search trajectory. During generation, they typically produce brief responses rather than comprehensive reports, while relying solely on base models provides limited user control over three report-level attributes: report granularity, multimodal composition, and evidence grounding. To address these challenges, we introduce MIRA, a Multimodal Illustrated Deep Research Agent for controllable reports. MIRA tightly integrates textual and visual evidence throughout the search process, enabling the agent to inspect, reason over, and reuse intermediate images while preserving strong web-search capabilities. At its core, we develop MIRA-Harness, which translates user requirements into generation contracts over report granularity, multimodal composition, and evidence grounding, and provides four complementary execution stages for improving report quality. Building on this harness, we develop MIRA-Data, a data construction pipeline for producing high-quality multi-hop multimodal research trajectories, and post-train foundation models to operate effectively within the harness. Together, MIRA achieves state-of-the-art performance among open agents on 7 representative, established multimodal-search and deep-research-agent benchmarks, while producing user-aligned illustrated reports.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Search Agent, MultiModel, Harness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7MoMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a multimodal deep-research agent for report generation at a requested level of detail (MIRA), together with an execution harness (MIRA-Harness) and a trajectory-construction pipeline (MIRA-Data). MIRA-Harness uses generation contracts, namely sets of hard acceptance criteria covering report granularity, multimodal composition, and evidence grounding; a report gate later checks the report draft against those contracts and provides feedback for iterative revision. It combines MIRA-Data report generation trajectories with general search data to conduct SFT on Qwen3.6-35B-A3B. In experiments, MIRA-SFT improves over MIRA-Base on all eight benchmarks, but the gain is small on the report-generation benchmark and larger on the seven multimodal search and VQA benchmarks.

Critical strengths
  1. The same-backbone comparisons provide evidence for the effectiveness of MIRA-Harness across the evaluated benchmarks, and SFT yields more pronounced gains on the seven multimodal search and VQA benchmarks.
Critical weaknesses
  1. The controllability of generation contracts is insufficiently validated. Their requirements are not adaptive, only instantiated through two fixed presets (concise or detail). And the evidence-grounding component checks source counts and citation provenance, but not whether cited sources support the report’s claims. The paper also doesn’t report MMDR’s TRACE/MOSAIC metrics, which can diagnose these failures.
  2. The contribution of MIRA-Data is not isolated. MIRA-SFT mixes 4.6K Harness-Native trajectories with 33K general search trajectories, so the gains cannot be directly attributed to the proposed data construction. This is particularly relevant because report generation is the paper’s core target, but MIRA-SFT improves over MIRA-Base by only 0.71 points on MMDeepResearch-Bench.
Questions for authors
  1. The evaluation includes only one multimodal deep-research report-generation benchmark with limited gains. Could the authors provide more results on additional report-generation benchmarks?
  2. The paper claims it yields competitive performance at the frontier scale. Could the authors report efficiency metrics (such as token usage, API cost, wall-clock time, and tool calls) for a comprehensive comparison with larger frontier models?
7 comments

Response to Reviewer 7MoM (Part 1/5)

We thank Reviewer 7MoM for the recognition that the same-backbone comparisons support the effectiveness of MIRA-Harness and the constructive questions.


(W1a): The controllability of generation contracts is not sufficiently validated: the requirements are not adaptive and are instantiated only through two fixed presets (Concise or Detail).

Concise and Detail are ranges rather than fixed presets, and MIRA-SFT also strictly meets user-defined values, including requirements that cross the two presets, with high report quality. Concise covers 300–2,000 characters and Detail 5,000–10,000 (Table 5), so each preset already spans a range. Beyond the two presets, the user can specify custom values with each request, which are written into the system prompt (Requirements: {L_min}–{L_max} characters, at least {N_img} images and {N_src} sources). We test four requirement sets on two queries drawn from MMDR: one inside Concise (C1), one inside Detail (C2), and two that cross Concise and Detail (C3, C4).

ContractRequired (chars / images / sources)Q41: delivered (MMDR)Q104: delivered (MMDR)
C1: inside Concise800–1,200 / ≥ 3 / ≥ 41,128 / 3 / 4 (39.7)989 / 3 / 7 (57.6)
C2: inside Detail6,000–7,000 / ≥ 10 / ≥ 86,668 / 10 / 10 (40.7)6,023 / 10 / 8 (53.1)
C3: Concise length, Concise images, Detail sources300–2,000 / ≥ 2 / ≥ 81,156 / 2 / 8 (59.2)1,645 / 2 / 8 (58.3)
C4: Detail length, Concise images, Detail sources5,000–10,000 / ≥ 2 / ≥ 87,842 / 2 / 8 (64.4)7,119 / 2 / 10 (53.1)

The generation contract is therefore a set of user-set values, not a choice between two fixed presets.

Response to Reviewer 7MoM (Part 2/5)

(W1b): The evidence-grounding check counts sources and verifies citation provenance but not whether the cited sources support the report's claims, and the paper does not report MMDR's TRACE/MOSAIC metrics, which could diagnose such failures.

On the same backbone, MIRA-Harness outperforms the Vanilla Harness (equipped with a basic search toolset) on TRACE, which measures whether cited sources support the report's claims, and on MOSAIC, which measures text–image consistency.

We break down the MMDR score of the Qwen3.6-35B-A3B backbone in three settings: without a harness, with the Vanilla Harness, and with MIRA-Harness (MIRA-Base). FLAE scores report quality, TRACE scores whether citations support and align with the claims (including visual-evidence fidelity), and MOSAIC scores text–image consistency:

Overall=0.2 FLAE+0.5 TRACE+0.3 MOSAIC.\text{Overall} = 0.2\,\text{FLAE} + 0.5\,\text{TRACE} + 0.3\,\text{MOSAIC}.

SettingFLAETRACEMOSAICOverall
Qwen3.6 w/o Harness40.1623.5223.3026.78
Qwen3.6 w/ Vanilla Harness (basic search toolset)43.4539.3332.0037.95
MIRA-Base (MIRA-Harness)46.4644.0335.3241.90
Δ (MIRA-Base − Vanilla)+3.01+4.70+3.32+3.95

With the backbone fixed, MIRA-Harness improves on all three metrics, and the setting without a harness is lowest on every metric. The report gate requires every citation to point to a page the agent opened and every image to keep its source-page provenance (Appendix A.2), so each claim and figure is tied to evidence the agent actually inspected.

Whether cited sources support the claims is therefore measured directly by TRACE, and MIRA-Harness improves it.

Response to Reviewer 7MoM (Part 3/5)

(W2): The contribution of MIRA-Data is not isolated: MIRA-SFT mixes 4.6K Harness-Native and 33K general search trajectories, so the gains cannot be attributed directly to the proposed data construction. This matters because report generation is the core target, yet MIRA-SFT improves over MIRA-Base by only 0.71 points on MMDR.

Each data source alone raises the average over MIRA-Base by about 3 points (59.36 and 58.61), and training on both gives the best average (61.34). We trained the same backbone on each source alone and on both.

SettingPost-training dataMMSearchMMSearch-PlusBC-VLSimpleVQALiveVQAFVQAVDR-BenchMMDRAvg.
(a) MIRA-Basenone73.1033.4456.6474.3378.3370.0019.0041.9055.84
(b) Harness-Native only4.6K Harness-Native74.8537.9857.6276.6782.0070.3330.8044.6059.36
(c) General only33K general search78.9535.3752.3880.0082.6776.3323.4039.7558.61
(d) MIRA-SFTboth (MIRA-Data)80.7038.9158.4079.0082.0079.3329.8042.6161.34

Training on either source alone is stronger on the tasks that match it: general data on short, VQA-style search, and Harness-Native data on report writing within the harness. Training on both gives the best average.

This follows the complementary design of Section 3 (Capability-Balanced Data Mixture). The gains of MIRA-SFT therefore do not come from general search supervision alone; the Harness-Native trajectories are an essential part of MIRA-Data.

Reviewer_zpYvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MIRA, a multimodal deep research agent designed to generate controlled reports. It introduces MIRA-Harness, which combines report generation constraints with feedback-driven revisions, and MIRA-Data, which provides training trajectories for supervised fine-tuning.

Critical strengths

The paper explores multimodal deep research for generating illustrated reports under specified requirements. The proposed framework is clearly described, and the experimental results show improvements over several evaluated baselines.

Critical weaknesses

(1) The experiments only consider two predefined modes, Concise and Detail, with fixed requirements for report length, image count, and citation count. Evaluating only these two configurations provides limited evidence for the claimed flexibility of controllable report generation.

(2) MIRA-Data combines 4.6K Harness-Native trajectories with 33K general search trajectories. While MIRA-SFT achieves substantial improvements on MMSearch (+7.60) and FVQA (+9.33), the gain on MMDeepResearch-Bench is relatively small (+0.71). This raises concerns about whether the improvements primarily come from general search supervision rather than the proposed report-oriented trajectories.

(3) Enabling the Report Gate improves the report quality score by 4.54 points, but also increases model calls by approximately 46%. It remains unclear whether the improvement results from the targeted feedback provided by the Report Gate or simply from additional opportunities for searching and revising reports.

Questions for authors

(1) Can the authors evaluate more flexible combinations of report length, image count, and citation requirements beyond the two predefined modes to better demonstrate controllable report generation?

(2) Could the authors provide separate ablations using only Harness-Native trajectories and only general search trajectories to clarify their respective contributions, particularly on MMDeepResearch-Bench?

(3) Could the authors introduce a compute-matched baseline without the Report Gate, allowing comparable model and tool calls, to distinguish the benefits of targeted feedback from those of additional inference-time computation?

4 comments

Response to Reviewer zpYv (Part 1/3)

We thank Reviewer zpYv for the recognition that the framework is clearly described and the precise questions.


(W1 & Q1): Only the two predefined modes, Concise and Detail, are evaluated, each with fixed requirements on report length, image count and citation count, which gives limited evidence for flexible controllable generation; more flexible combinations of these requirements beyond the two modes should be evaluated.

Concise and Detail are ranges rather than fixed settings, and MIRA-SFT also strictly meets user-defined values, including requirements that cross the two modes, with high report quality. Concise covers 300–2,000 characters and Detail 5,000–10,000 (Table 5), so each mode already spans a range. Beyond the two modes, the user can specify custom values with each request, which are written into the system prompt (Requirements: {L_min}–{L_max} characters, at least {N_img} images and {N_src} sources). We test four requirement sets on two queries drawn from MMDR: one inside Concise (C1), one inside Detail (C2), and two that cross Concise and Detail (C3, C4).

ContractRequired (chars / images / sources)Q41: delivered (MMDR)Q104: delivered (MMDR)
C1: inside Concise800–1,200 / ≥ 3 / ≥ 41,128 / 3 / 4 (39.7)989 / 3 / 7 (57.6)
C2: inside Detail6,000–7,000 / ≥ 10 / ≥ 86,668 / 10 / 10 (40.7)6,023 / 10 / 8 (53.1)
C3: Concise length, Concise images, Detail sources300–2,000 / ≥ 2 / ≥ 81,156 / 2 / 8 (59.2)1,645 / 2 / 8 (58.3)
C4: Detail length, Concise images, Detail sources5,000–10,000 / ≥ 2 / ≥ 87,842 / 2 / 8 (64.4)7,119 / 2 / 10 (53.1)

The generation contract is therefore a set of user-set values, not a choice between two fixed modes.

Response to Reviewer zpYv (Part 2/3)

(W2 & Q2): MIRA-Data combines 4.6K Harness-Native trajectories with 33K general search trajectories. MIRA-SFT improves substantially on MMSearch (+7.60) and FVQA (+9.33) but only slightly on MMDR (+0.71), so the gains may come mainly from general search supervision rather than the report-oriented trajectories. Please ablate each data source separately, particularly on MMDR.

Each data source alone raises the average over MIRA-Base by about 3 points (59.36 and 58.61), and training on both gives the best average (61.34). We trained the same backbone on each source alone and on both.

SettingPost-training dataMMSearchMMSearch-PlusBC-VLSimpleVQALiveVQAFVQAVDR-BenchMMDRAvg.
(a) MIRA-Basenone73.1033.4456.6474.3378.3370.0019.0041.9055.84
(b) Harness-Native only4.6K Harness-Native74.8537.9857.6276.6782.0070.3330.8044.6059.36
(c) General only33K general search78.9535.3752.3880.0082.6776.3323.4039.7558.61
(d) MIRA-SFTboth (MIRA-Data)80.7038.9158.4079.0082.0079.3329.8042.6161.34

As described in Section 3 of the paper (Capability-Balanced Data Mixture), the two sources are designed to be complementary. The general trajectories supervise short, VQA-style search and the Harness-Native trajectories supervise report writing within the harness; each alone is stronger on the benchmarks that match its role.

Combining the two balances both strengths and gives the best average, so the gains of MIRA-SFT do not come from general search supervision alone.

Response to Reviewer zpYv (Part 3/3)

(W3 & Q3): Enabling the Report Gate raises the report quality score by 4.54 points but also adds about 46% more model calls, so it is unclear whether the gain comes from the gate's targeted feedback or simply from extra opportunities to search and revise. Could the authors add a compute-matched baseline without the gate, with comparable model and tool calls, to separate the benefit of targeted feedback from that of additional inference-time computation?

The MMDR gain and most of the gain in contract adherence come from targeted feedback, not from additional inference-time computation. We ran a compute-matched comparison on the same 32 MMDR Detail-mode queries as Table 2, with four arms and model calls matched per query rather than on average:

  1. w/o Report Gate. The first submitted report is final.
  2. w/o Report Gate, compute-matched. If the agent submits before reaching the number of model calls that the same query used with the full Report Gate, it receives a neutral reply that states the remaining budget ("You still have {N} model calls remaining. You may continue researching or submit again.", where {N} is the target minus the calls already used), at most twice. The untrained model follows this budget hint only approximately, so its call count fluctuates around the target rather than matching it exactly.
  3. w/ Report Gate, w/o targeted feedback. The Report Gate checks and rejects exactly as in the full system, but returns only a fixed request to review and revise the report, without saying which check failed.
  4. w/ Report Gate, w/ targeted feedback. The Report Gate returns the specific checks that failed, which is the full setting of MIRA-Harness used throughout the paper.
ArmReport GateExtra model callsTargeted feedbackCompliant reportsChecks passed (of 4)Input images citedOpened sources citedModel callsTool callsMMDR OverallΔ MMDR
(1)✗✗✗0/321.7823.5%5.3143.542.543.19–
(2)✗✓✗2/322.1952.1%6.5365.363.143.14−0.05
(3)✓✓✗5/322.3839.6%7.0964.962.041.42−1.77
(4)✓✓✓25/323.6292.6%8.6963.661.047.73+4.54

With nearly the same number of model calls, arms (2) and (3) fall well short of arm (4) on every adherence metric and on MMDR.

Additional model calls, or a request to revise without targeted feedback, recover only part of the gain in contract adherence and none of the gain in MMDR.


To conclude, we thank the reviewer again for the precise questions.

Thanks for the detailed responses and additional experiments. The data-source ablations and Report Gate controls help address my concerns. However, the additional controllability evaluation covers only two MMDR queries, so the evidence remains limited. The Gate comparison also covers only 32 Detail-mode queries, leaving its effectiveness in other settings unclear. Overall, I appreciate the work and the effort put into the rebuttal, but the additional evidence is not sufficient to change my overall assessment. I will therefore keep my original score.

Reviewer_xA31OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MIRA, a multimodal deep research agent for controllable illustrated report generation. It introduces a carefully designed harness that integrates evidence collection, report submission, feedback-driven revision, and final delivery. The authors further construct MIRA-Data to train models to operate within this workflow and evaluate the approach on multiple multimodal search and deep research benchmarks.

Critical strengths

The paper is clearly written and well organized. The motivation and technical design are presented in a straightforward manner.

The proposed MIRA-Harness is carefully designed and solid. In particular, incorporating report requirements into the research process, rather than treating them merely as formatting constraints, is a reasonable and practical design choice.

The framework considers both report quality and evidence traceability, making it relevant to practical multimodal research applications.

Critical weaknesses

W1. The MIRA-Data pipeline is not described in sufficient detail. In particular, the query generation templates, rubric construction procedure, data generation steps, and quality-control mechanisms are unclear. It is difficult to understand how the generated queries and rubrics are produced and validated, raising concerns about the reproducibility and reliability of the data.

W2. The reported improvements may largely come from the carefully designed harness and its manually specified priors, rather than the proposed training data. The current experiments do not sufficiently disentangle the contributions of MIRA-Harness and MIRA-Data. It would be important to evaluate how much improvement can be achieved by applying the proposed harness alone, without additional post-training.

W3. The compared open-source search agents use different backbone models, making it difficult to attribute the performance differences to the proposed method. A more convincing comparison would use the same backbone under different harness and training configurations. For example, using the same base model with the WeAgent harness and training it on MIRA-Data would help isolate the contributions of the proposed harness and data.

Questions for authors

Could the authors provide the actual query generation prompts/templates, rubric construction procedure, and quality-control criteria used during data synthesis?

Can the authors conduct controlled experiments using the same backbone model under different harness configurations to disentangle the effects of harness design and training data?

How much of the reported improvement comes from MIRA-Harness alone, and how much comes from supervised fine-tuning on MIRA-Data?

4 comments

Response to Reviewer xA31 (Part 1/4)

We thank Reviewer xA31 for the positive assessment of MIRA-Harness and the constructive questions.


(W1 & Q1): The MIRA-Data pipeline is under-specified: the query-generation templates, rubric construction, generation steps and quality-control criteria are not given, so it is hard to see how queries and rubrics are produced and validated, or to reproduce the data.

We give the full procedure below: (1) the query-generation prompts, (2) the rubric construction procedure, and (3) the quality-control criteria. Together they make the construction of MIRA-Data fully specified and reproducible.

Every query-generation call concatenates two parts: messages = [{"role": "system", "content": Part 1}, {"role": "user", "content": Part 2}]. Part 1 is fixed; Part 2 holds the per-batch variables.

[Task]
- Create {count} original user requests for illustrated multimodal deep-research reports.
- The requests train report-harness behavior, not short-answer benchmark recall.
- Ground each batch in the seed type it is given: a knowledge-intensive topic, or a real-world information need.

[What each request must do]
- Define a clear question and scope.
- Serve one research purpose, varied across requests: explanation, comparison, development over time, or synthesis.
- Need information from multiple public sources.
- Benefit naturally from inspecting and discussing real webpage images, diagrams or charts.
- Have enough public material for the most detailed report mode (at least {min_images} distinct figures and {min_sources} source pages), while still allowing a useful shorter report.

[Rubric construction]
- For each request, write an internal rubric of at least {min_subquestions} subquestions that together cover what the request asks.
- For each subquestion, give the reference points a correct answer should contain and how they can be verified (the kind of source or image that would confirm them).
- Every subquestion should require externally retrieved evidence rather than general knowledge alone.
- At least one subquestion should require visual evidence; for it, state what needs to be seen in an image.
- Every subquestion should be answerable from publicly available sources.
- Keep the rubric out of the question text. It is not shown to the agent that produces the trajectory; it is used afterwards to check whether the trajectory's evidence covers each subquestion and agrees with its reference points.

[Constraints]
- Do not include answers, invented source URLs, a prescribed search path, tool instructions, or teacher-only hints.
- Do not state the report mode, length limits, image/reference counts or the submission protocol; the harness supplies these at runtime.
- Do not require rare image access, private data, or paywalled-only material.
- Do not ask for an exact output answer.
- Do not ask the agent to call every tool just to produce a long trace.
- If browsing tools are available, use them only to check that a request is feasible, not to build an answer key.

[Output format]
- Return a JSON array of request objects with: question, domain (multi-level topic label),
  seed_type, research_purpose, rubric (subquestions with reference points, a verification
  note, and whether visual evidence is needed), and query_images (if any).

Response to Reviewer xA31 (Part 2/4)

(continued from the previous comment)

Batch {index}. Seed type: {seed_type}. Domain focus: {domain}. Exploration seed: {seed}.
- Choose different specific subjects and research angles within this domain.
- Write natural user requests.
- Favor widely documented real-world subjects with public explanatory diagrams or photographs.
- Avoid subjects that need rare archives, historical course syllabi or unpublished facility
  floor plans; visual evidence must be readily obtainable.

To keep the query pool diverse, {seed_type} alternates between knowledge-intensive topics and real-world information needs across batches, {domain} sets each batch's topical focus, and other parameters such as {min_subquestions} are varied across generation rounds and batches. Every generated query is checked for format and de-duplicated.

Each rubric U(q)U(q) is a set of subquestions that a complete report to qq must answer. It is generated in the same call as the query, is never shown to the teacher model, and is used only afterwards to verify the finished trajectory.

The generator first splits the request into the aspects such a report must cover, then writes one subquestion for each, with its reference points, how they can be verified, and whether visual evidence is needed. For example:

Request:          Compare how arch and suspension bridges carry loads.
Subquestion:      How do the main cables pass the deck load to the towers and anchorages?
Reference point:  The cables work in tension.
Verification:     An inspected structural diagram (visual evidence required).

Before a query is used, its rubric is validated against four criteria: it covers the request, its reference points are verifiable, its subquestions depend on retrieved (including visual) evidence, and none of it leaks into the question. Queries that fail are regenerated or discarded.

The teacher model Kimi K2.6 generates the trajectories by running in MIRA-Harness, with the same tools and report gate as at inference. Each trajectory must meet three quality-control criteria:

(i) During rollout, the report gate GG checks each submitted report against the generation contract CmC_m of the selected mode (length, numbers of inspected images and cited sources). If a condition is unmet, the gate returns feedback, and the teacher model gathers further evidence or revises before resubmitting.

(ii) On the completed trajectory, the synthesis gate GsynG_{\text{syn}} checks it against the rubric generated with its query: the trajectory must inspect the images that the visual subquestions call for, cover every subquestion, and agree with the reference points.

(iii) GsynG_{\text{syn}} also runs a global check that the trajectory record is complete and sound: every tool call returns without error, inspected images open and are legible, the final report renders with its images and references, and image sources are attributed.

Only trajectories accepted by both GG and GsynG_{\text{syn}} enter MIRA-Data.

Response to Reviewer xA31 (Part 3/4)

(W2 & Q3): The improvements may come largely from the carefully designed harness and its manually specified priors rather than from the training data, and the experiments do not disentangle MIRA-Harness from MIRA-Data. How much improvement does the harness alone give without post-training, and how much comes from SFT on MIRA-Data?

Table 1 separates the two contributions: on the same backbone, MIRA-Harness alone adds 4.30 points on average over the Vanilla Harness, and training on MIRA-Data adds a further 5.50 points. All rows below use the same Qwen3.6-35B-A3B backbone and differ only in the harness and in whether the backbone is post-trained.

SettingHarnessPost-trainingMMSearchMMSearch-PlusBC-VLSimpleVQALiveVQAFVQAVDR-BenchMMDRAvg.
(a) Qwen3.6Vanilla Harnessnone67.8428.9451.0172.2473.6765.6715.0037.9551.54
(b) Qwen3.6 (MIRA-Base)MIRA-Harnessnone73.1033.4456.6474.3378.3370.0019.0041.9055.84
(c) Qwen3.6 (MIRA-SFT)MIRA-HarnessSFT on MIRA-Data80.7038.9158.4079.0082.0079.3329.8042.6161.34

From (a) to (b), only the harness changes: the untrained backbone gains 4.30 points on average in MIRA-Harness and is higher on every benchmark. From (b) to (c), the harness stays fixed and training on MIRA-Data adds a further 5.50 points.

Together, MIRA-SFT gains 9.80 points over the Vanilla Harness.

Response to Reviewer xA31 (Part 4/4)

(W3 & Q2): The compared open-source agents use different backbones, so the performance differences are hard to attribute to the method. A controlled comparison with the same backbone under different harness and training configurations (e.g., the same base model with the WeAgent-Harness, and trained on MIRA-Data) would separate the effects of harness design and training data.

We ran this controlled comparison: with the same untrained Qwen3.6-35B-A3B backbone, MIRA-Harness outperforms two other open-source multimodal search harnesses on every benchmark. The WeAgent-Harness of WeAgent-MMSearch suggested by the reviewer has not been publicly released, so we use the official harnesses of OpenSearch-VL (OSVL) and Vision-DeepResearch (VDR) instead. We run the comparison twice: on the untrained backbone, and on the same backbone after SFT on 33K general search data (without Harness-Native trajectories).

HarnessMMSearchMMSearch-PlusBC-VLSimpleVQALiveVQAFVQAVDR-BenchMMDRAvg.
Untrained backbone
OSVL52.0520.5837.5973.0065.6759.6711.6030.7043.86
VDR59.0629.5850.6364.0075.3359.3310.4029.8147.27
MIRA-Harness73.1033.4456.6474.3378.3370.0019.0041.9055.84
SFT on 33K general data
OSVL63.7421.2238.1078.0072.6773.6715.2037.4350.00
VDR62.5731.5148.8766.0080.0061.0016.6044.4651.38
MIRA-Harness78.9535.3752.3880.0082.6776.3323.4039.7558.61
SFT on MIRA-Data
MIRA-SFT80.7038.9158.4079.0082.0079.3329.8042.6161.34

With the backbone fixed, MIRA-Harness outperforms OSVL and VDR on average both before and after SFT on general data, so its advantage comes from the harness design and holds after post-training.


To conclude, we thank the reviewer again for the concrete suggestions.

Comments on the paper

1 comment

Dear Area Chairs and Reviewers,

Thank you for your time and service throughout the review process. We submit this brief summary of our contributions and the evidence added during rebuttal.


Reviews Summary

Reviewers xA31, zpYv, and 7MoM raised questions about reproducibility, the contributions of the harness and data, controllability, grounding, and evaluation coverage. We address these through controlled comparisons, data and feedback ablations, and an additional report-generation benchmark.


Main Concerns and Responses

Separating harness, training, and backbone effects (xA31).

On the same Qwen3.6-35B-A3B backbone, replacing the vanilla harness with MIRA-Harness raises the eight-benchmark average from 51.54 to 55.84. Training on MIRA-Data further raises it to 61.34. We also compare MIRA-Harness with the official OpenSearch-VL and Vision-DeepResearch harnesses using the same backbone, both before and after SFT on the 33K general-search subset. MIRA-Harness obtains the highest average in both settings. Together, these results demonstrate the effectiveness of our training data and harness.

Contribution of Harness-Native data to report generation (zpYv, 7MoM).

Training on either data source alone raises the eight-benchmark average by about 3 points over MIRA-Base, with Harness-Native data alone yielding a 3.52-point gain. Training on both sources gives the highest average (61.34). The Harness-Native and general-search data are designed to complement each other. General-search trajectories provide supervision for short, VQA-style search tasks, while Harness-Native trajectories provide supervision for report writing within the harness.

Whether Report Gate gains merely reflect additional computation (zpYv).

To separate targeted feedback from additional inference-time computation, we ran a compute-matched comparison on the same 32 MMDR Detail-mode queries as Table 2, and found that targeted feedback, not additional computation, drives both contract adherence and the MMDR gain. Targeted feedback achieves 47.73 MMDR and 25/32 compliant reports. With similar model-call budgets, encouraging additional calls without a gate yields 43.14 and 2/32, while rejection without targeted feedback yields 41.42 and 5/32. Additional calls alone therefore do not reproduce the gains from specific feedback about unmet requirements.

Flexible controllability and evidence grounding (zpYv, 7MoM).

MIRA-SFT also strictly meets user-specified numerical requirements, including combinations drawn from both presets, while maintaining high report quality. We test four custom combinations of report length, image count, and source count on two MMDR queries, including combinations across the default presets. All eight reports satisfy the specified constraints, demonstrating flexibility.

We also report MMDR component scores: on the same backbone, MIRA-Harness improves TRACE by 4.70 points and MOSAIC by 3.32 points over the vanilla harness. These metrics assess citation grounding and text-image alignment beyond the gate's provenance checks.

Reproducibility of data construction (xA31).

We provide query-generation templates, the rubric construction procedure, and trajectory quality-control criteria. We explain how rubrics are withheld from the trajectory-generating teacher and used for subsequent verification. These details will be incorporated into the revised appendix, and the synthesis code will be released alongside the training code.

Additional report-generation evaluation and efficiency (7MoM).

On TVIR-Bench, MIRA-SFT achieves 55.99 Overall, compared with 25.93 for OpenSearch-VL and 20.35 for Vision-DeepResearch, each evaluated in its own released harness. This is an end-to-end comparison, with much of the advantage arising from support for incorporating visual evidence.


We hope these results clarify the contributions of the harness, targeted feedback, and training data. We thank the reviewers for their constructive feedback and the Area Chairs for considering our submission and rebuttal.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.