acceptodds
Withdrawn from ICLR 2027

RefRoute: Learning to Search, Construct, and Verify References for Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video generators can condition on images, video clips, and audios, yet satisfying a video generation request often depends on choosing the right information and control signal before generation. We present RefRoute, a generator-agnostic agent that learns to search for external evidence or construct multimodal references, and then assigns each asset a role in a generator-ready request. Its reference routes cover retrieved media, 2D event timing, 3D scene and camera path, reconstructed body motion, and video depth. We train a reference-routing policy from 12k teacher trajectories; we also specify two group-relative reinforcement-learning stages for route and video feedback. RefRoute-Bench evaluates 600 requests across five request categories for different knowledge gaps and reference routes. In the evaluation, RefRoute with MiniMax H3 Base show apparent improvements compared with the generator alone, with the largest gain on video retrieval.

Primary area:
generative models
Keywords:
Agentic video generation, multimodal reference planning, tool use, trajectory learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XJbkOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes RefRoute, an agent that searches for or constructs references before video generation and assigns each asset a role in the final generation request. Its contributions include a reference-routing workflow covering retrieved media and 2D/3D previsualization, supervised training from 12k teacher trajectories, a two-stage GRPO procedure, and RefRoute-Bench. Experiments report improved generated-video scores over the same MiniMax H3 Base generator and better workflow completion after training. However, the manuscript gives inconsistent accounts of whether GRPO was completed and whether the benchmark contains 500 or 600 requests.

Critical strengths
  1. A useful formulation of reference selection. RefRoute treats retrieval, reference construction, verification, and asset-role assignment as explicit decisions. This addresses a practical gap between a user’s request and the conditions a video generator needs, making the workflow a potentially valuable contribution beyond prompt rewriting.

  2. Initial evidence of downstream utility. With the same MiniMax H3 Base generator, the reported overall score improves from 56.86 to 60.96 after SFT, with gains across all five request categories. This comparison provides preliminary evidence that preparing references can improve request fulfillment, particularly for procedural video requests.

Critical weaknesses
  1. The status and provenance of the central experiments are unclear. The method and Figure 2 describe the GRPO stages as specified rather than completed, while Table 3 and the conclusion report RL improvements. The benchmark also contains 600 requests in the abstract but 500 in the main text, and the “mixed-tier snapshot” is insufficiently explained. These inconsistencies prevent a reliable assessment of which results were actually obtained under comparable conditions.

  2. The experiments do not isolate the benefit of learned routing. The main comparison is against generation without the reference workflow, while the untrained agent completes only 1% of requests. Without a functional fixed-route, retrieval-only, or prompt-rewriting baseline, the improvements cannot be confidently attributed to learning when and how to select references rather than simply adding tools, reference inputs, or better prompts.

Questions for authors
  1. Were both GRPO stages actually executed? Please identify the checkpoints and evaluation configurations underlying Table 3, and reconcile these results with the “specified” stages in Figure 2.

  2. What exact request set and generation configuration produced each reported score? Please reconcile the 500/600 benchmark counts, explain “mixed-tier snapshot,” and clarify how unsuccessful requests enter the averages. Are there existing controlled results separating learned routing from a functional fixed workflow?

3 comments
AuthorsOpenReview ↗

Rebuttal by authors (1/2)

Weakness 1, Q1 and Q2: clarification of the reported results

A1: We thank the reviewer for pointing out the inconsistencies, and we sincerely apologize for them: different co-authors participate the the experimental write-ups, and the terminology was not unified in the final version. We have corrected every item.

  • Both GRPO stages were executed. Stage 1 ran on 2000 training requests with four on-policy rollouts each (8000 full M01–M04 trajectories); Stage 2 ran on the same 2000 requests with reviewed and fixed 2000 M01-M03 and four M04 continuations each, every continuation compiled and rendered for gated video feedback. The wording was confusing, and it is removed everywhere: Table 2 now reports the final policy (after SFT + GRPO-1 + GRPO-2), and Table 3 is the stage-wise ablation:
Reported modelCheckpoint
RefRoute-SFT (Table 3)Qwen3-8B after the complete SFT stage
RefRoute-RL-stage 1 (Table 3)SFT checkpoint + GRPO stage 1
RefRoute-RL-stage 2 (Table 3; Table 2 main row)SFT + GRPO-1 + GRPO-2 (final)
All rows of Tables 2–3 were obtained under identical conditions: same request set, same generator configuration, same rubric, with the model checkpoint as the only difference.
  • The benchmark has 500 requests. We corrected the typo in the abstract. The category split is:
Categoryn
Visual (text/image gap)100
Video150
2D previs100
3D scene100
3D body50
Total500

Video carries 150 because it spans more subcategories (procedural knowledge, style learning, direct reference, depth); 3D body carries 50 because it shares similar 3D-construction route family with 3D scene.

  • "Mixed-tier snapshot" was a misnomer: it referred to the fact that requests which never reach a compilable generation request are scored with the same-case vanilla-T2V score, no additional video is generated for them, so each row mixes successful outcomes with these fallbacks.
AuthorsOpenReview ↗

Rebuttal by authors (2/2)

Weakness 2: Isolating the benefit of learned routing

A2: We agree that the prompted baseline was not a functional fixed-route control (1% Ready), and we have now run three more controlled experiments with the same harness, tools, and backend on the same 500 requests:

  1. M04-only: the raw request goes directly to M04, with no search and no reference construction (pure prompt rewriting with the trained writing skills);
  2. Retrieval-only: M01/M02 run normally, but every transition into M03 is suppressed, so all routes end at M04 (search plus rewriting, no previs construction);
  3. RefRoute (final): the full learned policy after SFT + GRPO-1 + GRPO-2;
  4. Oracle route: the learned M01 plan is replaced by the reviewed gold plan we collected for benchmark evaluation as described in §3.2.3, and the trained policy executes the remaining stages.

In every row, requests that fail to reach a ready compilable generation request fall back to the vanilla T2V score. 95% confidence intervals are paired bootstrap over the 500 cases (20000 resamples); p-values are paired sign-flip permutation tests.

SettingScoreΔ vs H3Δ 95% CIReady
H3 (vanilla T2V)56.86–––
M04-only58.47+1.61[−1.12, +4.37]100%
Retrieval-only64.28+7.42[+5.15, +9.49]92%
RefRoute (final)65.83+8.97[+6.45, +10.88]90%
Oracle route66.75+9.89[+7.07, +12.76]82%

All p < 0.0001 except M04-only (p = 0.25).

The isolated benefit of learned routing:

  • First, prompt rewriting alone contributes +1.6 and is not the main source of the gain: M04-only applies MiniMax H3's prompt-writing skill to a request from which no new information was acquired.
  • Second, learned reference acquisition strategy is the dominant factor: +5.8 over the last row and recovers +7.42 of the total +8.97. This is not limited to correct appearance or knowledge searched online, for the categories of search-grounded 2D and 3D-scene requests, the retrieve images or videos often already carry some layout or spatial hints the request needs.
  • Third, learned routing and reference construction add +1.6 more: replacing the learned M01 plan with the gold moves the score by only a further +0.9, within overlapping CIs, the two end up similar because the gold route's main advantage, higher previs-route accuracy (it commissions previs when previs is genuinely needed), is offset by its costs: it leads by about 2 points over completed and ready request, but the expert is not stong enough and occasionally produces a mediocre render like a wrong camera control that scores no better than skipping the previs. And its Ready rate drops to 82% (vs. 90%) because the downstream stages are jointly calibrated with the learned plan. This is further evidence that reference construction, not routing, is the current bottleneck, and we state it as a direction for future work.

We would like to clarify on what our system has "learned": the fixed-route experiments above fix only the route, not the policy and strategy executing the tools. Retrieval-only still uses the learned M02 policy, including designing the query and modality for each gap, composing the VLM inspection question, deciding retain-vs-retry for the gap, and any request that reaches M03 uses the learned M03 policy, including choosing the specific previs construction route, and judging revise-vs-accept. These interaction skills are part of what is trained, not something merely "added" with the tools. The untrained baseline makes the contrast concrete: Qwen3-8B with the same tools and a system prompt reaches a compilable request on only 1% of rollouts (Table 3). Therefore tools alone fail without a learned policy.

AuthorsOpenReview ↗

Additional Response to Reviewer XJbk

Dear Reviewer XJbk:

We sincerely thank you again for your great efforts in reviewing this paper, especially for the precious advice that has helped us improve the quality of this paper significantly!

In our response we addressed every item you raised: (1) both GRPO stages were executed, and we provide an exact checkpoint map (SFT → GRPO-1 → GRPO-2) tying each row of Tables 2–3 to its training stage and evaluation configuration; (2) the benchmark contains 500 requests, the per-category split is now stated, and (3) we added controlled settings (M04-only, Retrieval-only, Oracle route) on the same 500 requests to separate learned routing from the tools and workflow themselves. All the modifications outlined in our response will be integrated into the revised version.

We would be most grateful if you could review our response and share whether these clarifications resolve your concerns; we are happy to discuss any remaining points.

Best,

Submission 5871 Authors

Reviewer_rZofOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces RefRoute, an agent that retrieves or constructs multimodal references for video generation. Its workflow identifies information gaps, searches for and inspects external media, optionally constructs references using 2D animation, 3D scenes, body-motion reconstruction, or depth extraction, and assigns each asset a role in the final generation request. The routing policy is trained through supervised fine-tuning on teacher trajectories, with two additional GRPO stages described for route-level and generated-video feedback. The paper introduces RefRoute-Bench and reports that RefRoute improves the MiniMax H3 Base score from 56.86 to 60.96 after SFT, with further gains reported in the RL ablations.

Critical strengths
  1. The paper addresses the practical challenge of determining which references a video generator needs to fulfill a request. This is particularly useful for unfamiliar subjects, procedural actions, and requests with specific timing or spatial constraints.

  2. The workflow makes useful distinctions between reference types and roles. Separating assets used as direct conditions from those used only as evidence is sensible. Combining retrieved media with constructed references also offers flexibility in addressing different information and control requirements.

  3. The implementation is described in considerable detail. Appendix B provides extensive decision templates, including explicit requirements for inspecting media and verifying references before accepting them. The reported SFT results improve upon the same generator across all five request categories, with the largest gain in video retrieval.

Critical weaknesses
  1. The reporting of the training stages and evaluated checkpoints needs clarification. Figure 2 distinguishes “completed SFT” from “specified GRPO stages,” whereas Table 3 reports results for both GRPO stages. This wording leaves the relationship between the described training procedure and the evaluated checkpoints unclear. Section 4.1 also describes additional SFT refinements without explicitly identifying which checkpoint produces each result. Please clarify this mapping and make the descriptions consistent.
  2. The experiments demonstrate improved workflow execution but do not isolate the contribution of learned routing to video quality. SFT increases the proportion of rollouts reaching a video-generator request from 1% to 84%, supporting improved execution of the tool workflow. However, the prompted baseline’s video scores are vanilla-generator fallback values, as the authors explicitly acknowledge. The video comparisons therefore do not distinguish the benefit of learned routing from those of reference acquisition, expert construction, and prompt rewriting. A comparison using the same tools and backend would help establish this distinction.
  3. The evaluation needs stronger validation and clearer reporting. Video scores rely on a single Gemini judge, without human validation or uncertainty estimates, making the robustness of the reported gains difficult to assess. The “mixed-tier snapshot” mentioned in Table 3 is unexplained, and the consistency of model configurations and generation settings, as well as the handling of incomplete workflows, needs clarification.
Questions for authors

The paper presents a useful system direction, and the reported SFT results support improved workflow execution and higher video scores. My main concerns are the limited attribution of video-quality gains to learned routing and the insufficient validation and specification of the evaluation protocol. The training descriptions and checkpoint mapping also require clarification. These concerns concern the strength and interpretability of the evidence; they do not establish that the reported improvements are invalid. A response addressing these issues could change my recommendation.

4 comments
AuthorsOpenReview ↗

Rebuttal by authors (1/3)

Weakness 1: Training-stage reporting and checkpoint mapping

A1: We thank the reviewer for pointing out the inconsistencies, and we sincerely apologize for them: different co-authors participate the the experimental write-ups, and the terminology was not unified in the final version. We have corrected every item below.

  • Figure 2's wording was only meant to separate the two training families (decision-level SFT vs. the RL stages), not to mark one as unexecuted. Both GRPO stages were executed and evaluated. Table 2 now reports the final policy after all training (SFT + GRPO-1 + GRPO-2); Table 3 is the stage-wise ablation. The checkpoint mapping is:
Reported modelCheckpoint
RefRoute-SFT (Table 3)Qwen3-8B after the complete SFT stage
RefRoute-RL-stage 1 (Table 3)SFT checkpoint + GRPO stage 1
RefRoute-RL-stage 2 (Table 3, and the Table 2 last row)SFT + GRPO-1 + GRPO-2 (final)
  • The "SFT refinements" in §4.1 are three sub-phases of one SFT stage:
    • (1) The first phase is one epoch over all training data with all decision points mixed.
    • (2) Then we added an additional M01-only phase. We found the model after (1) frequently failed to route into M03 on the previs categories. Since M01 is the most important stage which determines the entire route that follows, we additionally finetune the model on M01 decisions by using larger proportion of previs categories in the training data;
    • (3) an additional M01+M04 phase. This is because we mixed all the decision points together, while some points are simple like finishing a search or selecting an asset id in M02, but there are also long and difficult decisions like M04 request-writing task. Mixing them together made the training loss unstable. Since both refinements are ordinary SFT steps, equivalent to sampling decision points by different probability, these distinct phases were not evaluated as separate checkpoints, so we report them as one SFT stage.
  • The RL stage is split into two because each phase use different rewards so evaluated separately. Stage 1 rewards the executed M01–M04 route; stage 2 adds generated-video feedback and updates M04 alone. Once route accuracy is reasonable, the wording and information density of the M04 request become the dominant lever on video quality, which is why stage 2 shows a clear improvement.
AuthorsOpenReview ↗

Rebuttal by authors (2/3)

Weakness 2: Decomposing the benefit of learned routing

A2: We thank the reviewer for this constructive feedbacl and we have run controlled experiments with the same backend on the same 500 evaluation requests. We designed three new routing and ran the experiments using the model after GRPO-2:

  1. M04-only: the raw request is fed directly to M04, with no search and no reference construction (pure prompt rewriting with the trained writing skills);
  2. Retrieval-only: M01/M02 run normally, but every transition into M03 is suppressed, so all routes end at M04 (search plus rewriting, no previs construction);
  3. RefRoute (final): the full learned policy after SFT + GRPO-1 + GRPO-2;
  4. Oracle route: the learned M01 plan is replaced by the reviewed gold plan we collected for benchmark evaluation as described in §3.2.3 (more details below), and the trained policy executes the remaining stages.

In every row, requests that fail to reach a ready compilable generation request fall back to the vanilla T2V score. 95% confidence intervals are paired bootstrap over the 500 cases (20000 resamples); p-values are paired sign-flip permutation tests.

SettingScoreΔ vs H3Δ 95% CIReady
H3 (vanilla T2V)56.86–––
M04-only58.47+1.61[−1.12, +4.37]100%
Retrieval-only64.28+7.42[+5.15, +9.49]92%
RefRoute (final)65.83+8.97[+6.45, +10.88]90%
Oracle route66.75+9.89[+7.07, +12.76]82%

All paired permutation p-values are < 0.0001 except M04-only (p = 0.25).

The decomposed benefit of learned routing:

  • First, prompt rewriting is not the source of the gain: M04-only applies MiniMax H3's own prompt-writing skill to a request from which no new information was acquired, and its +1.61 gain is not significant.
  • Second, reference acquisition is the dominant factor: retrieval-only already recovers +7.42 of the total +8.97, and this is not limited to correct appearance or knowledge searched online, for the categories of search-grounded 2D and 3D-scene requests, the retrieve images or videos often already carry some layout or spatial hints the request needs.
  • Third, the learned route and the gold route end up similar because the gold route's gains are offset by its costs:
    • injecting the gold M01 adds only +0.92 over RefRoute, with overlapping CIs. The oracle's advantage over completed and ready request is about 2 points, but its Ready rate drops to 82% versus 90%. This is because the downstream stages are jointly calibrated with the learned plan, so gold plans are slightly out-of-distribution for them;
    • In addition, the gold route's main difference from the learned route is higher previs-route accuracy, it commissions previs when previs is genuinely needed. However, the expert is not stong enough and occasionally produces a mediocre render like a wrong camera control that scores no better than skipping the previs. This is further evidence that reference construction, not routing, is the current bottleneck, and we state it as a direction for future work.
AuthorsOpenReview ↗

Rebuttal by authors (3/3)

Weakness 3: Evaluation validation

A3: We thank the reviewer for this suggestion.

  • The A2 table now reports uncertainty estimates for every comparison: paired bootstrap 95% CIs and paired permutation p-values. The final gain (+8.97) and the retrieval-only gain (+7.83) are both significant at p < 0.0001.

  • "Mixed-tier snapshot" was a writing misalignment on our side. The phrase referred to the fact that requests failing to reach the generator are scored with their vanilla-T2V fallback, so a row mixes successful and fallback outcomes. We revised Table 3 caption.

  • To validate the Gemini judge, we sampled 30 requests × 3 settings (vanilla T2V, oracle route, RefRoute(ours)) = 90 videos, each scored by 3 annotators who were given the request and its gap-related text/image/video evidence. Each annotator scored every video twice: (1) overall prompt-following on a 1–5 scale; (2) our checklist-plus-rubric protocol, the same protocol the Gemini judge uses in evaluation.

    • Here we describe how the checklist-plus-rubric protocol is obtained. (1) We collect a high-quality trajectory for each request with the teacher policy, then have humans patch and verify it against the request, completing any missing evidence; (2) manually write one fixed instruction that teaches a scoring-plan designer (an LLM) how to write checklist and rubrics for stages in a trajectory and the final video, and how to aggregate the scores; (3) feed the scoring-plan designer the request, its gold trajectory, together with the instruction, it writes a request-specific plan and every checkpoint is paired with a 3–5-anchor rubric rather than a binary yes/no. For example, a trajectory checkpoint for search-requiring requests is scored as: 0 = no gap listed, or only unnecessary gaps; 1 = some relevant gaps but a required category omitted or unrelated searches added; 2 = all relevant gaps plus unnecessary ones; 3 = all and only the required gaps. A video checkpoint, e.g., "Does the core setting — a Victorian green cast-iron railway viaduct over Borough Market match the overhead viaduct in reference image 1?", is scored as: 0 = no part of the video matches; 1 = less than 30% matches; 2 = between 30% and 80% matches; 3 = more than 80% matches.
Human Scoring protocolKendall's τSpearman's ρ
Prompt-following (1–5)0.6840.815
Checklist + rubric (our protocol)0.7580.870

Inter-annotator agreement across the three annotators is 0.732. For each video and each scoring protocol, average over all annotator is calculated. The Gemini judge tracks human judgement well, and agreement is higher when humans score under the same rubric protocol, which also validates the rubric design itself.

AuthorsOpenReview ↗

Additional Response to Reviewer rZof

Dear Reviewer rZof:

We sincerely thank you again for your great efforts in reviewing this paper, especially for the precious advice that has helped us improve the quality of this paper significantly!

In our response we addressed each of your concerns: (1) we clarified the training-stage and checkpoint mapping, Table 2 now reports the final policy, and Table 3 is the stage-wise ablation; (2) we added the controlled baselines (M04-only, Retrieval-only, and an Oracle-route setting) with paired bootstrap confidence intervals, separating the contributions of prompt rewriting, reference acquisition, and learned routing; and (3) we strengthened the evaluation reporting with a human-validation study (90 videos, 3 annotators, plus judge-correlation statistics). All the modifications outlined in our response will be integrated into the revised version.

Could you kindly take a look at our response when you have a chance, and let us know whether these revisions address your concerns? We would be very happy to further discuss any remaining questions.

Best,

Submission 5871 Authors

Reviewer_WtY1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents RefRoute, an agent that prepares multimodal references to improve instruction fidelity in video generation. It retrieves or constructs references as needed, then inspects these references before specifying asset roles in a generation request. The policy is trained through supervised fine-tuning on teacher trajectories. The paper also describes a two-stage GRPO procedure with route-level and video-level feedback. RefRoute-Bench assesses agent performance through trajectory evaluation and video scoring. The reported results demonstrate that RefRoute helps the video generator follow user instructions more faithfully.

Critical strengths

RefRoute outperforms the evaluated baselines on RefRoute-Bench, including the same video generator used without RefRoute (Table 2). This comparison supports the effectiveness of the proposed workflow within the evaluated setting.

Critical weaknesses

RefRoute-Bench’s applicability to agents with different workflow structures remains unclear. Section 3.2.3 describes stage-specific trajectory evaluation but does not explain how agents whose module organization or routing differs from RefRoute’s M01–M04 structure would be evaluated. This leaves the benchmark’s applicability for comparing different agent architectures insufficiently established.

Questions for authors
  1. Does RefRoute-Bench’s stage-specific trajectory evaluation require agents to use RefRoute’s M01–M04 workflow structure? How would agents with different module organization or routing be evaluated under the protocol in Section 3.2.3? (p. 5, lines 257–263)
  2. It would be helpful to report the Seed-2.0-Pro teacher agent’s performance under the same experimental settings as RefRoute, with MiniMax H3 Base as the video generator. This comparison would clarify the student–teacher performance gap. Teacher trajectories are collected for the benchmark (p. 5, lines 257–259), but Tables 2 and 3 do not report the teacher’s performance. (p. 8, lines 378–385 and 412–426)
  3. What limitations of existing benchmarks motivate RefRoute-Bench? Section 3.2.3 does not fully explain this rationale (p. 5, lines 251–263).
  4. Why is the evaluation in Tables 2 and 3 limited to RefRoute-Bench? As a suggestion for future work, additional evaluation on a suitable existing benchmark would help clarify the scope of the findings.
3 comments
AuthorsOpenReview ↗

Rebuttal by authors (1/3)

Weakness 1 and Q1: Benchmark applicability

A1: We thank the reviewer for the constructive questions. RefRoute-Bench has two evaluation axes, and they are portable to different degrees by design.

  • The video axis is architecture-agnostic. It scores only the final video produced by any system, a plain text-to-video model, a reference-to-video model, or a full agent against the same user requests and its ground-truth evidence using a fixed rubric. Table 2 already uses it in exactly this way: five off-the-shelf generators with no agent harness at all are scored on the same 500 requests. This is the axis on which different architectures can be compared directly.
  • The trajectory axis is defined over our M01–M04 harness. It measures how well a policy makes the four decision types inside this scaffold, which is the level at which our training and ablations operate. It is therefore not a ranking of heterogeneous agent architectures. Within one harness, however, different policies are directly comparable on this axis, which is how we use it: all Qwen-8B variants and the Seed-2.0-Pro teacher run through the same scaffold.

An agent with a different harness can reuse the method by which our evaluation checkpoints are written. The method has three steps:

  • (1) construct gold trajectories: a strong teacher policy produces high-quality trajectories, which humans then patch and verify against the request, completing any missing evidence;
  • (2) write one fixed instruction that teaches a scoring-plan designer (an LLM) how to score each trajectory stage, how to combine the stage scores into a whole-trajectory score, and how to score the generated video;
  • (3) for each request, feed this instruction together with the request and its gold trajectory to the scoring-plan designer, which writes a request-specific scoring plan with two part: a trajectory scoring plan and a video scoring plan. Each plan is a checklist of checkpoints, instead of a binary yes/no, every checkpoint is paired with a rubric that defines 3–5 levels of anchor rubric. For example, the gap-identification checkpoint for search-requiring requests is scored as: 0 = no gap listed, or only unnecessary gaps; 1 = some relevant outside-information gaps, but some are omitted or unrelated searches are added; 2 = all relevant gaps listed, but unnecessary gaps also added; 3 = all required gap listed and nothing unnecessary. The rubric thus scores the reasonableness rather than an exact answer.
AuthorsOpenReview ↗

Rebuttal by authors (2/3)

Q2: Seed-2.0-Pro Teacher performance

A2: We have run the Seed-2.0-Pro teacher end-to-end under the same settings and add its in the table below. For clarity, the trajectory columns are: Gap: the M01 gap identification score; Search: the M02 search query & modality score; Valid and Ready are fraction of valid tool-calls and compilable final video generation request.

PolicyGapSearchValidReadyVisualVideo2D3D scene3D bodyMicro
Vanilla T2V (MiniMax H3 Base)––––51.1058.5052.8361.5462.1556.86
Qwen3-8B + system prompt508045.3151.10†58.50†52.83†61.54†62.15†56.86†
RefRoute-SFT789388.28455.2265.6156.0862.9764.2160.96
RefRoute-RL-stage 1859490.58956.7667.2762.9963.9764.7563.40
RefRoute-RL-stage 2879491.09058.0872.4565.2564.6464.9865.83
Teacher (Seed-2.0-Pro)939799.39567.6577.2867.7565.8062.3369.66
  • The teacher retains a lead on gap identification and search. The teacher's largest advantage is Text/Image (+9.57 over the final policy), because image search results are noisy, often contains the right item hidden among wrong ones, and after shortlisting and download attrition only a few remain to choose from. The teacher's more specific search queries are what let it select the right image more often than the student.

  • The notable part is the previs categories, essentially tied with the student. This is expected: the final video quality is determined primarily by previs quality which is built by the same expert models. We take this as evidence that (i) reference construction is the current bottleneck in the previs categories, (ii) better previs experts would lift all policies roughly together, and this is a direction for future work.

AuthorsOpenReview ↗

Rebuttal by authors (3/3)

Q3: What motivates RefRoute-Bench

A3: Our target is video generation that depends on external search or on constructed previs, that are requests with genuine knowledge gaps or strict layout/timing/geometry requirements.

  • Existing video benchmarks do not cover this regime: VBench-style [A,B] suites and similar prompt-following benchmarks [C] evaluate generation quality on prompts that require no external knowledge and no strict 2D or 3D constraints, so they cannot measure whether an agent found and verified the right evidence or whether it bound the right asset to the right role.
  • Search-augmented visual generation benchmarks, exist mostly for image generation [D,E,F], and are not directly reusable for video evaluation.

RefRoute-Bench fills this gap: 500 requests across five categories, each with a resolved information gap, a manually retrieved ground-truth evidence set, a per-request video rubric, and a reviewed gold trajectory, supporting both trajectory and video evaluation.

Q4: Evaluation scope (Q4)

A4: We agree that limiting Tables 2–3 to RefRoute-Bench bounds the claim, and we will state this scope explicitly. As a generalization check we will evaluate RefRoute against the vanilla generator on a suitable open video benchmarks; we expect this to confirm that the workflow does not hurt general prompt following while providing its gains where reference acquisition matters.


[A] VBench: Comprehensive Benchmark Suite for Video Generative Models. Huang et al.

[B] VBench-2.0: Advancing Video Generation Evaluation with Detailed Human Feedback. Huang et al.

[C] T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation. Sun et al.

[D] Gen-Searcher: Reinforcing Agentic Search for Image Generation. Feng et al.

[E] UnifyAgent: A Unified Multimodal Agent for World-Grounded Image Synthesis. Chen et al.

[F] Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation. Wang et al.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.