- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes RefRoute, an agent that searches for or constructs references before video generation and assigns each asset a role in the final generation request. Its contributions include a reference-routing workflow covering retrieved media and 2D/3D previsualization, supervised training from 12k teacher trajectories, a two-stage GRPO procedure, and RefRoute-Bench. Experiments report improved generated-video scores over the same MiniMax H3 Base generator and better workflow completion after training. However, the manuscript gives inconsistent accounts of whether GRPO was completed and whether the benchmark contains 500 or 600 requests.
Critical strengths
-
A useful formulation of reference selection. RefRoute treats retrieval, reference construction, verification, and asset-role assignment as explicit decisions. This addresses a practical gap between a user’s request and the conditions a video generator needs, making the workflow a potentially valuable contribution beyond prompt rewriting.
-
Initial evidence of downstream utility. With the same MiniMax H3 Base generator, the reported overall score improves from 56.86 to 60.96 after SFT, with gains across all five request categories. This comparison provides preliminary evidence that preparing references can improve request fulfillment, particularly for procedural video requests.
Critical weaknesses
-
The status and provenance of the central experiments are unclear. The method and Figure 2 describe the GRPO stages as specified rather than completed, while Table 3 and the conclusion report RL improvements. The benchmark also contains 600 requests in the abstract but 500 in the main text, and the “mixed-tier snapshot” is insufficiently explained. These inconsistencies prevent a reliable assessment of which results were actually obtained under comparable conditions.
-
The experiments do not isolate the benefit of learned routing. The main comparison is against generation without the reference workflow, while the untrained agent completes only 1% of requests. Without a functional fixed-route, retrieval-only, or prompt-rewriting baseline, the improvements cannot be confidently attributed to learning when and how to select references rather than simply adding tools, reference inputs, or better prompts.
Questions for authors
-
Were both GRPO stages actually executed? Please identify the checkpoints and evaluation configurations underlying Table 3, and reconcile these results with the “specified” stages in Figure 2.
-
What exact request set and generation configuration produced each reported score? Please reconcile the 500/600 benchmark counts, explain “mixed-tier snapshot,” and clarify how unsuccessful requests enter the averages. Are there existing controlled results separating learned routing from a functional fixed workflow?
3 comments
Rebuttal by authors (2/2)
Weakness 2: Isolating the benefit of learned routing
A2: We agree that the prompted baseline was not a functional fixed-route control (1% Ready), and we have now run three more controlled experiments with the same harness, tools, and backend on the same 500 requests:
- M04-only: the raw request goes directly to M04, with no search and no reference construction (pure prompt rewriting with the trained writing skills);
- Retrieval-only: M01/M02 run normally, but every transition into M03 is suppressed, so all routes end at M04 (search plus rewriting, no previs construction);
- RefRoute (final): the full learned policy after SFT + GRPO-1 + GRPO-2;
- Oracle route: the learned M01 plan is replaced by the reviewed gold plan we collected for benchmark evaluation as described in §3.2.3, and the trained policy executes the remaining stages.
In every row, requests that fail to reach a ready compilable generation request fall back to the vanilla T2V score. 95% confidence intervals are paired bootstrap over the 500 cases (20000 resamples); p-values are paired sign-flip permutation tests.
| Setting | Score | Δ vs H3 | Δ 95% CI | Ready |
|---|---|---|---|---|
| H3 (vanilla T2V) | 56.86 | – | – | – |
| M04-only | 58.47 | +1.61 | [−1.12, +4.37] | 100% |
| Retrieval-only | 64.28 | +7.42 | [+5.15, +9.49] | 92% |
| RefRoute (final) | 65.83 | +8.97 | [+6.45, +10.88] | 90% |
| Oracle route | 66.75 | +9.89 | [+7.07, +12.76] | 82% |
All p < 0.0001 except M04-only (p = 0.25).
The isolated benefit of learned routing:
- First, prompt rewriting alone contributes +1.6 and is not the main source of the gain: M04-only applies MiniMax H3's prompt-writing skill to a request from which no new information was acquired.
- Second, learned reference acquisition strategy is the dominant factor: +5.8 over the last row and recovers +7.42 of the total +8.97. This is not limited to correct appearance or knowledge searched online, for the categories of search-grounded 2D and 3D-scene requests, the retrieve images or videos often already carry some layout or spatial hints the request needs.
- Third, learned routing and reference construction add +1.6 more: replacing the learned M01 plan with the gold moves the score by only a further +0.9, within overlapping CIs, the two end up similar because the gold route's main advantage, higher previs-route accuracy (it commissions previs when previs is genuinely needed), is offset by its costs: it leads by about 2 points over completed and ready request, but the expert is not stong enough and occasionally produces a mediocre render like a wrong camera control that scores no better than skipping the previs. And its Ready rate drops to 82% (vs. 90%) because the downstream stages are jointly calibrated with the learned plan. This is further evidence that reference construction, not routing, is the current bottleneck, and we state it as a direction for future work.
We would like to clarify on what our system has "learned": the fixed-route experiments above fix only the route, not the policy and strategy executing the tools. Retrieval-only still uses the learned M02 policy, including designing the query and modality for each gap, composing the VLM inspection question, deciding retain-vs-retry for the gap, and any request that reaches M03 uses the learned M03 policy, including choosing the specific previs construction route, and judging revise-vs-accept. These interaction skills are part of what is trained, not something merely "added" with the tools. The untrained baseline makes the contrast concrete: Qwen3-8B with the same tools and a system prompt reaches a compilable request on only 1% of rollouts (Table 3). Therefore tools alone fail without a learned policy.
Additional Response to Reviewer XJbk
Dear Reviewer XJbk:
We sincerely thank you again for your great efforts in reviewing this paper, especially for the precious advice that has helped us improve the quality of this paper significantly!
In our response we addressed every item you raised: (1) both GRPO stages were executed, and we provide an exact checkpoint map (SFT → GRPO-1 → GRPO-2) tying each row of Tables 2–3 to its training stage and evaluation configuration; (2) the benchmark contains 500 requests, the per-category split is now stated, and (3) we added controlled settings (M04-only, Retrieval-only, Oracle route) on the same 500 requests to separate learned routing from the tools and workflow themselves. All the modifications outlined in our response will be integrated into the revised version.
We would be most grateful if you could review our response and share whether these clarifications resolve your concerns; we are happy to discuss any remaining points.
Best,
Submission 5871 Authors
Rebuttal by authors (1/2)
A1: We thank the reviewer for pointing out the inconsistencies, and we sincerely apologize for them: different co-authors participate the the experimental write-ups, and the terminology was not unified in the final version. We have corrected every item.
Video carries 150 because it spans more subcategories (procedural knowledge, style learning, direct reference, depth); 3D body carries 50 because it shares similar 3D-construction route family with 3D scene.