acceptodds
Under review as a conference paper at ICLR 2027

VideoGen-Agent: Reinforcing Video Generation Agents

13 views

Abstract

Recent advances in video generative models have enabled high-fidelity, temporally coherent video generation. However, these models often struggle to satisfy prompts requiring specialized knowledge, specific identities, physical consistency, or ordered events. In this paper, we present VideoGen-Agent, a multimodal agent trained through multitask agentic reinforcement learning to use external tools for video generation. The agent coordinates augmentation, generation, and verification tools through multi-turn interactions, using the prompt and intermediate observations to guide its decisions. We train a shared policy on a category-balanced dataset spanning six tasks. Supervised fine-tuning on teacher-generated trajectories establishes tool-use behavior, which is then refined through reinforcement learning. A category-aware hybrid reward evaluates tool-call validity, task-appropriate tool use, and generated video quality. We further introduce VABench, a held-out benchmark of 600 prompts designed to evaluate video generation agents across six capability categories: procedural knowledge, single- and multi-entity identity preservation, physical consistency, scene composition, and multi-shot temporal structure. On VABench, VideoGen-Agent improves over its base video generator by points, from to . Upgrading the generation tools further raises the score to without additional agent training. Human raters prefer the Toolset 1 configuration over the strongest standalone baseline in of comparisons. These results support learning tool use across video-generation tasks and show that the trained agent can benefit from subsequent advances in generation tools.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Generation, Multimodal Agents, Agentic Reinforcement Learning, Tool-Augmented Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vMQCOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
3(+2)
Read review
Paper summary

VideoGen-Agent introduces a multimodal agentic framework designed to overcome the limitations of standalone video generation models in handling complex prompts that require specialized procedural knowledge, specific entity identities, physical consistency, compositional scene structures, and multi-shot temporal orderings. Operating within a multi-turn reasoning-action-observation loop, the agent coordinates tools across three functional tiers: augmentation (text/image search, physics simulation code), generation (text-to-video, image-to-video, motion-to-video, action-to-video), and verification (object detection, depth estimation).

The training pipeline consists of SFT on 16K teacher-generated trajectories, followed by multitask RL using GRPO. This optimization is guided by a hybrid reward function combining format validity, VLM judge quality scores, and task-appropriate tool use, alongside task advantage normalization across categories. Additionally, the authors curate VABench, a benchmark comprising 600 complex prompts across six capability categories. Empirical results show that VideoGen-Agent significantly outperforms standalone open-source and proprietary video models, achieving an overall score of 75.6 with Toolset 1, and rising to 86.1 when upgrading generation models without agent retraining. Human evaluators preferred the Toolset 1 agentic setup over the strongest standalone baseline in 84.3% of comparisons.

Critical strengths
  1. The paper identifies critical failure modes of current standalone video generation base models (identity drift, physical implausibility, and multi-shot errors) and establishes VABench as a well-structured, category-balanced evaluation harness spanning six distinct capability axes.

  2. Decoupling tools into augmentation, generation, and verification allows the agent to flexibly retrieve external knowledge, ground visual identities, and refine outputs based on visual feedback. A notable engineering highlight is that the trained agent policy can seamlessly leverage upgraded generation models (Toolset 2) without requiring policy retraining.

Critical weaknesses
  1. As reported in Table 2, the performance improvement of VideoGen-Agent-Toolset 1 over its direct base model (Seedance 1.0 Pro Fast) is substantial, yielding a 19.1-point gain (from 56.5 to 75.6). However, when moving to a stronger standalone baseline model (Seedance 2.0 Fast), which scores 73.2, the performance gap narrows drastically to just 2.4 points. This trend highlights a critical question of diminishing returns: as foundation video generation models rapidly improve their native instruction-following and compositional capabilities, the marginal utility of complex multi-turn agentic scaffolding and tool coordination may shrink.

Furthermore, the evaluation omits comparisons against the most advanced frontier video generation models available (such as Seedance 2.5, Wan 3.0, or Minimax H3). Without evaluating the agent framework on top of these top-tier foundation models, it remains unproven whether explicit tool-use loops and simulation/retrieval augmentations provide meaningful gains over native generation, or if advanced base models render such multi-step pipelines redundant. The paper should discuss how agentic value scales with model generation capability and incorporate stronger contemporary baselines to validate the framework's long-term relevance.

  1. As detailed in Section 2.2, the prompt construction process purposely simplifies non-target aspects of each prompt and uses stripped-down subjects or backgrounds to ensure that the intended capability remains the primary challenge. While this controlled curation is effective for isolating specific bottlenecks (such as physics simulation or single-entity identity preservation), it raises questions about the framework's ecological validity.

In practical video generation workflows and real-world industrial use cases, user prompts are rarely neatly categorized or artificially isolated; instead, they are messy, open-ended, and cross-cutting, frequently demanding the simultaneous synthesis of complex multi-entity identities, accurate physics, compositional spatial structures, and narrative transitions. By simplifying non-target attributes during data and benchmark creation, the evaluation may overestimate the agent's performance in unconstrained, real-world production environments. The paper should discuss how the framework scales when faced with uncurated, multi-faceted prompts that combine these dimensions concurrently, or provide evaluation results on wild, non-isolated prompts.

  1. Multi-turn agentic rollouts - involving web searches, image retrieval, physics code execution, object detection, depth estimation, and multiple video generation passes - incur significantly higher latency and computational costs compared to single-pass video generation. A detailed runtime as well as token cost analysis would strengthen the practical deployment discussion.
Questions for authors
  1. As reported in Table 2, while VideoGen-Agent-Toolset 1 yields a substantial 19.1-point improvement over its direct base model (Seedance 1.0 Pro Fast), the performance gap narrows sharply to just 2.4 points when compared against a stronger standalone baseline (Seedance 2.0 Fast). This trend highlights a critical concern regarding diminishing returns: as foundation video generation models rapidly improve their native instruction-following and compositional capabilities, the marginal utility of complex multi-turn agentic scaffolding and tool coordination may shrink. The paper should discuss how agentic value scales with base model capability and incorporate top-tier contemporary baselines (such as Seedance 2.5, Wan 3.0, Minimax H3, etc) to validate the framework's long-term relevance.

  2. As detailed in Section 2.2, the prompt construction process purposely simplifies non-target attributes and uses stripped-down subjects or backgrounds to ensure that each task category isolates a single primary challenge. While this controlled curation is effective for isolating specific bottlenecks, it raises questions regarding the framework's ecological validity. In real-world production environments and industrial use cases, user prompts are rarely neatly categorized or artificially isolated; they are typically messy, open-ended, and cross-cutting, frequently demanding the simultaneous synthesis of complex multi-entity identities, accurate physics, spatial compositions, and multi-shot narrative transitions. The paper is encouraged to discuss how the framework scales when exposed to uncurated, multi-faceted prompts, or provide evaluation results on wild, non-isolated inputs.

  3. Multi-turn agentic rollouts—which interleave web searches, image retrieval, physics code execution, object detection, depth estimation, and multiple sequential video generation passes - incur significantly higher latency and computational expenses than standard single-pass video generation. To better substantiate the practical deployment viability of VideoGen-Agent, the paper should include a detailed analysis of end-to-end inference latency, execution time, and token/compute costs relative to baseline methods.

4 comments

We thank the reviewer for recognizing that VABench tests important problems in video generation. We also thank the reviewer for noting that our agent can use better generation tools without retraining. We respond to the concerns and questions below.

[Q1 & W1] Agentic gains with stronger generators and newer frontier video models.

The 2.4-point gap uses the older Toolset 1 agent against standalone Seedance 2.0 Fast. It does not measure the agentic gain with the stronger generator. Table 2 provides that comparison: Toolset 2 scores 86.1 against 73.2, a 12.9-point gain. The trained policy is unchanged, so it does not shrink when using stronger generators We have added frontier-backend comparisons on the same 600 prompts (Appendix H, Table 9). The agent scores 85.4 to 88.2, with gains of 11.1 to 14.0 points over the corresponding standalone models. These gains use unrounded category means. PS retains Wan 2.1-VACE-14B and reuses the Toolset 2 score of 69.2, as marked in the table. The comparisons measure complete systems with external conditioning. They do not isolate policy gains at matched cost or test native flow support in the frontier generators.

[Q2 & W2] Generalization from controlled prompts to complex real-world requests.

VABench isolates capabilities to make failures diagnosable. We have clarified this scope. GenArtist [1] and CompAgent [2] also use capability-specific T2I-CompBench evaluations. Gen-Searcher [3] uses a curated search-grounded benchmark. Controlled prompts help identify what a method improves. They do not establish performance on unrestricted production requests.

We also evaluate 100 held-out Procedural Identity prompts that combine specialized actions with named-subject identity (Appendix E.3). No combination-specific training is used. RL scores 73.2, compared with 57.6 for SFT and 72.4 for the composed workflow. This tests one unseen combination. More complex requests may introduce conflicting constraints or propagate errors between tools. We discuss these limits without claiming unrestricted real-world generalization.

[Q3 & W3] Inference cost of multi-turn tool interactions.

We have reported the inference trade-off in Tables E/F (Appendix I, Tables 14/15). RL takes 115.5 s per prompt. Fixed workflow takes 83.2 s, while direct generation takes 46.1 s. Their quality scores are 75.6/64.1/56.5. RL uses 20.1k input tokens and 0.81k output tokens per prompt. Fixed workflow uses 0.2k of each.

RL's video cost is 0.04 USD per prompt. Auxiliary tools cost 0.01 USD, giving a reported total of 0.05 USD. Fixed workflow has the same rounded total. Direct generation costs 0.03 USD. These figures exclude local Qwen compute. Table F gives the task-level breakdown. Table C shows that RL retains its quality advantage under matched generation-call caps.

Table E. Inference latency and cost. Appendix I, Table 14. Values use the same Toolset 1 evaluations as Table B. Token counts are summed across model calls per prompt and reported in thousands. E2E covers the full execution, including queues and retries. Costs cover video generation and auxiliary tools, including failed calls. They exclude local Qwen compute. Training and post-hoc scoring are also excluded. "-" means inapplicable.

MethodE2E mean (s)Input tokens (k)Output tokens (k)Video cost ($)Auxiliary cost ($)Total ($)
Direct generation46.1--0.030.000.03
Fixed workflow83.20.20.20.040.010.05
VideoGen-Agent-RL115.520.10.810.040.010.05

Table F. Runtime by task for VideoGen-Agent-RL with Toolset 1. Appendix I, Table 15. Each category averages 100 prompts, including failures. Overall averages all 600 prompts. Unused components contribute zero. "-" means unused. E2E covers the full workflow, including retries. It also includes last-frame extraction and video assembly. Component times may overlap.

CategoryNGeneration (s)Text/image retrieval (s)Simulation (s)Grounding/object detection/depth (s)Agent/planner/critic (s)E2E (s)Total cost ($/prompt)
PK10040.75.6--19.276.50.05
SI10041.86.4--18.772.70.05
MI10050.220.3--50.1130.20.05
PS10054.1-17.7-25.3102.30.00
CS100100.3--4.542.9153.40.06
MS100120.9---31.7157.90.09
Overall60068.05.43.00.831.3115.50.05

We hope the corrected comparison and supporting results resolve the main concerns and support a reconsideration of the assessment.

References

[1] Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing. arXiv preprint arXiv:2407.05600, 2024.

[2] Zhenyu Wang, Enze Xie, Aoxue Li, Zhongdao Wang, Xihui Liu, and Zhenguo Li. Divide and Conquer: Language Models Can Plan and Self-Correct for Compositional Text-to-Image Generation. arXiv preprint arXiv:2401.15688, 2024.

[3] Kaituo Feng, Manyuan Zhang, Shawn Chen, Yunlong Lin, Kaixuan Fan, Yilei Jiang, Hongyu Li, Dian Zheng, Chenyang Wang, and Xiangyu Yue. Gen-Searcher: Reinforcing Agentic Search for Image Generation. arXiv preprint arXiv:2603.28767, 2026.

Thank you for the clarifications. I have increased my score to a 2. I have chosen not to raise it to a 3 for the following reasons:

  1. Limited novelty: The concept of identifying weaknesses in standalone models and addressing them via post-training or custom agents is well-explored. Furthermore, the value of this specific approach may diminish as baseline models continue to rapidly evolve.
  2. Practical overhead: In real-world applications, creators and companies weigh speed, cost, and performance simultaneously. While the proposed method enhances the generation quality of baseline models, it also introduces significant time and computational costs.

Thank you for the follow-up and for raising your score. We address the two remaining concerns below.

1. Novelty and relevance as generators improve.

We agree that tool augmentation itself is an established idea. Our contribution is multitask reinforcement learning that teaches a shared video-generation agent to use different tools to solve different problems. The policy learns to select tools, incorporate their outputs, and respond to feedback across six heterogeneous video tasks. To our knowledge, prior work has not demonstrated this form of multitask RL for video-generation tool use. VISTA [1] uses test-time prompt optimization, while Gen-Searcher [2] trains a policy for search-augmented image generation. Recent concurrent methods also target specific tasks. MVAgent [3] applies RL to multi-shot generation, while Video-Coder [4] trains an agent for code-driven scene construction. With the backbone and tools held fixed, performance improves from 64.1 with a fixed workflow to 69.2 after SFT and 75.6 after RL, supporting the value of learned coordination. The open-weight 8B policy is one implementation of this methodology, which is not inherently restricted to that backbone or scale.

We agree that stronger generators may reduce the need for particular tools on some tasks. However, the same policy, trained once with Toolset 1, achieves gains of 11.1 to 14.0 points over standalone Seedance 2.0 Fast, Seedance 2.5, Wan 3.0, and MiniMax H3 without retraining (Table 9). This shows that learned coordination remains complementary to the stronger generators evaluated. Some tasks also require information supplied at inference time, such as specific reference images or explicit simulation conditions, which stronger pretrained capabilities alone do not guarantee access to. A standalone generator is unlikely to satisfy every task requirement on its own. Our Physics Simulation task provides an initial example of combining executable code with video generation, where simulated motion supplies explicit physical structure. This points toward combining generative models with coding capabilities to tackle more complex tasks. We acknowledge that future models may narrow individual gaps, while our results demonstrate continued value as generation backends improve.

2. Practical overhead.

We agree that quality, speed, and cost must be considered together. Commercial video agents such as Moodio [5] and Utopai’s PAI [6] also coordinate generation workflows, illustrating the practical relevance of agentic video creation. Our contribution is a methodology for training open-weight models to perform this coordination, giving developers control over adaptation and deployment. The demonstrated policy runs on a single A100 40GB GPU without per-token planner API fees. Tool costs increase by only USD 0.02 per prompt, from USD 0.03 to USD 0.05, excluding local policy compute. End-to-end latency increases from 46.1 s to 115.5 s, with video generation accounting for most of the runtime (Tables 14 and 15). We acknowledge this latency increase and do not claim cost or speed superiority over proprietary agents without matched measurements.

Training makes the coordination policy itself amenable to improvement. Developers can adapt it to their tools and recurring user trajectories if they have agent platform, and potentially optimize it to avoid unnecessary calls rather than depend on a proprietary planner. Such efficiency improvements remain future work. Our current experiments show that the quality advantage over a fixed workflow persists under matched generation call caps (Table 12), supporting the value of learned coordination within a controlled generation budget. We will clarify the demonstrated quality gains and their deployment trade-offs in the revision.

[1] VISTA: A Test-Time Self-Improving Video Generation Agent. arXiv:2510.15831, 2025.

[2] Gen-Searcher: Reinforcing Agentic Search for Image Generation. arXiv:2603.28767, 2026.

[3] MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization. arXiv:2609.30609, 2026.

[4] Video-Coder: Reinforcing Agentic Coding for Video Generation. arXiv:2611.09894, 2026.

[5] Moodio. https://www.moodio.art/

[6] Utopai Studios. PAI. https://www.utopaistudios.com/en/pai

Reviewer_K3ySOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors propose VideoGen-Agent, a multimodal agent trained through supervised fine-tuning and multitask reinforcement learning to coordinate external tools for video generation. The agent uses retrieval, simulation, generation, and verification tools across six task categories. The authors also introduce VABench, a 600-prompt benchmark for evaluating these capabilities. Experiments on VABench demonstrate substantial gains over standalone video generators, supported by ablation studies, human preference evaluations, and further improvements with upgraded generation tools.

Critical strengths
  1. Adaptive tool orchestration. The proposed VideoGen-Agent learns to select and coordinate tools across different generation tasks rather than following a fixed pipeline. The improvement over the fixed-workflow baseline supports the benefit of learned tool use.

  2. Comprehensive task coverage. The proposed VABench covers six challenging video generation tasks, including identity preservation, physical consistency, and temporal structure. This provides a useful setting for evaluating tool-augmented generation.

  3. Impressive empirical results. Based on the experimental results, VideoGen-Agent improves over its base generator by 19.1 points and achieves an 84.3% human preference rate against Seedance 2.0 with Toolset 1.

  4. Flexible tool integration. Replacing the generation tools improves the overall score from 75.6 to 86.1 without retraining the agent, demonstrating compatibility with upgraded generation models.

Critical weaknesses
  1. Unmatched inference budgets. The fixed-workflow baseline uses predefined execution budgets, while the RL agent allows iterative refinement. Without matched generation calls or runtime costs, it is unclear how much of the improvement comes from learned tool use.

  2. Limited agentic baselines. Most comparisons are against standalone video generators, with only a hand-designed fixed workflow as an agentic baseline. Comparisons with existing video generation agents, such as VISTA, are missing.

  3. Unclear multitask benefits. Single-task agents slightly outperform the shared multitask agent (76.3 vs. 75.6), while gains on the unseen task combination are marginal (73.2 vs. 72.4). This provides limited evidence for the benefits of multitask RL beyond using a shared policy.

  4. Potential evaluation bias. Gemini 3.1 Pro is used for both RL rewards and benchmark evaluation with the same rubrics. Judge–human agreement drops to 72% for Procedural Knowledge, raising concerns about the reliability of improvements measured by this evaluator.

Questions for authors
  1. Could the authors report the number of generation calls and inference costs for each method? A comparison under matched budgets would help clarify the gains from learned tool use.

  2. How does VideoGen-Agent compare with existing agentic video generation methods, such as VISTA, under comparable tool and inference settings?

  3. Table 3 shows that single-task agents slightly outperform the multitask agent. What are the main benefits of using a shared policy, particularly in terms of generalization?

  4. Since Gemini 3.1 Pro is used for both training rewards and evaluation, could the authors validate the results with an independent judge, especially for Procedural Knowledge, where judge–human agreement is lower?

3 comments

We thank the reviewer for recognizing the value of learning to use tools across different tasks. We also thank the reviewer for noting the strong results and the ability to use better generation tools without retraining. We respond to the concerns and questions below.

[Q1 & W1] Inference budgets and generation usage.

RL achieves higher quality under the same generation-call caps. We have reported these comparisons in Table C (Appendix I, Table 12). RL scores 74.3/75.1/76.4 at B1/B2/B3. Its lead over fixed workflow is 10.2 to 12.0 points. Its lead over SFT is 5.5 to 7.2 points. Table B reports actual generation usage. Table E reports the corresponding inference costs.

The fixed workflow uses the prescribed task-specific procedures with Toolset 1. CS permits up to three verification iterations. MS generates its requested segments sequentially. The learned agent can revise its actions using tool feedback within 20 interaction steps. Every generated candidate counts toward the cap, including technical retries and MS segments. Equal caps limit the allowed generation budget. They do not imply equal realized usage or cost.

Table B. Generation usage under the standard evaluation protocols. Appendix I, Table 11. Values are means per prompt over 600 test prompts, with 100 per category. Failures remain included. Every candidate counts as an attempt. This includes technical retries and MS segments. A successful generation means a returned video, not correct content. Other-tool counts exclude generation. All methods use Toolset 1. Table E reports costs from the same evaluations.

MethodNGeneration attempts, meanSuccessful generations, meanTechnical retries, meanOther tool calls, meanQuality (0 to 100)
Direct generation6001.001.000.000.0056.5
Fixed workflow6001.231.230.001.6164.1
VideoGen-Agent-SFT6001.591.410.182.1369.2
VideoGen-Agent-RL6001.801.760.042.5375.6

Table C. Quality under matched generation-call caps. Appendix I, Table 12. Evaluation uses the same 600 prompts. The base cap is B1=1B_1=1 for PK/SI/MI/PS. CS uses B1=3B_1=3. MS uses B1=KB_1=K, where K∈{2,3}K\in\{2,3\} is the requested shot count. B2=2B1B_2=2B_1 and B3=4B1B_3=4B_1. Every candidate counts toward the cap, including technical retries and MS segments. Equal caps specify allowed generation usage. Realized calls and monetary costs may differ.

CategoryB1B2 = 2 × B1B3 = 4 × B1
PK124
SI124
MI124
PS124
CS3612
MS, K = 2248
MS, K = 33612
Budget levelFixed workflow (budgeted)SFTRL
B164.168.874.3
B2 = 2 × B163.969.575.1
B3 = 4 × B164.469.276.4

[Q2 & W2] Limited comparisons with existing video-generation agents.

We have added a VISTA comparison in Table D (Appendix I, Table 13). VISTA [1] uses Gemini 2.5 Flash to refine prompts through video feedback. Veo 3 generates its videos. It does not train a tool-use policy or retrieve external references. Our open-weight 8B policy learns tool use through SFT and multitask RL. It can retrieve procedural details or visual references before generation. Our Toolset 2 system scores 86.1, compared with 77.8 for adapted VISTA. This is a comparison of complete systems, not an isolated test of policy training.

VISTA's original setup generates 150 videos per prompt. To keep evaluation practical, we retain its refinement loop but use three rounds with two candidates per round. This gives six videos per input. In qualitative PK/SI/MI cases, repeated generation with Veo 3 still failed to recover the requested content without external retrieval. More candidates did not consistently resolve these failures. These observations suggest that extra sampling alone may not supply missing task-specific evidence. They do not establish that further sampling can never help. The 1.8-call mean in Table B is for Toolset 1, not the Toolset 2 system in Table D. We do not claim matched generation cost for the VISTA comparison.

Table D. Overall quality of agentic baselines. Appendix I, Table 13. Both methods use the same 600 VABench prompts and scoring protocol. Our agent uses Toolset 2. VISTA uses Gemini 2.5 Flash with Veo 3. Adapted VISTA runs for three rounds, starting with initialization. Two refinement rounds follow. Each round generates two candidates, giving six videos per input. Scores compare complete systems with different agent backbones and tool inventories. Generation budgets are not matched.

MethodOverall quality (0 to 100)
VideoGen-Agent-RL + Toolset 286.1
VISTA77.8

[Q3 & W3] Limited evidence for multitask benefits beyond policy consolidation.

The main benefit is one policy covering six tasks at near-specialist quality. The shared policy scores 75.6, compared with 76.3 for six specialists. It leads on three categories. We do not claim uniform gains over specialization. Specialists use the same backbone and training settings but train only on their own category (Appendix E.4). AgentRL [2] reports a similar pattern. Its shared policy scores 67.7, close to 67.8 from the task-wise best single-task results.

Appendix E.3 tests 100 held-out Procedural Identity prompts without combination-specific training. RL scores 73.2, compared with 57.6 for SFT. The composed workflow scores 72.4. The main evidence is the 15.6-point gain over SFT, not the small lead over that workflow. Dual-retrieval usage rises from 40% to 84%. These results support generalization to this combination. They do not isolate multitask training as its cause.

[Q4 & W4] Independent evaluation and procedural-knowledge judge reliability.

We have evaluated a second judge on PK. Gemini 4 Argon agrees with the human majority on 81% of the same 100 PK pairs. Gemini 3.1 Pro agrees on 72%. We use the same judge prompt and human-majority labels. PK scoring assigns 90% weight to factual correctness and 10% to aesthetics (Appendix D.1). The factual references come from Claude Web Search and are manually checked. All methods use the same references. These are separate from the agent's retrieval outputs (Appendix D.2).

We have also added human preference studies for both toolsets (Appendix E.1). Each compares 100 pairs against standalone Seedance 2.0 Fast. Four raters provide 400 judgments per comparison. Toolset 1 receives 88.25% (353/400) agent preference. Toolset 2 receives 91.50% (366/400). These rates measure overall preference. PK judge reliability is measured separately by the agreement experiment above. The 81% agreement still leaves some disagreement with humans.

We hope these comparisons and validation results address your concerns and clarify the contribution of the shared policy.

References

[1] Do Xuan Long, Xingchen Wan, Hootan Nakhost, Chen-Yu Lee, Tomas Pfister, and Sercan Ö. Arık. VISTA: A Test-Time Self-Improving Video Generation Agent. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.

[2] Hanchen Zhang, Xiao Liu, Bowen Lv, Xueqiao Sun, Bohao Jing, Iat Long Iong, Zhenyu Hou, Zehan Qi, Hanyu Lai, Yifan Xu, Rui Lu, Hongning Wang, Jie Tang, and Yuxiao Dong. AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework. arXiv preprint arXiv:2510.04206, 2025.

I appreciate the efforts of authors' on rebuttal. After carefully reviewing the rebuttal and revisiting the paper, most of my concerns have been solved. I'm willing to maintain my current positive rating score.

Reviewer_nYY2OpenReview ↗
Rating
3
Confidence
2(−1)
Informativeness
2(−1)
Presentation
3
Soundness
2(−1)
Read review
Paper summary

This paper presents VideoGen-Agent, a multimodal agent trained to coordinate external tools for video generation across six task categories: procedural knowledge, single- and multi-entity identity preservation, physical consistency, scene composition, and multi-shot temporal structure. The agent uses retrieval, simulation, generation, and verification tools through multi-turn interactions. Training consists of supervised fine-tuning on teacher-generated trajectories, followed by multitask GRPO with a hybrid reward covering output validity, tool use, and video quality. The authors also introduce VABench, a held-out benchmark containing 600 prompts. VideoGen-Agent achieves 75.6 with its training-time toolset, compared with 56.5 for its base text-to-video generator. Upgrading the generation tools without further policy training increases the score to 86.1.

Critical strengths
  1. Clear motivation and coherent framework. The paper addresses video generation fidelity beyond visual quality, including procedural accuracy, identity preservation, physical plausibility, and temporal ordering. The proposed tools provide appropriate support for these distinct challenges within a shared agent framework.

  2. Evidence supporting learned tool use. The experiments compare direct generation, prompt rewriting, fixed workflows, SFT, and RL. Under Toolset 1, performance improves from 64.1 with fixed workflows to 69.2 after SFT and 75.6 after RL, supporting the value of training the agent policy.

  3. Practical modularity design. Replacing generation tools while keeping the trained policy fixed improves performance from 75.6 to 86.1. This demonstrates that the agent can benefit from compatible tool upgrades without additional policy optimization.

  4. Evaluation across multiple capabilities. Category-specific evaluation helps identify where tool augmentation is most effective. Human preference judgments and a separate judge–human agreement study provide additional support for the automated results.

Critical weaknesses
  1. failure identification and masking during SFT. The paper states that failed action spans are excluded from the supervised loss while their context and subsequent recovery actions are retained. I'd like to ask what kind of failures are excluded, execution failures like format errors or semantically incorrect but executable actions, such as selecting an incorrect identity reference or supplying simulation parameters inconsistent with the prompt?And how these failures are identified?

  2. GRPO task-level normalization and trajectory length. Task-level advantage statistics are computed over agent-emitted tokens, so a trajectory’s advantage is repeated according to its length. Longer trajectories therefore receive greater weight in the category statistics, while the policy loss also aggregates over tokens. This does not necessarily incentivize longer outputs, but it couples trajectory length with optimization weight. Have you analyzed response lengths, tool-call counts, and their relationship with rewards? Or a direct ablation removing category normalization would also clarify its contribution.

  3. training rewards and benchmark evaluation using the same VLM.
    RL rewards and the main benchmark scores use the same VLM judge (Gemini 3.1 Pro) and scoring rubrics, leaving open the possibility of adaptation to that evaluator. Human evaluation only partially addresses this concern, the preference study covers only 100 pairs for Toolset 1 and does not validate the highest-performing Toolset 2 configuration.

Questions for authors

please see weaknesses

4 comments

We thank the reviewer for recognizing that our paper addresses an important problem and presents a clear approach. We also thank the reviewer for noting the gains from learned tool use and the ability to upgrade tools without retraining. We respond to the concerns and questions below.

[W1] Failure identification and loss masking in SFT.

We have clarified error identification in Appendix C.9. We use schema validation and explicit tool-reported invocation errors. Executable semantic mistakes are outside this detector's scope. A valid reference image with the wrong identity is one example. A timeout or HTTP 500 response alone does not establish an incorrect action.

We mask the entire erroneous assistant turn, including its reasoning and action tokens. The failed turn remains in context alongside the error observation. Correct recovery turns are supervised. Tool observations never contribute to the loss. Trajectories with at least three identified errors are discarded before SFT.

[W2] Trajectory-length weighting in task-level advantage normalization.

Trajectory length affects both category statistics and policy-loss weight. We have clarified this in Appendix C.6 and Appendix I. We follow AgentRL's token-pooled normalization [3]. Let LiL_i count valid agent tokens in trajectory ii. Its group-relative advantage A^i\hat A_i is repeated over those tokens. The sample standard deviation uses Bessel's correction. Statistics are pooled across data-parallel workers. Tool observations are excluded. The policy loss averages over valid agent tokens, so longer trajectories receive greater weight.

\sigma_c^2=\frac{\sum_{i\in c}L_i(\hat A_i-\mu_c)^2}{\sum_{i\in c}L_i-1},\qquad \widetilde A_i=\frac{\hat A_i-\mu_c}{\sigma_c+\epsilon}.$$ We have added the analysis in Table A (Appendix I, Table 10). Removing category normalization lowers Toolset 1 quality from 75.6 to 72.1, a 3.5-point drop. The group-relative advantage computation is unchanged, as is the token-level loss reduction. Mean output length decreases from 0.81k to 0.75k. Within-category training correlations between length and reward range from −0.14 to +0.17. The measured rollouts show no strong monotonic association. This does not rule out other effects of token-length weighting. **Table A. Output length and category-normalization ablation.** Appendix I, Table 10. Overall rows use the same 600 held-out prompts with Toolset 1. Token counts include assistant reasoning and actions, in thousands. Call counts match Table B. PK to MS rows report within-category Spearman correlations with raw training reward $R$. Correlations are computed separately at each sampled update. The ablation removes only category normalization. The group-relative advantage computation and token-level loss reduction are unchanged. Other training settings are held fixed. "-" denotes inapplicable entries. | Variant / category | Agent output tokens (k), mean | Generation calls, mean | Other-tool calls, mean | $\rho(L,R)$ | $\rho(\text{generation calls},R)$ | $\rho(\text{other-tool calls},R)$ | Test quality (0 to 100) | |---|---:|---:|---:|---:|---:|---:|---:| | Full RL / Overall | 0.81 | 1.80 | 2.53 | - | - | - | 75.6 | | RL w/o category norm / Overall | 0.75 | 1.71 | 2.37 | - | - | - | 72.1 | | Full RL / PK | - | - | - | +0.06 | -0.14 | -0.03 | - | | Full RL / SI | - | - | - | -0.04 | +0.00 | +0.07 | - | | Full RL / MI | - | - | - | +0.17 | -0.13 | +0.06 | - | | Full RL / PS | - | - | - | -0.14 | -0.11 | +0.03 | - | | Full RL / CS | - | - | - | +0.05 | +0.09 | +0.06 | - | | Full RL / MS | - | - | - | -0.04 | -0.09 | +0.05 | - |

[W3] Shared training and evaluation judge and limited Toolset 2 human validation.

We use the same judge and rubrics for training rewards and held-out evaluation. Gen-Searcher [1] uses the same GPT-4.1 judge and K-Score for RL and KnowGen evaluation. Flow-GRPO [2] also evaluates with optimization metrics such as PickScore. This measures generalization to unseen prompts under a fixed objective. It does not exclude adaptation to the evaluator.

We have added human preference studies for both toolsets (Appendix E.1). Each uses 100 pairs rated by four people, giving 400 judgments. Toolset 1 receives 353/400 (88.25%) preferences against standalone Seedance 2.0 Fast. Toolset 2 receives 366/400 (91.50%). The separate 600-pair agreement study uses three blinded raters per pair. Overall agreement is 92%, with 72% on PK. We have also evaluated Gemini 4 Argon on the same 100 PK pairs with the same judge prompt and human-majority labels. Agreement reaches 81%. This is agreement with humans, not an agent preference rate.

We hope these clarifications and results address your concerns about training and evaluation.

References

[1] Kaituo Feng, Manyuan Zhang, Shawn Chen, Yunlong Lin, Kaixuan Fan, Yilei Jiang, Hongyu Li, Dian Zheng, Chenyang Wang, and Xiangyu Yue. Gen-Searcher: Reinforcing Agentic Search for Image Generation. arXiv preprint arXiv:2603.28767, 2026.

[2] Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-GRPO: Training Flow Matching Models via Online RL. Advances in Neural Information Processing Systems (NeurIPS), 2025.

[3] Hanchen Zhang, Xiao Liu, Bowen Lv, Xueqiao Sun, Bohao Jing, Iat Long Iong, Zhenyu Hou, Zehan Qi, Hanyu Lai, Yifan Xu, Rui Lu, Hongning Wang, Jie Tang, and Yuxiao Dong. AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework. arXiv preprint arXiv:2510.04206, 2025.

Dear Reviewer nYY2,

Thank you again for your time and feedback. As the discussion period is approaching its end, we would greatly appreciate hearing whether our responses have addressed your main concerns, or if there are any remaining questions we could clarify.

Thank you very much for your time and consideration!

Thanks for the author's efforts in addressing my concerns. I think this is a well-rounded work with comprehensive experiments. I will discuss with other reviewers and AC whether to modify the score.

Comments on the paper

1 comment

We thank the reviewers for their feedback. We have clarified the training procedure and expanded the evaluation. Revised text is highlighted in blue.

  • Stronger baselines (vMQC). Toolset 2 improves over standalone Seedance 2.0 Fast by 12.9 points. Frontier-backend gains range from 11.1 to 14.0 points (Appendix H, Table 9). The unchanged PS backend is marked explicitly.
  • Training analysis (nYY2). We have specified the SFT loss mask in Appendix C.9. Removing category normalization reduces Toolset 1 quality from 75.6 to 72.1. Table A examines the effect of trajectory length.
  • Human validation (nYY2, K3yS). Human preference is 88.25% for Toolset 1 and 91.50% for Toolset 2. Gemini 4 Argon reaches 81% agreement with humans on PK, compared with 72% for Gemini 3.1 Pro. Both judges use the same 100 PK pairs and judge prompt (Appendix E.1).
  • Multitask learning (K3yS, vMQC). One policy scores 75.6, close to 76.3 for six specialists. On the held-out Procedural Identity task, RL improves over SFT from 57.6 to 73.2. We state the limits of this evidence.
  • Budget and efficiency (K3yS, vMQC). RL leads fixed workflow by 10.2 to 12.0 points under matched generation-call caps. Mean latency is 115.5 s, compared with 83.2 s for fixed workflow. We have also included adapted VISTA, which scores 77.8 against our Toolset 2 score of 86.1 (Appendix I, Tables 10 to 15).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.