acceptodds
Under review as a conference paper at ICLR 2027

ReFactor: Reflection-Augmented Agentic Video Generation

13 views

Abstract

Narrative video generation requires coordinating multiple tools and adapting subsequent operations to intermediate outputs. Existing approaches typically rely on fixed pipelines or one-shot plans, which often fail when generated outputs deviate from expectations and lack mechanisms for autonomous recovery. We propose ReFactor, a multi-turn agentic framework that formulates narrative video generation as a Markov decision process: At each step, the agent selects a tool call conditioned on the narrative prompt, execution history, and generated artifacts to maximize narrative completeness. Since video generation requires multiple sequential tool calls without informative feedback, the ability to reflection becomes crucial for agents to inspect generated artifacts, diagnose failures, and guide corrective actions. To enhance the agent's reflection ability, we construct 21K reflection-augmented trajectories grounded in actual tool executions and enriched with evaluator-accepted repair demonstrations for SFT. We further develop an agentic RL approach that introduces a reflection-anchored process reward to measure the relative quality improvements from intermediate repairs. This provides denser supervision for learning when and how to reflect across tasks of varying difficulties. On StoryEval, ReFactor achieves an 18.9% relative improvement in narrative completion over the strongest baseline. Beyond this gain, RL makes the policy learn selective reflection, cutting reflection frequency on easy prompts from 0.32 to 0.13 while raising it on hard prompts from 0.46 to 0.61 relative to the SFT policy, with scorer-defined repair success improving in both groups. Code is available at: https://anonymous.4open.science/r/Refactor-6BC2

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agentic video generation, Multimodal agents, Reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LSWvOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes ReFactor, a multi-turn agentic framework for narrative video generation. It formulates video generation as an MDP where a multimodal LLM controller selects tool calls based on the prompt, execution history, and generated artifacts. The framework includes structured visual memory, post-hoc reflection for repairing defective artifacts, SFT on 21K reflection-augmented trajectories, and online RL with GRPO using a reflection-anchored process reward. Experiments on StoryEval and VideoScore2.0 show clear improvements over strong baselines, especially on hard prompts.

Critical strengths

1.Solid experimental validation and detailed analysis. The paper provides comprehensive experiments, ablations, human evaluation, and behavior analysis. The results are consistent and provide useful empirical insights for the community.

2.Well-written and easy to follow. The system design, training pipeline, and evaluation protocol are clearly presented.

Critical weaknesses

None. I do not identify a rejection-level weakness. The main concerns are moderate novelty and missing compute/cost details, which I raise as questions rather than as grounds for rejection.

Questions for authors
  1. I am concerned about the practical cost and scalability of online GRPO in this setting. Because GRPO is on-policy, each update requires fresh rollouts, and each rollout involves repeated calls to slow video-generation tools. The paper reports GPU counts for controller training but not the cost of tool execution or reward evaluation, so it is unclear whether the reported RL time reflects total wall-clock time and whether the method remains practical at larger scale. Could you clarify: (1) Is rollout collection synchronous or asynchronous? (2) How many workers/GPUs are used for tool execution? (3) What is the average wall-clock time to collect one rollout batch? (4) What are the total GPU hours for RL training, including tool execution and reward evaluation, and the estimated cost per generated video?

  2. For SFT reflection injection, my concern is causal consistency. If a repair is spliced at step t and subsequent actions are kept from the original trajectory, those actions were generated conditioned on the original artifact, not the repaired one. Could you clarify whether subsequent actions are kept or re-generated, and share your view on whether this may introduce inconsistency between the repaired artifact and later decisions? If so, how do you handle it?

Reviewer_j7qsOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents ReFactor, an agentic framework designed to tackle the long-horizon challenge of narrative video generation. Unlike fixed-pipeline or one-shot models, ReFactor formulates video generation as a Markov decision process (MDP). The agent is capable of post-hoc reflection, meaning it can inspect generated artifacts, diagnose visual failures, and execute corrective tool calls before finalizing the video.

Critical strengths

The paper addresses an important problem: video-generation systems should adapt when intermediate outputs fail. ReFactor is evaluated on a narrative benchmark, includes a human preference study, and reports improvements (which are reported to be quite substantial) with the same generation tools as its agent baselines.

Critical weaknesses
  • Limited novelty. The method combines well-established components, including multimodal tool use, trajectory collection, and GRPO. The before/after quality reward is also simple. I find it hard to see the new points proposed in the framework.

  • With this design, the process-level credit assignment is pretty weak. Repair scores are averaged into a single trajectory return for GRPO, so successful repairs can reinforce unrelated actions in the same rollout. Moreover, the local score can measure generic artifact quality instead of whether the diagnosed defect or missing narrative event was fixed.

  • The evidence for and advantage of selective reflection are unclear to me. It's worth noting that different reflection frequencies on Easy and Hard prompts do not show that the policy responds appropriately to the observed artifact.

  • The evaluation also remains quite limited, focusing on only one benchmark with an automatic judge. Also, no compute-equivalent comparisons are conducted.

  • Finally, can the authors provide some examples of generated videos? It's hard to justify the improvements by just looking at llm judge scores.

Questions for authors
  • How many repair examples and reflection steps are actually present in the final 21K SFT trajectories?
  • Does the process reward correlate with human judgments that a repair fixed the stated defect or completed the relevant story event?
  • What happens to the full-RL advantage when process reward is ablated with matched RL training budget and multiple seeds?
  • Can the authors provide more examples of real generated videos?
Reviewer_x7TnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

ReFactor uses a multimodal agent to generate narrative videos through repeated tool calls. After inspecting each generated artifact, the controller decides whether to continue, repair it, extract frames, or compose clips. It keeps the newest artifact as visual input and summarizes older artifacts with captions. Training starts with approximately 21K executed trajectories augmented with evaluator-accepted repairs, followed by GRPO using final-video rewards and a process reward for relative quality improvement after repairs. The trained 9B model scores 64.8 on StoryEval, compared with 54.5 for MultimediaAgent, a gain of 10.3 points.

Critical strengths
  1. The controller makes subsequent tool decisions from actual generated outputs, addressing a practical problem with planning an entire video before seeing any results. Executed trajectories and repair demonstrations provide supervision for handling the failures that arise during generation.

  2. The final-output gains are supported by human preferences, prompt-level paired bootstrap intervals, and a test-time narrative judge from a different model family than the training reward evaluator. These checks strengthen the evidence provided by the automatic scores and cumulative ablations.

Critical weaknesses
  1. The experiments leave the contribution of learned reflection difficult to separate from the other changes. The architecture ablation adds reflection and visual memory together, while the complete-system comparison also adds repair supervision and RL. These results support the full training and inference procedure, but they do not show how much each mechanism contributes. The reported repair success rates also use the artifact scorer used during training. The human study supports better final videos; it does not separately check whether individual repairs fix the semantic failures that prompted them.

  2. The evaluation leaves open how well the trained policy transfers and what it costs to run. StoryEval prompts contain only two to four events. The alternative-generator experiment tests the training-free Gemini architecture, so it does not establish transfer of the trained 9B policy. Keeping only one raw artifact also makes longer-range identity checks depend on the captions. For efficiency, generation-call counts and calls per StoryEval point omit controller, captioning, and utility overhead. Those measurements describe one part of the cost, but are insufficient to assess the end-to-end resource trade-off.

Questions for authors
  1. Can you ablate reflection and visual memory separately at the same training stages, and compare with a non-reflection controller given comparable RL training and inference budgets?

  2. In a sample of before-and-after repair pairs, how often is the diagnosed failure actually corrected according to blinded human judgments or a separate evaluator?

  3. Does the trained 9B policy retain its gains with a different video generator, with narratives longer than four events, or with recurring characters whose identity must remain consistent?

  4. At matched output duration and resolution, what are the end-to-end latency, controller and captioner usage, and generation compute? How does performance compare when the total resource budget is matched?

2 comments
Reviewer_x7TnOpenReview ↗

Rebuttal Follow-up

Thank you for the additional experiments. The separate memory and reflection ablations and the comparison under the same outcome-RL training setup address my main questions about where the gains come from. The blinded human evaluation also provides direct evidence of how often repairs correct the diagnosed defects. I am raising my rating from 2 to 3.

Thank you for your thoughtful follow-up and for updating your score! We are glad that our additional experiments and analyses addressed your main concerns. We also appreciate your careful assessment and your support for acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.