- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes ReFactor, a multi-turn agentic framework for narrative video generation. It formulates video generation as an MDP where a multimodal LLM controller selects tool calls based on the prompt, execution history, and generated artifacts. The framework includes structured visual memory, post-hoc reflection for repairing defective artifacts, SFT on 21K reflection-augmented trajectories, and online RL with GRPO using a reflection-anchored process reward. Experiments on StoryEval and VideoScore2.0 show clear improvements over strong baselines, especially on hard prompts.
Critical strengths
1.Solid experimental validation and detailed analysis. The paper provides comprehensive experiments, ablations, human evaluation, and behavior analysis. The results are consistent and provide useful empirical insights for the community.
2.Well-written and easy to follow. The system design, training pipeline, and evaluation protocol are clearly presented.
Critical weaknesses
None. I do not identify a rejection-level weakness. The main concerns are moderate novelty and missing compute/cost details, which I raise as questions rather than as grounds for rejection.
Questions for authors
-
I am concerned about the practical cost and scalability of online GRPO in this setting. Because GRPO is on-policy, each update requires fresh rollouts, and each rollout involves repeated calls to slow video-generation tools. The paper reports GPU counts for controller training but not the cost of tool execution or reward evaluation, so it is unclear whether the reported RL time reflects total wall-clock time and whether the method remains practical at larger scale. Could you clarify: (1) Is rollout collection synchronous or asynchronous? (2) How many workers/GPUs are used for tool execution? (3) What is the average wall-clock time to collect one rollout batch? (4) What are the total GPU hours for RL training, including tool execution and reward evaluation, and the estimated cost per generated video?
-
For SFT reflection injection, my concern is causal consistency. If a repair is spliced at step t and subsequent actions are kept from the original trajectory, those actions were generated conditioned on the original artifact, not the repaired one. Could you clarify whether subsequent actions are kept or re-generated, and share your view on whether this may introduce inconsistency between the repaired artifact and later decisions? If so, how do you handle it?
Rebuttal Follow-up
Thank you for the additional experiments. The separate memory and reflection ablations and the comparison under the same outcome-RL training setup address my main questions about where the gains come from. The blinded human evaluation also provides direct evidence of how often repairs correct the diagnosed defects. I am raising my rating from 2 to 3.