- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents CineForge, a self-improving agent framework for long-horizon story-driven video generation. The framework consists of CineForge-Produce, which organizes video production into structured stages and records production trajectories, and CineForge-Evolve, which uses Case-to-Pattern-to-Policy Evolution (CPPE) to identify recurring failures and update persistent production policies. Experiments show that CineForge improves CineScope scores after three evolution rounds, outperforming adapted GenMAC and VideoWeaver baselines. The results also suggest improvements on unseen stories and reduced review workloads.
Critical strengths
-
An interesting and practical research problem. The paper addresses an important limitation of existing agentic video generation systems: the difficulty of transferring lessons from previous production failures to future stories. The distinction between instance-level repair and persistent production-policy improvement is meaningful.
-
A well-structured production and evolution framework. The separation between CineForge-Produce and CineForge-Evolve is clear. The use of production trajectories, cross-story failure patterns, bounded policy patches, and replay-based validation provides a reasonably systematic framework for self-improvement.
-
Encouraging experimental results. The proposed method achieves improvements over several video generation and optimization baselines. The evaluation includes human judgments, external metrics, unseen-story comparisons, and component ablations. The CineScope benchmark also provides a useful perspective on story-level video quality.
Critical weaknesses
-
Limited methodological novelty. Ideas are closely related to existing work. Although the paper discusses these connections, it does not sufficiently establish which components are fundamentally new. The novelty appears to lie mainly in the integration of existing mechanisms into a video production pipeline.
-
A gap between the theoretical guarantees and their empirical verification. The paper presents confidence-controlled replay and structural improvement guarantees as important contributions. However, Appendix B.9 acknowledges that the reported experiments do not provide the complete per-candidate numerical audit needed to verify every deployed update.
-
Potential fairness issues in the video generation comparisons. Although the methods share the same underlying models and generation budgets, their realized output lengths differ substantially. In Table 7, CineForge-Produce generates videos with a median duration of 12.0 minutes, compared with 3.52 minutes for ViMax and 2.18 minutes for AniMaker. Such differences may affect narrative coverage, character development, and other story-level evaluation dimensions. It is therefore difficult to attribute all improvements solely to the proposed production framework.
-
Limited evaluation of generalization and evaluator reliability. CineScope-Metric achieves reasonable agreement with human judgments, but its held-out alignment evaluation uses only 15 distinct stories, and some individual dimensions show moderate correlations. The proposed method is evaluated primarily on a relatively small fixed test set, while the evolution experiments do not report uncertainty across independently learned policies.
Questions for authors
-
Could the authors further clarify the main methodological differences between CPPE and existing trajectory-driven agent or workflow optimization approaches? A comparison with a general-purpose policy optimization baseline would also be helpful.
-
Could the authors provide additional details on the numerical verification of deployed policy updates, including acceptance margins and regression-test results? It would also be helpful to clarify how the theoretical guarantees relate to the actual sampling protocol.
-
Could the authors discuss how differences in generated video duration might influence the comparisons? A duration-controlled evaluation would help strengthen the conclusions.
-
Could the authors provide further evidence of CineScope-Metric's reliability on a larger held-out set? Additional independent evolution runs would also help assess the stability and generalization of the proposed method.
-
Could the authors provide a more comprehensive analysis of the computational costs associated with policy evolution, including candidate validation and replay? This would help clarify the practical efficiency benefits of the framework.
Thank you very much to the authors for the detailed response. The additional clarifications and results regarding computational cost, stability across independent runs, and the extended validation of CineScope have alleviated my concerns to some extent. However, I still have reservations regarding the overall soundness of the work and the degree of human verification in the manuscript, as there remain several instances of overly defensive or self-contradictory phrasing. Below are a few representative examples:
N/Aentries.must) rather than a descriptive statement of what was conducted reads somewhat like unedited prompt instructions or internal guidelines. Similar prescriptive patterns appear on Page 30, Line 1617 ("must be computed from... Issue counts cannot be substituted...") and Page 31, Lines 1667–1673 (a sequence of five consecutive "X [claims] require Y" statements).unknownmass." However, this expression does not appear anywhere in the preceding 30 pages. Could the authors clarify the origin of this formula?The examples above are illustrative rather than exhaustive. Taken together, they leave me with the impression that human verification and end-to-end consistency checking of the manuscript remain somewhat limited, so I will maintain my current score.
Although I do not find the paper ready for publication at ICLR in its current form, I sincerely appreciate the efforts in the rebuttal, and encourage the authors to thoroughly audit the full manuscript and further strengthen the rigor of both the methodology and its implementation. I believe this work holds genuine potential to make a meaningful contribution to long-horizon video generation in the future.