- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Current video generation benchmarks primarily focus on temporal coherence, physical fidelity and semantic alignment, while overlooking the artistic expressiveness of multi-shot filmmaking, which is very important in current art creativity. To address this gap, CINEMA-Bench constructs high-quality prompts from real, professionally produced film clips and evaluates six leading video generation methods using a taxonomy grounded in film theory. This work represents an early exploration toward a comprehensive definition and quantification of cinematic expressiveness, bridging artistic expression and computational evaluation.
Critical strengths
It examines multi-shot video generation from the distinctive perspective of artistic expressiveness.
The prompts cover Mikels’ eight emotion categories, and the evaluation taxonomy is grounded in film theory, providing a well-founded methodological basis.
Critical weaknesses
In Table 4, cinematic diversity and rhythm rationality show limited agreement with human judgments. Many of the paper’s automatic metrics rely on Gemini, which interprets videos through sampled frames. Whether this approach can yield an understanding of cinematic content comparable to that of human viewers remains to be established.
Questions for authors
-
Cinematic expression is closely intertwined with music and sound. Could CINEMA-Bench’s exclusion of audio lead to a biased assessment of artistic quality? I think a comprehensive evaluation should be multimodal.
-
Could the evaluation metrics in CINEMA-Bench be used as reward signals to train video generation models?
-
As our understanding of art and cinematic conventions evolves, could this evaluation framework become outdated? How could it adapt to such changes?
-
Could you include new models like LTX in the model evaluation?
-
Different models may respond differently to prompt styles, especially descriptions of shot transitions. Were the prompts tailored to each model during evaluation? If not, could the observed performance differences partly reflect biases arising from model-specific prompt preferences rather than differences in cinematic capabilities? How to seperate this bias?
Since the authors did not submit a rebuttal, I will maintain my original score.