- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper propose a FilmTogether, a video generative system for transforming low cost captured videos in to high-cost cinematic productions. At the core of the method is two main contributions: 1. CineCraft-120k, a large scale dataset of aligned low and high cost video pairs annotated with editing prompts, synthetically generated by transforming high quality production ready datasets into low quality videos. 2. instruction-routed text memory and layer-wise hint memory which enables such low quality to high quality video. Temporal consistency, PickScore, and VLM judge scores are provided on the synthetic datasets. User study is also performed.
Critical strengths
- The dataset is very large, and building it takes about 1500 gpu-days, which cannot be easily reproduced by many groups. If released, this can be useful to the community.
Critical weaknesses
- I am concerned about a lack of qualitative examples of the technique and the dataset. Across the entire paper, only 4 qualitative examples are shown which are output of the technique. Unfortunately this is not enough for me to build sufficient intuition about what the performance of the proposed method or the quality of the dataset is like. Therefore, I am strongly uncertain about the quality of the work and cannot recommend acceptance.
- Evaluations are all in domain on the proposed synthetic dataset. It is not clear if the technique will generalize to open domain videos, which is very critical for modern video generative methods.
- Even on the in-domain dataset, the quantitative performance often very low or questionable. The temporal consistency metrics (CLIP-F, DINO-F) is all very saturated for all tested methods. On the other hand, pickscore is only 18.83 and lags behind Bernini-R-14B(19.05). Moreover, the pickscores of all methods are very low. Why?
- Only CLIP-F, DINO-F ablation scores are provided for the proposed method, and the improvement is extremely marginal. This only suggests slightly improved temporal consistency, but not editing quality. It does not seem to support author's claim that "text-memory expands semantic capacity for low2high appearance changes, and a hint memory supplies layer-wise visual feature corrections of high-cost element before context injection".
- it is not clear what is the motivation for converting low cost movie into high cost movie vs e.g. reference subject motion driven video generation (e.g. WAN-Animate-2) or identity conditioned video generation (such as WildActor Unconstrained Identity-Preserving Video Generation). In fact I think the author should benchmark against these two categories of techniques.
Questions for authors
The biggest difference between low cost captures vs professional productions might be the storyboard, the footage cuts, and the camera poses, which the proposed method cannot handle. I wonder what is the authors thought on that? Why is the proposed method still very useful despite such shortcomings?