- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CineRef, a reference-guided audio-video generation framework built on LTX-2.3 for cinematic scenes with large camera motions and shot transitions. It adds explicit ID supervision with a face-aware content loss, a trajectory-level ID consistency loss, and RoPE-based audio-visual reference binding. The authors also introduce CineSet, a 100-sample benchmark, and report improved face similarity on OpenS2V and CineSet.
Critical strengths
- On OpenS2V, FaceSim-Cur improves from 55.23% (second best) to 74.13%, and the gain over LTX-Ingredients with the same backbone is substantial.
- The data construction pipeline (cross-trace reference retrieval, TTS-augmented reference audio, structured captions) is thorough and practically useful.
- Open-source reference-guided joint audio-video generation is a meaningful direction.
Critical weaknesses
-
The ArcFace ID loss and the mask-weighted flow-matching loss are common practices. Aligning audio markers via temporal RoPE is close to the synchronized RoPE in DreamID-Omni. L_traj is largely redundant with L_id, and since only four frames are sampled per clip with K_traj = 3, the trajectory set nearly coincides with the frames used for L_id.
-
ArcFace is used for the ID loss and also for FaceSim-Arc evaluation, which risks metric gaming. FaceSim-Cur uses a different model but is highly correlated.
3.ID gains may come at the cost of motion and naturalness. On OpenS2V, MotionAmplitude is only 14.77%, NaturalScore (68.32%) is below RefAlign, VINO, and Kaleido, and GmeScore (65.04%) is among the lowest of open-source methods. The TotalScore lead comes almost entirely from FaceSim, which conflicts with the claim that the improvement is not due to copy-and-paste behavior.
Questions for authors
-
Eq. 5 uses a generated frame as the trajectory anchor rather than the reference image. Why is this choice made, and how do you prevent the loss from propagating ID drift if the anchor itself deviates from the reference? Please also clarify the inconsistency with Fig. 2, where the trajectory anchor appears to be the reference image.
-
ArcFace is used both in training and in FaceSim-Arc evaluation. Can you report ID similarity with an independent evaluator or a user study to rule out metric gaming?
-
On OpenS2V, MotionAmplitude, NaturalScore, and GmeScore are relatively low. Does the stronger ID supervision reduce motion dynamics or naturalness?
-
Can you provide supplementary videos (I don't know if it is allowed), especially for multi-shot and large camera motion cases?