- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CAVE evaluates geometric consistency in generated videos by asking whether each new frame makes earlier observations harder to explain. It updates a scene representation sequentially and checks how its geometric predictions for historical views change, identifying when and where inconsistencies arise. The resulting score also serves as preference supervision for improving video generation through DPO post-training.
Critical strengths
- The method is effective at identifying when persistent inconsistencies begin.
- The score provides useful signal for DPO and improves generation.
Critical weaknesses
- Metric reliability and interpretation are insufficient. CAVE measures increases in a learned observer’s historical reconstruction error, so its scores can reflect observer limitations and estimation failures in addition to video inconsistency. Although the clean–conflict analysis demonstrates sensitivity to injected perturbations, it does not characterize how clean-score distributions vary with scene, sequence length, camera motion, etc. Moreover, the alternative-observer experiments report localization accuracy without establishing agreement on video level rankings, the robustness of DPO is therefore unverified. The authors should report clean-video baseline distributions and false-positive rates, and test whether the same candidate-video pairs retain their ordering across observers and controlled evaluation variations.
- The validation mainly uses synthetic deformations of static scenes, leaving uncertainty about dynamic scenes with fast camera movement, newly revealed objects, and legitimate object motion, etc. The dynamic post-training experiment does not directly validate the metric in these conditions. The authors could independently vary camera motion, visibility, and object motion in controlled sequences, testing whether CAVE distinguishes valid changes from injected inconsistencies.
Questions for authors
Please see weaknesses.
3 comments
Thanks for additional experiments, most of my concerns are addressed.
We thank the reviewer for the positive feedback. We have incorporated the additional experiments and analyses into the revised manuscript. We appreciate your time and thoughtful consideration.
Response to Reviewer DLjs
We thank the reviewer for recognizing CAVE's persistent-conflict localization and post-training benefits. We provide paired analyses, human evaluation, cross-observer comparisons, and visual evidence on our anonymous supplementary website.
W1: Clean-Video Baselines and Observer Reliability
CAVE does not require zero reconstruction error on clean videos. Instead, it measures whether a new observation further degrades the geometric explanation of previously observed history.
Clean baselines and excess damage. We analyzed 100 matched CO3D clean–conflict pairs. Clean videos have a nonzero baseline (median 0.271, IQR [0.142, 0.424], 95th percentile 0.798). Within each pair, the scene, camera trajectory, and observation history are identical; only the geometric perturbation differs. At conflict onset, the mean response increases from 0.282 (clean) to 0.407 (conflict), with paired excess 0.125 and a clip-clustered bootstrap 95% CI of [0.091, 0.161].
We further grouped pairs by visually identified viewpoint changes and newly revealed regions in clean frames:
Even with substantial viewpoint changes or new visibility, introduced conflicts produce additional damage relative to matched clean observations. This directly reflects CAVE's before/after verification of the same historical views, rather than reliance on absolute reconstruction errors.
Agreement with human judgments. Ten annotators each blind-rated 100 triplets of naturally generated videos (three candidates under the same generation conditions; 1,000 group-level evaluations). We compared metric rankings with human judgments:
CAVE leads on both measures, supporting its relevance to geometric consistency in naturally generated videos, beyond controlled perturbations.
Cross-observer consistency. Beyond the localization results in Appendix A.1, we instantiated CAVE with CUT3R, VGGT, and TTT3R on the same generated-video triplets and compared their complete three-video rankings:
The 82%–88% agreement supports stable video-level judgments across observer implementations, rather than dependence on CUT3R alone.
W2: Camera Motion, Newly Revealed Geometry, and Object Motion
Normal observation changes. CAVE revisits the same historical views after assimilating each new frame. Geometry that becomes visible for the first time is therefore not automatically considered inconsistent merely because it was absent from earlier observations. Supplement §1 provides frame-by-frame analyses of naturally generated videos:
These examples show CAVE's response to benign revelation versus newly introduced structural damage; the matched CO3D analysis provides complementary controlled evidence.
Genuine object motion. As explicitly stated in the paper, CAVE and the compared detectors target geometry expected to remain stable across observations. On DAVIS 2017, CAVE's response concentrates on moving foregrounds, with lower responses on relatively static backgrounds. This characterizes behavior outside the static-scene setting (Supplement §3).
Post-training on dynamic content. The evaluator's static-geometry scope does not restrict its preference signal to training generators of static videos. On PDI-Bench, CAVE-DPO reduces PDI from 0.651 to 0.591, while Dynamic Degree remains 82.1%. First-frame-conditioned I2V comparisons also cover animal and human motion, illustrating improvements in structural preservation and local deformation (Supplement §2).