acceptodds
Under review as a conference paper at ICLR 2027

CAVE: A Generated World Should Still Explain Its Past

Abstract

Video generation models can produce visually realistic sequences, yet object structures and spatial relationships may still drift over time. Existing geometric metrics compare frame pairs or jointly model the entire sequence under a shared interpretation, making conflicts difficult to localize or allowing anomalous observations to be absorbed into the overall solution. We introduce an observer-centric principle: video geometric consistency should be evaluated in observation order. The historical prefix provides an internal geometric reference, and a new observation should preserve geometric relationships already supported by history. Building on this principle, we develop CAVE, which maintains an evolving world state and adopts an “assimilate-then-look-back” verification strategy. At each timestep, CAVE queries identical historical views before and after the update and measures the resulting increase in historical explanation error. These responses form a geometric damage tensor that supports video scoring, temporal conflict localization, and current-view spatial attribution. On controlled geometric conflicts, CAVE improves temporal and spatial localization over prior metrics, with particularly strong gains under persistent conflicts. We further use the CAVE video score as preference supervision for DPO post-training, improving geometric consistency across multiple independent metrics while maintaining video quality and dynamics. CAVE therefore turns directional historical verification into a practical signal for both geometric diagnosis and post-training.

Primary area:
generative models
Keywords:
video generation, Geometric Consistency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_DLjsOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CAVE evaluates geometric consistency in generated videos by asking whether each new frame makes earlier observations harder to explain. It updates a scene representation sequentially and checks how its geometric predictions for historical views change, identifying when and where inconsistencies arise. The resulting score also serves as preference supervision for improving video generation through DPO post-training.

Critical strengths
  1. The method is effective at identifying when persistent inconsistencies begin.
  2. The score provides useful signal for DPO and improves generation.
Critical weaknesses
  1. Metric reliability and interpretation are insufficient. CAVE measures increases in a learned observer’s historical reconstruction error, so its scores can reflect observer limitations and estimation failures in addition to video inconsistency. Although the clean–conflict analysis demonstrates sensitivity to injected perturbations, it does not characterize how clean-score distributions vary with scene, sequence length, camera motion, etc. Moreover, the alternative-observer experiments report localization accuracy without establishing agreement on video level rankings, the robustness of DPO is therefore unverified. The authors should report clean-video baseline distributions and false-positive rates, and test whether the same candidate-video pairs retain their ordering across observers and controlled evaluation variations.
  2. The validation mainly uses synthetic deformations of static scenes, leaving uncertainty about dynamic scenes with fast camera movement, newly revealed objects, and legitimate object motion, etc. The dynamic post-training experiment does not directly validate the metric in these conditions. The authors could independently vary camera motion, visibility, and object motion in controlled sequences, testing whether CAVE distinguishes valid changes from injected inconsistencies.
Questions for authors

Please see weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer DLjs

We thank the reviewer for recognizing CAVE's persistent-conflict localization and post-training benefits. We provide paired analyses, human evaluation, cross-observer comparisons, and visual evidence on our anonymous supplementary website.

W1: Clean-Video Baselines and Observer Reliability

CAVE does not require zero reconstruction error on clean videos. Instead, it measures whether a new observation further degrades the geometric explanation of previously observed history.

Clean baselines and excess damage. We analyzed 100 matched CO3D clean–conflict pairs. Clean videos have a nonzero baseline (median 0.271, IQR [0.142, 0.424], 95th percentile 0.798). Within each pair, the scene, camera trajectory, and observation history are identical; only the geometric perturbation differs. At conflict onset, the mean response increases from 0.282 (clean) to 0.407 (conflict), with paired excess 0.125 and a clip-clustered bootstrap 95% CI of [0.091, 0.161].

We further grouped pairs by visually identified viewpoint changes and newly revealed regions in clean frames:

ConditionPairsCleanConflictPositive excess
All1000.2820.40781/100
Large viewpoint change390.3200.51636/39
Newly revealed regions400.3180.49937/40

Even with substantial viewpoint changes or new visibility, introduced conflicts produce additional damage relative to matched clean observations. This directly reflects CAVE's before/after verification of the same historical views, rather than reliance on absolute reconstruction errors.

Agreement with human judgments. Ten annotators each blind-rated 100 triplets of naturally generated videos (three candidates under the same generation conditions; 1,000 group-level evaluations). We compared metric rankings with human judgments:

MetricPairwise ranking ↑Full ranking ↑
CAVE86.36%62.91%
3DCS79.46%55.81%
GeCo80.47%54.77%
Epipolar77.67%52.44%

CAVE leads on both measures, supporting its relevance to geometric consistency in naturally generated videos, beyond controlled perturbations.

Cross-observer consistency. Beyond the localization results in Appendix A.1, we instantiated CAVE with CUT3R, VGGT, and TTT3R on the same generated-video triplets and compared their complete three-video rankings:

Observer pairFull-ranking agreement ↑
CUT3R / VGGT82%
CUT3R / TTT3R88%
VGGT / TTT3R84%

The 82%–88% agreement supports stable video-level judgments across observer implementations, rather than dependence on CUT3R alone.

W2: Camera Motion, Newly Revealed Geometry, and Object Motion

Normal observation changes. CAVE revisits the same historical views after assimilating each new frame. Geometry that becomes visible for the first time is therefore not automatically considered inconsistent merely because it was absent from earlier observations. Supplement §1 provides frame-by-frame analyses of naturally generated videos:

  • Camera motion and wall change: newly revealed scene content elicits low response, while a later structural change in the city wall triggers localized damage.
  • Sofa revelation and deformation: the newly exposed right seating surface shows low response; deformation of the foreground surface concentrates the response on the affected region.
  • Disappearance and persistence: response rises when a structure disappears, then diminishes when the inconsistency persists without further deterioration.

These examples show CAVE's response to benign revelation versus newly introduced structural damage; the matched CO3D analysis provides complementary controlled evidence.

Genuine object motion. As explicitly stated in the paper, CAVE and the compared detectors target geometry expected to remain stable across observations. On DAVIS 2017, CAVE's response concentrates on moving foregrounds, with lower responses on relatively static backgrounds. This characterizes behavior outside the static-scene setting (Supplement §3).

Post-training on dynamic content. The evaluator's static-geometry scope does not restrict its preference signal to training generators of static videos. On PDI-Bench, CAVE-DPO reduces PDI from 0.651 to 0.591, while Dynamic Degree remains 82.1%. First-frame-conditioned I2V comparisons also cover animal and human motion, illustrating improvements in structural preservation and local deformation (Supplement §2).

Thanks for additional experiments, most of my concerns are addressed.

We thank the reviewer for the positive feedback. We have incorporated the additional experiments and analyses into the revised manuscript. We appreciate your time and thoughtful consideration.

Reviewer_RbTXOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

CAVE is a metric for the geometric consistency of generated videos, based on an observer-centric principle: a new frame may reveal previously unseen geometry, but should not contradict geometry that earlier frames have already established. A frozen recurrent geometry observer (CUT3R) maintains a world state over the video; at each timestep, it assimilates the new frame and re-queries every historical view from the states before and after the update. After rigid alignment to the fixed historical reference, any increase in historical explanation error is recorded as "geometric damage," yielding a damage tensor that supports video scoring, temporal localization, and spatial attribution. On the GeCo WarpBench protocol, CAVE outperforms MEt3R, VGGT-Reprojection, Epipolar, and GeCo variants — 71% exact-frame accuracy on single-frame conflicts and 59% under persistent conflicts, with the best AP/SRCC for spatial localization. Finally, CAVE scores serve as preference supervision for DPO post-training of CogVideoX-5B (CAVE-DPO), which improves all four independent geometric metrics while maintaining VBench-Q and Dynamic Degree.

Critical strengths
  1. A new evaluation principle with demonstrated diagnostic value. Directional attribution — which frame introduced the conflict, and where — is something existing metrics structurally cannot provide (pairwise symmetry; full-sequence absorption). The 32-point Exact-accuracy gap under persistent conflicts is strong evidence the principle matters in precisely the regime where prior metrics break.
  2. Closes the loop from measurement to optimization. CAVE-DPO is the only preference variant that improves all four independent geometric metrics, showing the signal is not just self-consistent but transfers — the key test for a reward metric.
Critical weaknesses
  1. Validation is confined to synthetic warp conflicts. All quantitative localization results use TPS deformations of real videos. Real generators fail differently, and it is unverified that CAVE's damage ranking correlates with human judgments of geometric inconsistency on actual generated videos. A human-correlation study on generated content would materially strengthen the central claim.
  2. Static-scene assumption and observer dependence bound applicability. Any legitimate non-rigid motion risks being scored as damage, and the entire measurement is mediated by one frozen observer whose failure modes become the metric's failure modes. The paper discusses observers (App. A) but does not quantify observer-induced false positives on dynamic content, and gives no cost analysis for long videos.
Questions for authors
  1. What is CAVE's runtime/GPU cost as a function of video length T, and can historical verification be windowed without losing the principle?
  2. How does CAVE behave on dynamic scenes involving articulated human motion?
  3. On real generated videos (not TPS-injected), does the CAVE score correlate with human rankings of geometric consistency?
  4. The current-view attribution normalizes the carrier to match the frame-level score (Eq. 12) — how sensitive is spatial localization to this normalization choice vs. raw revision magnitudes?
3 comments
AuthorsOpenReview ↗

Response to Reviewer RbTX

We thank the reviewer for recognizing CAVE's sequential verification, conflict-onset localization, and cross-metric DPO gains. Videos are available on our anonymous supplementary website.

W1/W2 / Q3: Human Agreement and Cross-Observer Consistency

Human judgments. Ten annotators each blind-rated 100 triplets of naturally generated videos (three candidates under identical generation conditions; 1,000 group-level evaluations), judging the consistency of object structure and spatial relations over time.

MetricPairwise ranking ↑Full ranking ↑
CAVE86.36%62.91%
3DCS79.46%55.81%
GeCo80.47%54.77%
Epipolar77.67%52.44%

CAVE leads on both measures, supporting human-aligned ranking beyond controlled conflict localization. Supplement §1 provides examples of structural changes, object disappearance, and persistent errors.

Observer dependence. We instantiated CAVE with CUT3R, VGGT, and TTT3R on the same triplets and compared complete three-video rankings:

Observer pairFull-ranking agreement ↑
CUT3R / VGGT82%
CUT3R / TTT3R88%
VGGT / TTT3R84%

Agreement of 82%–88%, alongside Appendix A.1's localization results, supports consistency across observers rather than reliance on CUT3R-specific updates.

W2 / Q2: Scope Regarding Dynamic Scenes

As stated in the paper, CAVE and the compared methods target geometry expected to remain stable across observations. To characterize responses to genuine object motion, we additionally visualize detections on real DAVIS videos. CAVE responds primarily around moving foreground objects, while relatively static backgrounds exhibit lower responses (Supplement §3). This characterizes behavior outside the static-scene setting, rather than demonstrating general dynamic-scene evaluation.

Post-training with dynamic content. On PDI-Bench, CAVE-DPO reduces PDI from 0.651 to 0.591, while Dynamic Degree remains 82.1%. We also provide first-frame-conditioned I2V comparisons covering animal and human motion, showing improvements in structural preservation and local deformation. These results support the utility of CAVE preference supervision for generating videos containing moving objects (Supplement §2).

Q1: Runtime, Sequence Length, and Historical Sampling

Full-history verification performs T(T−1)/2 update–history comparisons for a T-frame video, giving O(T²) verification complexity. For 49 frames, our implementation takes approximately 109 s on one RTX 4090, with peak GPU memory of about 5 GB. It batches world-state queries, uses eight CPU workers for independent alignment and residual computations, and aggregates dense responses on the fly without explicitly storing an O(T²HW) damage tensor.

For longer sequences, historical views can be sampled while retaining the same before/after verification operation. Rather than using only a recent sliding window, we favor retaining early references and sampling additional views uniformly across the history to cover initial geometry and later revisits. Verifying at most k historical views per update reduces comparison complexity to O(Tk). This provides an approximate acceleration while preserving the core historical verification operation.

Q4: Spatial Normalization and Historical Damage

Eq. (12) rescales the current-view carrier ata_t defined in Eq. (11):

Lt(p)=stCAVEat(p)RMS⁡wˉt(at),RMS⁡wˉt(Lt)=stCAVE.L_t(p)=s_t^{\mathrm{CAVE}}\frac{a_t(p)}{\operatorname{RMS}_{\bar w_t}(a_t)},\qquad \operatorname{RMS}_{\bar w_t}(L_t)=s_t^{\mathrm{CAVE}}.

With identical valid support and a positive scaling factor, this preserves within-frame pixel rankings. Appendix A.3 confirms that within-frame AP is essentially unchanged, whereas historical-damage calibration improves clean–conflict paired AP from 25.3% to 33.3%. Thus, normalization helps distinguish normal state revisions from conflict-inducing updates through their historically verified response magnitude, rather than changing within-frame spatial rankings.

To connect displayed locations directly to verified damage, we additionally project historical damage into the current view: damage detected in historical views is attached to corresponding frozen historical reference 3D points, projected into the current view, and combined using confidence-weighted aggregation. Consequently, both displayed location and intensity can be traced to historical verification rather than current-view revision alone. Examples are provided in Supplement §1.

Thank the authors for the additional experiments. The human study on naturally generated videos addresses my main concern, providing evidence that CAVE agrees with human geometric-consistency rankings better than the compared metrics. The matched clean–conflict analysis is also helpful for separating conflict-induced damage from observer noise. The runtime analysis answers my efficiency question. Overall, the response addresses my central concern, and I still lean slightly positive. I encourage the authors to include the human study and the revised attribution results.

We thank the reviewer for the positive feedback and constructive suggestions. We have incorporated the human evaluation and revised spatial attribution into the updated manuscript. We appreciate your careful consideration of our work.

Reviewer_1VWGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAVE, a metric for the geometric consistency of generated videos. CAVE uses a frozen CUT3R as a recurrent world state. After each new frame is assimilated, it measures how much worse the state explains the earlier views and treats this increase as damage. The damage is then used for video-level scoring and for localizing conflicts in time and space. On synthetic TPS conflicts, CAVE localizes conflicts better than prior metrics, especially when the conflict persists. Using CAVE as a DPO preference signal on CogVideoX-5B also improves several geometry metrics. Overall, the idea is reasonable, but the metric is validated only on synthetic perturbations, and some of the post-training gains are small.

Critical strengths
  • Comparing the explanation error before and after each update is a simple and intuitive idea that differs from existing pairwise and full-sequence metrics. The ablation in Table 3 supports this design: Exact accuracy is 14% when only the post-update error is used, compared with 71% for the full definition. The results also remain similar with VGGT and TTT3R as the observer (Table 4).
  • CAVE clearly outperforms the baselines when the conflict persists, with 59% Exact accuracy compared with 27% for the best baseline. CAVE-DPO also improves all four other geometry metrics over the base model on both prompt sets (Tables 2 and 6).
Critical weaknesses
  • I find the validation of the metric itself limited. It is tested only on small TPS warps applied to 100 static, object-centric CO3D clips, without comparison to human judgments or to annotated failures in real generated videos. In the single-frame setting, the gain over Epipolar is small (71% versus 68% accuracy), and its MAE is worse. It is also unclear how CAVE behaves when objects actually move in the scene.
  • Some of the post-training gains are small. For example, 3DCS only decreases from 0.508 to 0.501 in Table 2. Since only one base model is used and no variance across seeds, significance tests, or human evaluation are reported, it is hard to judge how reliable these gains are.
Questions for authors
  • Have you checked whether CAVE agrees with human judgments on real generated videos? How does it behave on videos with genuine object motion?
  • Could you report variance or significance for Tables 2 and 6? In addition, the CAVE scores are around 1.0 in Tables 2 and 6 but on the order of 0.001 in Fig. 5. Are they on the same scale?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 1VWG

We thank the reviewer for recognizing CAVE's historical verification design, ablation results, and advantage in localizing persistent conflicts. We provide additional evidence on naturally generated videos, genuine object motion, and post-training gains. Videos are available on our anonymous supplementary website.

W1 / Q1: Reliability on Naturally Generated Videos and Conflict Localization

Human ranking agreement. Ten annotators independently evaluated the same 100 triplets of naturally generated videos, each containing three candidates under identical generation conditions, yielding 1,000 blinded group-level evaluations.

MetricPairwise ranking ↑Full ranking ↑
CAVE86.36%62.91%
3DCS79.46%55.81%
GeCo80.47%54.77%
Epipolar77.67%52.44%

CAVE achieves the highest agreement on both measures, showing that sequential historical verification yields rankings consistent with human geometric judgments on naturally generated videos, beyond controlled TPS perturbations.

Isolated vs. persistent conflicts. Geometric failures in generated videos often persist: once object structure or spatial relations drift, subsequent frames tend to preserve or accumulate the error rather than immediately recover. Experiment 1B models this important failure pattern. On isolated conflicts, CAVE is competitive with the strongest baseline (71% vs. 68% Exact accuracy). When conflicts persist, CAVE identifies the observation that first introduced the inconsistency, achieving 59% vs. 27% Exact accuracy. This larger gap directly reflects the benefit of sequential historical verification.

Genuine object motion. CAVE and the compared geometric detectors use static-scene cross-view constraints. To characterize their response to real motion, we provide DAVIS 2017 detection examples: CAVE's responses are concentrated around moving objects, while relatively static backgrounds show lower responses (Supplement §3).

W2 / Q2: Reliability of Post-Training Improvements

Statistical analysis. We performed 10,000 paired-bootstrap resamples on 3DCS and VBench-Q evaluation results from Tables 2 and 6. The 95% confidence intervals for improvements from Base to CAVE-DPO are:

EvaluationMetric95% CI for improvement
Table 23DCS ↓[0.00192, 0.01093]
Table 2VBench-Q ↑[−0.008713, 0.434897]
Table 63DCS ↓[−0.001044, 0.015918]
Table 6VBench-Q ↑[−0.114326, 0.562153]

The Table 2 3DCS improvement has a 95% CI entirely above zero; the other metrics also show mean improvements consistent with the main results. More broadly, under the same base model, candidate pool, training configuration, and budget, CAVE-DPO consistently improves the mean scores of independent geometric metrics. Together, the paired analysis and cross-metric improvements support the effectiveness of the CAVE preference signal.

I2V extension. To examine applicability beyond the original T2V setting, we additionally post-trained CogVideoX-5B-I2V and compared four methods on 100 image-conditioned generation cases:

MethodCAVE ↓3DCS ↓GeCo ↓Epipolar ↓SGC ↓
Base1.10950.56594.65154.10710.02156
VideoGPA-DPO1.05570.55854.96444.02820.02487
Epipolar-DPO1.08670.56144.55134.13540.02475
CAVE-DPO1.02710.55434.07813.83390.02017

CAVE-DPO achieves the lowest mean error on all five metrics, extending the cross-metric T2V gains to I2V. The I2V first frame also establishes an explicit scene reference, making preservation of geometry in later frames a direct evaluation target.

On PDI-Bench, CAVE-DPO reduces PDI from 0.651 to 0.591 while maintaining Dynamic Degree at 82.1%, showing that preference supervision targeting stable geometry can benefit generation involving moving objects without suppressing motion. See Supplement §2 for T2V, I2V, and dynamic-object comparisons.

Q3: Scale Difference Between Fig. 5 and Tables 2/6

We thank the reviewer for identifying the axis-label issue. The 10−310^{-3} multiplier in Fig. 5 was left over from an earlier plotting version. Fig. 5 and Tables 2/6 use the same CAVE score definition; the issue concerns only the axis annotation, not the underlying calculations, comparisons, or conclusions. We will correct it in the revision.

Thank you for the detailed response. I've read the rebuttal and other reviewers' comments, found that most of my concerns regarding human evaluation and post-training improvements. Therefore, I'll raise my rating to 3 (weak accept). I recommend the authors to include these rebuttal experiments to the final version.

We sincerely thank the reviewer for reconsidering our work and for the positive feedback. We have incorporated the human evaluation and additional post-training experiments into the revised manuscript (§5.2 and §5.3). We appreciate your constructive suggestions and thoughtful consideration.

Reviewer_cY2iOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CAVE, a sequential framework for evaluating geometric consistency in generated videos. Its central idea is to assess whether assimilating a new observation damages the geometric interpretation supported by previous observations. Using a frozen geometry observer, CAVE queries identical historical views before and after each state update and measures positive increases in error relative to stored historical references. The resulting damage tensor supports video scoring, temporal conflict localization, and spatial attribution. The authors additionally use CAVE scores to construct preference pairs for diffusion DPO. Validation includes controlled geometric perturbations, alternative geometry observers, component ablations, and post-training evaluations using independent geometric and video-quality metrics. Overall, I lean toward weak acceptance: the sequential verification principle is compelling, and the experiments demonstrate useful diagnostic and optimization benefits, although the scope and interpretation of the metric require further qualification.

Critical strengths

A meaningful formulation of geometric inconsistency with convincing evidence for temporal attribution. Evaluating how a new observation affects the explanation of history provides a useful distinction between detecting incompatible observations and identifying when a conflict enters a sequence. This is particularly consequential under persistent perturbations, where CAVE achieves 59% exact localization compared with 27% for the strongest baseline in Table 1. The damage-definition ablation further supports the specific choice of positive historical-error increments over absolute error, direct state revision, or unsigned change. These results substantiate the central contribution beyond a conceptual reframing.

The proposed signal demonstrates utility beyond its own evaluation metric. Under a controlled post-training setup, CAVE-DPO improves all four independent geometric metrics over the base model while preserving the reported Dynamic Degree. Improvements also extend to GeCo-Eval, and the supplementary PDI-Bench results provide encouraging evidence for generation involving dynamic objects. Together with the alternative-observer experiments, these findings strengthen the case that CAVE captures a useful geometric signal rather than merely exploiting one reconstruction model or optimizing its own score.

Critical weaknesses

Validation of the metric remains concentrated on synthetic deformations of static scenes. The direct localization benchmark contains 100 CO3D sequences with injected TPS warps. This provides controlled ground truth, but covers only a subset of naturally occurring failures, such as object disappearance, inconsistent scene revisits, or changing spatial relationships. Moreover, the paired analysis confirms that clean observations can produce nonzero damage because the observer itself revises imperfect geometry. It therefore remains unclear how reliably CAVE separates actual inconsistency from legitimate geometric refinement or motion in unconstrained generated videos. The dynamic post-training results are useful, but do not directly validate the metric’s discrimination in those settings.

Current-view spatial attribution has a weaker connection to historical damage than temporal scoring does. The spatial pattern comes from current-view state revision and is scaled by a scalar historical-damage score. Consequently, historical verification changes the response magnitude but not the within-frame pixel ordering, as Appendix A.3 explicitly confirms. The paired-AP improvement supports better calibration across clean and conflicting observations, but does not establish that the highlighted pixels are specifically responsible for the historical damage. This limits the interpretation of the map as conflict-source attribution, particularly when benign newly revealed geometry and a genuine inconsistency coexist.

Questions for authors

Do you have human-annotated results on naturally generated inconsistencies, or clean-video false-positive measurements under substantial viewpoint change and disocclusion? Such evidence would materially strengthen confidence that historical damage reflects video inconsistency rather than observer uncertainty.

When a frame contains both substantial benign geometric revelation and a localized contradiction, does the current-view carrier localize the contradiction? A controlled example would help distinguish damage attribution from general state revision.

Are the smaller improvements in Table 2, particularly 3DCS and VBench-Q, consistent across sampling seeds or supported by paired confidence intervals? This would clarify which benefits are robust beyond the stronger gains on GeCo and VideoReward.

1 comment
AuthorsOpenReview ↗

Response to Reviewer cY2i

We thank the reviewer for recognizing CAVE's sequential verification, persistent-conflict localization, and cross-metric DPO gains. Additional video evidence is available on our anonymous supplementary website.

W1 / Q1: Reliability on Naturally Generated Videos

Human judgments. Ten annotators independently evaluated the same 100 triplets of naturally generated videos (three candidates under identical generation conditions), yielding 1,000 group-level blind evaluations.

MetricPairwise ranking ↑Full ranking ↑
CAVE86.36%62.91%
3DCS79.46%55.81%
GeCo80.47%54.77%
Epipolar77.67%52.44%

CAVE achieves the highest agreement on both measures, extending evidence beyond controlled TPS perturbations to human judgments of naturally generated geometry.

Matched clean–conflict pairs. We analyzed 100 CO3D pairs with identical scenes, camera trajectories, and histories, differing only in whether the current observation contains a geometric perturbation. Clean videos show a nonzero baseline (median 0.271, IQR [0.142, 0.424]). At conflict onset, mean response rises from 0.282 to 0.407 (paired excess 0.125, 95% CI [0.091, 0.161]), with positive excess in 81/100 pairs. Thus, newly introduced conflicts cause measurable additional damage to historical explanations beyond normal observer estimation errors. Supplement §1 shows natural examples with large viewpoint changes and newly revealed geometry.

Genuine object motion. As explicitly scoped in the paper, CAVE evaluates geometry expected to remain static. On DAVIS 2017, responses concentrate on moving foregrounds while static backgrounds exhibit lower responses. These cases characterize behavior outside that scope (Supplement §3).

W2 / Q2: Linking Spatial Responses to Historical Damage

We agree that the original within-frame spatial distribution comes from current-view revisions. This follows from CAVE's decision to detect damage in historical views: it re-queries the same historical viewpoint before and after updating the world state. This avoids treating newly revealed geometry as a conflict or repeatedly attributing persistent errors to subsequent frames.

To link displayed responses directly to verified damage, we refine the visualization: damage measured in historical views is attached to the corresponding frozen historical reference 3D points, projected into the current view, and aggregated with confidence weights. Both response location and magnitude now derive from historical verification rather than current-view revisions alone. The damage definition and temporal/video scores are unchanged.

To address your questions about benign scene revelation, localized contradictions, and rapid camera motion, we provide visual examples in the Supplement §1.

  • Sofa: gradual revelation of the right seating surface yields low response; deformation of the foreground surface produces localized response.
  • City wall: rapid camera motion and benign scene revelation yield low response, while a subsequent structural change produces localized high response.

These cases illustrate how historical damage is mapped into the current view.

Q3: Reliability of Post-Training Gains

Paired statistics. On 200 paired Table 2 evaluations, 10,000 paired-bootstrap resamples give 3DCS means of 0.50756 (Base) and 0.50111 (CAVE-DPO): improvement 0.00645, 95% CI [0.00192, 0.01093]. VBench-Q rises from 79.242 to 79.456, with improvement CI [−0.008713, 0.434897], likewise a positive mean change. Despite different metric scales and sensitivities, CAVE-DPO improves all four external geometric metrics (3DCS, GeCo, Epipolar, SGC) on both evaluation sets (Tables 2 and 6).

I2V extension. We also post-trained CogVideoX-5B-I2V and evaluated four methods on 100 image-conditioned cases:

MethodCAVE ↓3DCS ↓GeCo ↓Epipolar ↓SGC ↓
Base1.10950.56594.65154.10710.02156
VideoGPA-DPO1.05570.55854.96444.02820.02487
Epipolar-DPO1.08670.56144.55134.13540.02475
CAVE-DPO1.02710.55434.07813.83390.02017

CAVE-DPO yields the lowest mean error on all five metrics, extending its T2V gains to I2V. For dynamic-object generation, PDI-Bench PDI improves from 0.651 to 0.591, with Dynamic Degree unchanged at 82.1%. Supplement §2 shows T2V, I2V, and dynamic-object comparisons.

Comments on the paper

1 comment

We thank the Area Chair and all reviewers for their constructive feedback. We have uploaded a revised manuscript incorporating the additional experiments, methodological refinements, and analyses discussed during the rebuttal.

For clarity, we summarize the main revisions below:

  1. History-grounded spatial attribution (§4.3, Eqs. 10–11). We refined current-view localization by projecting verified historical damage through frozen historical reference 3D points. We also updated the controlled spatial localization results (Table 1) and qualitative comparisons (Figure 3).

  2. Human evaluation on naturally generated videos (§5.2, Table 2). We added a blinded study with 10 annotators evaluating 100 groups of three candidate videos. CAVE achieves the highest agreement with human geometric-consistency judgments (86.36% pairwise and 62.91% complete-ranking agreement). Figure 4 provides qualitative comparisons across geometric metrics.

  3. T2V and I2V post-training (§5.3, Table 3). We extended CAVE-DPO to CogVideoX-5B-I2V under a controlled comparison with other preference objectives. CAVE-DPO achieves the lowest mean error across all five geometric metrics in I2V, complementing the consistent cross-metric improvements observed in T2V. Figure 5 presents additional qualitative comparisons.

  4. Cross-observer robustness (Appendix A.1.3, Table 6). We added video-level ranking comparisons across CUT3R, VGGT, and TTT3R, obtaining 82%–88% complete-ranking agreement.

  5. Clean–conflict analysis (Appendix A.2, Figure 6). We corrected the score-scale presentation and aligned the accompanying numerical description with the figure. The analysis shows that geometric conflicts produce additional historical damage beyond the nonzero clean baseline.

Additional video examples, including natural geometric inconsistencies, post-training comparisons, and behavior under genuine object motion, are available on our anonymous supplementary website.

We appreciate the reviewers’ suggestions, which helped us improve the clarity and supporting evidence of the paper.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.