acceptodds
Under review as a conference paper at ICLR 2027

DiVid: Diagnosing Dimension-Specific Diversity Collapse in Video Generation Models

Abstract

Despite remarkable progress, video generation models often produce highly similar outputs when repeatedly sampled from the same prompt, limiting their usefulness for creative exploration. Existing diversity evaluations primarily rely on global scalar metrics, which obscure where diversity collapses in the spatiotemporal space of videos. We introduce DiVid, a dimension-level diagnostic framework that decomposes video generation diversity into six interpretable dimensions: Semantic, Style, Subject, Scene, Motion, and Camera. Each dimension is measured through a reproducible computer-vision pipeline and analyzed alongside quality and instruction faithfulness to examine potential trade-offs. Systematic evaluation of representative video generation models reveals that diversity is highly dimension-specific: models with strong global diversity scores still collapse on specific factors, particularly Motion and Camera. These rankings persist after filtering unfaithful generations, indicating genuine capability differences rather than off-prompt outputs. Beyond measurement, controlled prompt interventions identify two fundamental bottlenecks: default mode convergence, where models fall back to dominant patterns under open-ended prompts; and realization gaps, where models fail to faithfully realize diverse, explicitly requested alternatives, particularly for temporal factors. The larger faithfulness losses for temporal factors highlight the difficulty of controlling motion and camera variation through text alone. DiVid thus shifts the study of diversity from measuring whether it exists to diagnosing where and why it collapses, and provides actionable directions for dimension-aware training objectives and control signals. The framework will be released to facilitate future research on diverse and controllable video generation.

Primary area:
datasets and benchmarks
Keywords:
video generation, diversity, homogeneity, evaluation methodologies

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_yYb3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a new metric, "Divid", to evaluate the diversity of generated videos. Unlike existing metrics, Divid aims to assess diversity across multiple dimensions. The authors argue and demonstrate that a single score is insufficient to fully capture video diversity. The paper also identifies two fundamental bottlenecks to improving diversity: default mode convergence and realization gap, and suggests a future research direction to address these challenges.

Critical strengths
  • The idea of separating diversity into different aspects for evaluation is interesting. This provides users with a more comprehensive understanding of how the model behaves across different dimensions and helps identify areas for further improvement. In particular, Figure 1 provides a strong starting point and clearly illustrates the motivation behind this work.
Critical weaknesses
  • Although the motivation is clear, the evaluation results for the proposed metrics are relatively weak. Specifically, the Spearman correlations with human evaluations are generally weak to moderate (Table 5), while an ideal correlation would be above 0.8. Similarly, Krippendorff's alpha values are also low, indicating limited agreement among human annotators. These results raise concerns about how reliably the proposed metrics can reflect human judgments of diversity.
  • I understand that this is primarily a benchmark paper, so proposing a new approach to enhance diversity is not necessarily required. However, the paper lacks concrete experiments demonstrating how existing diversity enhancement techniques (e.g., prompt rewriting, as mentioned by the authors) can actually improve diversity and how these improvements are reflected in the proposed metrics. The authors attempt to provide some "actionable" directions for improving models, but these discussions are somewhat incomplete. For instance, the discussion at the end of "Model Convergence" in Section 5.2 does not really suggest a concrete solution. Overall, while the authors are not expected to develop a new diversity enhancement method, I believe the paper would be strengthened if they applied existing techniques and examined how the resulting changes are captured by the proposed metrics. This would help validate whether the identified diversity issues are indeed reflected in practice and whether the proposed metrics can reliably measure improvements in these areas.
Questions for authors
  • It would be helpful if the authors could further explain why the proposed metrics are still considered reliable despite their relatively low correlations with human evaluations, and whether this limitation affects the validity of their findings.
  • I recommend that the authors include additional experiments using existing diversity enhancement techniques to demonstrate whether these approaches can effectively address the identified issues and lead to measurable improvements in the corresponding Divid dimension scores.
7 comments
AuthorsOpenReview ↗

Authors' Response: W1&Q1 (1/4)

Dear Reviewer yYb3, thank you for your careful comments and suggestions. We address your concerns as follows:

W1&Q1: The model-human correlation and human correlation are not ideal; Whether this will affect the paper's findings

Re: Thank you for raising this concern about metric reliability. We would like to clarify that our correlations and annotator agreement are in line with existing evaluation work. We further support the reliability of both the metrics and the human annotations through an additional stratified analysis and the validation already presented in the paper.

1) Our metric–human correlations (ρ) and human–human agreement (α) are comparable to those reported in existing evaluations of open-ended generation.

For subjective, open-ended judgments, ρ>0.8 is not a universal reliability threshold. Under Cohen’s (1988) convention, a correlation of 0.5 represents a large effect. This convention is routinely applied to rank correlations in evaluation research, including CodeJudge (Tong and Zhang, 2024) and PASTA (Yang et al., 2026). Moreover, our correlations are computed at the item level (two models’ videos for one prompt), which is stricter than the model-level win-ratio correlations used by video benchmarks such as VBench. The table below lists ρ and α reported in recent evaluation studies: our ρ (0.531–0.604) and α (0.679–0.703) lie within the reported range.

PaperTaskSpearman ρ(model-human)Human Krippendorff’s αRelevant statement
DiVid (ours)Video-set diversity, 6 dims0.531–0.6040.679–0.703
i) Established open-ended generation studies: most describe comparable results as high correlations
G-Eval, EMNLP 2023Summary quality0.506, 0.582\outperforming all previous methods by a large margin
Prometheus 2, EMNLP 2024Open-ended response evaluation0.491, 0.496\achieving high correlations with humans and proprietary LMs
Measuring LLM Novelty, ICLR 2026Quality of poetry, story continuations, and creative tool use0.50, 0.520.59, 0.64, 0.68correlates highly with human annotations
FLASK, ICLR 2024Fine-grained skill evaluation of open-ended responses0.680\a high correlation between model-based and human-based evaluations
ii) Our correlations are comparable to or higher than those reported for related diversity evaluation tasks
Describing Like Humans, CVPR 2019Diversity of image-caption sets0.602, 0.617\largely consistent with human evaluation
Rethinking and Refining the Distinct Metric, ACL 2022Diversity of dialogue responses0.62, 0.65\more consistent with human judgments
Hypernymy Understanding Evaluation, arXiv 2023Subcategory diversity in generated images0.40, 0.44, 0.520.58, 0.63moderate yet statistically significant correlation
Verbalized Sampling, arXiv 2025Diversity of story plots\0.71moderate to high reliability
iii) Spearman correlations of 0.8 or higher are often achieved in GT-based evaluation.
RoSE, ACL 2023ROUGE-1 recall0.857-0.9760.7571a relatively high correlation with the ACU scores, especially at the system level
AuthorsOpenReview ↗

Authors' Response: W1&Q1 (2/4)

2) The additional stratified analysis shows that DiVid correlates strongly with human judgments when annotators agree; lower correlations mainly come from items that even humans find hard to distinguish.

We analyze the 90 items rated by two annotators separately for each of the six dimensions. We encode the five preference levels as −2, −1, 0, 1, and 2. Items whose two ratings differ by at most one level form the high-agreement group; those whose ratings differ by two or more levels form the low-agreement group. For diversity differences that humans can distinguish consistently, DiVid shows high correlations with human judgments; lower correlations are mainly observed in the group with greater disagreement between annotators. For example, in the case for Wan2.2-5B, DiVid assigns HappyHorse a slightly higher subject diversity score, while one human rater considers Wan2.2-5B more diverse and the other favors HappyHorse. In fact, the two sets exhibit comparable subject diversity.

DimensionFull-sample DiVid–human ρHigh-agreement DiVid–human ρHigh-agreement human–human ρLow-agreement DiVid–human ρLow-agreement human–human ρ
Semantic0.6040.6940.9490.340−0.208
Style0.5630.6970.8820.325−0.137
Subject0.5490.7640.9140.217−0.190
Scene0.5410.7420.9100.3300.077
Motion0.5310.7530.8920.205−0.291
Camera0.5560.7200.8740.276−0.165

3) Quality control and complementary validations support metric validity.

  • Annotation quality control (Appendix B.4). The original study incorporates standardized training, blinding and randomization, separate displays for different dimensions, and video-viewing requirements. All annotators correctly answered every gold item, providing direct support for annotation quality.
  • Complementary experiments further support metric validity. Table 4 shows that Motion responds as expected to differences in direction, speed, and action. Table 8 shows that, when variation in the other factors is small, Motion and Camera identify video pairs with pronounced differences in the target factor for 92.0% and 93.3% of prompts, respectively, supporting their ability to provide distinct diagnostic information. Moreover, the leading model in each factor remains unchanged under stricter validity filtering (Appendix B.3), and the key cross-dimensional ranking reversals are supported by paired statistical tests (Appendix C.1).

Together, these findings support DiVid’s use for model-level diversity comparisons and factor-level diagnosis, as well as our central finding that models exhibit different strengths across diversity dimensions.

AuthorsOpenReview ↗

Authors' Response: W2&Q2 (3/4)

W2&Q2: Need to display the results of diversity enhancement techniques, showing how DiVid captures their improvements across dimensions; need to show whether DiVid can capture the intended improvements in diversity dimensions; better show more concrete directions for improving current methods.

Re: Thank you for the suggestion. To address your concerns, we: 1) add experiments showing that existing techniques produce measurable diversity gains, and that DiVid captures both these improvements and their dimension-specific effects; 2) provide controlled, single-factor interventions to test whether the corresponding DiVid dimension responds as intended; 3) give more concrete recommendations.

1) Dimension-level diagnosis of existing methods.

We evaluate sampling-based methods (Bell CFG, FreqCFG, CADS) and two prompt-rewriting baselines (DPP, DIVA) on Wan2.2-5B across all 206 prompts, generating 10 videos per prompt for each method. We report six-dimensional diversity alongside quality and faithfulness, with IF evaluated against the original prompt.

  • Sampling-based interventions: Bell CFG (TV-CFG) uses a guidance schedule that first increases and then decreases; FreqCFG applies separate guidance strengths to low-frequency structure and high-frequency details; CADS perturbs the text conditioning early in sampling and gradually restores it.
  • Prompt rewriting: DPP, an inference-time variant of DPP-GRPO, that selects candidates using the original method’s diversity and semantic-relevance scoring, replacing GRPO training because the code and the trained rewriting model are not open-source; DIVA, our re-implementation of its Director → Architect → Critic → Producer workflow. Both use DeepSeek-v4.1-flash-1017.
MethodSemanticStyleSubjectSceneMotionCameraTCETIEIFAQIQTFMS
CFG=50.1690.2340.4470.4260.3260.0975.2913.360.8480.5430.6720.9800.989
Bell CFG0.2030.2460.4840.4930.3730.1196.4114.320.8420.5360.6730.9750.988
FreqCFG0.1950.2480.4770.4770.3610.1046.0813.910.8430.5360.6710.9790.989
CADS0.2430.3300.5600.5860.3570.0547.0615.750.7600.5150.7000.9850.990
DPP0.2400.2790.4920.5460.3010.0346.7414.430.8930.5990.7020.9880.993
DIVA0.2160.2300.4340.4660.3060.0216.1913.940.8970.5830.7150.9880.994

Bold indicates superior performance over the CFG=5 baseline.

The results directly address whether existing techniques can alleviate the identified issues:

  • Bell CFG and FreqCFG improve all six DiVid dimensions. For example, Bell CFG increases Motion from 0.326 to 0.373 and Camera from 0.097 to 0.119, demonstrating measurable improvements in temporal diversity.
  • Prompt rewriting improves diversity while preserving quality and faithfulness. DPP increases Semantic from 0.169 to 0.240 and Scene from 0.426 to 0.546, while increasing IF from 0.848 to 0.893 and improving all four quality metrics. DIVA also improves Semantic, Scene, IF, and all four quality metrics.
  • DiVid identifies the scope and trade-offs of these improvements. Although DPP and DIVA improve overall diversity and quality, their Motion and Camera diversity decreases. CADS improves five dimensions but reduces Camera, IF, and AQ. In particular, CADS, DPP, and DIVA all increase TCE/TIE while reducing Camera diversity—a distinction that the overall scores do not reveal.
Reviewer_AVAiOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DiVid proposes a dimension-level diagnostic framework for evaluating diversity collapse in video generation models. It decomposes diversity into six dimensions: Semantic, Style, Subject, Scene, Motion, and Camera, using dimension-specific computer vision pipelines. Experiments across representative video generation models reveal substantial differences in diversity across dimensions, while controlled prompt interventions identify default-mode convergence and realization gaps. Although the framework provides a systematic approach to diversity diagnosis, the novelty of its metrics and the reliability of its dimension-specific measurements require further examination.

Critical strengths
  1. The six-dimensional decomposition offers a more interpretable analysis of video diversity than global scalar metrics and reveals differences that aggregate scores may overlook.
  2. The paper evaluates multiple video generation models and includes human assessments, controlled prompt interventions, and statistical analyses, providing useful evidence for the observed diversity patterns.
Critical weaknesses
  1. The framework primarily combines existing feature representations and computer vision pipelines. The paper does not sufficiently establish the methodological advances over the closest existing fine-grained diversity evaluation approaches.
  2. The Motion and Camera metrics depend on tracking and geometric estimation, which may introduce measurement errors or confound different sources of variation. The current validation does not fully establish that the resulting scores consistently isolate the intended diversity dimensions across challenging scenarios.
Questions for authors
  1. How does DiVid differ technically from the closest existing fine-grained video diversity evaluation methods, and what specific diagnostic capability does it provide that those methods cannot?
  2. How robust are the Motion and Camera diversity rankings to tracking errors, camera-motion estimation errors, and alternative feature representations? Could these errors materially change the reported model rankings?
6 comments
AuthorsOpenReview ↗

Authors' Response: W1&Q1 (1/3)

Dear Reviewer AVAi, thank you for your meaningful comments and reviews. We address your concerns as follows:

W1&Q1: The representations applied in each dimension are existing feature representations; need to further display the advantages over existing fine-grained diversity evaluation methods.

Re: Thank you for asking us to clarify DiVid’s technical contribution and diagnostic advantages. We add more diversity metrics and compare them with DiVid, highlighting our additional diagnostic capability.

1) Contribution Clarification.

We would like to claim that our main contribution is not new encoding features (the metrics compared below also build on VGG/CLIP/DINO/ViPE), but: i) the six-dimensional decomposition of video diversity, and the correlated evaluation protocol design**; ii)** Our decomposition reveals model ranking reversals across diversity dimensions and Motion/Camera differences obscured by global metrics such as TCE and TIE.

2) How existing fine-grained metrics work, and what DiVid adds. We compare with VBench-2.0 Diversity and the diversity metrics used in DFD. We summarize their settings and DiVid’s additions below.

  • VBench-2.0: It evaluates VGG-19 content-feature distances and Gram-matrix style distances, reported as content, style, and aggregate scores.
  • DFD: It extracts CLIP ViT-L and DINOv2 ViT-L features. For camera metrics, they use the default pipeline of official ViPE to estimate poses and extract trajectories, from which the paper's mean pairwise endpoint distance and time-averaged pairwise trajectory distance are computed.
  • How DiVid differs: i) Both VBench and DFD measure full-frame appearance differences, while DiVid extracts subject and background features separately, identifying whether content variation comes from subject appearance or the scene; ii) DFD’s ViPE camera metrics measure differences in camera-translation endpoints and trajectories. DiVid separately measures camera translation and zoom features, as well as the subject’s own actions and trajectories.
AuthorsOpenReview ↗

Authors' Response: W1&Q1 (2/3)

3) DiVid's additional diagnostic capability.

Our full evaluation across all prompts shows strong agreement between DiVid and DFD in overall model rankings (DFD-CLIP vs. Semantic: Spearman ρ = 0.96). VBench Content/Style measure distances between VGG activations / Gram statistics, whose different definitions can yield different rankings.

Existing Methods:

ModelsVBench ContentVBench StyleVBench DiversityTIETCEDFD CLIPDFD DINODFD ViPE EndpointDFD ViPE Trajectory
Wan2.2-5B3.09916.4540.54713.3615.2860.1500.4478.5074.482
Wan2.2-14B3.85423.5900.71713.2914.5350.1350.41117.67034.469
CogVideo2.23310.7440.36212.6634.1180.1270.3964.5032.641
Hunyuan3.01912.4660.43713.0685.0550.1290.3925.0312.656
Wan2.73.89619.8510.66312.7374.9420.1440.4058.8884.835
HappyHorse3.53112.5620.45411.4482.7710.0830.22215.0978.976
Seedance3.35710.4810.39113.0915.1640.1470.38616.5369.709

Ours:

ModelsSemanticStyleSubjectSceneMotionCamera
Wan2.2-5B0.1690.2340.4470.4260.3260.097
Wan2.2-14B0.1440.2240.4120.3940.3440.148
CogVideo0.1420.2140.4140.3770.3230.045
Hunyuan0.1420.1970.3920.3780.4330.490
Wan2.70.1580.2010.3970.3900.4130.303
HappyHorse0.0930.1230.2500.2230.3260.193
Seedance0.1600.1620.3800.3640.4250.463

DiVid's additional diagnostic capabilities are demonstrated in three ways:

  • Distinguishing subject motion when appearance scores are similar. Hunyuan and CogVideo differ by less than 2% in their mean DFD-CLIP, DFD-DINO, and DFD-Traj, while their DiVid Motion scores are 0.433/0.323. The motion example further illustrates this capability: for "a black dog wearing a Halloween costume," the left pair containing a head turn and the right pair remaining nearly frontal receive DFD DINO scores of 0.219/0.216, respectively, while DiVid Motion scores them at 0.578/0.199, clearly reflecting the visible motion difference.
  • Identifying scene variation when full-frame scores are similar. The Hunyuan video pairs for the same prompt have similar CLIP/DINO scores, but one contrasts woodland with a yard, while the other contrasts two indoor backgrounds. DiVid Scene assigns scores of 0.718/0.472, identifying scene variation separately from overall appearance differences.
  • Revealing different model strengths when aggregate scores are tied. For "an apartment building with balcony" in the example, Wan2.2-5B, Wan2.2-14B, and Wan2.7 all receive a VBench Diversity score of 1.000. Among these three models, DiVid identifies Wan2.7 as highest in Subject diversity and Wan2.2-5B as highest in Scene diversity, providing a clear basis for selecting models for different generation needs.

These results demonstrate DiVid’s additional diagnostic capability: identifying which factor varies and revealing dimension-specific strengths obscured by overall scores.

AuthorsOpenReview ↗

Authors' Response: W2&Q2 (3/3)

W2&Q2: The Motion and Camera metrics rely on tracking and geometric estimation, which may introduce measurement errors or confound different sources of variation, especially in challenging scenarios. Thus, need to display the robustness of Motion/Camera diversity rankings to tracking errors, camera-motion estimation errors, and alternative feature representations.

Re: We thank the reviewer for raising concerns about measurement reliability and factor separation. We conducted systematic experiments addressing tracking errors, camera-motion estimation errors, and alternative feature representations. The results show that the overall Motion and Camera rankings are robust, and the paper’s key model comparisons remain consistent.

1) We agree that challenging scenarios (strong 3D parallax and dynamic backgrounds) may cause 2D geometric estimation to confound different sources of motion; however, they are rare and will not affect model-level conclusions. Screening the text of all original prompts identified:

  • 3D depth parallax: No prompt explicitly requires both camera translation and rigid backgrounds at different depths. 7 candidate prompts (3.4%) involve spatial depth or camera movement, such as "drone footage of Amsterdam metro station."
  • Dynamic backgrounds: 3 prompts (1.5%) explicitly describe environmental motion or change outside the primary subject, such as "A frog resting under a leaf during a rainstorm."

Each category accounts for fewer than 3.4% of the prompts. The impact is inherently limited. To test this, we replace the 2D estimator with a 3D one (MegaSaM), where the model rankings are robust (see below).

2) Motion and Camera rankings are robust to tracking errors, camera-estimation errors, and alternative representations.

Table 4 already demonstrates Motion’s sensitivity to changes in direction, speed, and action type: controlled changes in motion produce the expected differences in diversity scores, supporting the metric’s sensitivity to the intended dimension.

We further evaluated three categories of perturbations and alternative representations using the existing videos from all 206 prompts and seven models. Random tracking and camera-estimation perturbations were repeated five times; each remaining setting was evaluated once.

  • Tracking errors: Add Gaussian jitter with a standard deviation of 1 px to subject points, background points, or both; alternatively, randomly drop 20% of point observations.
  • Camera-motion estimation errors: Add a translation bias of 1 or 2 px to the estimated camera transformation between adjacent frames. The modified transformation is used for both the Camera descriptor and camera-motion compensation in Motion.
  • Alternative feature representations: Change the trajectory/deformation fusion weights, retain early/middle/late temporal bins, or recompute the metrics using 3D camera poses and depth estimated by MegaSaM.

As shown in the results below, we find that:

The key model comparisons remain stable. Across all ten settings, the overall-ranking correlation is at least 0.821 for Motion and 0.964 for Camera. Camera preserves the complete original ranking in nine settings. Both dimensions retain the same top-three models across all settings.

The direction of the paper’s key model comparisons remains unchanged. For example, Hunyuan’s Camera MPD is 10.8 times that of CogVideo in the original results. This ratio remains 11.28 with 1 px jitter on background points and 6.44 with a 2 px translation bias in camera transformations. Thus, even with tracking or camera-estimation errors, the metric clearly distinguishes the camera diversity of these two models. This preserves a key factor-level difference reported in the paper and supports the diagnostic value of factor-specific evaluation. The key diagnostic findings remain stable under the tested errors and representation changes.

CategorySettingMotion overall-ranking ρCamera overall-ranking ρ
Tracking errors1 px jitter on subject points only0.8571
Tracking errors1 px jitter on background points only0.8211
Tracking errors1 px jitter on both subject and background points0.8211
Tracking errorsRandomly drop 20% of point observations0.8931
Camera-motion estimation errors1 px translation bias in camera transformations0.9641
Camera-motion estimation errors2 px translation bias in camera transformations0.9291
Alternative representationsTrajectory/deformation weights: 1:1 → 1:211
Alternative representationsTrajectory/deformation weights: 1:1 → 2:10.8571
Alternative representationsReplace whole-video statistics with early/middle/late temporal bins0.8931
Alternative representationsMegaSaM0.8570.964
Reviewer_VvqgOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DiVid, a six-dimensional diagnostic framework for evaluating and localizing mode/diversity collapse in text-to-video (T2V) generation models. Instead of collapsing video diversity into a single global scalar (such as TCE or TIE), DiVid decomposes set-level variation (N=10 videos sampled per prompt across random seeds) into two holistic dimensions—Semantic (CLIP ViT-L/14 with Scendi Schur-complement prompt projection) and Style (InceptionV3 pool3)—and four disentangled factor-level dimensions: Subject (DINOv2 features on GroundingDINO + SAM 2 foreground crops), Scene (DINOv2 features on foreground-blurred backgrounds), Camera (16-D RANSAC partial-affine/homography statistics on background CoTracker trajectories), and Motion (camera-compensated subject CoTracker trajectories combined with canonicalized non-rigid RAFT optical flow residuals). Diversity is quantified via Mean Pairwise Distance (MPD) and Vendi Score, and jointly analyzed with VBench perceptual/temporal quality metrics (AQ, IQ, TF, MS) and factor-specific Instruction Faithfulness (IF, scored by Qwen3.6-27B).

The authors validate DiVid against blind human preference judgments across 150 prompt–model pairs (Spearman ρ = 0.531–0.604, p < 0.001; Krippendorff's α > 0.67) and a 300-video human mask audit (93.3% pass rate). Benchmarking 7 open- and closed-source T2V models (Wan2.2-5B, Wan2.2-14B, CogVideoX1.5-5B, HunyuanVideo-1.5, Wan2.7, HappyHorse-1.5, Seedance-1.5-pro) over 206 prompts reveals a striking rank reversal (q < 10-8): models leading in spatial/content diversity (e.g., Wan2.2-5B) collapse in temporal diversity (Motion and Camera), whereas temporal leaders (HunyuanVideo-1.5, Seedance-1.5-pro) lag in spatial content diversity, and the highest-faithfulness/quality model (HappyHorse-1.5) exhibits severe spatial mode collapse. Finally, a controlled 4-condition prompt intervention study (Specified, Omitted, Hinted, Enumerated) across 20 semantic groups isolates two distinct bottlenecks: default-mode convergence under open prompts and a realization gap where LLM prompt enumeration unlocks Subject/Scene diversity but degrades Motion/Camera faithfulness (-0.690 and -0.393).

Critical strengths

Rigorous Spatiotemporal Disentanglement and Statistical Validation Revealing Actionable Foundation-Model Bottlenecks: The paper solves a genuine diagnostic blind spot in multimodal foundation model evaluation. By decoupling foreground subject appearance/motion from background scene/camera movement and gating diversity on factor-level faithfulness (τ ≥ 0.8 equal-count MPD, Figure 4a), the authors convincingly prove that global diversity metrics (TCE/TIE) mask a 10.8× spread in Camera diversity and a full rank reversal between spatial and temporal exploration (q < 10-8, Table 11). Insightful Controlled Prompt Intervention Study Aligned with the Multimodal LLM / Post-Training Roadmap: Section 5 cleanly separates prior collapse (default-mode convergence caused by post-training alignment, where models default "an animal" to a golden retriever or static camera) from execution bottlenecks (where LLM prompt rewriting via Enumerated candidates successfully diversifies discrete spatial factors like Subject/Scene, but fails on continuous temporal factors like Motion/Camera due to sharp drops in execution faithfulness, Table 23). This provides strong empirical evidence that LLM prompt expansion alone cannot resolve temporal homogenization in video world models without trajectory- or pose-level conditioning.

Critical weaknesses

Order-Insensitive Bag-of-Statistics Temporal Descriptors and 2D Planar Camera Assumptions Under Parallax / Dynamic Backgrounds: In Appendix B.2.1, both Camera (16-D) and Motion (24-D trajectory + 20-D deformation) compress an entire video's temporal trajectory into global temporal summary statistics (mean, standard deviation, 90th percentile, and an 8-bin direction histogram) before applying an RBF kernel with fixed bandwidths (σ=2.0, 3.0). Because temporal pooling discards temporal ordering and phase, two opposite temporal sequences (e.g., panning left-then-right vs. right-then-left, or sitting-down-then-jumping vs. jumping-then-sitting-down) produce nearly identical summary statistics and direction histograms, underestimating genuine temporal sequence diversity. Furthermore, estimating Camera motion via a 2D partial-affine / homography fit on background CoTracker points assumes a rigid, near-planar background. In scenes with strong 3D depth parallax or dynamic non-rigid backgrounds (e.g., ocean waves in "A person is swimming in ocean", Figure 1, or moving crowds/foliage), background point motion violates the 2D affine camera model, causing dynamic background motion to leak into both the Camera descriptor and the camera-compensated subject Motion residual rp,t = xp,t+1 - Ht(xp,t). Purely Diagnostic Scope Without Quantitative Validation of Any Mitigation or Factor-Aware Sampling Baseline: While Appendix A.2 mentions testing three inference-time diversity interventions (bell-shaped CFG schedule, frequency-decoupled guidance of Sadat et al. 2026, and condition-annealed sampling CADS of Sadat et al. 2024), no quantitative numbers are reported anywhere in the paper or appendix, nor are existing diversity-promoting video methods (such as DPP-GRPO, Kazimi et al. 2026a, or DIVA, Kazimi et al. 2026b) evaluated on DiVid to see which specific dimensions they actually improve. Including quantitative results for these inference/RL diversity methods—or demonstrating a simple hybrid baseline combining LLM prompt enumeration for Subject/Scene with trajectory/camera sampling for Motion/Camera—would significantly strengthen the paper's practical impact.

Questions for authors

Clarification / Core Assessment — Quantitative results for the inference-time diversity methods in Appendix A.2 and existing diverse video baselines: In Appendix A.2 (lines 857–863), you note that you ran pilot experiments with bell-shaped CFG, frequency-decoupled guidance (Sadat et al., 2026), and CADS (Sadat et al., 2024). Can you provide the quantitative 6-dimension DiVid table for these methods (as well as DPP-GRPO or DIVA if available)? Seeing exactly which dimensions these methods improve or degrade would directly illustrate DiVid's value for evaluating diversity-enhancing algorithms. Clarification / Core Assessment — Sensitivity of Motion/Camera descriptors to temporal order inversion and dynamic backgrounds: Since the 16-D Camera and 44-D Motion descriptors rely on time-aggregated statistics (mean, std, 90th percentile, 8-bin direction histogram), how does DiVid distinguish temporally reversed actions (e.g., push-in followed by pull-out vs. pull-out followed by push-in, or standing-up vs. sitting-down)? Would preserving coarse temporal pyramid bins (e.g., early/middle/late temporal chunks) or 3D camera pose estimation (e.g., MegaSaM, Li et al. 2025) change the Motion/Camera rankings on dynamic-background prompts?

6 comments
AuthorsOpenReview ↗

Authors' Response: W1&Q2 (1/4)

Dear Reviewer Vvqg, thank you for your insightful comments and questions. We address your concerns as follows:

W1 & Q2: Current Motion/Camera representations may not handle two challenging conditions: 1) They may not distinguish videos that contain similar movements in a different temporal order (e.g., panning left-then-right vs. panning right-then-left); 2) In scenes with strong parallax or dynamic backgrounds (e.g., ocean waves), the 2D affine camera model may attribute background motion to the camera, further affecting the Motion representation through camera compensation, and consequently, affecting the Camera/Motion Diversity.

Re: We agree that these challenging conditions do exist, and your suggested temporal pyramid bins and MegaSaM can improve the measurement for these challenging cases. However, our findings focus on general-domain prompts. Our evaluation uses 206 prompts from two general-domain benchmarks, covering diverse motions and activities, attribute changes, spatial relationships, and multi-subject interactions. Each challenging category accounts for less than 4% of these prompts. We replaced our original features with the suggested ones and compared the resulting rankings. On the challenging set, the new rankings correlated strongly with the original rankings (0.821 for Motion and 0.929 for Camera). We also show results on the full set, demonstrating that our main conclusions remained unchanged.

1) For most conditions, our current measurement can capture motion and camera distinctions. In the specific challenging conditions you raised, the suggested methods further improve the Motion/Camera diversity measurement.

Table 4 verifies that Motion diversity scores respond as expected to controlled changes in movement amplitude, direction, speed, and deformation. To further test the challenging conditions, we constructed paired videos in Blender, keeping other factors consistent while varying only the temporal order, background motion, or camera motion.

  • Single reversed actions. We compared six standing-up videos, six sitting-down videos, and a mixed group containing three of each in Blender. Motion increased from 0.012 for the single-action groups on average to 0.247 for the mixed group. Camera scores remained near zero. This controlled experiment shows that the original method distinguishes single-action direction reversals, such as standing up versus sitting down. Blender video cases: stand up, sit down.
  • Reordered composite motions. We tested four types: camera movement left-then-right versus right-then-left; push-in-then-pull-out versus pull-out-then-push-in; subject motion crouch-then-stand versus stand-then-crouch; and left-then-right versus right-then-left. Mixed-order groups should exhibit more temporal diversity than single-order groups. With the original method, the measured diversity increments were near zero. Preserving early/middle/late temporal bins increased these increments to 0.060–0.166 across the four cases. Blender video cases: camera left-then-right, camera right-then-left.
  • 3D depth parallax. We tested: A stationary subject stands between a nearby railing and distant buildings while the camera translates laterally. With MegaSaM, the subject’s Motion diversity score decreased from 0.245 to 0.170. Blender video cases: parallax case 1, parallax case 2.
  • Dynamic backgrounds. We tested: A person stands on a dock with both the subject and camera fixed, while only the background water moves. The Camera diversity score is near zero under both methods (6e-5 and 1e-7), and MegaSaM reduces the Motion diversity score from 0.277 to 0.078. Blender video cases: dynamic background 1, dynamic background 2.

In all, the original measurement already distinguishes single reversed actions and keeps Camera diversity robust to dynamic backgrounds. The suggested methods improve these cases. As shown below, these cases are uncommon in general conditions and do not change the main conclusions.

AuthorsOpenReview ↗

Authors' Response: W1&Q2 (2/4)

2) The challenging conditions are infrequent (each under 4% in general prompts), making the impact inherently limited.

We carefully screened the general prompts using explicit textual criteria:

  • Temporal ordering and phase: 8 prompts (3.9%) involve back-and-forth, cyclic, or rhythmic motion, e.g., "girls jumping up and down in the bedroom" and "One tree stands tall, swaying gently in the breeze."
  • 3D depth parallax: No prompt explicitly requires camera translation over rigid backgrounds at different depths. 7 prompts (3.4%) involve spatial depth or camera movement that may induce parallax, e.g., "drone footage of Amsterdam metro station."
  • Dynamic backgrounds: 3 prompts (1.5%) explicitly describe environmental motion outside the primary subject, e.g., "A frog resting under a leaf during a rainstorm."

Each category accounts for fewer than 4% of the prompts. The impact is inherently limited.

3) On the challenging set, the new rankings correlate strongly with the original rankings; On the full set, our main conclusions remained unchanged: Leading models and key factor differences are preserved when using the suggested method.

We separately evaluated Spearman’s rank correlation between rankings under the original and new methods on all challenging prompts. The correlations were 0.821 for Motion and 0.929 for Camera, with most model orderings preserved. Since these conditions may also arise when the prompt does not specify them, we further verify their impact on the full benchmark. We re-evaluated the existing videos across all prompts using temporal pyramid bins and MegaSaM. We compared the methods on matched valid videos to assess changes in model rankings. The table reports the diversity score/rank.

ModelMotion: OriginalMotion: Temporal pyramidMotion: MegaSaMCamera: OriginalCamera: Temporal pyramidCamera: MegaSaM
Wan2.2-5B0.317 / 60.409 / 70.331 / 70.094 / 60.104 / 60.218 / 6
Wan2.2-14B0.335 / 40.423 / 50.340 / 40.145 / 50.156 / 50.229 / 5
CogVideo0.308 / 70.412 / 60.336 / 50.045 / 70.050 / 70.185 / 7
Hunyuan0.426 / 10.515 / 20.434 / 20.483 / 10.497 / 10.543 / 2
Wan2.70.404 / 30.499 / 30.408 / 30.297 / 30.306 / 30.387 / 3
HappyHorse0.319 / 50.426 / 40.332 / 60.190 / 40.190 / 40.281 / 4
Seedance0.424 / 20.554 / 10.439 / 10.463 / 20.483 / 20.571 / 1
  • The leading models remain consistent. All three methods identify Hunyuan, Seedance, and Wan2.7 as the top three models for both Motion and Camera diversity, with no model moving between this leading group and the remaining four models. Across the full benchmark, the temporal pyramid yields Spearman’s ρ of 0.893 for Motion and 1.000 for Camera relative to the original rankings, and the corresponding correlations for MegaSaM are 0.857 and 0.964. Rank swaps occur only between models with nearly identical scores. For example, Hunyuan and Seedance obtain Motion scores of 0.426 / 1 and 0.424 / 2 under the original method, versus 0.434 / 2 and 0.439 / 1 under MegaSaM. Their order changes, while their scores remain close.
  • The main conclusions remain unchanged: Models with high content diversity do not necessarily exhibit high Motion or Camera diversity. Global metrics fail to fully reveal these distinctions. As discussed in Section 4.2 Global Metrics Conflate Opposite Diversity Collapse, Wan2.2-5B ranks first on both global metrics (TIE and TCE) and across four content dimensions. Yet Hunyuan’s MPD is 1.33× that of Wan2.2-5B for Motion and 5.05× for Camera. Both alternative methods preserve this pattern. With temporal pyramid bins, Hunyuan’s Motion and Camera MPD are respectively 1.26× and 4.79× those of Wan2.2-5B. With MegaSaM, the corresponding ratios remain 1.31× and 2.49×.

4) Conclusion

In all, the challenging conditions you raised are real but infrequent. The suggested methods improve the measurement in these cases; however, they will not affect our central findings: the leading Motion/Camera models are preserved, content diversity does not imply equally diverse motion or camera behavior, and a single global metric can obscure such differences. Additionally, because DiVid is modular, both suggested alternatives work as drop-in replacements for the Motion/Camera representations, without changing our factor definitions, aggregation, or evaluation protocol. We keep the efficient 2D pipeline as the default, and will add both alternatives to the revision as optional modules for targeted evaluation of these conditions. We thank you for helping us strengthen DiVid.

AuthorsOpenReview ↗

Authors' Response: W2&Q1 (3/4)

W2 & Q1: Need to report the full quantitative results of inference-time diversity methods, such as those tested in Appendix A.2 (bell-shaped CFG, FreqCFG, and CADS), as well as existing diversity methods (e.g., DPP-GRPO and DIVA), showing exactly which dimensions each method improves or degrades.

Re: Thank you for your suggestion. We have added the full quantitative results for these diversity-enhancement methods, showing which dimensions they improve or reduce, alongside faithfulness and quality change. As shown in the table below, higher overall diversity can coexist with lower diversity in specific dimensions, which DiVid can reveal but global metrics (TIE/TCE) cannot. For instance, both prompt-rewriting baselines increase TIE/TCE scores but reduce Motion and Camera diversity relative to CFG=5. DiVid can capture such limitations. More detailed discussion is shown below:

MethodSemanticStyleSubjectSceneMotionCameraTCETIEIFAQIQTFMS
CFG=50.1690.2340.4470.4260.3260.0975.2913.360.8480.5430.6720.9800.989
Bell CFG0.2030.2460.4840.4930.3730.1196.4114.320.8420.5360.6730.9750.988
FreqCFG0.1950.2480.4770.4770.3610.1046.0813.910.8430.5360.6710.9790.989
CADS0.2430.3300.5600.5860.3570.0547.0615.750.7600.5150.7000.9850.990
DPP0.2400.2790.4920.5460.3010.0346.7414.430.8930.5990.7020.9880.993
DIVA0.2160.2300.4340.4660.3060.0216.1913.940.8970.5830.7150.9880.994

1) Setup. We use Wan2.2-5B throughout and generate 10 videos per prompt for each method across all 206 prompts.

  • Sampling-based interventions: Bell CFG (TV-CFG) uses a guidance schedule that first increases and then decreases; FreqCFG applies separate guidance strengths to low-frequency structure and high-frequency details; CADS perturbs the text conditioning early in sampling and gradually restores it.
  • Prompt rewriting: DPP, an inference-time variant of DPP-GRPO, that selects candidates using the original method’s diversity and semantic-relevance scoring, replacing GRPO training because the code and the trained rewriting model are not open-source; DIVA, our re-implementation of its Director → Architect → Critic → Producer workflow. Both use DeepSeek-v4.1-flash-1017.
  • Factor-specific prompting (Suggested baseline). Regarding the suggested baseline, our Enumerated method in Section 5 already explores factor-specific prompt augmentation (Table 2 in our paper). We summarize the main results and conclusions below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.