- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper points out that the continuous coordinate-time deformation fields used in video-to-4D generation carry no explicit motion state and no way to change how strongly each Gaussian contributes over time. To fix this, it gives each Gaussian a motion state that evolves through a learned drift and an expected event response under soft motion regimes, and uses this state to condition the deformation before decoding attributes. It further adds FMint for step-consistent motion and TVGE for time-varying opacity. Experiments on Consistent4D report the best performance over the STAG4D backbone.
Critical strengths
- The state is used to condition the deformation before attributes are decoded, rather than to correct the geometry after the fact. This is a clean idea, and it directly addresses the missing motion state in continuous deformation fields.
- The ablation is very thorough. The capacity-controlled experiments (Tables 5 and 8) make sure the gains are not just from extra parameters, which is rare in this area.
- The theory and limitations are honest. The appendix is clear about what the bounds do and do not guarantee, and the extra training time and memory are reported openly.
Critical weaknesses
The evaluation feels a bit thin for how strong the claims are.
- All the quantitative results are on Consistent4D, which has only 7 objects. The text-to-4D and Internet-video results are qualitative only, and the headline FVD is computed from very few samples, which the paper itself admits is high-variance. The gain over the strongest external baseline (MVTokenFlow) is also modest: LPIPS ties, and only FVD and CLIP improve.
- The reported 793.10 uses a cached regime prior from the previous frame, while the strict-reset setting gives 813.61. Since the paper claims inference can be queried at arbitrary timestamps, this gap is worth clarifying.
- The "event, jump, hybrid dynamics" wording is a bit overstated. The event response is deterministic, and the paper itself notes that it reduces to a single effective field. The real contribution is the structured state conditioning, not stochastic dynamics.
Questions for authors
- The strict-reset evaluation gives 813.61, while the reported number is 793.10 with a cached regime prior. For arbitrary or non-sequential timestamp queries, is that cached prior available? What is the performance without it?
- All numbers are on seven Consistent4D objects, and FVD uses few samples. Can you report the spread across objects or seeds, and add quantitative results on a second benchmark or the text-to-4D setting?
We thank the reviewer for the constructive comments. We address each point below.
W1:Beyond the seven-object Consistent4D quantitative benchmark, we have also obtained results on examples from STAG4D's official releases. These releases contain 28 video-to-4D sequences and 21 text-to-4D sequences (49 examples in total). Our additional qualitative results are not included in the seven-object quantitative averages. We will incorporate more of them into the manuscript and supplementary material. Component ablations and controlled studies in Tables 2–8 provide complementary evidence.
Each object contributes four predicted and four GT clip features (Appendix D and p. 28). The 32 frames define each clip, not 32 independent samples. FVD is therefore a benchmark-compatible relative metric, not a low-variance population estimate. The main comparison is summarized below.
FVD is approximately 6.0% lower, while LPIPS ties. This is a metric-specific point-estimate improvement, not evidence of statistical significance.
We additionally conducted a text-to-4D user study following STAG4D. Preference shares for visual quality (Vis.), temporal consistency (Cons.), and text alignment (Align.) are shown below.
These support the applicability of the text-conditioned extension without resolving video-to-4D FVD's small-sample uncertainty.
W2:Independent timestamp queries use strict-reset inference, without full motion-state rollout or a cached regime prior. Without a prior, regime probabilities are predicted from the current state, timestamp, and context. In chronological hybrid evaluation, motion state and participation logit are initialized at each timestamp, while the preceding frame-level regime distribution is reused as a cached prior. Table 4 reports the following results.
Thus, the best reported FVD uses additional temporal information. We will make this distinction explicit beside the inference description and main results.
W3:We agree that the contribution is structured state conditioning, not stochastic dynamics. Section 3.2 uses a deterministic expected response; Eq. (80) expresses an effective field, bθ = fθ + λθ∑ₘρ⁽ᵐ⁾θJ⁽ᵐ⁾θ. The contribution is its structured factorization and coupling to deformation and temporal participation.
Table 5 provides parameter-matched controls.
The residual controls have slightly lower LPIPS. These results support the structured design in the tested setting, not unique expressivity or a stochastic mechanism. We agree that event and hybrid terminology should consistently reflect this narrower interpretation.
Q1:A preceding-frame prior cannot be assumed available for isolated or non-sequential queries. The applicable setting is strict reset, with FVD 813.61, CLIP 0.925, and LPIPS 0.118. The 793.10 result belongs to chronological hybrid evaluation. Independent querying does not imply identical performance between reset and cached inference.
Q2:The selected results underlying our reported best-FVD result are shown below.
Across seven objects, FVD ranges from 235.55 to 1789.35. The sample SD describes variation across objects, not training seeds, and does not eliminate the small-sample uncertainty of each FVD estimate.
We re-rendered 10 objects from Diffusion4D's curated Objaverse-1.0 list [1], excluding identifiable Consistent4D UIDs. Methods used 16 input frames, four held-out views and fixed input-only scale/XY alignment. Mean ± sample SD over seeds 42–44 (object means first):
References
[1] H. Liang et al. Diffusion4D. NeurIPS 2024.
[2] Z. Li et al. SS4D. TOG 2025.