acceptodds
Under review as a conference paper at ICLR 2027

Beyond Continuous Deformation: State-Aware Hybrid Dynamics for 4D Gaussian Generation

Abstract

Recent approaches to 4D generation represent dynamic objects using deformable 3D Gaussians whose transformations are predicted by continuous spatiotemporal fields. Such direct coordinate-time mappings, however, propagate no motion state between adjacent timestamps, and provide no mechanism for adapting how strongly each primitive contributes at a given timestamp. We propose State-Aware Hybrid Dynamics (SAHyD), which assigns each Gaussian a 3-D motion state in center-coordinate space and updates it through a hybrid rule combining a learned continuous drift with an expected latent-event response under soft motion regimes. The resulting regime and event features condition the continuous deformation representation before Gaussian attributes are decoded, so the state shapes the predicted offsets rather than correcting them afterwards. To regularize the learned transition across step sizes, Fine-Grained Motion Integration (FMint) learns a residual center-motion correction from a full-step versus two-half-step consistency reference. Time-Varying Gaussian Existence (TVGE) then predicts a soft participation probability that gates opacity and suppresses residual primitives, while preserving the number of Gaussians. Extensive experiments on video-to-4D generation demonstrate consistent improvements in temporal quality and overall reconstruction fidelity over the continuous-deformation backbone.

Primary area:
generative models
Keywords:
4D Generation, 3D Gaussian Splatting, Dynamic Gaussian Representations, Video-to-4D Generation, State-Aware Dynamics, Temporal Modeling, Generative Models, Neural Rendering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_e5jFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper points out that the continuous coordinate-time deformation fields used in video-to-4D generation carry no explicit motion state and no way to change how strongly each Gaussian contributes over time. To fix this, it gives each Gaussian a motion state that evolves through a learned drift and an expected event response under soft motion regimes, and uses this state to condition the deformation before decoding attributes. It further adds FMint for step-consistent motion and TVGE for time-varying opacity. Experiments on Consistent4D report the best performance over the STAG4D backbone.

Critical strengths
  • The state is used to condition the deformation before attributes are decoded, rather than to correct the geometry after the fact. This is a clean idea, and it directly addresses the missing motion state in continuous deformation fields.
  • The ablation is very thorough. The capacity-controlled experiments (Tables 5 and 8) make sure the gains are not just from extra parameters, which is rare in this area.
  • The theory and limitations are honest. The appendix is clear about what the bounds do and do not guarantee, and the extra training time and memory are reported openly.
Critical weaknesses

The evaluation feels a bit thin for how strong the claims are.

  • All the quantitative results are on Consistent4D, which has only 7 objects. The text-to-4D and Internet-video results are qualitative only, and the headline FVD is computed from very few samples, which the paper itself admits is high-variance. The gain over the strongest external baseline (MVTokenFlow) is also modest: LPIPS ties, and only FVD and CLIP improve.
  • The reported 793.10 uses a cached regime prior from the previous frame, while the strict-reset setting gives 813.61. Since the paper claims inference can be queried at arbitrary timestamps, this gap is worth clarifying.
  • The "event, jump, hybrid dynamics" wording is a bit overstated. The event response is deterministic, and the paper itself notes that it reduces to a single effective field. The real contribution is the structured state conditioning, not stochastic dynamics.
Questions for authors
  • The strict-reset evaluation gives 813.61, while the reported number is 793.10 with a cached regime prior. For arbitrary or non-sequential timestamp queries, is that cached prior available? What is the performance without it?
  • All numbers are on seven Consistent4D objects, and FVD uses few samples. Can you report the spread across objects or seeds, and add quantitative results on a second benchmark or the text-to-4D setting?
1 comment
AuthorsOpenReview ↗

We thank the reviewer for the constructive comments. We address each point below.

W1:Beyond the seven-object Consistent4D quantitative benchmark, we have also obtained results on examples from STAG4D's official releases. These releases contain 28 video-to-4D sequences and 21 text-to-4D sequences (49 examples in total). Our additional qualitative results are not included in the seven-object quantitative averages. We will incorporate more of them into the manuscript and supplementary material. Component ablations and controlled studies in Tables 2–8 provide complementary evidence.

Each object contributes four predicted and four GT clip features (Appendix D and p. 28). The 32 frames define each clip, not 32 independent samples. FVD is therefore a benchmark-compatible relative metric, not a low-variance population estimate. The main comparison is summarized below.

MethodFVD ↓LPIPS ↓
MVTokenFlow844.100.120
Ours793.100.120

FVD is approximately 6.0% lower, while LPIPS ties. This is a metric-specific point-estimate improvement, not evidence of statistical significance.

We additionally conducted a text-to-4D user study following STAG4D. Preference shares for visual quality (Vis.), temporal consistency (Cons.), and text alignment (Align.) are shown below.

MethodVis. ↑Cons. ↑Align. ↑
Consistent4D16.3%18.0%15.2%
STAG4D34.3%32.0%38.2%
Ours49.3%50.0%46.5%

These support the applicability of the text-conditioned extension without resolving video-to-4D FVD's small-sample uncertainty.

W2:Independent timestamp queries use strict-reset inference, without full motion-state rollout or a cached regime prior. Without a prior, regime probabilities are predicted from the current state, timestamp, and context. In chronological hybrid evaluation, motion state and participation logit are initialized at each timestamp, while the preceding frame-level regime distribution is reused as a cached prior. Table 4 reports the following results.

InferenceCLIP ↑LPIPS ↓FVD ↓
Strict reset0.9250.118813.61
Hybrid (cached prior)0.9240.120793.10

Thus, the best reported FVD uses additional temporal information. We will make this distinction explicit beside the inference description and main results.

W3:We agree that the contribution is structured state conditioning, not stochastic dynamics. Section 3.2 uses a deterministic expected response; Eq. (80) expresses an effective field, bθ = fθ + λθ∑ₘρ⁽ᵐ⁾θJ⁽ᵐ⁾θ. The contribution is its structured factorization and coupling to deformation and temporal participation.

Table 5 provides parameter-matched controls.

DynamicsParamsLPIPS ↓FVD ↓
Direct residual26,9910.118893.25
Gated residual26,8020.118885.13
Structured26,9160.120793.10

The residual controls have slightly lower LPIPS. These results support the structured design in the tested setting, not unique expressivity or a stochastic mechanism. We agree that event and hybrid terminology should consistently reflect this narrower interpretation.

Q1:A preceding-frame prior cannot be assumed available for isolated or non-sequential queries. The applicable setting is strict reset, with FVD 813.61, CLIP 0.925, and LPIPS 0.118. The 793.10 result belongs to chronological hybrid evaluation. Independent querying does not imply identical performance between reset and cached inference.

Q2:The selected results underlying our reported best-FVD result are shown below.

ObjectFVD ↓
Aurorus620.91
Crocodile848.53
Guppie412.26
Monster948.26
Pistol235.55
Skull1789.35
Trump696.87
Mean ± sample SD793.10 ± 502.41

Across seven objects, FVD ranges from 235.55 to 1789.35. The sample SD describes variation across objects, not training seeds, and does not eliminate the small-sample uncertainty of each FVD estimate.

We re-rendered 10 objects from Diffusion4D's curated Objaverse-1.0 list [1], excluding identifiable Consistent4D UIDs. Methods used 16 input frames, four held-out views and fixed input-only scale/XY alignment. Mean ± sample SD over seeds 42–44 (object means first):

MethodCLIP ↑LPIPS ↓FVD-16 ↓
STAG4D0.9038±0.00250.1199±0.0003762.25±13.85
SS4D [2]0.9142±0.00290.1360±0.0114677.16±70.74
Ours0.9088±0.00100.1170±0.0001655.93±19.17

References

[1] H. Liang et al. Diffusion4D. NeurIPS 2024.

[2] Z. Li et al. SS4D. TOG 2025.

Reviewer_WPQyOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes to do dynamic 3DGS from a video. They propose a model to regularize the temporal consistent by decoupling it into global motion in SAHyD and refine it in FMint. They evaluate against the baselines and ablate each components on Consistent4D dataset.

Critical strengths

None

Critical weaknesses
  • The paper proposes to do video-to-4D but it lacks of video results which makes it hard to judge whether there is improvement. Figure 3 only shows the same camera angles as the input.
  • The paper is really hard to follow. There are so many coined terms and notations. I don't fully understand 3.2, 3.3, 3.4
  • The paper ablate on Consistent4D and test it against other appraoches on the same dataset. This is unfair.
Questions for authors
  • Why don't compare with newer video-to-4D appraoches like SS4D, ActionMesh, ...? STAG4D, GVFD are very old.
  • I don't understand the motivation of doing dynamic Gaussians when it will be more useful if we get a mesh + deformation field, where you could use it for game assets. But doing dynamic 3DGS is only for novel view synthesis problem.
3 comments
AuthorsOpenReview ↗

We thank the reviewer for the constructive comments. We address each point below.

W1:We acknowledge that Figure 3's selected frames do not sufficiently demonstrate temporal behavior or novel-view quality. We have generated MP4 videos for both video-to-4D and text-to-4D, including multi-view renderings. Figure 9 shows text-to-4D results across timestamps and viewpoints. We will supplement the video-to-4D presentation with clearly labeled novel-view results and corresponding MP4 supplementary videos, enabling direct inspection of motion consistency and local structural quality.

W2:We agree that the terminology and notation obscure the computational flow. SAHyD updates a per-Gaussian motion state and uses state/event features to condition deformation before attribute decoding. FMint learns a residual correction from a self-generated two-half-step reference; the refined state increment adjusts the decoded centers. TVGE controls temporal participation through opacity modulation, without inserting or deleting primitives during rendering.

We will introduce each component through its purpose, inputs, and outputs before the equations, reduce unnecessary terminology, define essential symbols at first use, and align notation across the text, Figure 2, and algorithms.

W3:Ablations test individual components, whereas comparisons with other methods evaluate the complete framework. Using the same benchmark provides a common evaluation setting and does not itself make the comparison unfair. However, data use and model selection require clear documentation. Our task optimizes each scene from a monocular video and evaluates rendered sequences from four viewpoints (Section 4.1 and Appendix D). We will explicitly document optimization observations, evaluation views, and configuration/checkpoint selection rules. Our conclusions concern the Consistent4D protocol and do not establish cross-dataset generalization.

Our task involves scene-specific optimization from a monocular video, followed by evaluation of rendered sequences from four viewpoints, as described in Section 4.1 and Appendix D. Optimizing a scene from its input observations and evaluating it against benchmark ground truth are distinct steps. We agree that these distinctions should be presented more clearly. We will explicitly document the observations used for optimization, the evaluation views, and the rules for selecting configurations and checkpoints. Our quantitative conclusions concern performance under the Consistent4D protocol; they do not establish cross-dataset generalization.

Q1:We evaluated SS4D [1] and ActionMesh [2], averaging each comparison over three random seeds. Official SS4D code and pretrained weights, evaluated on seven Consistent4D objects with 32 input frames and four viewpoints, yielded the following results.

MethodCLIP ↑LPIPS ↓FVD-32 ↓
SS4D0.900.151103.44

For ActionMesh, all methods used the same 16 input frames (indices 0,2,...,30) and four novel viewpoints. STAG4D and Ours were retrained with these inputs and generated guidance, without reusing 32-frame checkpoints. Mean silhouette IoU is shown below.

MethodSilhouette IoU ↑
STAG4D0.72
Ours-reset0.72
ActionMesh0.61

We thresholded rendered and GT alpha at 0.5; alignment used only the first input frame and remained fixed across time, without GT fitting. Consistent4D lacks the GT surfaces and temporal correspondences needed for ActionMesh's CD-3D/CD-4D/CD-M metrics. Thus, this comparison assesses projected silhouette agreement, not complete 3D geometry or motion accuracy.

Q2:We agree that meshes with deformation fields are attractive for game-asset workflows. We use dynamic Gaussians for their explicit primitive representation, efficient differentiable rendering, and suitability for optimization from monocular observations.

Our contribution concerns temporal modeling within this representation. Motion-state conditioning, transition-consistency correction, and time-dependent primitive participation address local structural inconsistencies in coordinate–time deformation. The output is an explicit, time-varying Gaussian representation evaluated through novel-view sequences.

Our experiments do not establish mesh-equivalent editing or game-asset functionality. We appreciate this constructive suggestion. In future work, we will consider extending the framework toward temporally consistent meshes with associated deformation fields, exploring their potential for asset editing and game-engine integration.

[1] Z. Li et al. SS4D. ACM TOG 44(6), 244, 2025. DOI.

[2] R. Sabathier et al. ActionMesh. CVPR 2026. Code.

  • I still think using the same dataset is unfair. This is like tuning hyperparameters on the test set which I believe it is not common.
  • What is silhouette IoU? I feel like it is hard to believe that ActionMesh is worse than STAG4D and yours, given that the results in the papers are not strong. If your approach is better than STAG4D, why is the number of silhouette IoU similar?
  • Do you have visualization of these ActionMesh, SS4D versus yours?

C1:To extend our evaluation beyond the seven Consistent4D objects, we conducted an additional comparison on ten animated objects selected from the Objaverse-1.0 assets curated by Diffusion4D. These assets were re-rendered under a common protocol. Two separate development objects were excluded from the reported results.

STAG4D, SS4D, and our method received the same 16-frame monocular input sequences and were evaluated from four held-out viewpoints at a resolution of 512 × 512. All three methods used the same input-only alignment procedure, with a single scale and XY translation determined from the first conditioning frame and fixed throughout the sequence. Held-out reference images were not used for alignment.

We report the mean and sample standard deviation across three seeds, after averaging equally over the ten objects within each seed.

MethodCLIP ↑LPIPS ↓FVD-16 ↓
STAG4D0.9038 ± 0.00250.1199 ± 0.0003762.25 ± 13.85
SS4D0.9142 ± 0.00290.1360 ± 0.0114677.16 ± 70.74
Ours0.9088 ± 0.00100.1170 ± 0.0001655.93 ± 19.17

Our method obtains lower LPIPS and FVD-16 in this comparison, while SS4D obtains higher CLIP similarity. These results provide complementary evidence on additional objects beyond the original Consistent4D evaluation. This is a separately constructed object subset, rather than the complete official Diffusion4D benchmark, and we do not claim that the assets are entirely unseen by the pretrained models.

C2:Silhouette IoU measures the intersection-over-union between predicted and reference foreground masks. Rendered and reference alpha values were thresholded at 0.5, and scores were averaged across timestamps, viewpoints, and objects.

The reported scores are 0.72 for STAG4D, 0.72 for our method, and 0.61 for ActionMesh. We do not claim an improvement over STAG4D on this metric. Silhouette IoU measures projected foreground occupancy, but does not directly evaluate texture, complete 3D geometry, surface correspondence, or temporal coherence. Similar silhouette scores therefore do not imply identical reconstruction quality.

ActionMesh primarily reports geometry and motion metrics, including CD-3D, CD-4D, and CD-M, on ActionBench. Our silhouette experiment is a supplementary comparison under a local rendering protocol, not a reproduction of that evaluation or evidence of overall superiority over ActionMesh. Incomplete original GT calibration and differences between mesh and Gaussian boundary rasterization further limit its interpretation.

Separately, in our original Consistent4D evaluation, FVD decreases from 995.11 to 793.10, CLIP increases from 0.909 to 0.924, and LPIPS decreases from 0.126 to 0.120 relative to STAG4D. These results should not be conflated with the 16-frame silhouette experiment or interpreted as evidence of improved silhouette overlap.

C3:We agree that numerical summaries alone are insufficient for assessing dynamic reconstruction. We have organized synchronized four-view MP4 comparisons for all seven Consistent4D objects, with method names, frame indices, and evaluation-view angles explicitly labelled.

Four-view visual comparisons — public preview ,A backup link is available at https://visionlize.asia/.

The page separates two experiments. The 32-frame RGB section compares GT, STAG4D, SS4D, and our method. The 16-frame silhouette section compares GT, STAG4D, ActionMesh, and the independently optimized Ours-reset control using source indices 0,2,...,30. Each section provides a four-view overview and individual-view videos. The two conditioning protocols are not mixed.

For ActionMesh, the primary comparison uses silhouettes; its untextured geometry previews are not treated as an appearance-quality disadvantage. The page retains the limitations of SS4D's native-scale framing and the available camera provenance.

Reviewer_7pU9OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a novel video-to-4D generation framework with hybrid dynamics. Established on the STAG4D, a 3D motion state is incorporated with explit motion modeling. Also, latent and event-induced features are integrated as the state-aware dynamic bias. FMint decomposes a full-step transition into two half-step transitions. Moreover, a TVGE module assigns each primitive a differentiable temporal participation probability.

Critical strengths
  1. The paper's designs seem to be reasonable. although for some designs the motivation is not so clear.

  2. The method part is easy to follow, where authors also provide the pseudo-code for better understanding.

  3. The qualitative results are sufficient and the comparisons can reveal the strengths of proposed methods.

Critical weaknesses
  1. The paper only evaluates the performance in video-to-4D tasks, and it is still unclear how the performance is in text-to4D generation.

  2. The comparison baselines are somehow outdated. More SoTA methods published in 2026 should be compared.

  3. The paper lacks of the efficiency comparison with other methods, especially with STAG4D. Becuase the work is established on STAG4D, I guess the latency will be bigger than that.

  4. In Section 3.3, I feel confused why authors reduce the step into two half. What about reducing it even more fine-grained? i.e. into three or four steps.

  5. In the event-based evolution, how does the dynamic module predict the event-based attributes? More descriptions are required. Also, what is the meaning of 'event' in the context of 4D generation task? Since in many cases of the used dataset, the avatars only move a little without servere motions.

Questions for authors

For the temporal Gaussian adjustments, some works utilize pruning or densification techqniue to dynamically adjust the Gaussian primitives. I would like to see the comparison between pruning-based methods and TVGE.

5 comments
AuthorsOpenReview ↗

We thank the reviewer for the constructive comments. We address each point below.

W1:Following STAG4D, we use SDXL text-to-image generation, SVD image-to-video generation, and our video-to-4D pipeline with the same dynamics modules. Beyond Appendix H/Fig. 9, a user study following STAG4D evaluated visual quality, temporal consistency, and text alignment. Preference shares (Vis./Cons./Align.) are shown below.

MethodVis. ↑Cons. ↑Align. ↑
Consistent4D16.3%18.0%15.2%
STAG4D34.3%32.0%38.2%
Ours49.3%50.0%46.5%

These results support the applicability of the text-conditioned extension; video-to-4D remains our primary task.

W2:We evaluated SS4D [1] and ActionMesh [2], averaging each comparison over three random seeds. Official SS4D code and pretrained weights, evaluated on seven objects with 32 input frames and four viewpoints, yielded the following results.

MethodCLIP ↑LPIPS ↓FVD-32 ↓
SS4D0.900.151103.44

For ActionMesh, all methods used the same 16 input frames (indices 0,2,...,30) and four novel viewpoints. STAG4D and Ours were retrained with these inputs and generated guidance, without reusing 32-frame checkpoints. Mean silhouette IoU is shown below.

MethodSilhouette IoU ↑
STAG4D0.72
Ours-reset0.72
ActionMesh0.61

We thresholded rendered and GT alpha at 0.5; alignment used only the first input frame and remained fixed across time, without GT fitting. Consistent4D lacks the GT surfaces and temporal correspondences needed for ActionMesh's CD-3D/CD-4D/CD-M metrics. Thus, this comparison assesses projected silhouette agreement, not complete 3D geometry or motion accuracy. W3:On seven objects at 512 × 512 on an NVIDIA L40S, using checkpoints corresponding to FVD 793.10, we timed the complete rendering forward pass after warm-up with synchronized wall-clock timing. This includes dynamics, deformation, FMint, TVGE, and rasterization, excluding loading and image export. Rendering latency is shown below.

MethodLatency (ms/frame) ↓
STAG4D2.75
Ours6.49

Ours adds 3.74 ms/frame (2.36× latency), with approximately 154 FPS. This is separate from Appendix J's optimization cost. Our current implementation incurs additional rendering cost despite avoiding full-sequence motion-state rollout; we will clarify this trade-off.

W4:Two half steps provide the simplest refined reference for FMint's consistency target. The total interval and timestamp conditioning remain fixed, while the state updates sequentially. Three-seed means are shown below.

SubstepsCLIP ↑LPIPS ↓FVD ↓
20.9240.120793.10
30.9220.118793.01
40.9260.119792.99

Finer subdivision requires more dynamics evaluations but offers only small FVD gains and no consistent monotonic image-metric improvement. We therefore retain two half steps as the default.

W5:An “event” denotes an internal latent response, not an observed discrete event or a requirement for severe motion. In Section 3.2, the module is conditioned on the 3-D motion state, timestamp, shared temporal context, and regime embedding. Its prediction heads produce the drift, a nonnegative intensity, probabilities over latent response types, type-specific response vectors, and a feature that conditions Gaussian deformation. Equation (3) uses the deterministic expected contribution, λΔt∑ₘρ⁽ᵐ⁾J⁽ᵐ⁾, rather than sampling jumps. Section 3.2 and Appendix C specify this deterministic setting, and Appendix G interprets the intensity as state-dependent modulation rather than a detector of physical events. Consequently, small local changes can be represented without assigning a semantic action label. We agree that “latent response component” is a clearer interpretation than language suggesting detected abrupt events.

Q1:Directly incorporating 4DSTR's per-frame densification/pruning, TAD-GS's temporal centers/lifespans , or LPM's multiview geometric matching would change our representation or supervision, complicating controlled comparisons. We instead adapted Temporal Sensitivity Pruning (TSP) from SpeeDe3DGS [3]. Its official scoring code operates on deformed Gaussian attributes and removes primitives without replacing the motion representation.

With TVGE disabled and other dynamics components retained, adapted TSP gives the three-seed means below; the full-method result is from the paper.

MethodCLIP ↑LPIPS ↓FVD ↓
Adapted TSP, no TVGE0.9200.119877.19
Full method (paper)0.9240.120793.10

The full method has higher CLIP and lower FVD, while TSP has slightly lower LPIPS. This evaluates the adapted pruning strategy, not the complete SpeeDe3DGS pipeline.

[1] Z. Li et al. SS4D. ACM TOG 44(6), 244, 2025. DOI.

[2] R. Sabathier et al. ActionMesh. CVPR 2026. Code.

[3] A. Tu et al. SpeeDe3DGS. CVPR 2026.

Thanks for the author's additional experiments and clarifications. However, there are still some concerns:

  1. In W1, how is the user study designed to assess the performance? What is exactly the group size?

  2. In terms of efficiency, compared to the baseline STAG4D, there is over double computational overhead. It may influence real-time application in other downstream tasks.

  3. The distinction between TSP and this paper's performance is quite small, which may instead weaken the effectivenss and significance to propose a novel module.

AuthorsOpenReview ↗

We thank the reviewer for the constructive comments. We address each point below.

C1:Thank you for requesting clarification. Our user study involved 14 test scenarios and 30 evaluators, comparing Consistent4D, STAG4D, and our method. Participants viewed dynamic videos rendered from multiple viewpoints. Following STAG4D’s evaluation dimensions, they assessed visual quality, temporal consistency, and alignment with the input text.

Visual quality concerns rendered appearance, visible detail, and structural artifacts. Temporal consistency concerns the coherence of appearance and motion over time, while text alignment concerns agreement with the object, appearance, and motion described in the prompt. Preferences were summarized separately for each criterion and reported as percentages. These results provide supplementary perceptual evidence for our text-to-4D extension. We will explicitly describe the study size and evaluation criteria in the revision.

C2:We agree that the additional rendering cost matters for downstream applications with tight latency budgets. Our current implementation takes 6.49 ms/frame compared with 2.75 ms/frame for STAG4D, corresponding to 2.36× the latency and an additional 3.74 ms/frame. We acknowledge this as a limitation and a quality–efficiency trade-off.

Under the measured setting of seven objects rendered individually at 512 × 512 on an NVIDIA L40S, our method achieves approximately 154 FPS, with a frame-level p95 latency of 7.10 ms. These measurements characterize the rendering stage after scene optimization. The latency available for other downstream components depends on the application workload and hardware, so these results do not establish end-to-end real-time performance. We will make the measured conditions, additional cost, and application scope explicit in the revision.

C3:Thank you for raising this concern. TVGE’s contribution is its state-conditioned formulation of temporal Gaussian participation, coupling opacity modulation with the learned motion and regime/event features. Adapted TSP physically removes selected primitives from the shared Gaussian set. TVGE preserves them and controls their contribution over time, allowing the same retained primitive to have different contributions across timestamps. This formulation complements the inherited densification and pruning mechanisms.

We acknowledge that the image-level differences from adapted TSP are modest. CLIP increases from 0.920 to 0.924, while LPIPS is slightly worse, 0.120 versus 0.119. The reported FVD decreases from 877.19 to 793.10, approximately 9.6%. We interpret this comparison as a metric-specific trade-off.

The component-level controls in Tables 2 and 8 provide more direct evidence for the proposed design:

SettingCLIP ↑LPIPS ↓FVD ↓
SAHyD + FMint, without TVGE (Table 2)0.9220.131834.67
Time-conditioned opacity head (Table 8)0.9200.120846.47
Full method with TVGE0.9240.120793.10

The time-conditioned head and TVGE have closely matched module parameter counts, 8,501 and 8,454, respectively. These controls support the usefulness of TVGE’s combined temporal-participation design within our framework. We will clarify this design distinction and its empirical contribution, while presenting the TSP comparison as supplementary evidence and limiting our conclusions to the evaluated settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.