- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper addresses the failure mode in dynamic 3D Gaussian Splatting (4DGS) where small, fast-moving objects (such as juggling balls or baseballs) fade, tear, or completely vanish during reconstruction. Attributing this degradation to suboptimal spatiotemporal capacity allocation rather than model capacity limits, the authors introduce two heuristic allocation strategies integrated into the FreeTimeGS framework: (1) Seed Injection, which samples candidate regions from adjacent-frame temporal appearance differences and multi-view photometric errors to initialize new Gaussians directly in under-reconstructed dynamic 3D regions; and (2) Temporally Biased Relocation, which preserves screen-space positional gradient magnitudes across timestamps along a Gaussian's trajectory to construct a temporal relocation distribution, guiding when and where pruned/under-utilized Gaussians should be reallocated. The approach is evaluated on N3DV, CMU Panoptic, and a newly collected multi-camera Custom dataset featuring high-velocity motions (Baseball, Break).
Critical strengths
S1: Relevant Practical Problem: The paper focuses on an acknowledged, high-impact failure mode in dynamic volumetric rendering—specifically, the tendency of gradient-based 4D Gaussian representations to prune or fail to track small, high-velocity foreground elements (e.g., athletic gear, small thrown objects). Investigating this failure regime is practically relevant for dynamic view synthesis.
S2: Lightweight Execution with Negligible Computational Overhead: The engineering realization avoids auxiliary heavy neural networks or architectural overhauls of the underlying representation. As shown in the ablation benchmarks, integrating the two modules into FreeTimeGS incurs minimal training overhead (increasing average training time marginally from 5.69 minutes to 5.91 minutes on CMU sequences) while remaining within standard GPU memory constraints.
Critical weaknesses
W1: Severe Overclaim of "Framework-Agnostic" Generality vs. Single-Baseline Confinement: The abstract and introduction pitch the core contributions as general allocation strategies that "can be incorporated into existing dynamic Gaussian frameworks without modifying their underlying scene representation". In reality, the entire technical formulation and experimental validation are built strictly on top of FreeTimeGS. The temporal relocation mechanism (Equations 8 and 9) relies directly on FreeTimeGS's linear trajectory parametrization () and its internal opacity-pruning/resampling cycle. It cannot be directly applied to deformation-field-based representations (e.g., Deformable-GS, 4DGS with HexPlane/MLP) or explicit correspondence trackers (e.g., Dynamic 3DGS) without substantial methodological restructuring. Validating solely on one codebase while asserting architectural agnosticism constitutes a material overclaim that misrepresents the scope of the contribution.W2: Fundamental Disconnect in Quantitative Metrics and Circular Evaluation Mask Bias: The paper centers its narrative on reconstructing small, high-speed objects that vanish in prior work. However, such objects occupy less than 0.2% of the overall image pixels; their presence or total absence barely moves full-frame PSNR or SSIM metrics. Furthermore, in Section 4.1, the authors acknowledge that SAM3 fails on fast-moving objects and state: "we additionally incorporate the dynamic masks used for Seed Injection to ensure that these regions are included in the evaluation". This introduces circular evaluation bias: the authors' own heuristic candidate regions are fed directly into the ground-truth evaluation mask, heavily rewarding their method for placing primitives there while systematically penalizing baselines that lacked access to that exact injection mask prior. To soundly validate the core claim, the paper requires unbiased, manually annotated or independently tracked bounding-box metrics (e.g., ball/bat-only PSNR/LPIPS) and detection/tracking recall, rather than self-referential dynamic masks.W3: Lack of Theoretical Depth and Superficial Conceptual Packaging: The manuscript packages the proposed pipeline under the conceptual umbrella of "Spatiotemporal Gaussian Allocation". However, from a technical perspective, the work merely combines two disconnected engineering heuristics: (1) classic temporal frame-differencing combined with multi-view unprojection (a standard visual hull / multi-view stereo initialization trick dating back decades), and (2) an argmax positional gradient selection along linear trajectories within an existing relocation framework. There is no rigorous analytical investigation into why Gaussian optimization fails in transient high-velocity regimes—such as loss landscape flatness under large temporal displacements, opacity shrinkage driven by temporal smoothness/TV regularization, or gradient starvation. The paper patches the symptoms with external geometric injection rather than diagnosing or solving the underlying optimization breakdown.W4: Susceptibility to Multi-View Artifacts, Floaters, and Marginal Quantitative Impact: Seed Injection triggers whenever appearance changes and reconstruction errors co-occur. In real multi-view setups, temporal appearance differences are heavily polluted by view-dependent specular highlights, complex shadow transience, and inter-camera hardware sync jitter. Injecting 3D Gaussians based on 2D frame-difference thresholds introduces a severe risk of scattering floating artifacts across empty 3D space. The paper provides no geometric cleanliness analysis, novel-view depth maps, or floater quantification. More tellingly, on the authors' Custom dataset—specifically designed to demonstrate this capability—the full PSNR increases by a negligible 0.14 dB (20.84 to 20.98), and Masked PSNR increases by merely 0.10 dB (27.37 to 27.47). Such minuscule quantitative gains suggest that while small objects may qualitatively emerge in selected crops, the injected Gaussians likely degrade surrounding space, canceling out metric improvements.W5: Temporal Cannibalization and Missing Longitudinal Consistency Validation: Temporally Biased Relocation shifts under-utilized Gaussians toward the single timestamp exhibiting the highest screen-space gradient magnitude (). By redistributing representational capacity toward high-motion peak timestamps, this mechanism risks depriving calmer, non-peak timestamps of sufficient geometric support. The paper only reports sequence-averaged metrics and isolated snapshot figures. It lacks per-frame PSNR/LPIPS variance plots across full sequences and includes no temporal stability evaluations (such as temporal warping error or optical flow consistency) to verify that fixing peak frames does not induce high-frequency temporal flickering elsewhere.
Questions for authors
Q1: How can the claim of "framework agnosticism" be substantiated when the relocation formulation (Eq. 8–9) is fundamentally coupled to the linear trajectory equation and resampling loop of FreeTimeGS? Have you implemented and benchmarked either Seed Injection or Temporally Biased Relocation on fundamentally different dynamic paradigms, such as deformation-field networks (e.g., Deformable-3DGS) or spacetime plane grids (e.g., SpaceTimeGS)?Q2: Regarding the evaluation bias in Section 4.1: When the test-view dynamic mask incorporates the heuristic masks derived from your own Seed Injection module, how much does this mask disparity artificially favor your method over baselines? Could you provide isolated, ground-truth-annotated bounding-box metrics (e.g., localized PSNR, LPIPS, and tracking/detection recall strictly computed over the bounding boxes of the baseball, bat, and juggling balls) across all compared methods?Q3: In Table 4, on the Custom dataset designed specifically to showcase the advantages of your method on fast-moving objects, the improvement in dynamic masked PSNR is only 0.10 dB (27.37 vs. 27.47) and full PSNR is only 0.14 dB. Given the qualitative prominence of the baseball in Fig. 1, why are the sequence-wide quantitative gains so marginal? Does the heuristic injection introduce floating artifacts or background inaccuracies that degrade metrics elsewhere?Q4: What safeguards prevent Seed Injection from triggering false positives and generating persistent 3D floaters under view-dependent specular highlights, non-rigid shadow movements, or camera exposure fluctuations? Could you provide novel-view depth map visualizations and floater point-count statistics?Q5: When Temporally Biased Relocation reallocates Gaussians toward peak-gradient timestamps (), does it degrade reconstruction quality at the timestamps from which capacity was redirected? Please provide a per-frame PSNR variance plot across the full 250/300 frames of the Baseball and Break sequences to demonstrate temporal consistency across the entire trajectory.