- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes a feed-forward method for reconstructing compact 4D Gaussians from unposed monocular video. The authors point out that existing pixel-aligned methods create a Gaussian for every pixel, leading to redundancy and ghosting, while query-based methods are compact but do not explicitly track motion, resulting in weak correspondence. To address this, the method uses a three-stage design. First, a DA3-based backbone predicts per-pixel 3D points and cameras, and a trajectory is predicted for every pixel as a B-spline displacement that, through a null-space constraint, always passes through the source point at its source time. Second, these dense trajectories are clustered based on 3D proximity, relative-distance variation, and color to extract dynamic anchors, which are combined with static anchors to form a small set of anchor trajectories. Third, a query-based decoder attaches a learnable token to each anchor, reads image features, and generates child Gaussians that move along the anchor trajectory with additional time-varying offsets. Training is done in two stages, where the foundation model is trained first and then frozen while the decoder is trained. As a result, the method performs rendering and tracking with far fewer Gaussians than dense pixel-aligned methods.
Critical strengths
- Hierarchical design that separates motion tracking from rendering.
Existing feed-forward 4D methods either create a Gaussian per pixel, which tracks motion well but duplicates surfaces, or decode Gaussians from a few queries, which is compact but weak at tracking. This paper combines both by predicting per-pixel trajectories and using a selected subset as a scaffold for the Gaussians. With far fewer Gaussians, it achieves the highest PSNR and SSIM among external baselines and outperforms other query-based methods on all tracking metrics. Showing that a representation can be compact without losing motion tracking is the main reason I view this paper positively.
- Simple yet effective null-space constraint (NSC).
Trace Anything’s absolute control points let the learned motion shift the trajectory’s starting position. This paper instead splits each trajectory into a source point and a displacement, using a null-space basis so that the displacement is exactly zero at the observation time by construction, at negligible extra cost. The ablation shows that removing it degrades both rendering and tracking, and the idea can be readily adopted by other trajectory-based methods.
Critical weaknesses
- Heavy reliance on external papers for describing key components.
The paper combines several existing methods, including TraceAnything, SuperTrajectory, TokenGS, and LocusGS, and many of its components are described only by reference. For example, the DPC procedure and static anchor selection in anchor extraction, the geometric bias and the separate image encoder in the decoder, and the geometry, reprojection, static, and rendering losses in Stage I are stated to follow prior work without concrete definitions. Since the contribution relies heavily on combining existing components, this lack of detail not only hurts reproducibility but also makes it difficult to assess which parts are new contributions of this paper.
- No efficiency evaluation despite the emphasis on compactness.
Although the final number of Gaussians is greatly reduced, the compact variant first predicts trajectories for every pixel and then additionally runs clustering, a separate image encoder, and a query-based decoder, so its computational cost may actually exceed that of the dense variant. Moreover, since the positions and rotations of the child Gaussians are decoded again for each target time, it is unclear whether the Gaussian count fully reflects the actual storage size of the representation. Nevertheless, no measurements of inference time, memory, or storage are provided. Since the Gaussian count is used as a key point of comparison, the paper should show that compactness translates into actual efficiency for its central claim to be convincing.
Questions for authors
- Efficiency
Could the authors compare the inference time, memory, and rendering speed of the full pipeline, including dense trajectory prediction, clustering, the image encoder, and the decoder, against the baselines? In addition, since the positions and rotations of the child Gaussians are decoded again for each target time, how does the actual storage size scale with the number of frames? I would like to understand whether “compact” brings practical efficiency benefits beyond the number of Gaussians.
- Scope of contribution in the decoder and anchor extraction
The query-based decoder builds on TokenGS and LocusGS, and the anchor extraction builds on SuperTrajectory and GroupFlow. The paper presents several changes, such as replacing LocusGS’s static anchors with time-dependent anchors kept fixed during decoding, adding a color term to the grouping criterion, and separating the static and dynamic budgets. Beyond these, could the authors clearly identify which elements are newly designed compared to existing methods?