- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
FluxSplat extends MoVieS to streamable dynamic view synthesis through history-aware depth prediction, consistency losses, query-aware frame sampling, and a Compact Gaussian Adapter. The method aims to preserve temporal consistency with a bounded input budget and improve high-resolution rendering with fewer Gaussians.
Critical strengths
- The proposed method achieve a SOTA performance through simple but effective mechanism.
- The benchmark comparisons and component ablations support improved rendering quality over the evaluated baselines.
- Motion-FVS requires no training and can use pose and motion estimates from off-the-shelf models. This modular design makes the sampler reusable beyond the proposed network.
Critical weaknesses
-
The core contribution "constant-memory streaming" is limited, as the baseline methods can also be "constant-memory" by simply adding a sampling module.
-
Limitations of Motion-FVS. The relevance filtering identifies the nearest reference view based solely on Euclidean distance between camera centers, ignoring camera orientation and actual field-of-view overlap. This may lead to unreliable frame selection under significant camera rotation or limited view overlap. Also, using global mean motion magnitude may overlook small but important moving objects.
-
Questionable effectiveness of CGA. As shown in Table 1, introducing CGA consistently reduces PSNR and worsens LPIPS across all three benchmarks. It also increases inference latency and GPU memory consumption (Table 6). Furthermore, the paper lacks comparisons with simpler training-free alternatives, such as average pooling followed by Gaussian scale correction, making it unclear whether the learned adapter provides benefits beyond basic Gaussian aggregation.
-
Limited methodological novelty. The overall architecture and dynamic Gaussian formulation are largely inherited from MoVieS. The proposed contributions primarily consist of a ConvGRU-based depth head, a convolutional aggregation adapter (weakness 3), and a heuristic frame sampling strategy (weakness 2), making the methodological novelty relatively incremental.
Questions for authors
-
Regarding scalability: Can the authors report the end-to-end memory footprint (including historical caches) and frame-selection latency as video length increases? How would the scalability comparison change if MoVieS were evaluated with the same fixed frame budget?
-
Regarding CGA: How does CGA compare against a simple training-free baseline using average pooling and Gaussian scale correction, under the same Gaussian count? Can the authors clarify why CGA consistently degrades reconstruction quality at the original resolution despite introducing additional computational overhead?
-
Regarding Motion-FVS: Why is camera orientation excluded from the nearest-view selection criterion? Have the authors evaluated alternative sampling strategies that explicitly consider camera rotation and field-of-view overlap?
-
Regarding motion-aware sampling: Could the authors provide evaluations on the robustness of Motion-FVS?
7 comments
Response (part 2)
Q3. Why is camera orientation excluded from the nearest-view criterion? Have authors considered alternatives incorporating rotation and FoV overlap?
A: Our design targets dense streaming videos with in-distribution queries. Dense camera trajectories provide substantial viewpoint redundancy, making translation distance an efficient geometric prior for candidate selection and sufficient FoV overlap for the selected anchor view. Subsequent content-based filtering and motion-aware sampling select informative and complementary reference frames.
To assess the benefit of explicitly considering orientation, we augment Motion-FVS with camera rotation similarity (Note that, for rotation similarity formula, please check Eq. 12 in Appendix Line 1012-1013). The similarity accounts for yaw, pitch, and roll, with higher values indicating more similar rotation. Using ours w/o CGA and following the evalution setting in main Table.1, we obtain same PSNR values across all evaluated datasets:
| Sampling strategy | Nerfies & HyperNeRF | Dynamic NVIDIA | DyCheck |
|---|---|---|---|
| Motion-FVS | 17.7 | 18.5 | 19.9 |
| Motion-FVS + rotation similarity | 17.7 | 18.5 | 19.9 |
These results suggest that explicit rotation similarity provides no additional benefit in our dense streaming setting, supporting the simpler translation-based criterion.
About considering FoV overlap in sampling, camera-based geometric overlap does not account for independently moving foreground objects. When both the camera and foreground move, limited overlap may still capture the same moving object. We therefore incorporate foreground content similarity, inspired by DINO-Tracker [1], to identify dynamically relevant observations beyond geometric proximity.
Action:
- Added an analysis of incorporating camera rotation similarity into Motion-FVS in Appendix Sec. A.3.2, “Motion-FVS Analysis.”
References:
[1] Taming DINO for Self-Supervised Point Tracking in a Single Video. ECCV, 2024.
Q4. Does mean motion magnitude overlook small moving objects?
A: We agree that mean motion magnitude is a coarse approximation that may underestimate motion in small image regions, particularly objects with limited pixel coverage. Our current approach primarily focuses on prominent foreground objects, and we acknowledge that it may be less effective at capturing small moving objects. We acknowledge this limitation and identify more fine-grained motion-aware sampling as a direction for future work.
Action:
- Added the potential neglect of small-object motion as a limitation in Appendix Sec. A.4, “Limitations and Future Work.”
Response (part 3)
Q5. Why does CGA reduce native-resolution quality, and is it better than simple pooling with scale correction?
A: CGA trades some native-resolution quality for a compact Gaussian representation, particularly in scenes with high-frequency details where per-pixel Gaussians offer greater representational capacity. Its learned coordination and scale prediction support rendering across resolutions while reducing hole artifacts at higher resolutions.
We thank the reviewer for this suggestion. Following the high-resolution evaluation setting in Table 2, we compare CGA with a training-free baseline that applies average pooling to the outputs of our model without CGA, followed by global scale correction (AvgPool + SC) with multipliers {1,2 } to control scale enlargement. For each factor {2,3}, both methods use the same Gaussian budget, input frames, and target views. We define the corrected scale in Eq. 13 in Appendix Line 1103-1105: We report results on DyCheck and Dynamic NVIDIA below.
DyCheck
| Method | k | γ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|---|
| Ours (CGA) | 2 | — | 17.36 | 0.590 | 0.370 |
| AvgPool + SC | 2 | 1 | 6.76 | 0.069 | 0.875 |
| AvgPool + SC | 2 | 2 | 8.55 | 0.139 | 0.831 |
| Ours (CGA) | 3 | — | 17.06 | 0.450 | 0.390 |
| AvgPool + SC | 3 | 1 | 6.36 | 0.066 | 0.854 |
| AvgPool + SC | 3 | 2 | 8.04 | 0.131 | 0.797 |
Dynamic NVIDIA
| Method | k | γ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|---|
| Ours (CGA) | 2 | — | 15.52 | 0.470 | 0.460 |
| AvgPool + SC | 2 | 1 | 8.02 | 0.061 | 0.919 |
| AvgPool + SC | 2 | 2 | 9.59 | 0.179 | 0.836 |
| Ours (CGA) | 3 | — | 15.12 | 0.440 | 0.530 |
| AvgPool + SC | 3 | 1 | 7.88 | 0.063 | 0.894 |
| AvgPool + SC | 3 | 2 | 9.47 | 0.191 | 0.794 |
We can see that CGA consistently outperforms both tested scale-correction settings across all metrics. Compared with , it improves PSNR by 8.81–9.02 dB on DyCheck and 5.65–5.93 dB on NVIDIA. We also present a comparison of the visualization results in Figure 9 in Appendix. These results show that simple pooling with global scale correction cannot substitute for learned Gaussian aggregation, even at the same Gaussian budget.
Action:
- Added a comparison of CGA with a training-free pooling and scale-correction baseline in Appendix Sec. A.3.3, “Compact Gaussian Adapter Analysis.”
Q6. Is the methodology merely an incremental extension of MoVieS?
A: While FluxSplat builds upon MoVieS, its central contribution is a history-aware framework for streamable dynamic view synthesis (DVS) from monocular streams. To our knowledge, prior feed-forward dynamic Gaussian splatting methods have not jointly exploited explicit and implicit historical memory in this setting.
Our framework introduces hybrid memory: historical images provide explicit memory of past observations, while recurrent latent states provide implicit memory that carries historical information into subsequent predictions. This enables query-dependent retrieval of past observations alongside history-conditioned Gaussian prediction.
The CGA in our framework, it learns a dynamic kernel that models local interactions among spatially neighboring dynamic Gaussians, enabling coordinated aggregation into a compact representation. This learned coordination complements the history-aware framework by reducing storage requirements and supporting high-resolution rendering.
Within this framework, Motion-FVS combines motion relevance with spatial coverage to retrieve informative frames under a fixed input budget through a training-free sampling strategy. To our knowledge, FluxSplat is the first to introduce this motion-guided, query-centric sampling strategy into feed-forward dynamic Gaussian splatting for streamable DVS.
Thus, our contribution is a framework for integrating, retrieving, and exploiting historical information, complemented by learned Gaussian coordination for compact representations. These components are modular and potentially applicable to other VGGT-based feed-forward Gaussian splatting frameworks.
Response (part 1)
We sincerely thank the Reviewer for the thorough evaluation and constructive suggestions. Below, we provide detailed responses to all questions. We have updated the manuscript accordingly, with newly added content indicated by blue [NEW] labels for ease of reference.
Q1. Is constant-memory streaming a meaningful contribution if MoVieS can also use a fixed-budget sampler?
A: We agree that MoVieS can also achieve constant inference memory with a fixed-budget sampler. Our contribution, however, is the carefully designed application of query-centric, motion-guided frame selection to feed-forward dynamic Gaussian splatting for streaming DVS from monocular video streams—to our knowledge, the first such integration.
Motion-FVS jointly considers motion relevance, temporal information, and spatial coverage to select historical observations. Constant model-inference memory streaming is one benefit of this design, alongside the ability to retrieve relevant information from an expanding observation history.
Furthermore, fixed-budget sampling alone does not ensure temporal consistency. FluxSplat combines history-conditioned prediction with cross-sample-set alignment losses to maintain temporally consistent Gaussian predictions as the selected observations change. Our contribution therefore encompasses both query-aware frame selection and temporally consistent reconstruction.
Q2. End-to-End Memory Scalability, Frame-selection Latency, and Fair Comparison with MoVieS
A: Historical caches are stored incrementally, while model inference uses a fixed input-frame budget. The following results we reported are averaged over 10 videos, each containing 150 timesteps:
Historical cache storage grows approximately linearly; our constant-memory claim applies to model inference under a fixed frame budget, not total memory.
With K = 12, Motion-FVS latency increases modestly and remains below 10 ms across all evaluated timesteps:
Regarding fair comparison, as stated in Sec. 5 (Baselines), MoVieS in Tabs. 1–3 already incorporates our Motion-FVS and uses the same fixed input-frame budget. With K = 12 and an input resolution of 210 × 378, the comparison about inference is:
These forward-pass timings exclude Motion-FVS overhead. Fixed-budget MoVieS is faster and uses less GPU memory; our improvements concern reconstruction quality and temporal fidelity under the same frame budget.
Action: