acceptodds
Under review as a conference paper at ICLR 2027

Snapshot-to-Flow: Residual Dynamics for Adapting Pretrained 4D Scenes

Abstract

Feed-forward 4D reconstruction supplies a reusable scene prior, but individual videos retain motion and appearance errors. We introduce Snapshot-to-Flow, a scene-level adapter that learns residual dynamics around pretrained trajectories. A shared velocity field integrates source-referenced position residuals; local appearance and scene-wide color controls address complementary rendering errors before an unchanged rasterizer. Complete sequence records with dynamic-branch updates show mean novel-frame PSNR gains of 1.29, 2.25 and 1.12 dB on STAG4D, HyperNeRF and Neural 3D Video. These validation gains retain all evaluated sequences, including negative-gain cases. An eight-scene motion-query study reports a mean gain of 1.31 dB beyond the conditioning interval. Component and scene-attribute diagnostics characterize the complete adaptation procedure; separate cross-scene controls distinguish its benefits from integration-specific superiority.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D Reconstruction, Gaussian Splatting, Neural ODE, Scene Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_foc9OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Snapshot-to-Flow, a scene-level adaptation framework that learns residual dynamics around pretrained 4D Gaussian trajectories. Built upon StreamSplat, the method introduces a shared velocity field for source-referenced motion correction, along with local appearance and scene-wide color controls to refine the reconstructed scene before rendering. Experiments demonstrate improvements in novel-frame reconstruction quality.

Critical strengths

The proposed method achieves improvements in reconstruction quality over the pretrained StreamSplat baseline across three datasets.

Critical weaknesses

Although the proposed framework achieves some performance gains through per-scene optimization, it compromises a key advantage of feed-forward reconstruction: reconstruction speed. More importantly, this approach shifts the framework toward conventional optimization-based reconstruction, yet the manuscript lacks direct comparisons with such methods, leaving the necessity and practical value of the proposed approach insufficiently justified.

Questions for authors
  1. Feed-forward reconstruction aims to eliminate per-scene optimization, yet the proposed framework reintroduces it, seemingly reverting to the conventional optimization-based paradigm. Does this not undermine the fundamental motivation of feed-forward reconstruction?

  2. How does the proposed framework differ from conventional optimization-based 4D reconstruction methods, and what advantages does it offer?

  3. In Table 2, M-ODE alone yields limited improvements, while ODE-Full achieves substantially larger gains, particularly after updating the pretrained model's dynamic branch. Could the authors provide a comparison with directly fine-tuning StreamSplat without the ODE modules to better isolate the contribution of residual dynamics?

Reviewer_GjgCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies sequence-specific adaptation of pretrained feed-forward 4D reconstruction models. Rather than replacing the pretrained scene dynamics, the authors propose Snapshot-to-Flow, which treats the predicted 4D scene as a reference and learns residual motion and appearance corrections before rasterization. Motion corrections are parameterized as a shared residual velocity field integrated along pretrained Gaussian trajectories, while local appearance and scene-level color controls compensate for rendering errors.

Critical strengths
  1. The paper addresses a meaningful adaptation problem for feed-forward 4D reconstruction. The idea of retaining the pretrained scene as a reference and learning sequence-specific residual corrections is well motivated and avoids unnecessarily relearning the entire reconstruction.
  2. The proposed scene-space parameterization is interpretable. Motion, local appearance, opacity, and scene-wide color are corrected through explicit Gaussian attributes, allowing the authors to analyze both trajectory-level and image-level interventions.
  3. The evaluation covers three dynamic-scene datasets and reports complete-cohort results, including negative-gain sequences.
Critical weaknesses
  1. The evaluation protocol raises concerns about the independence of the reported results. Section 4 states that Novel validation frames are used for checkpoint selection, while Table 2 reports the main reconstruction improvements on these same frames. Although these frames are excluded from the adaptation loss, their use in model selection may introduce selection bias. I suggest reporting additional results on an independent held-out test set that is not used for checkpoint selection.
  2. The experimental comparison does not sufficiently isolate the benefit of the proposed residual-dynamics parameterization. The main results primarily compare the adapted model against the unchanged StreamSplat backbone. Since the proposed method performs several minutes of sequence-specific optimization, improvement over a non-adapted baseline does not establish that Neural-ODE-based residual dynamics are preferable to simpler or existing adaptation strategies. Comparisons against direct trajectory residuals, per-frame residuals, polynomial parameterizations, or existing 4D refinement/adaptation methods would substantially strengthen the contribution.
  3. The motion-ODE component alone provides relatively limited gains. In Table 2, M-ODE improves Novel PSNR by only +0.15 and +0.34 dB on STAG4D and HyperNeRF and slightly decreases performance on N3DV. Much larger gains appear after adding appearance controls and especially after updating the pretrained dynamic branch. This makes it unclear how much of the final improvement should be attributed specifically to the proposed flow/ODE formulation.
  4. Generalization is demonstrated only with one pretrained reconstruction backbone, StreamSplat. While the paper appropriately acknowledges this limitation, validation on another feed-forward 4D reconstruction model would provide stronger evidence that the proposed adaptation interface generalizes beyond one architecture.
Questions for authors

same as weaknesses

Reviewer_tCPVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Snapshot-to-Flow, a scene-level adapter that improves the output of a pretrained feed-forward 4D reconstruction model for a specific video. The base predictor is StreamSplat, which takes two frames with estimated depths and predicts dynamic Gaussians with linear motion. The method keeps these predicted trajectories as a reference and learns residual corrections before an unchanged rasterizer. Motion is corrected by a shared velocity field that is integrated relative to each Gaussian’s source time. Appearance errors are handled by local color and opacity residuals and a scene-wide color transform. A compact configuration freezes the predictor, while an extended configuration (ODE-Full) also updates its dynamic branch. The controls are fitted once per sequence with additional frames, which takes 273–998 seconds. On STAG4D, HyperNeRF, and Neural 3D Video, ODE-Full improves novel-frame PSNR over the unadapted StreamSplat by 1.29, 2.25, and 1.12 dB, respectively. An eight-scene study also reports gains for queries beyond the conditioning interval.

Critical strengths

The method consistently improves over the feed-forward baseline on all three datasets. Including sequences with negative gains, ODE-Full improves novel-frame PSNR by 1.29, 2.25, and 1.12 dB on STAG4D, HyperNeRF, and Neural 3D Video, respectively. The beyond-pair queries also improve in all eight evaluated scenes.

Critical weaknesses
  1. Limited novelty and unclear contribution of the proposed modules. The method is essentially a per-scene post-optimization of a feed-forward prediction. This is already common practice, for example in the post-optimization stages of AnySplat and NoPo4D and in the test-time adaptation of LoRA3D and Test3R. Yet it takes 273–998 seconds per video, far longer than existing post-optimization stages that finish within seconds. Motion is built on StreamSplat’s linear motion between two frames, and Gaussians are defined only within short local intervals, so trajectory information across multiple frames is not exploited. The correction velocity also depends only on the reference path, which makes the ODE effectively a time-dependent position offset. In Table 2, the motion ODE alone accounts for only 12%, 15%, and −4% of the final gains, while 56–81% of the gains come from fully updating the dynamic branch. The contribution of the proposed modules is therefore unclear.

  2. Insufficient experimental validation and unclear presentation. The only quantitative baseline is the unadapted StreamSplat. The paper does not compare against the feed-forward 4D methods it cites (BTimer, MoVieS, 4D-LRM), post-optimization approaches, optimization-based 4DGS methods (e.g., ODE-GS, 3DGStream, MoRGS), or frame interpolation methods (RIFE, FILM, AMT). The evaluation is limited to PSNR at the input viewpoint, so the benefit of scene-level correction is not demonstrated. The only motion-accuracy experiment (Table 4) is declared invalid by the authors themselves. The validation frames used for reporting are also used for checkpoint selection, which may make the numbers optimistic. Finally, the paper contains excessive caveats, keeps archived or unreconciled tables in the main text, and defers key details to the appendix, which makes it difficult to understand what the paper actually claims.

Questions for authors
  1. Under the same frames and time budget, how does the proposed adaptation compare with simple post-optimization of the predicted Gaussians (as in AnySplat or NoPo4D)? How does it compare with fine-tuning only StreamSplat’s dynamic branch without the adapter?

  2. What does the motion ODE contribute when the dynamic branch is also updated? How does ODE integration compare with directly predicting a time-dependent residual?

  3. Can the method use trajectory information across multiple frames or intervals instead of two-frame linear motion, and how would this affect the results?

  4. Under the same input and evaluation protocol, how does the method compare with feed-forward 4D methods, optimization-based dynamic Gaussian methods, and frame interpolation methods?

  5. Can you provide evaluations beyond same-view PSNR, such as novel-viewpoint rendering, SSIM/LPIPS, or a valid motion-accuracy evaluation? Can you also report results on a held-out test split that is not used for checkpoint selection?

  6. Were the frames used for extrapolation in Table 5 excluded from adaptation training? Why are the archived and unreconciled results (Tables 1 and 4) kept in the main paper?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.