acceptodds
Withdrawn from ICLR 2027

Motion-MoE-GS: Decomposing Local Motion Laws for Dynamic 3D Gaussian Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Complex object motion is locally coherent but globally heterogeneous: units subject to the same structural constraints, part relations, or interactions share a local motion law, while different regions can obey different laws. Existing dynamic 3D Gaussian methods commonly decode all primitive updates with one shared time-conditioned function. Conditioning this function on location, appearance, or a global dynamic state changes its input, but does not allocate different motion laws to different primitives. We propose Motion-MoE-GS, which casts dynamic 3D Gaussian reconstruction as conditional decomposition and allocation of local motion laws. The method evolves a continuous scene-level dynamic state with a Neural ODE and combines it with each Gaussian's static state to form a local motion descriptor. A motion-conditioned router sparsely selects two of four deformation experts. Each expert represents a learned local motion law and predicts position, rotation, and scale updates. Per-scene evaluations on NVFI-Obj and DynMultiParts support conditional motion-law decomposition as an effective inductive bias for heterogeneous 3D Gaussian dynamics.

Primary area:
generative models
Keywords:
3dgs, MoE, Motion-Aware

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vKPiOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the problem of dynamic 3d gaussian reconstruction. This paper proposes Motion-MoE-GS, a dynamic 3D Gaussian method for reconstruction. The method can also perform motion extrapolation. It uses a Neural ODE to evolve a scene-level dynamic state, which each Gaussian combines with its own local feature. They introduce a router that selects the top 2 deformation experts to predict delta position, rotation, and scale. The paper considers these experts as reusable local motion laws and argues that decoding all deformations with one shared network is suboptimal. On the NVFI-Obj and DynMultiParts datasets, they report improved PSNR, SSIM, and LPIPS on most scenes, for both reconstruction and extrapolation. They provide ablations on expert count, routing, conditioning, and the ODE.

Critical strengths
  1. clear and intuitive motivation. The idea that motion is locally coherent but globally heterogeneous is easy to understand. Replacing a single shared deformation decoder with a small bank of experts for different motions is a straightforward method following this idea.

  2. Reasonable method design. The router is conditioned on both the Gaussian's local feature and the Neural ODE's global state.

Critical weaknesses
  1. Per-Gaussian Top-K gating over multiple deformation experts, composed as weighted attribute offsets on shared canonical Gaussians, is already the central design of MoDE and MoE-GS in [1]. However, the authors do not compare against them. The remaining differences are an ODE-conditioned router, homogeneous experts, and an extrapolation focus. Experiments should separate these from the prior design, for example by swapping in a MoDE-style gate on the same backbone. Moreover, some related prior works are missing. ODE-GS [2] already argues that time-conditioned deformation fails out of distribution and uses a latent ODE for extrapolation, and MoGaF [3] also decomposes motion explicitly for dynamic scene forecasting.

[1] On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting.

[2] ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting

[3] Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Grouping.

  1. The idea of decomposing local motion laws is intuitive and reasonable. However, the paper does not clearly (e.g., quantitatively) demonstrate that the experts effectively specialize in different motion regimes. Moreover, the definition of the balance loss is not given in the paper. The experiments show that "Removing the balance loss likewise reduces extrapolation PSNR by 0.4107 dB and 3.2910 dB, respectively, and degrades the remaining extrapolation metrics". A deeper analysis of the reason behind this result would be very helpful to understand how the method works.
Questions for authors
  1. As shown in Tables 1 and 2, LPIPS is worse than the baselines in many scenes. Could the authors explain these results? Also, providing video demos would help assess the actual quality of the results.

  2. Evaluation covers only two synthetic datasets. Experiments on more datasets, as in ParticleGS, would improve the evaluation.

Reviewer_HiZJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Motion-MoE-GS is a per-scene dynamic 3D Gaussian reconstruction and extrapolation method. It encodes each canonical Gaussian into a static local descriptor and aggregates Gaussian neighborhoods into an initial scene-level state. A Neural ODE evolves this shared state through time. For each Gaussian and time, a router uses the local descriptor and evolved global state to select the top two of four homogeneous deformation MLPs; their weighted outputs update position, rotation, and scale. Training uses rendering supervision and an expert-balancing term. The method is evaluated on NVFI-Obj and DynMultiParts using published baseline results and a set of controlled ablations.

Critical strengths
  1. The paper tests a clear architectural hypothesis with relevant controlled ablations. Under fixed initialization and training settings, it varies expert count, Top-1/Top-2/dense routing, local and global conditioning, the ODE evolver, and load balancing. The full model improves extrapolation PSNR over both a single expert and dense routing on both datasets, so the reported benefit is not attributable solely to adding an ODE or increasing the number of decoder branches.

  2. The empirical reporting is comparatively transparent. The paper gives per-scene PSNR, SSIM, and LPIPS rather than only an aggregate, obtains the best extrapolation PSNR on all six NVFI-Obj scenes and three of four DynMultiParts scenes, and acknowledges both the fallingball failure mode and cases where perceptual metrics favor another method. This makes the scope of the gain easier to assess

Critical weaknesses
  1. The central “local motion law” interpretation is not demonstrated by the formulation or experiments. The Neural ODE evolves only the shared scene state; each expert is a feed-forward decoder that maps a static Gaussian descriptor and that shared state directly to a canonical-to-time deformation. There is no expert-specific local state equation, velocity, history, or integration of an individual Gaussian trajectory. In a per-scene fit to one observed trajectory, the router can therefore learn a scene-specific spatial-temporal partition while the experts memorize different deformation regions.

  2. The incremental contribution and source of the gain are insufficiently isolated. ParticleGS already combines per-Gaussian static features, a scene-level Neural ODE state, and a shared Gaussian deformation decoder; Motion-MoE-GS mainly replaces that decoder with a sparse Top-2 bank. More importantly, the cited MoDE work already jointly optimizes multiple candidate deformation experts over a shared canonical Gaussian set and uses per-Gaussian temporally varying sparse gating. Homogeneous experts and ODE-conditioned routing are genuine differences, but the paper neither compares with MoDE nor includes a clearly parameter- and compute-matched shared or multi-head decoder.

Questions for authors
  1. Does each expert predict an independent absolute deformation from the canonical Gaussian at every queried time, while only the global state is integrated by the ODE? If so, what operational criterion makes an expert a motion law rather than a time-conditioned decoder?
  2. How does the method compare with the jointly trained MoDE variant of Jin et al. under the same data and protocol?
  3. Is the initial ODE state computed only from the canonical Gaussian parameters, and are all encoder, ODE, router, and expert weights optimized separately for each scene? If so, where are initial motion direction and phase represented, and how could the model distinguish two sequences with the same canonical geometry but different velocities?
Reviewer_FVJaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Motion-MoE-GS, a dynamic 3D Gaussian Splatting method for reconstruction and future-frame extrapolation. It builds on the observation that object motion is locally coherent but globally heterogeneous, and aims to replace the shared deformation decoder of prior work with a set of reusable local motion experts. An encoder summarizes the canonical Gaussians into a scene-level state, which a Neural ODE evolves continuously over time. For each Gaussian, a router conditioned on its local descriptor and the evolved global state selects two of four deformation experts, whose weighted outputs update position, rotation, and scale. Experiments on NVFI-Obj and DynMultiParts report improved reconstruction and extrapolation over prior dynamic Gaussian and dynamics-aware methods.

Critical strengths
  1. The idea of treating dynamic Gaussian deformation as conditionally allocating a small set of reusable local motion patterns through expert routing is interesting. It is well motivated by the observation that object motion is locally coherent but globally heterogeneous.
  2. The per-scene tables report all metrics, including cases where the method is not the best, and the paper explicitly discusses its limitations.
Critical weaknesses
  1. The central claim is that the experts specialize in reusable local motion laws, but no analysis is provided to support this interpretation. Such analysis could include visualizations of routing assignments, their correspondence to object parts, or functional differences between experts. From the reported metrics and ablations alone, it is not possible to determine whether the improvement arises from motion-law decomposition or simply from the increased capacity or ensembling effect of the mixture.
  2. The reconstruction protocol is not clearly specified. In the original Dynamic Multipart protocol [1], the 552 frames form the training split, and novel-view evaluation is performed on 138 separate frames from three held-out views. The paper, however, states that “reconstruction uses 552 training frames,” and does not state whether held-out views are used for NVFI-Obj either. Notably, the reconstruction PSNR exceeds that of ParticleGS by 7.1–16.6 dB on three DynMultiParts scenes in Table 2, and even the single-decoder variant (E = 1) is about 6 dB higher on average. This suggests that these gains may stem from differences in training or evaluation conditions rather than from the MoE design.
  3. The evaluation is limited to synthetic object-level scenes, and the real-world PanopticSports dataset used in the ParticleGS evaluation is omitted. Results on a standard real-world benchmark such as N3DV [2] would help assess the generality of the approach.
  4. Although temporal consistency during extrapolation is presented as a central motivation and contribution, no video results are provided. Static frame comparisons and a trajectory visualization of a single scene are insufficient to assess flickering or temporal discontinuities. In addition, LPIPS is mostly worse than that of ParticleGS in extrapolation on both datasets, but the paper only describes this as a “perceptual trade-off” without further analysis. Since PSNR can remain high for roughly aligned but blurry renderings, while LPIPS is sensitive to the loss of boundaries and fine detail, this result is inconsistent with the claimed sharper boundaries in Figs. 3,4.
  5. Implementation details for several key components are missing. The paper states only that z0z_0 is obtained by aggregating KNN groups with cross- and self-attention, without describing the architecture. The structure of F_ODE, the solver used for ODESolve, and the role of the starting time −0.01 in Eq. 1 are also not specified. Most importantly, the formulation of the load-balancing loss, its weight λbal\lambda_{bal}, and the scheme it follows are not given, even though removing this loss reduces the NVFI-Obj extrapolation PSNR by 3.29 dB.

[1] Li et al. TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos. ICCV 2025 [2] Li et al. Neural 3D Video Synthesis from Multi-view Video. CVPR 2022

Questions for authors
  1. If possible, could the authors provide evidence that the experts specialize, such as visualizations or videos of routing assignments over time?
  2. Please clarify which views and frames were used for reconstruction evaluation on each dataset, in particular whether the held-out test views of the original protocol were used.
  3. Could the authors evaluate on real-world data (e.g., PanopticSports as in ParticleGS)? Reconstruction results on a standard real-world benchmark such as N3DV would also show whether the MoE decoder generalizes to real scenes.
  4. Could the authors provide extrapolation videos, ideally side by side with ParticleGS, and explain why LPIPS is mostly worse than that of ParticleGS in extrapolation despite higher PSNR?
  5. Please provide complete implementation details (e.g., architectures, loss formulations, and training and ODE settings).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.