acceptodds
Withdrawn from ICLR 2027

InstanceFlow-SLAM: Instance-Guided Continuous Motion Reconstruction for Dynamic Gaussian SLAM

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Keyframe-based Gaussian SLAM leaves dynamic motion between map updates underconstrained, while increasing the mapping frequency raises computational cost. We introduce InstanceFlow-SLAM, which reconstructs continuous motion from intermediate RGB-D observations without additional Gaussian mapping up- dates. Instance identity restricts motion sharing among Gaussians, observations, and nodes, while adaptive node allocation captures local motion variation within each object. Continuous SE(3) B-splines represent node motion and are fitted to image tracks and depth-supported 3D observations. Reliable observations deter- mine each trajectory’s temporal support, preventing unsupported endpoints from distorting the fitted motion. After keyframe mapping, boundary-consistent mo- tion transfer preserves the recovered interior variation while aligning it with up- dated Gaussian states. Experiments on TUM RGB-D and Bonn demonstrate im- proved dynamic reconstruction with competitive camera localization. On Bonn, our method achieves the best average full-image reconstruction scores and the lowest average trajectory error among the compared methods, improving average dynamic-region PSNR by 1.44 dB over Flow4DGS-SLAM.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Dynamic Gaussian SLAM, Dynamic Scene Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QBWrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents InstanceFlow-SLAM, an RGB-D Gaussian SLAM system that reconstructs nonlinear object motion between keyframes while retaining the underlying keyframe mapping pipeline. It associates dynamic Gaussians, observations, and motion nodes within object instances, allocates nodes according to observed motion variation, and fits continuous SE(3) B-spline motion to image tracks and depth-supported 3D observations. Observation-supported endpoints limit unreliable trajectory extrapolation, and a residual transfer step preserves fitted interior motion after keyframe mapping changes the Gaussian states. Experiments on TUM RGB-D and Bonn evaluate rendering quality, camera localization, and runtime. The strongest reported result is a 1.44 dB improvement in average dynamic-region PSNR over Flow4DGS-SLAM on Bonn.

Critical strengths
  1. A coherent solution to motion reconstruction between mapping updates.

    The paper addresses a concrete limitation of keyframe-based dynamic Gaussian mapping: accurate keyframe states can still miss motion reversal and other nonlinear variation inside a window. Combining instance-constrained motion sharing, observation-fitted SE(3) splines, and supported trajectory endpoints is well motivated. The position transfer in Eqs. (7)–(8), further explained in Appendix A.6, provides a transparent way to preserve the fitted non-affine motion component while matching updated map boundaries. This is a useful design for reconstructing moving content without adding Gaussian mapping updates.

  2. Consistent dynamic-region gains on Bonn, supported by a relevant controlled ablation.

    Table 5 reports improved dynamic-region PSNR on all nine Bonn sequences, with the reported average increasing from 24.34 to 25.78 dB. Average full-image PSNR improves from 26.33 to 26.63 dB, while average camera ATE decreases from 3.9 to 3.7 cm (Tables 3 and 6). Table 8 also includes a useful ablation with fixed observation inputs and mapping budget: the full model improves PSNR by 1.155 dB over the linear variant on Bonn balloon. These results support the value of recovering nonlinear motion. The specific failure analysis in Appendix A.7.1 also helps delineate the system’s limitations.

Critical weaknesses

Critical Weaknesses

  1. Insufficient methodological novelty and inadequate technical presentation.

The proposed framework relies heavily on integrating existing components, including Flow4DGS-SLAM, pretrained instance segmentation and tracking models, SE(3) B-splines, and dual-quaternion blending. Although the authors introduce instance-constrained motion fitting and boundary-consistent transfer, the experiments do not convincingly isolate their contributions from the stronger segmentation and tracking pipeline.

Figure 2 is visually elaborate but does not sufficiently clarify the computational dependencies and implementation details of the proposed modules. A more rigorous technical diagram would improve the reproducibility and transparency of the method.

  1. Reconstruction quality and empirical improvements are insufficiently compelling.

The reconstruction results do not convincingly demonstrate a sufficiently substantial advance over existing methods. On TUM RGB-D, average PSNR improves by only 0.07 dB, while LPIPS becomes slightly worse. On Bonn, the average full-image PSNR improvement is only 0.30 dB.

Although the reported dynamic-region improvement on Bonn is more substantial, the qualitative results still reveal visible reconstruction artifacts. Figure 7 additionally demonstrates a failure involving overlapping static and dynamic geometry, producing a doubled and blurred reconstruction.

More importantly, these limited overall gains come with a substantial computational penalty: throughput decreases from 0.51 to 0.34 FPS. The paper does not convincingly demonstrate that the proposed architecture provides a sufficiently meaningful improvement in reconstruction quality, robustness, or computational efficiency.

Questions for authors
  1. Controlled contribution assessment.

    Could you compare against Flow4DGS-SLAM using the same dynamic masks and mapping schedule on a small representative subset? Extending the existing fixed-input linear-versus-continuous ablation to a different motion pattern, such as a box or TUM walking sequence, would also help establish whether the result in Table 8 generalizes.

  2. Ablation evaluation protocol.

    Which frames, image regions, and aggregation rules are used in Table 8? Its Full result is 26.844 PSNR / 0.8032 SSIM / 0.0806 LPIPS, whereas Table 3 reports 26.50 / 0.892 / 0.207 for Bonn balloon and Table 5 reports dynamic-region PSNR of 26.65. Please clarify whether these results use different subsets or evaluation settings.

  3. Computation and output delay.

    Could you provide a targeted quality-versus-total-runtime comparison with a baseline that uses more frequent mapping, and specify the hardware and per-frame/per-keyframe/per-window units in Table 7, including the accounting for Track-On-R and SAM2? Please also distinguish the two-frame observation look-ahead from the delay in producing a finalized reconstruction, since Algorithm 1 fits motion after the keyframe window closes.

Reviewer_Jd5rOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This manuscript presents a pipeline to map a dynamic scene as a set of dynamic 3D Gaussians given streaming RGB-D inputs. It relies on point tracking and segmentation to partition 2D inputs for constructing instance-specific constraints during optimization. The kinematics of the 3D Gaussians are modeled by cubic SE(3) B-spline. Experiments on TUM RGB-D and Bonn demonstrate improved dynamic reconstruction with competitive camera localization.

Critical strengths
  1. Higher PSNR (+1.4 dB) compared to Flow4DGS-SLAM on dynamic regions.
  2. Adaptive motion node allocation for non-rigid objects.
Critical weaknesses
  1. Heavily relies on the performance segmentation and point tracking.
  2. low FPS.
  3. Lack of analysis on critical hyperparameters, including the number of B-spline nodes of Gaussians, the length of intervals, etc.
  4. Lack of details on keyframe selection.
Questions for authors
  1. How to determine keyframes?

  2. When to add nodes for the B-spline of the 3D Gaussians?

  3. What are the values of a and b of the interval [a, b]? The above questions are coupled and would affect the system's performance.

  4. Is the rendering fidelity (PSNR/SSIM) only measured on training views? How about the results of novel view rendering?

  5. Are the hyperparameters the same for all the sequences?

Reviewer_hvfiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses motion reconstruction between Gaussian SLAM keyframes. Building on Flow4DGS-SLAM, it uses instance masks to group dynamic Gaussians and fits local SE(3) spline motion to point tracks and depth observations from intermediate RGB-D frames. A residual transfer step then adjusts the fitted trajectories to the Gaussian states produced by keyframe mapping. The method improves dynamic-region PSNR on the Bonn sequences, while its average full-image gains are small on both Bonn and TUM RGB-D. Camera trajectory accuracy changes little, and runtime is lower than the direct baseline.

Critical strengths
  1. The method uses intermediate observations without requiring another Gaussian mapping update for every frame. Instance-local motion and the boundary transfer step fit together: the former constrains which observations can influence a Gaussian, while the latter keeps the fitted motion compatible with updated keyframe states.

  2. Dynamic-region PSNR improves on all nine Bonn sequences. The motion-reversal example also helps show the type of error that linear propagation can make. I see these as evidence that the approach can help in its intended setting, though the broader gains are modest.

Critical weaknesses
  1. The added complexity has a limited effect on the overall results. Average full-image PSNR changes from 24.30 to 24.37 dB on TUM and from 26.33 to 26.63 dB on Bonn; TUM LPIPS becomes slightly worse. The 1.44 dB dynamic-region gain on Bonn is much larger than the 0.35 dB gain on TUM. Average camera error is essentially unchanged, while throughput falls from 0.51 to 0.34 FPS. I am unsure that the reported improvement justifies this cost.

  2. It is also hard to tell which parts of the system account for the gains. Table 8 tests the components on only one Bonn sequence, and its “Full” numbers do not match the corresponding results in Tables 3 or 5 without an explanation of the evaluation protocol. Results across the other sequences, with masks, tracks, and keyframe schedules held fixed, would make the case for continuous motion fitting more convincing.

Questions for authors
  1. What is the evaluation protocol for Table 8? Its “Full” results differ from the balloon results in Tables 3 and 5, so I cannot directly relate the ablation to the main comparison.

  2. Could you report the linear-motion and other component ablations on more than one sequence, keeping the masks, tracks, and keyframe schedule fixed? A breakdown of non-keyframe quality by distance from a keyframe would help establish where continuous fitting provides the reported gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.