acceptodds
Under review as a conference paper at ICLR 2027

Dynamic Vision Optical Flow-Guided Gaussian Splatting for Dynamic Scene Reconstruction

Abstract

Deformation-based dynamic Gaussian Splatting models every point of a monocular video with a single deformation field, whether the point is static, part of a rigid object, or part of a fluid. We found that such deformation fields cause rapidly moving objects to lose pattern details or become blurred, rather than exhibiting a clear motion trajectory. Existing event-based methods such as E-4DGS and Event-boosted D3DGS utilize events for photometric loss, which requires multiple rendering passes during training, whereas we use an event camera to decompose the scene. We estimate dynamic vision optical flow (DVOF) from events recorded by a dynamic vision sensor using contrast maximization, and derive an observability matrix from the contrast curvature to make the aperture problem explicit. Patches are grouped by whether a single affine motion explains them along their observable directions, and the groups are linked into tracks over time. This divides the scene into static, rigid, and free regions. Gaussians are assigned to the regions by rendering-weight voting, and consistency-checked spatial completion recovers event-sparse object interiors. Static Gaussians optimize time-invariant geometry and appearance, rigid ones follow explicit rigid trajectories supervised by the flow, and free ones keep a deformation field. On real RGB-event sequences, our method improves quality of full image and dynamic region over deformation-based and event-based methods, and improves the consistency of image-plane motion between rendered and observed sequences.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
dynamic gaussian splatting, event camera, monocular dynamic scene reconstruction, motion decomposition

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_yrbDOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an event-guided Gaussian Splatting framework for dynamic scene reconstruction, addressing motion inconsistencies and blurred details caused by shared deformation fields, particularly for fast-moving objects. The method estimates dynamic vision optical flow from event streams and uses an observability-aware motion grouping strategy to identify regions with shared motion patterns. It assigns Gaussians to static, rigid, or freely deforming branches through rendering-weight voting, while introducing spatial completion and track-guided seeding to handle event-sparse object interiors and underrepresented transient regions.

Critical strengths
  1. The paper clearly identifies motion inconsistency and detail loss in deformation-based dynamic Gaussian Splatting, motivating the use of event-guided motion constraints.

  2. The paper presents the proposed method in a coherent sequence, from event-based optical flow estimation and motion grouping to Gaussian assignment and reconstruction.

Critical weaknesses
  1. Since the proposed method groups Gaussians by shared motion, comparisons with anchor-based or motion-sharing Gaussian methods would help clarify the benefits of its observability-aware grouping and event-derived motion supervision.

  2. Reporting stage-wise runtime and peak GPU memory usage would clarify the computational costs of event preprocessing, motion grouping, and reconstruction.

  3. The reliance on synchronized RGB frames and event-camera data introduces additional hardware and synchronization requirements. Discussing the trade-off between reconstruction quality and these requirements would clarify practical applicability.

Questions for authors
  1. How does the proposed observability-aware motion grouping differ from existing anchor-based or motion-sharing Gaussian representations in enforcing coherent object motion?

  2. Which stages contribute most to the overall computational cost, and could the authors provide a breakdown of runtime and peak GPU memory usage?

  3. In what scenarios does the use of event-camera data provide sufficient benefits to justify the additional hardware and synchronization requirements compared with RGB-only approaches?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors

Q1. "How does the proposed observability-aware motion grouping differ from existing anchor-based or motion-sharing Gaussian representations in enforcing coherent object motion?"

A1. Thank you for this question. Existing anchor-based and motion-sharing representations already allow multiple Gaussians to share motion parameters. We do not claim shared transformations themselves as our novelty. Our contribution concerns how motion groups are constructed from observed event evidence and how that evidence constrains their shared motion.

We have added SpeeDe3DGS with GroupFlow enabled as a representative motion-sharing baseline. Across our ten scenes, its mean dynamic-region PSNR is 22.59 dB, compared with 26.67 dB for our method; its mean rendered-flow EPE is 9.337 px, compared with 6.244 px. We complement it with the H=IH=I ablations to examine the roles of directional observability and event-derived motion evidence within our pipeline.

MethodFull image PSNR ↑Full image SSIM ↑Full image LPIPS ↓Dynamic region PSNR ↑Dynamic region SSIM ↑Dynamic region LPIPS ↓
SpeeDe3DGS28.440.83920.279322.590.82470.2416
Ours w/ H = I30.240.85760.255324.820.85360.2104
Ours w/ optical flow30.350.85760.259124.920.85520.2125
Ours w/ event flow30.950.86490.229026.670.89960.1608

Q2. "Which stages contribute most to the overall computational cost, and could the authors provide a breakdown of runtime and peak GPU memory usage?"

A2. Thank you for pointing out the missing computational analysis. We have conducted a stage-wise profiling of runtime and peak GPU memory usage and added the results to the Tab.11 in appendix. The breakdown is presented below.

StageTimePeak allocated (GiB)Peak reserved (GiB)
Event coherence estimation00:540.3070.342
Event velocity and Hessian estimation00:150.3150.361
Affine grouping, temporal association, and completion00:02N/A—CPUN/A—CPU
Reconstruction initialization00:060.8261.037
Initial training stage6:142.2344.297
Gaussian assignment, seeding, and transform initialization00:501.0862.010
Joint training stage13:021.6052.010
Final testing and checkpoint saving00:072.7063.455

Q3. "In what scenarios does the use of event-camera data provide sufficient benefits to justify the additional hardware and synchronization requirements compared with RGB-only approaches?"

A3. Thank you for raising this practical trade-off. We consider event sensing most valuable when object motion is insufficiently constrained by RGB observations at the available frame rate and exposure, particularly for rapid rotation and short-lived motion. Asynchronous events provide additional motion evidence between RGB frames, while RGB supplies appearance information.

These benefits come with additional sensor, spatial-calibration, and timestamp-synchronization requirements. We therefore view the method as most applicable to calibrated RGB–event acquisition systems where recovering fast motion is important. For slow motion with sufficiently sampled, sharp RGB observations, the additional hardware may offer less benefit. Beyond the methodology itself, event cameras also offer the advantage of low power consumption—though this is a factor that would only be considered during actual deployment.

The authors have adequately addressed my concerns, and I will raise my rating.

Dear Reviewer yrbD,

We sincerely appreciate your valuable suggestions. Thank you for your time and effort in reviewing our paper.

Best regards,

The Authors

Reviewer_bmCEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an event-guided dynamic Gaussian Splatting framework for reconstructing dynamic scenes from monocular RGB images and event streams. It estimates event-based optical flow using contrast maximization and derives an observability matrix to group patches with consistent affine motion. The resulting motion groups are assigned to Gaussians, which are modeled using static, shared-rigid, or free-deformation branches. Track-guided Gaussian seeding is also introduced to improve the reconstruction of transient objects. The method reports improvements in dynamic-region reconstruction quality and rendered motion consistency.

Critical strengths

The paper addresses a meaningful limitation of deformation-based dynamic Gaussian Splatting, where independently deforming Gaussians may fail to preserve coherent object motion. Using directional observability to handle the aperture problem is well motivated, and combining event-based motion grouping with shared rigid transformations is an interesting approach. The experiments demonstrate improvements in dynamic-region PSNR and LPIPS over several baselines. The paper also includes component ablations and reports training time, rendering speed, and Gaussian counts.

Critical weaknesses
  1. The comparison with closely related methods is incomplete. DEGS [1] also uses event-derived optical flow to guide dynamic Gaussian reconstruction but is omitted from the quantitative evaluation. Without this comparison, it is difficult to assess the benefit of the proposed motion grouping over existing event-guided deformation methods. A comparison under matched settings, or an explanation of why it is infeasible, is needed.
  2. The contribution of directional observability is not sufficiently isolated. Table 3 evaluates observability weighting for motion grouping, but Table 4 does not show its effect on final reconstruction quality independently of affine grouping and shared rigid motion. A controlled ablation comparing identity weighting (H = I) with the proposed Hessian-based weighting would better support the central contribution.
  3. Motion grouping accuracy is not independently validated. Table 3 evaluates grouping consistency against reference regions generated by the full method rather than ground-truth motion labels. Moreover, 2D affine consistency does not necessarily imply 3D rigidity. Without independent validation, it remains unclear whether the estimated groups correspond to actual object motion.

MoRGS [2] and SpeeDe3DGS [3] also explore explicit Gaussian motion modeling and groupwise rigid transformations, respectively. The authors should discuss these methods to better clarify the novelty of the proposed approach.

[1] DEGS: DEGS: Deformable Event-based 3D Gaussian Splatting from RGB and Event Stream. IEEE TVCG 2025.

[2] MoRGS: Efficient Per-Gaussian Motion Reasoning for Streamable Dynamic 3D Scenes. CVPR 2026.

[3] SpeeDe3DGS: Speedy Deformable 3D Gaussian Splatting with Temporal Pruning and Motion Grouping. CVPR 2026.

Questions for authors
  1. How does the method compare with DEGS under comparable RGB-event settings? What benefit comes specifically from observability-aware motion grouping rather than direct event-flow supervision?
  2. Can you report a reconstruction ablation comparing affine grouping with H = I and the proposed Hessian-based observability weighting, keeping all other components unchanged?
  3. Can you provide an independent evaluation of motion grouping accuracy? How does the method handle nonrigid regions that exhibit locally consistent 2D affine motion?
3 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part 1)

We thank Reviewer bmCE for the constructive feedback and insightful suggestions to improve our work.

Q1. "How does the method compare with DEGS under comparable RGB-event settings? What benefit comes specifically from observability-aware motion grouping rather than direct event-flow supervision?"

A1. Thank you for highlighting this closely related work. DEGS is already discussed in our Introduction and Related Work. Since DEGS is not open-source yet, we could not locate a public official implementation of DEGS in our current check, so a verified comparison under our evaluation protocol is currently unavailable. Its published results have not been established to use our exact sequence intervals, RGB sampling, held-out split, and metric implementation, and therefore do not establish a matched comparison.

DEGS adapts a pretrained event-flow estimator and employs geometry-aware event–Gaussian association to supervise deformation. In contrast, our method estimates patchwise motion directly through event alignment and uses directional observability and affine consistency to construct shared motion fields. These fields subsequently guide the assignment of Gaussians to static, shared rigid, or free deformation branches.

The key distinction is therefore not merely the use of event-derived motion, but how the reliability of this motion evidence determines the representation and the extent of motion sharing.

Q2. "Can you report a reconstruction ablation comparing affine grouping with H = I and the proposed Hessian-based observability weighting, keeping all other components unchanged?"

A2. Thank you for pointing out the current shortcomings in the ablation studies. We have now completed a H=I ablation under the same event flow, replacing H while keeping the other components and their rules unchanged.

MethodFull image PSNR ↑Full image SSIM ↑Full image LPIPS ↓Dynamic region PSNR ↑Dynamic region SSIM ↑Dynamic region LPIPS ↓
D-3DGS29.430.84990.266623.410.84940.2152
Ours w/o motion grouping29.270.85410.259923.380.83010.2292
Ours w/ H = I30.240.85760.255324.820.85360.2104
Ours w/o spatial completion30.310.85560.260824.350.84480.2153
Ours w/o seeding and visibility29.880.85690.241425.560.86980.2079
Ours w/o seeding30.230.85790.255324.780.85370.2102
Ours w/ optical flow30.350.85760.259124.920.85520.2125
Ours w/ event flow30.950.86490.229026.670.89960.1608

We have incorporated this new ablation and the corresponding discussion into the revised manuscript.

AuthorsOpenReview ↗

Official Comment by Authors (Part 2)

Q3. “Can you provide an independent evaluation of motion grouping accuracy? How does the method handle nonrigid regions that exhibit locally consistent 2D affine motion?”

A3. Thank you for identifying the limitations of our original grouping evaluation. We agree that the reference regions in the original Table 3 are not independent ground truth. We have therefore evaluated the same four grouping variants against manually annotated rigid-object masks on five held-out RGB frames per scene.

SceneGroupingGroups ↓Cover (%) ↑IoU (%) ↑Persist (%) ↑Shared
UmbrellaVelocity similarity3775.57 ± 0.3132.56 ± 3.14×
+ Affine consistency78470.15 ± 0.8467.00 ± 7.25✓
+ Observability H49075.31 ± 1.9887.33 ± 2.33✓
+ Spatial completion110084.10 ± 0.5287.33 ± 2.33✓
SchaffVelocity similarity38555.03 ± 2.1841.41 ± 2.86×
+ Affine consistency38464.44 ± 0.7245.42 ± 6.55×
+ Observability H19569.93 ± 1.8681.78 ± 2.69✓
+ Spatial completion110080.61 ± 2.1081.78 ± 2.69✓
BellevueVelocity similarity5.54929.99 ± 3.2039.03 ± 7.39×
+ Affine consistency38636.50 ± 3.6228.10 ± 3.79×
+ Observability H19560.24 ± 1.8952.12 ± 3.32✓
+ Spatial completion110073.82 ± 2.5152.12 ± 3.32✓

We have added this independent mask-based evaluation to the revised manuscript. Our tests evaluate the measurement support for affine or translational motion parameters over a track. They do not constitute a test of physical 3D rigidity. Regions without a qualifying track retain the free-deformation branch, but a nonrigid region with persistent, sufficiently observed affine image motion can still pass these tests and be assigned an inappropriate shared transform. This is a limitation of the model-selection heuristic.

W4. "MoRGS and SpeeDe3DGS also explore explicit Gaussian motion modeling and groupwise rigid transformations, respectively. The authors should discuss these methods to better clarify the novelty of the proposed approach."

A4. These are relevant works that deserve discussion. MoRGS addresses online reconstruction from streaming multi-view RGB inputs, using sparse-view optical flow, per-Gaussian motion offsets, and motion confidence. SpeeDe3DGS uses GroupFlow to distill learned Gaussian trajectories into shared SE(3) transformations for efficiency. Our work also shares motion within groups; the intended contribution is the use of event-derived directional constraints to construct motion groups and select motion models in fixed-camera monocular RGB-event reconstruction. Shared rigid transformations, which is employed by many existing methods, are not our novelty claim.

MoRGS targets online reconstruction from synchronized multi-view RGB streams and evaluates held-out camera views, whereas our protocol uses monocular RGB-event input. Its released implementation does not directly support our evaluation protocol.

Since SpeeDe3DGS explicitly implements groupwise rigid motion sharing, we consider it a particularly relevant comparison method. We have therefore added quantitative and qualitative comparison with SpeeDe3DGS to our experiments. Thanks again.

MethodFull image PSNR ↑Full image SSIM ↑Full image LPIPS ↓Dynamic region PSNR ↑Dynamic region SSIM ↑Dynamic region LPIPS ↓
SpeeDe3DGS28.440.83920.279322.590.82470.2416
Ours w/ optical flow30.350.85760.259124.920.85520.2125
Ours w/ event flow30.950.86490.229026.670.89960.1608

Dear Reviewer bmCE,

Thank you again for your feedback. We would appreciate any follow-up on our responses, particularly whether the added H=I ablation, independent manual-mask grouping evaluation, and SpeeDe3DGS comparison address your concerns.

If any issues remain unresolved, we would be happy to provide further clarification before the discussion period closes. Even a brief indication of the remaining concerns would be very helpful.

Best regards,

The Authors

Comments on the paper

6 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part 1)

Q1. "Does Eq. (14) use directional observability beyond constructing its target? Why should unobservable components receive the same supervision weight?"

A1. Thank you for raising this point. Eq. (14) uses an unweighted Euclidean residual; directional observability enters the preceding grouping, field fitting, and motion-model selection stages.

The reason for this design is that the supervision target is a shared motion field estimated from multiple patches. A direction that is unobservable at one edge can be constrained by measurements at other positions and edge orientations within the same group. We therefore use Eq. (14) to encourage the projected Gaussian motion to follow this group-level estimate, rather than directly enforcing each patch’s ambiguous raw flow.

We nevertheless agree that group-level selection does not guarantee equal directional confidence in every window. The full-vector loss should be understood as a shared-motion prior, with weakly constrained components potentially influenced by the model assumption and regularization. We will clarify this distinction in the paper.

Q2. "Is there a motion or geometry check independent of GMFlow on the same camera images? Otherwise, please narrow the geometry claim."

A2. Thank you for highlighting this concern.

We agree that Balloon contributes substantially to the average improvement. Recomputing the averages from the per-scene values in Table 8, excluding Balloon, gives an EPE of 5.243 px for D-3DGS versus 4.324 px for our method. Our method improves over D-3DGS in eight of ten scenes, or seven of nine after excluding Balloon. Thus, the improvement is not confined to Balloon, although its contribution to the overall mean is substantial.

We also agree that Rendered-flow EPE is intended as a complementary diagnostic of image-plane motion agreement. Both the reference and rendered flows are estimated by GMFlow, so the metric can be affected by appearance quality and does not independently validate metric depth or physical 3D trajectories. We will make this limitation explicit and consistently restrict our interpretation to motion agreement in the observed image plane.

Q3. "Can the seeding ablation retain the same opacity schedule, so depth seeding and visibility changes are not removed together?"

A3. Thank you for identifying this limitation in our previous ablation design. We have renamed the original ablation as "Ours w/o seeding and visibility" to clarify that it removes both depth seeding and the visibility-related schedule. We have additionally conducted a new "Ours w/o seeding" experiment, in which depth seeding is disabled while the visibility schedule is retained as in the full model.

MethodFull image PSNR ↑Full image SSIM ↑Full image LPIPS ↓Dynamic region PSNR ↑Dynamic region SSIM ↑Dynamic region LPIPS ↓
Ours w/o seeding and visibility29.880.85690.241425.560.86980.2079
Ours w/o seeding30.230.85790.255324.780.85370.2102
Ours w/ optical flow30.350.85760.259124.920.85520.2125
Ours w/ event flow30.950.86490.229026.670.89960.1608

We have updated the ablation table, variant names, and corresponding discussion in the revised manuscript.

AuthorsOpenReview ↗

Official Comment by Authors (Part 2)

Q4. "What are the repeat count and dispersion for the full model and grouping ablation, and the interval-selection and threshold settings?"

A4. Thank you for pointing out these omissions and the limitations of our original interval selection. We have expanded the experimental details in the main paper and appendix to clarify the repeated-evaluation protocol, the dispersion statistics, and the threshold (ρ\rho=1.15).

We trained 3 times for each of them.

MethodRunsStatisticFull image PSNR ↑Full image SSIM ↑Full image LPIPS ↓Dynamic region PSNR ↑Dynamic region SSIM ↑Dynamic region LPIPS ↓
D-3DGS3Mean29.430.84990.260623.410.84940.2152
Std. dev.0.290.00310.00340.270.00500.0052
MotionGS3Mean29.680.86110.237623.570.84520.2128
Std. dev.0.050.00060.00180.050.00070.0022
DeGauss3Mean28.830.84860.233023.350.83710.1991
Std. dev.0.110.00140.00080.120.00140.0008
SpeeDe3DGS3Mean28.440.83920.279322.590.82470.2416
Std. dev.0.250.00140.00260.350.00290.0054
E-D3DGS3Mean30.380.86260.237224.760.85200.2092
Std. dev.0.120.00130.00110.090.00150.0009
Ours w/o motion grouping3Mean29.270.85410.259923.380.83010.2292
Std. dev.0.230.00250.00610.360.00560.0081
Ours w/ H = I3Mean30.240.85760.255324.820.85360.2104
Std. dev.0.150.00100.00140.090.00170.0024
Ours w/o seeding and visibility3Mean29.880.85690.241425.560.86980.2079
Std. dev.0.090.00140.00410.170.00220.0059
Ours w/o seeding3Mean30.230.85790.255324.780.85370.2102
Std. dev.0.060.00050.00040.140.00090.0008
Ours w/ optical flow3Mean30.350.85760.259124.920.85520.2125
Std. dev.0.090.00180.00150.120.00080.0010
Ours w/ event flow3Mean30.950.86490.229026.670.89960.1608
Std. dev.0.110.00150.00170.090.00110.0031
SceneGroupingGroups ↓Cover (%) ↑IoU (%) ↑Persist (%) ↑Shared
UmbrellaVelocity similarity3775.57 ± 0.3132.56 ± 3.14×
+ Affine consistency78470.15 ± 0.8467.00 ± 7.25✓
+ Observability H49075.31 ± 1.9887.33 ± 2.33✓
+ Spatial completion110084.10 ± 0.5287.33 ± 2.33✓
SchaffVelocity similarity38555.03 ± 2.1841.41 ± 2.86×
+ Affine consistency38464.44 ± 0.7245.42 ± 6.55×
+ Observability H19569.93 ± 1.8681.78 ± 2.69✓
+ Spatial completion110080.61 ± 2.1081.78 ± 2.69✓
BellevueVelocity similarity5.54929.99 ± 3.2039.03 ± 7.39×
+ Affine consistency38636.50 ± 3.6228.10 ± 3.79×
+ Observability H19560.24 ± 1.8952.12 ± 3.32✓
+ Spatial completion110073.82 ± 2.5152.12 ± 3.32✓
AuthorsOpenReview ↗

Official Comment by Authors (Part 3)

W4. "Selected event-dense intervals from nine HS-ERGB scenes and one CED scene do not establish performance through weak-event periods."

A5. Regarding weak-event scenarios, we consider event sparsity from both spatial and temporal perspectives.

Spatially, scenes such as Water bomb ETH, Water bomb floor, and Plate contain event activity concentrated in relatively small image regions, leaving large parts of the image with limited event measurements.

Temporally, we have extended the evaluation intervals for Water bomb ETH and Balloon popping to include additional low-activity periods following the main motion. In both scenes, the objects remain approximately stationary for a period after the primary event. We selected the extended intervals such that the duration of the approximately stationary period matches that of the active-motion period. In the appended low-activity segments, the event rate drops to approximately 7.4% of that in the original active interval, while the proportion of event-dense blocks decreases from 32.2% to 3.2%.

We conducted three independent runs on these extended sequences and report the mean and standard deviation of both full-image and dynamic-region reconstruction metrics.

SceneMethodFull image PSNR ↑Full image SSIM ↑Full image LPIPS ↓Dynamic region PSNR ↑Dynamic region SSIM ↑Dynamic region LPIPS ↓
Water bomb ETH extendedD-3DGS34.71 ± 0.210.9488 ± 0.00060.1037 ± 0.002329.79 ± 0.790.9492 ± 0.00140.0794 ± 0.0035
Ours w/ event flow35.21 ± 0.020.9535 ± 0.00050.0908 ± 0.000830.99 ± 0.200.9518 ± 0.00030.0736 ± 0.0010
Balloon extendedD-3DGS34.59 ± 0.230.8448 ± 0.00030.3527 ± 0.000327.95 ± 0.710.9214 ± 0.00190.1857 ± 0.0030
Ours w/ event flow34.87 ± 0.120.8460 ± 0.00020.3507 ± 0.000228.68 ± 0.380.9241 ± 0.00090.1799 ± 0.0021

These additional experiments extend the evaluation beyond the originally selected event-dense intervals and provide evidence that the method can remain effective when event activity becomes substantially weaker.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.