acceptodds
Withdrawn from ICLR 2027

Event4D: Dense 4D Reconstruction from RGB and Neuromorphic Event Streams

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reconstructing dynamic 3D scenes over time (4D reconstruction) from a single moving camera is important for robotics and AR/VR. However, existing RGBonly feed-forward methods are vulnerable to motion blur and temporal aliasing under rapid camera or object motion. Event cameras asynchronously capture per-pixel brightness changes at microsecond resolution, providing complementary temporal information between RGB frames. We introduce Event4D, the first single-stage feed-forward 4D reconstruction pipeline that fuses events and RGB images. Event4D pairs event voxels at fine-grained reconstruction time steps with the most recently captured RGB frame and fuses their features through a learned RGB-staleness conditioning. Using five benchmark datasets, we evaluate Event4D under three challenging conditions for RGB-based 4D reconstruction: fast object motion, low illumination, and temporal aliasing. Event4D improves threshold accuracy over the state-of-the-art by 10.11% on fast-motion sequences, 59.29% under temporal aliasing, and 11.61–24.77% across low-light sequences.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D Reconstruction, Neuromorphic Computing, Event Cameras

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GoSROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Event4D extends PAGE-4D with a recurrent event encoder, staleness-conditioned feature fusion, and event-to-camera attention. It trains on synthetic and real event datasets and evaluates depth and camera pose across five datasets. The depth results are strong under several challenging conditions, including reduced RGB frame rates.

Critical strengths
  1. Events can provide useful motion information between RGB captures, particularly in fast motion and low light.
  2. Explicitly encoding RGB staleness is a sensible way to handle repeated or delayed images.
  3. The paper tests multiple datasets and conditions, with useful ablations for input modalities, the event encoder, losses, and staleness conditioning.
Critical weaknesses
  1. The RGB-only results in Table 4 match the PAGE-4D baseline in Table 1, while Event4D is trained on seven datasets with additional losses. A same-data, same-training RGB-only baseline is needed to determine how much improvement comes from events and fusion.
  2. The frame-rate experiments evaluate repeated RGB inputs, but the paper does not say whether training includes stale frames or nonzero staleness values. Please report the training schedule and the staleness distribution; these details are necessary to interpret the F/2–F/8 results.
  3. Event4D has higher ATE than PAGE-4D on N-Sintel (0.225 vs. 0.179) and MVSEC (0.552 vs. 0.298). The pose table also omits EAG3R, a relevant event-based 4D method with reported MVSEC nighttime pose results EAG3R. These regressions and the baseline choice need discussion.
  4. The quantitative results report depth and pose, but not point-map accuracy, global 3D reconstruction quality, or temporal consistency. The point-map evidence is qualitative.
  5. Synthetic events are generated from videos interpolated with SuperSloMo before applying v2e. Since temporal performance is a central claim, a comparison across real-event and synthetic-event training, or an analysis of sensitivity to this simulation pipeline, would strengthen the evidence.
Questions for authors

Please refer to the concerns raised in the Weaknesses section.

Reviewer_EmE9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript presents Event4D, a framework that augments frozen PAGE-4D components with a recurrent event encoder, RGB-staleness-conditioned fusion adapters, and event-to-camera attention. Event4D can predict depth, pointmaps, and camera poses. It targets fast motion, low illumination, and sparse RGB observations.

Critical strengths
  1. Problem formulation is clear: Combining events with the latest RGB observation addresses missing temporal information, while explicit staleness conditioning provides asynchronous sensor availability.

  2. Useful adaptation strategy: Learnable event modules and adapters introduce a complementary modality while preserving the pretrained geometric knowledge from backbones and prediction heads.

  3. Obvious gains in several settings: Video-depth AbsRel improves from 0.229 to 0.125 on EVIMO and from 0.078 to 0.047 on DSEC relative to PAGE-4D.

Critical weaknesses
  1. Unclear training settings: Sequence-level train/test splits are insufficient in the manuscript, and the RGB-only comparison does not clearly control for additional training data and supervision.

  2. Incomplete 4D evaluation: Depth and camera-pose metrics provide limited evidence about dynamic pointmap accuracy, temporal correspondence, and consistency in the 4D coordinate system.

  3. Performance inconsistency: Compared with PAGE-4D, ATE increases on N-Sintel and MVSEC.

Questions for authors
  1. Please provide sequence-level dataset splits and an RGB-only baseline trained with identical datasets and supervision.

  2. Can you evaluate dynamic-region 3D accuracy and temporal consistency, and specify sequence lengths, depth ranges, and alignment procedures?

  3. Please explain the performance issue of the increased ATE metrics.

  4. Please report the parameters, memory usage, and runtime relative to PAGE-4D and other compared methods under matched resolutions and sequence lengths.

Reviewer_F41KOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Event4D fuses event streams with RGB in a feed-forward 4D reconstruction model. It freezes PAGE-4D and trains an RVT event encoder, eleven staleness-conditioned adapters and an event-to-camera cross-attention, plus two pose losses. Training mixes v2e-simulated sets (N-*) with real ones (EVIMO, DSEC, EventKitchen). It reports depth and pose gains under fast motion, reduced RGB frame rate and low light (vs. EAG3R on MVSEC night).

Critical strengths
  • Fusing events with a strong 4D backbone is a sensible fix for blur, low light and frame gaps.

  • The staleness embedding is simple and backed by an ablation (Table 4).

  • The frame-rate sweep (Fig. 4) and the EAG3R comparison target the claimed benefits directly.

Critical weaknesses

Major:

  1. Likely train/test overlap. EVIMO, DSEC and EventKitchen are in both training and Table 1, and no split is given. Baselines run zero-shot, and the “RGB only” row in Table 4 is identical to PAGE-4D in Table 1. There is no matched RGB-only model trained on the same data and losses, so gains can’t be credited to events. Gains only on seen data. On held-out MVSEC, video AbsRel is 0.451 vs 0.344 for MonST3R. On N-Sintel, δ<1.25 is 0.735 vs 0.763 for PAGE-4D.

  2. The aliasing claim is weak. Holding RGB frames makes a no-event model fail by construction. There is no fresh-frame-plus-interpolation or staleness-aware RGB baseline. The 59.29% figure is in no table. The fast-motion subset has 8 sequences and no variance anywhere.

Other:

  • “State of the art” is overstated. In Table 2, Event4D is worst on N-Sintel and has the highest MVSEC ATE (0.552). EVIMO ATE gains are tiny (0.036 to 0.034). In Table 3, Night1 AbsRel (0.441) is worse than all baselines, and PAGE-4D and VGGT are missing.

  • Suspicious ablations. Dropping the pose losses or the RVT gives DSEC AbsRel 0.095–0.101, worse than frozen PAGE-4D (0.078). That needs explaining.

  • Dynamic pointmaps are never evaluated, though EVIMO has moving objects. No compute numbers (params, FLOPs, runtime, memory) are given.

Minor:

The aligned-grid assumption is glossed over (DSEC has two sensors, MVSEC is low-res grayscale). Headline gains mix absolute points and relative percentages. The AI-use statement is garbled. Please confirm dataset licenses allow redistributing the derived data. Also qualitative results can’t judge quality (Figs. 5–11). There is no GT depth, no GT point map, and no baseline outputs side by side. Per-frame AbsRel is printed, but depth colormaps and sparse point clouds say little. There is no video, which matters a lot for a temporal method.

Questions for authors

Can you please:

  1. Give the train/test splits per dataset and held-out-only results.

  2. Add an RGB-only adapter model with the same data, losses and compute.

  3. Trace 10.11% and 59.29% to a table, and give the alignment protocol and the sequence length.

  4. Let me know if staleness is simulated during training? Also, how were the loss weights chosen?

  5. Explain why the Table 4 variants fall below frozen PAGE-4D.

  6. Report params, latency and memory vs PAGE-4D and EAG3R, and add PAGE-4D and VGGT to Table 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.