acceptodds
Under review as a conference paper at ICLR 2027

DyRAD: Radar Novel View Synthesis for Dynamic Driving Scenes

19 views

Abstract

Reconstructing dynamic driving scenes from recorded sensor data supports closed-loop evaluation of autonomous driving systems by synthesizing observations beyond the original trajectory. Unlike cameras and LiDAR, radar measures radial velocity directly through Doppler. Yet existing radar novel-view synthesis fails to exploit this capability: methods addressing dynamic scenes reconstruct only range–azimuth tensors, while methods that render Doppler assume static scenes. Moreover, because radar processing spreads each reflection across multiple bins, existing representations absorb this spread into scene geometry, causing it to render incorrectly when the viewpoint moves. We present DyRAD, which models dynamic driving scenes using static background reflectors and motion-tracked dynamic point reflectors to render complete range–azimuth–Doppler (RAD) tensors. Reflector velocities are derived from object tracks and projected onto the line of sight, making Doppler both a rendered output and supervision for those tracks. Crucially, we render reflectors through a fixed analytic point-spread function (PSF) derived from the radar's signal-processing chain, preventing sensor-induced spread from being baked into the scene representation. Beyond improving scene reconstruction, this separation also enables zero-shot sensor-configuration transfer, allowing the same reconstructed scene to be rendered under different radar specifications without refitting. We evaluate DyRAD on RADIal, Boreas, and a synthetic benchmark across both on-path poses and displaced viewpoints untested by prior work. On RADIal, DyRAD recovers radar detections in of reference-detected objects, compared with for the strongest baseline.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
neural fields, radar simulation, sensor simulation, automotive radar, dynamic scene reconstruction, differentiable rendering, Doppler, autonomous driving

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_n8x6OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DyRAD, a method for generating radar measurements from new viewpoints in dynamic driving scenes. It models static and moving objects using point reflectors and renders range-azimuth-Doppler (RAD) measurements. The method also uses a fixed point-spread function (PSF) to separate the scene representation from the radar sensor response. Experiments are conducted on RADIal, Boreas, and synthetic driving scenes.

Critical strengths
  1. The paper addresses an interesting and challenging problem. Existing radar novel-view synthesis methods either do not model Doppler for moving objects or assume static scenes. DyRAD addresses this gap by modeling object motion and Doppler together. The use of a fixed sensor PSF is also a sensible design choice.
  2. The experimental results are strong. On RADIal, DyRAD achieves a foreground hit rate of 90.7%, compared with 26.9% for the strongest prior baseline. The improvements in object-region reconstruction are also substantial. The authors also evaluate the method on Boreas, which uses a different radar system.
  3. I appreciate the effort to evaluate the method beyond the original sensor trajectory. The authors introduce a render-and-refit test on real data and a synthetic benchmark with ground truth at new viewpoints. The ablation studies are useful and cover the main components of the method. The paper also reports training time, rendering speed, and memory usage.
Critical weaknesses
  1. My main concern is the evaluation at new viewpoints. This is the central goal of the paper, but the authors do not have real radar measurements from the displaced viewpoints. Instead, they use a render-and-refit procedure. This is a useful test, but it does not directly verify whether the synthesized measurements at the new viewpoint are accurate. The direct off-path evaluation uses synthetic data with an idealized radar model. It is therefore difficult to judge how well the method would work in real driving scenes.
  2. The method uses existing object bounding-box annotations to initialize moving objects and their tracks. This is a reasonable assumption for offline scene reconstruction. However, the paper does not evaluate the impact of inaccurate or missing annotations. Since object assignments remain fixed during optimization, annotation errors could affect the reconstructed scene.
Questions for authors
  1. Have the authors considered evaluating DyRAD using real radar measurements collected from different viewpoints of the same scene? This would provide stronger evidence for the accuracy of the synthesized measurements.
  2. How sensitive is the method to errors in the provided object bounding boxes? What happens when a moving object is not annotated?
  3. In Table 10, DyRAD improves foreground hit rate considerably, but the joint detection F1 is almost unchanged compared with DyRAD-static (0.227 versus 0.223). The number of false-positive boxes also increases. Could the authors explain this result? Does it affect the usefulness of the synthesized data for downstream perception tasks?
  4. The sensor-configuration experiment in Table 16 is interesting. However, both configurations come from the same radar recordings. Would the method also work with a physically different radar sensor?
2 comments
AuthorsOpenReview ↗

n8x6 Response

We sincerely thank the reviewer for their insightful feedback and comments. Please see General Response 1 for sensitivity to box errors and for unannotated moving objects. We address the remaining questions below.

1. Real off-path validation

n8x6, Questions: “Have the authors considered evaluating DyRAD using real radar measurements collected from different viewpoints of the same scene? This would provide stronger evidence for the accuracy of the synthesized measurements.”

Answer. We agree that real measurements from displaced viewpoints would provide stronger validation. However, to the best of our knowledge, no public dataset provides synchronized RAD radar measurements of the same dynamic driving scene from different viewpoints.

Within this constraint, we evaluate off-path performance in two complementary ways that go beyond the on-path evaluation of prior radar NVS work. A real-data cycle protocol (Sec. 4.2) keeps the evaluation anchored in real measurements, and a synthetic benchmark (Appendix C.4) provides exact ground truth at displaced viewpoints under an idealized sensor model. We will release the synthetic benchmark to support future work on this problem.

2. Hit rate, F1 and false positives

n8x6, Questions: “In Table 10, DyRAD improves foreground hit rate considerably, but the joint detection F1 is almost unchanged compared with DyRAD-static (0.227 versus 0.223). The number of false-positive boxes also increases. Could the authors explain this result? Does it affect the usefulness of the synthesized data for downstream perception tasks?”

Answer. Joint F1 is computed over all detections in the scene, and most of the scene is static, so better vehicle reconstruction barely changes global F1. Hit rate measures the vehicles directly, and there DyRAD improves from 22% to 75%.

A false-positive box is a vehicle box that contains predicted detections but no reference detections. For DyRAD, these are real vehicles whose rendered returns are just above the detection threshold while their reference returns are just below it (median 2.41 vs. 1.82 dB, threshold 2.0 dB). DyRAD-static has few such boxes because it rarely renders the vehicles at all. As noted in Sec. 4.2, this is the cost of the higher vehicle coverage. Its effect on downstream perception models remains to be evaluated.

3. Physical-sensor transfer

n8x6, Questions: “The sensor-configuration experiment in Table 16 is interesting. However, both configurations come from the same radar recordings. Would the method also work with a physically different radar sensor?”

Answer. In principle, yes. DyRAD separates the scene representation from the sensor response, so a fitted scene can be rendered with another sensor’s PSF and sampling grid, as in the sensor-configuration experiment (Table 18 in the revised manuscript). To demonstrate this, we rendered two held-out RADIal frames with the PSF and sampling grid of the RaDelft radar, without refitting the scene (Appendix E.1, Figs. 12–13 of the revised manuscript). Validating the accuracy of such transfer requires paired recordings of the same scene from different physical sensors, which we plan to explore in future work.

Reviewer_n8x6OpenReview ↗

Response to Authors and Final Assessment

Thank you for the detailed responses and additional experiments. The annotation robustness results and other clarifications address most of my concerns.

My main remaining concern is the lack of direct validation using real radar measurements from displaced viewpoints. I understand the dataset limitations and appreciate the proposed evaluation protocols.

Overall, the rebuttal has strengthened the paper. I will maintain my current rating.

Reviewer_qqK7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on the task of radar novel view synthesis for dynamic driving scenes. Existing work usually synthesizes range-azimuth maps only, but this work can synthesize range-azimuth-doppler maps.

This work proposes representing the whole scene as multiple point reflectors, and each reflector has four attributes including position, reflected power, view-dependent coefficients and a static/dyamic class.

In addition, the paper implements an off-path evalution pipeline for radar NVS and verifies the effectiveness of their method on two public datasets (Boreas and RADIal).

Critical strengths

The motivation behind is clearly illustrated. Extensive quantitative and qualitative results are provided to demonstrate their strong performence. And the proposed method is validated on different radar settings.

Critical weaknesses
  1. This work relies on GT bounding boxes of moving vehicles during training. Although it's reasonable to use bounding boxes to focus on dynamic targets, this makes the task setting relatively easier compared to other baseline methods since other methods do not require these bounding boxes. I raise this concern because this work uses the Boreas sequences with GT bounding boxes, but the sequences used in RadarSplat/RadarFields do not have GT bounding boxes. Then in practice, to synthesize novel views using the proposed method, we need first to obtain detection results on the observed frames, then train the reflector primitives and finally synthesize the novel views. This introduces additional dependency on detection results.

  2. I understand that the scope of this work is set to novel view synthesis, but I am concerned about whether this trained scene (point reflector primitives) is able to differentiate real objects in the scene from radar artifacts. Baseline methods (RadarFields, RadarSplat) also report occupancy estimation results, but the proposed method does not report this metric. Is it because all reflectors are regarded as real objects?

Questions for authors
  1. As radar measurements usually contain different kinds of artifacts (multipath, receiver saturation, spectral leakage, scatter reflection). These artifacts are transient and the qualitative results still show that proposed method could synthesize part of them, in this case, are these artifacts considered as static or dynamic reflectors as well?

  2. In radar sensing principles, the received power is related to the distance between radar and target. However, in Equation 2, the reflected power is not related to the distance. Is this design on purpose?

  3. In the provided HTML, for Boreas dataset, there are some frames that DyRAD-static is even better than DyRAD, what causes this?

  4. Regarding the motion modeling (Equation 1), dynamic reflectors' positions are dertmined by three variables p, R and \mu? Why not just predict the coordinate directly?

5 comments
AuthorsOpenReview ↗

qqK7 Response - Comment 1 of 2

We sincerely thank the reviewer for their insightful feedback and comments. Please see General Responses 1–2 for annotation dependence and baseline fairness. We address the remaining questions below.

1. Occupancy and physical geometry

qqK7, Weaknesses: “I understand that the scope of this work is set to novel view synthesis, but I am concerned about whether this trained scene (point reflector primitives) is able to differentiate real objects in the scene from radar artifacts. Baseline methods (RadarFields, RadarSplat) also report occupancy estimation results, but the proposed method does not report this metric. Is it because all reflectors are regarded as real objects?”

Answer. To address your concern, we evaluated occupancy using RadarSplat’s protocol on three Boreas sequences and ten RADIal sequences. Compared with RadarSplat (the stronger baseline), DyRAD achieves higher precision at lower recall on both datasets, and substantially higher moving-object recall (0.64 vs. 0.14 on Boreas; 0.60 vs. 0.09 on RADIal). RadarSplat has lower geometric error on Boreas, and DyRAD on RADIal. The full results and visualizations are in Appendix F (Table 19, Fig. 14) of the revised manuscript.

Lidar agreement is informative, but radar and lidar can detect different surfaces, so some discrepancies may reflect sensor differences rather than reconstruction errors. We consider CFAR detection agreement with measured radar a more direct test of whether the reconstructed scene preserves radar-detectable objects.

2. Radar artifacts

qqK7, Questions: “As radar measurements usually contain different kinds of artifacts (multipath, receiver saturation, spectral leakage, scatter reflection). These artifacts are transient and the qualitative results still show that proposed method could synthesize part of them, in this case, are these artifacts considered as static or dynamic reflectors as well?”

Answer. DyRAD assumes that all reflectors correspond to real scene content, either static or moving with an annotated object. The PSF models part of the sensor’s own effects; for example, the range kernel h_R captures range-FFT spectral leakage (Eq. 2). Unmodeled effects, such as multipath, receiver saturation, and scattering, may be absorbed into the reflectors. We acknowledge these unmodeled effects in the paper’s limitations section (Sec. 5), where we identify modeling further radar effects, such as multipath and speckle, as future work.

3. Range attenuation

qqK7, Questions: “In radar sensing principles, the received power is related to the distance between radar and target. However, in Equation 2, the reflected power is not related to the distance. Is this design on purpose?”

Answer. Yes, this is deliberate. In theory, a point target’s return falls as R⁻⁴ in power, or R⁻² in amplitude. However, radar systems can apply range compensation during acquisition or processing, and specular reflections and constructive and destructive interference of signals from large objects such as cars cause large variations.

Examining the recorded datasets, we found much shallower vehicle-return falloff: approximately R⁻¹ in amplitude on RADIal and R⁻⁰·⁵ in power on Boreas. The available documentation does not let us rule out that range compensation is applied. Enforcing the theoretical falloff hurt reconstruction in our experiments, and the baselines likewise omit it in their released code despite including it in their papers: RadarFields’ released configuration disables its range term (approximate_fft = True), and RadarSplat’s released renderer contains none. We therefore fit observed return strengths through learned reflectivity without imposing an explicit range-attenuation term.

AuthorsOpenReview ↗

qqK7 Response - Comment 2 of 2

4. Boreas frames favoring static

qqK7, Questions: “In the provided HTML, for Boreas dataset, there are some frames that DyRAD-static is even better than DyRAD, what causes this?”

Answer. Thank you for examining the videos closely. Nearly all of these frames are in Sequence 1 (29 of its 50 frames, and only one frame across the other two sequences). This sequence contains mostly static structure and slow traffic, which limits the benefit of motion modeling. There, DyRAD-static’s advantage in full-frame RA correlation is at most 0.006, and the sequence averages are nearly identical (0.738 for DyRAD-static and 0.737 for DyRAD). Thus, motion modeling improves most evaluated frames, but does not improve every case, particularly when motion is limited.

5. Rigid motion design

qqK7, Questions: “Regarding the motion modeling (Equation 1), dynamic reflectors’ positions are dertmined by three variables p, R and μ? Why not just predict the coordinate directly?”

Answer. Our design uses a prior suited to driving scenes: most of the background is static and vehicles move approximately rigidly. The rigid track moves all of an object’s reflectors along one shared trajectory, which preserves the object’s structure. In Eq. 1, p and R describe object translation and rotation, while μ is each reflector’s position within the object.

Following your question, we tested an MLP that predicts each reflector’s position over time directly, with Doppler computed from the trajectory derivatives. The annotations identify the dynamic reflectors and their initial positions, and the MLP learns each reflector’s offset from its initial position over time. The vehicles then remain nearly static: across all ten RADIal sequences, object-region RAD correlation drops from 0.66 to −0.01 and hit rate from 91% to 9%, matching DyRAD-static. We added this ablation to the revised manuscript (Sec. 4.4; Tables 12–13; Appendix D).

Thank you for the provided additional experiments and clarifications. Could the authors further explain how the predicted radar point clouds are obtained for occupancy estimation? RadarSplat applies a threshold to the predicted occupancy to extract radar point clouds. Then based on the proposed scene primitives, how does your method determine which regions are occupied?

In addition, to my knowledge, RadarSplat applies an intensity filter to LiDAR point clouds, retaining only those whose corresponding radar range-azimuth bins have high intensity. However, CFAR may detect multipath ghost targets, in this case, CFAR may not be a good reference compared to RadarSplat's occupancy reference.

Reviewer_XbZEOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a method for synthesizing radar measurements of dynamic driving scenes from novel viewpoints. It combines static and motion-tracked point reflectors with a fixed sensor response to separate scene structure from measurement spreading. Aside from scene reconstruction, this separation also enables zero-shot sensor-configuration transfer, allowing the same reconstructed scene to be rendered under different radar specifications without refitting. The authors evaluate their method on experimental and synthetic benchmarks across both on-path poses and displaced viewpoints with overall convincing results.

Critical strengths

Solid contribution in an underexplored field. The paper is technically novel and provides strong experimental results.

Novelty:

  • The work is the first radar NVS method to reconstruct dynamic driving scenes with Doppler measurements.
  • Reflector velocities are derived from object tracks and projected onto the line of sight, making Doppler both a rendered output and supervision for these tracks. The authors render reflectors through a fixed analytic point-spread function (PSF) derived from the radar’s signal-processing chain, preventing sensor-induced spread from being baked into the scene representation.

Experimental Evaluation:

  • The experimental results are convincing and show that the method is effective and outperforms static scene method such as RadarFields and RadarSplats. The authors should further clarify that these methods do not model dynamic objects (and they do not have access to track information) but this doesn't take away from the strong margins.
Critical weaknesses

The method lacks extensive ablation experiments and is incremental:

  • The method's dependence on tracking inputs is not adequately evaluated, and ablation experiments are missing. The authors should remove tracking information and perturb the accuracy of the track outputs to fully evaluate the method.

  • Ablation experiments aside, one could argue that the technical approach is rather incremental (tracked object simulation + static background model). However, as the results speak for themselves, I would not hold this against the authors.

Questions for authors

See ablation experiments missing above.

1 comment
AuthorsOpenReview ↗

XbZE Response

We sincerely thank the reviewer for their insightful feedback and comments. We address the requested annotation ablations in General Response 1, including new experiments with detector outputs, perturbed boxes, and sparse annotations. General Response 2 clarifies the differences in task and supervision between DyRAD and the baselines. In the revised manuscript, we clarified these differences in Sec. 4.1 and the Table 1 caption, and added the annotation experiments to Sec. 4.4 and Appendix D.1.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response 1

We thank all reviewers for their thoughtful and constructive feedback. We are encouraged that they recognize DyRAD’s technical novelty, namely the joint modeling of dynamic scenes and Doppler and the separation of the scene from the sensor response through a fixed PSF (XbZE, n8x6). All reviewers note the strong experimental results, including substantial gains in moving-object reconstruction (n8x6), extensive evaluation (qqK7), and validation across radar settings (qqK7, n8x6). We also appreciate the clear motivation noted by qqK7.

Below, we address the shared concerns about annotation dependence and baseline fairness, followed by individual responses to each reviewer. We have uploaded a revised manuscript in which all changes are marked in red.

1. Annotation dependence and robustness

XbZE, Weaknesses: “The authors should remove tracking information and perturb the accuracy of the track outputs to fully evaluate the method.”

n8x6, Questions: “How sensitive is the method to errors in the provided object bounding boxes? What happens when a moving object is not annotated?”

qqK7, Weaknesses: “This introduces additional dependency on detection results.”

Answer. We thank the reviewers for highlighting the need to evaluate annotation dependence. DyRAD uses object boxes to initialize dynamic reflectors and tracks, which are then jointly optimized against radar measurements. An entirely unannotated moving object is represented by static reflectors and may smear or disappear. Without any boxes, the model reduces to DyRAD-static. We acknowledge this dependence on annotations as a limitation in the paper (Sec. 5).

When annotations are not available, the boxes can come from an off-the-shelf radar detector. To test this, we ran a pretrained radar detector, T-FFTRadNet (Giroux et al., 2023), on three RADIal sequences that were not used to train it, and initialized DyRAD from its detections instead of the annotations. Its box centers lie on average 0.17–0.29 m from the annotated ones; since the annotations are themselves noisy, this is the disagreement between two estimates rather than the detector’s true error (example frames are shown in Fig. 10 of the revised manuscript). The detector-initialized DyRAD performs close to the annotation-initialized one (table below, top), and the full-scene metrics are nearly unchanged (Table 16 of the revised manuscript).

We additionally evaluated Gaussian box-center noise (σ = 0.25 and 0.5 m), a constant 1 m range offset, and sparse annotations retaining one box every 10 frames (plus the first and last box of each track), on the five RADIal sequences with the most moving objects. Box-center noise lowers the object metrics as σ grows, and the object Doppler error rises from 0.73 to 1.11 and 1.57 bins, while a range offset or sparse boxes barely affect them (table below, bottom).

Annotations (RADIal, on-path, 3 seeds; DyRAD-static 1 seed)Object-region RAD ρ ↑Hit rate ↑
Detector boxes (3 sequences)
Annotations0.6494.8%
Detector boxes (T-FFTRadNet)0.6691.4%
No boxes (DyRAD-static)0.0115.3%
Perturbed annotations (5 sequences)
Annotations0.6385.6%
Box-center noise σ = 0.25 m0.5675.4%
Box-center noise σ = 0.5 m0.4670.3%
Constant 1 m range offset0.6285.3%
One box every 10 frames0.6483.9%
No boxes (DyRAD-static)0.0211.4%

The annotation rows differ between the two blocks because they average different sequences and training runs. In every setting, DyRAD stays far above DyRAD-static. We added these experiments to the revised manuscript (Sec. 4.4; Appendix D.1, Tables 16–17).

AuthorsOpenReview ↗

General Response 2

2. Baseline fairness

XbZE, Experimental Evaluation: “The authors should further clarify that these methods do not model dynamic objects (and they do not have access to track information) but this doesn’t take away from the strong margins.”

qqK7, Weaknesses: “Although it’s reasonable to use bounding boxes to focus on dynamic targets, this makes the task setting relatively easier compared to other baseline methods since other methods do not require these bounding boxes.”

qqK7, Weaknesses: “I raise this concern because this work uses the Boreas sequences with GT bounding boxes, but the sequences used in RadarSplat/RadarFields do not have GT bounding boxes.”

Answer. RadarSplat and RadarFields reconstruct static scenes, while DyRAD additionally reconstructs moving objects and their Doppler signatures. To support this added complexity, we use boxes to initialize object tracks. For a direct comparison with prior methods, we report DyRAD-static, which requires no annotations and outperforms both baselines in full-scene reconstruction on RADIal and Boreas (Table 1). We thank the reviewers for highlighting the need to clarify these differences in task and supervision and have made them explicit in Sec. 4.1 and the Table 1 caption of the revised manuscript.

To address the concern about sequence selection, we evaluated DyRAD and retrained RadarSplat on three of RadarSplat’s original Boreas sequences. These sequences have no box annotations, so we run DyRAD on them without boxes, i.e., as DyRAD-static; alternatively, boxes could come from an off-the-shelf detector, as we show on RADIal in General Response 1. DyRAD achieves higher mean correlation (0.772 vs. 0.746) and PSNR (25.96 vs. 25.47 dB), with comparable SSIM (0.491 vs. 0.494).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.