acceptodds
Under review as a conference paper at ICLR 2027

Imaging the Radar: Recovering 4D Radar Tensors from Images with a Latent Diffusion Bridge

Abstract

Millimeter-wave radar's all-weather robustness and Doppler velocity sensing make it increasingly important for autonomous perception. However, the scarcity of high-quality 4D radar data limits the development of radar perception systems. Images provide a scalable source for radar data expansion: camera sequences are abundant and inexpensive to acquire, preserve scene structure shared with radar, and support scene-level editing. We therefore introduce image-to-4D radar tensor generation and propose the Radar-aligned Visual Prior Bridge (RaViP-Bridge) to recover complete radar tensors from images. However, direct translation faces a substantial modality gap. Images record scene appearance on a 2D perspective plane, whereas native 4D radar tensors encode spatial and Doppler responses amid noise and clutter. To narrow this gap, we use visual foundation models to extract geometry, semantic, and motion cues from image sequences and organize them into a radar-aligned visual-prior tensor. Given the high dimensionality and noise of native radar tensors, we use dual VAEs to compress the visual-prior and radar tensors into compact latent spaces. We then formulate latent translation as an endpoint-constrained diffusion bridge from the visual-prior latent to the radar latent. On K-Radar, we find that RaViP-Bridge achieves high-fidelity 4D radar generation and improves downstream detection through data augmentation.

Primary area:
generative models
Keywords:
Radar Generation, Diffusion Models, Vision for Robotics, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TJdaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RaViP-Bridge for generating complete 4D radar tensors from camera sequences. It extracts depth, semantic, and motion cues from images, projects them into a radar-aligned visual prior, and translates the visual latent into a radar latent using a diffusion bridge. Experiments on K-Radar show improved tensor reconstruction over controlled baselines and modest gains when the generated radar is used for downstream detection augmentation.

Critical strengths

The paper explores an interesting cross-modal generation problem and targets complete 4D radar tensors rather than simplified radar maps or point clouds. The downstream experiments with both radar-only and camera-radar detectors are also useful.

Critical weaknesses

1.The practical value of image-to-radar generation is not yet convincing. The model itself is trained on paired camera–radar data, while the claimed motivation is large-scale radar data expansion from inexpensive camera data. The paper does not show whether a model trained on K-Radar can generate useful radar for a new camera-only dataset or an unseen domain. Without such an experiment, the data-scarcity argument is somewhat circular.

2.The reliance on visual priors is problematic for the scenarios where radar is most useful. RaViP-Bridge depends heavily on monocular depth, segmentation, and optical flow. These cues can become unreliable in adverse weather or poor illumination, while radar is specifically valuable under such conditions. The paper does not provide weather-wise results on K-Radar, and the 9,008-sample subset only retains samples for which the visual prior can be constructed. It is therefore unclear how the method behaves on visually degraded K-Radar scenes.

Questions for authors

1.Can the authors evaluate the model on a truly camera-only dataset or an unseen domain to demonstrate actual radar dataset expansion?

2.How does performance vary across K-Radar weather conditions, especially scenes where the camera is severely degraded?

3.Are the train/validation/test splits sequence-disjoint? Since K-Radar contains temporally adjacent frames, this is important for assessing possible train-test overlap.

4.How should the generated tensors be interpreted when radar responses arise from effects not observable in RGB, such as multipath or material-dependent reflections?

2 comments

Thank you for your careful review and your questions about dataset expansion and low visibility.

For the first question, paired camera and radar data are used for training. Once trained, the model needs only image pairs and calibration to generate radar. We have added generation examples from Nexar driving videos using the model trained on K-Radar (Figure 4, p. 10). These videos have no paired radar for computing generation error, so we provide qualitative results. A key advantage of generating radar from images is that the input images are highly editable. We can add, remove, or replace objects before generation to control the object configurations in the added training samples, as shown in Figure 5 (p. 10) and Appendix G.5 (p. 24). We further added a detection experiment with scarce Bus/Truck examples. With the same training sample count, edited generation outperforms class repetition; Table 5 (p. 10) reports mean ±\pm std over three seeds.

For the second question, our model needs visible scene information. The dark tunnel case in Appendix H.2 (p. 26) shows how low visibility affects generation. Weather affects generation when it obscures the scene. When severe frost makes the camera image nearly white, the visual prior cannot provide useful scene information. Light rain and light snow cause little radar power attenuation over short automotive ranges; we discuss the supporting measurements in Appendix H.2 (p. 26). We can therefore use images with sufficient visibility to generate additional radar training samples.

We also evaluated RTNH in severe weather using models trained on normal weather data. The model trained with augmentation achieves higher AP for both classes across the 37 severe weather test scenes (Table 4, p. 9). We can also edit the input images to add examples of rare object configurations. The class scarcity experiment in Table 5 (p. 10) shows the resulting detection gains.

For the third question, we use a chronological split within each scene. Training, validation, and test frames occupy successive time blocks and do not overlap. The split is described in Section 4.1 (p. 6) and Appendix A.1 (p. 14), with the split files provided in the supplementary material.

For the fourth question, images do not uniquely determine multipath or material differences between individual objects. We assign representative permittivity priors by semantic category, while the model learns the remaining effects from paired data. We therefore interpret the generated tensor as an estimate of radar responses given the visible scene cues (Section 4.6, p. 10; Appendix B.2, pp. 14–15). We would like to study more detailed material and propagation models.

I appreciate the authors' efforts in providing detailed additional experiments. However, I remain concerned about the practical value of generating radar data from images, as its significance has not been sufficiently demonstrated. Furthermore, I have doubts about the generalization capability of a model trained solely on the Kradar dataset. Therefore, I will maintain my original rating.

Reviewer_XQM1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RaViP-Bridge, a framework for generating 4D radar tensors from image sequences. It constructs radar-aligned visual priors using depth, semantic, and motion cues, and employs dual VAEs and a latent diffusion bridge to generate radar power and Doppler-spectrum responses. Experiments on K-Radar demonstrate improvements over the reported baselines in radar generation quality. Downstream experiments further show that the generated data can support detector training and improve 3D object detection through data augmentation.

Critical strengths
  1. Practically valuable problem. The paper addresses the shortage of high-quality 4D radar data by exploring image-to-radar tensor generation, offering a potentially scalable approach to radar dataset expansion.

  2. Promising experimental results. The method achieves clear improvements over the reported baselines in radar generation quality. Downstream experiments further demonstrate its utility for detector training and data augmentation.

Critical weaknesses
  1. Unclear technical novelty. The method builds on closely related works, particularly RadarGen, L2RLDB, and BBDM. The paper does not sufficiently distinguish its novel technical contributions from existing components, making it difficult to assess the methodological significance.

  2. Insufficient experimental validation. The baselines mainly compare different translation mechanisms without adequately establishing advantages over the closest prior works. Additionally, the mapping from 64 native Doppler bins to 63 response channels remains unclear, and the current metrics provide limited evidence for complete 4D radar recovery, particularly regarding false Doppler responses.

Questions for authors
  1. Technical novelty. What are the key technical contributions of RaViP-Bridge beyond RadarGen, L2RLDB, and BBDM? Which components are inherited, and which are specifically designed for image-to-4D radar tensor generation?

  2. Comparison with prior work. Can the authors provide comparisons with RadarGen using a common point cloud representation or downstream detection task, or adapt L2RLDB to visual inputs? If direct comparisons are infeasible, what targeted ablations can demonstrate the advantages of the proposed designs?

  3. Doppler representation and evaluation. How are the original 64 Doppler bins mapped to the 63 Doppler-response channels, and can the original spectrum be reconstructed? Additionally, can the authors evaluate false Doppler responses outside valid radar-hit regions to better support the claim of complete 4D radar recovery?

2 comments

Thank you for your careful reading and your questions. The original manuscript did not clearly distinguish our designs from the work we build on. We have clarified this in Sections 2.2–2.3 (p. 3) and 3.2–3.4 (pp. 4–6).

For the first question, RadarGen inspired us to use visual foundation models to extract scene information for radar generation. Drawing on radar simulation, we organize geometry, representative permittivity, and radial velocity into three separate channels on the radar grid. These physical cues can be edited or ablated separately. RadarGen uses visual cues as conditions for denoising. We encode the visual prior as the source endpoint of the bridge and generate radar from this representation. The visual prior therefore provides interpretable physical cues that we can manipulate and serves as the starting point for bridge translation. In Table 1 (p. 8), removing permittivity has a larger effect on spatial fidelity, while removing radial velocity has a larger effect on Doppler fidelity.

Our main contribution is the visual prior module. For translation, we follow BBDM's Brownian bridge formulation and use L2RLDB's dual VAE design and radar representation.

For the second question, we did not have suitable implementations of RadarGen or L2RLDB for comparison at the time of our experiments. We therefore report the published L2RDaS spatial power results as a LiDAR reference and include visual prior ablations and controlled translation comparisons. These comparisons use the same visual prior, dual VAEs, latent cache, and data split, changing only the translation method. Table 1 (p. 8) shows that bridge translation outperforms standard conditional latent diffusion on both spatial and Doppler generation metrics.

For the third question, the original manuscript omitted the channel mapping. We follow K-Radar/RTNH preprocessing: the first Doppler bin is skipped, and the remaining 63 bins retain their original order. Together with the spatial power channel, they form the 64-channel radar tensor. The retained spectra can be recovered by inverting normalization and the logarithmic transform; the omitted bin is not reconstructed. The mapping is detailed in Appendix C (pp. 15–16). We have also added false-positive rate outside real hit regions and hit recall at several mask thresholds (Table 2, p. 8; Appendix Table 10, p. 18).

Thank you for the detailed rebuttal and additional experiments. I appreciate the clarifications regarding the Doppler representation and the improved downstream evaluation, which address several of my concerns.

However, my main concern regarding the methodological novelty remains. The core diffusion bridge and dual-VAE components largely follow existing approaches, while the main contribution lies in constructing the radar-aligned visual prior. Although this is a useful design, I am not fully convinced that it represents a sufficiently substantial methodological advance. The lack of stronger comparisons with closely related work also limits my assessment.

Therefore, I will maintain my original rating.

Reviewer_GhwCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces the task of image-to-4D-radar-tensor generation and proposes RaViP-Bridge. Frozen vision foundation models first extract geometric, semantic and motion cues from a pair of front-view images (Depth Anything 3 for depth, SAM 3 for semantic segmentation, UniMatch for optical flow). These cues are projected through the camera intrinsics/extrinsics onto the radar's range–azimuth–elevation polar grid and voxelized into a three-channel "radar-aligned visual prior tensor" (point density, relative permittivity obtained from a semantic-class lookup table, and radial velocity). Two separate 3D VAEs then compress the visual prior tensor and the 64-channel real radar tensor (one dense log-power channel plus 63 Doppler channels kept only at valid hit locations, with a hit mask) into latent spaces of identical shape. Finally, a Brownian bridge is built in latent space from the visual-prior latent to the radar latent; a 3D U-Net regresses the drift term, and at inference 30 deterministic sampling steps produce the radar latent, which is then decoded.

On 9,008 camera–radar pairs from K-Radar, RaViP-Bridge outperforms three controlled baselines built by the authors across multiple metrics. For downstream detection, training RTNH on generated radar alone reaches 71–85% of the real-data reference, and adding generated validation-split radar to the real training set improves the AP of RTNH and SGDet3D by 2–7% relative to the real-data reference.

Critical strengths
  1. The task is practically valuable: camera data are far easier to obtain than 4D radar data, and generating complete range–azimuth–elevation–Doppler tensors (rather than point clouds or RA maps) from images would be meaningful for enlarging radar datasets.

  2. The construction of the visual prior is the most substantive design in the paper: depth, semantics (mapped to a permittivity prior) and optical flow (projected to radial velocity) are organized on the radar polar grid so that source and target are spatially aligned position by position. The ablations show that the permittivity and velocity channels contribute to spatial fidelity and Doppler fidelity respectively, which is a reasonable conclusion.

  3. The controlled baselines are well designed: the three latent-space baselines share the same visual prior, the same pair of VAEs, the same latent cache and the same split, so the comparison in Table 1 does isolate the translation mechanism as the only variable.

Critical weaknesses

The central claim of the paper is that camera images alone suffice to generate 4D radar tensors realistic enough to train radar detectors directly. Only two pieces of evidence support this claim: Table 1 shows that the generated tensors are close to the real ones, and Table 2 shows that a detector trained on generated tensors reaches 85% of real-data performance, and that adding generated data to real data yields a further 1–3 AP. Unfortunately, neither piece of evidence is convincing at present:

  1. Is the data split performed randomly at the frame level? If so, Table 1 and the synthetic setting of Table 2 measure memorization rather than generation. K-Radar consists of a few dozen driving sequences recorded continuously at 10 Hz, and adjacent frames have nearly identical images and radar tensors. The paper selects 9,008 frames and "split[s] them 4:2:1"; the words "sequence-level" or "scene-level" split appear nowhere in the paper. If the split is random at the frame level, almost every validation and test frame has a near-identical neighbor in the training set, a few tenths of a second away. This has two consequences. First, the generator only needs to recognize which training frame an image corresponds to and reproduce that frame's radar in order to obtain high PSNR/SSIM in Table 1; these metrics would then no longer reflect the ability to infer radar from images. Second, the synthetic-setting result (a detector trained only on generated validation-split radar reaches 85% of the real-data AP on the test set) can equally be explained by the generated validation radar being essentially the real radar of neighboring training frames, so that the detector is in effect trained on real radar. In other words, under a frame-level split, no number in the paper demonstrates that the method has learned an image-to-radar mapping. This is a well-known pitfall in almost all mmWave radar generation/reconstruction work, and the paper does not mention it at all. Please state the split rule explicitly; if the split is frame-level, the data should be re-split by sequence and all experiments repeated.

  2. The augmentation experiment conflates "2,574 additional labeled real scenes" with "the quality of the generated radar", and the paper attributes the entire gain to the latter. In the augmentation setting, the generated radar of the 2,574 validation frames, together with the real 3D bounding-box labels of those frames, is added to the training set and compared against training on real data alone, yielding a 1–3 AP improvement. However, adding 2,574 new scenes with real labels to the training set will almost always improve a detector, regardless of whether the radar of those scenes is generated or real, or even of mediocre quality. The experiment therefore does not answer "is the generated radar good?"; it only answers "does adding 2,574 labeled scenes help?". The only control that separates the two is to add the real radar of the same 2,574 frames to the training set and measure the improvement. Only if the gain from generated radar is close to the gain from real radar can one claim that the generated tensors "support detector learning"; if it is much smaller, the contribution of the generated tensors is limited. The paper does not include this control. The case for SGDet3D is even weaker: it is a camera–radar fusion detector, and the 2,574 added samples use real images and real labels, with only the radar being generated, so the gain is more likely to come from the extra real data seen by the camera branch.

If the rebuttal confirms (or switches to) a sequence-level split, adds the upper-bound control with real validation-split radar, provides multi-seed results, and the conclusions still hold, I would be willing to raise my score.

In addition, I have a few minor comments:

  1. The semantic-to-permittivity lookup table (classes, materials, values) is given neither in the main text nor in the appendix and should be provided.

  2. The font sizes within the figures vary too much, which makes the figures visually unpolished. Please fix this.

Questions for authors

Please see weakness.

2 comments

Thanks for your careful reading and helpful comments.

Page numbers refer to the revised manuscript.

For the first question, we use a chronological split within each scene. Radar frames are assigned to training, validation, and testing in that order, with no shared frames between the three sets. We have clarified this in Section 4.1 (p. 6) and Appendix A.1 (p. 14) and provided the split files in the supplementary material.

Two observations also provide supporting evidence against memorization. In Figure 5 (p. 10), the edited object configurations are absent from the training set, and the generated radar responds to the addition or removal of vehicles. In the dark tunnel example in Appendix H.2 (p. 26), the visual prior deteriorates in low visibility, and the generated radar also deteriorates in the same region. Simply recalling training radar is difficult to reconcile with both the responses to new object configurations and the failures that follow degraded visual input.

For the second question, we added real radar from the same 2,574 validation frames as a control upper bound and repeated the RTNH experiments with six random seeds. The table below reports 3D AP at IoU 0.5 as mean ±\pm std; full results are in Table 3 (p. 9).

Training data3D AP
Real training set only50.00±2.4250.00 \pm 2.42
Add generated radar54.82±1.0654.82 \pm 1.06
Add real radar from the same frames59.09±0.5459.09 \pm 0.54

You also pointed out that the additional camera images could contribute to the SGDet3D gains. We have compared our generated radar with Latent Diffusion using the same images and annotations (Table 3, p. 9). Our method achieves higher AP in both synthetic and augmentation settings, showing the effect of radar quality on detection. SGDet3D training is slow, so we have so far completed only the Latent Diffusion baseline. The setup is described in Appendix F.5 (p. 19).

We have added the semantic permittivity lookup table in Appendix B.2, Table 7 (p. 15). The mapping follows material modeling in traditional physics simulation. We chose permittivity because it is readily assigned from semantic categories and strongly influences radar reflection strength. We have also revised the figure font sizes and labels.

Thanks for helping us improve the paper. We hope our responses address your concerns. We would be grateful if you could reconsider your score.

Thank you for the detailed rebuttal and additional experiments. I appreciate the real-radar upper-bound comparison and multi-seed results, which address important concerns.

However, the chronological split within each scene is not sequence-disjoint, leaving my original concern about generalization to unseen sequences unresolved. Moreover, generated radar achieves only about half the augmentation gain of real radar, while RGB images cannot uniquely determine material-dependent and multipath responses. Thus, the evidence remains insufficient to establish reliable, generalizable 4D radar generation from images.

Although the rebuttal provides meaningful improvements, my original conditions are not fully met, and I will maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.