- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper adapts vision image generation model, which can produce multi-task outputs such as depth and segmentation, to support radio-frequency (RF) sensor input. The main idea is to consider align the RF input embedding with latent embedding of vision image generation model such that the output capability of image generation model can be preserved. The experiment shows that the proposed method can achieve better depth estimation performance than task-specific model and demonstrate strong zero-shot performance on other tasks.
Critical strengths
The motivation of adapting the vision image generation model to support multi-task prediction of RF sensor is clear. I think it is a way to address the limited data issue of RF sensing.
Critical weaknesses
I think the main limitation of this work is that it does not clearly make ablations of the proposed alignment modules. Another thing is the paper does not demonstrate the proposed method can be useful across different vision generation models.
Questions for authors
-
The overall method pipeline figure is not very clear to me to understand how the proposed alignment method is implemented. Could you please provide a detailed architecture illustration figure in the paper/appendix or clarify on some details for reproduction, eg, what parts of the model are frozen or trained during the proposed alignment stage?
-
From the qualitative results of the proposed method, it looks like some of the predicted outputs have distortion effects especially if you zoom into some geometries that are flat/straight in line. Since vision image generation models typically do not use non-planar images as input, could you provide more analysis on how well the proposed adaptation method address the planar image input to non-planar image input domain gap?
2 comments
Kind Follow-up on Rebuttal Discussion
Dear Reviewer,
Thank you for your constructive feedback. We have carefully addressed your concerns in our rebuttal and provided additional experimental results. As the discussion period is approaching its end, we would greatly appreciate it if you could review our responses and let us know whether any concerns remain.
Thank you for your time and consideration.
Response to Reviewer LPsG
We thank the reviewer for the detailed review and address the points one by one below.
Q1. Training details and ablations
The revised Figure 2(a) explicitly shows training and gradient paths. The RF adapter is a nine-residual-block ResNet mapping a 256×192×64 heatmap to a 16×96×96 condition latent. Representation alignment compares this with the frozen VAE encoding of the paired LiDAR crop. The RF condition tokens, noisy task-target tokens, and text embeddings are supplied to the generative backbone. The training proceeds in three parts.
Table R1 quantifies each loss and task-set variant, and Table R2 separates the RF adapter from LoRA. These controls show that functional optimization of the RF adapter enables transfer even when the entire pretrained model is fixed. Relative-depth results bypass the metric decoder to avoid conflating these stages.
Table R1. Ablation of the alignment objectives. Lmse and Ldense are the two terms of Lrep (Eq. 4). K is the functional alignment task set, with D for depth and E for entity segmentation. Relative depth is the direct output of DICEPTION.
Table R2. Separating the RF adapter from LoRA. The adapter Aϕ and LoRA Δθ columns give the objective each component is trained with. Normal, pose, and instance are zero-shot in all rows. Relative depth is the direct output of DICEPTION.
Q2. Straight-line distortion
We analyze the distortion from three aspects.
(1) The curvature is expected. PanoRadar represents RF and LiDAR on an azimuth-elevation grid. Its wide field of view causes wide-angle distortion, so a straight 3D edge can appear curved in this spherical projection. The LiDAR ground truth shows the same curvature.
(2) Distortion also depends on RF accuracy. Since RF quality degrades with distance, we evaluate the depth error at different distances (Table R3). Depth MAE increases from 0.145 m at 0 to 3 m to 0.393 m at 6 to 10 m. This reflects a real limitation. Accuracy degrades as distance grows.
(3) Depth error reflects 3D error. Each pixel of the depth image corresponds to a ray with a known azimuth and elevation. Using the angular parameters of the LiDAR, the depth image can be projected into a 3D point cloud. The 3D error of each point then equals its depth error along the ray. Therefore, depth error reflects the geometric error in 3D space.
Table R3. Depth error of RFception by distance.