- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Combining a triplane representation with LiDAR physical ray sampling to form a two-stage LiDAR scene generation framework, LiTDM, is reasonable. The main experimental results are somewhat competitive. However, there are shortcomings in the incremental novelty, the evidence for conditional generation, and the consistency between training and inference.
Critical strengths
- Performing coarse-to-fine sampling along simulated LiDAR rays is reasonable, coarse sampling covers the ray, and fine sampling refines around the coarse point with the highest occupancy.
- It can be extended to camera/text control and offers good controllability.
Critical weaknesses
- The practicality of Triplane diffusion is limited, and it has already been applied in other fields. Why is it particularly suitable for LiDAR generation tasks?
- During LSPS training, the fine sampling interval is determined by the ground truth (GT), so the model always refines near the true surface; at inference, it is determined by the predicted occupancy. If the coarse sample with the highest occupancy selects the wrong bin, the fine sampling will move to the wrong region, and since points are ultimately selected only from fine samples, how can the coarse samples correct this?
- The occupancy labels used during training are Gaussian soft labels. How is their accuracy ensured?
- During LSPS training, points are selected from M_c + M_f, whereas at inference they are selected from M_f, which leads to a distribution mismatch.
- The triplane resolution and average pooling may create an information bottleneck, could this lead to a loss of fine-grained information?
- The definition of empty-ray labels and the source of the max-pool features in the mask decoder are only introduced at a general conceptual level, the specific operational definitions and sources are not clearly specified.
Questions for authors
Please refer to the weaknesses
5 comments
Official Comment for Reviewer ii2a (2/2)
W4. We thank the reviewer for the careful reading, and we would like to clarify this point. No point is selected during training. The decoder is trained with the occupancy loss on all samples (Eq. 4), and the depth loss, which determines where the surface lies, is computed from the fine samples only (Eqs. 5–7). At inference, the point is likewise selected from the fine samples (Eq. 15), and its intensity is computed from the same fine samples as in training (Eq. 16). Training and inference therefore use the fine samples in the same way.
We believe the confusion comes from our description in Sec. 3.1, which states that the point is selected among the candidates. This sentence is inconsistent with Eq. 15, and we apologize for the confusion.
We also verified that this choice makes no practical difference. On 4,000 generated scans, selecting among all samples returns exactly the same point as selecting among the fine samples on more than 99.99% of rays, and all generation metrics are identical to two decimal places. This is because the fine samples are placed densely around the coarse sample with the highest occupancy, so the highest occupancy along a ray is almost always found among them.
Revision. We will correct the description in Sec. 3.1 to state that the point is selected among the fine samples, consistent with Eq. 15.
W5. We thank the reviewer for raising this point, and we understand the concern that a 1.25 m cell and average pooling may look too coarse for fine-grained geometry. We would like to explain why the point precision is not tied to the cell size in our design.
The triplane cells store features rather than points. The decoder predicts the occupancy at any continuous 3D position from the bilinearly interpolated features of the three planes and the positional encodings, and LSPS then locates each point with fine samples along its ray. Similarly, the average pooling is applied to features that the sparse convolutional encoder extracts at a voxel size of 0.05 m (Table 6), rather than to raw coordinates.
To check this, we reconstructed test scans through the triplane. The median Chamfer distance between the reconstruction and the input is 5 cm, about 4% of the 1.25 m cell size (please refer to our respond to reviewer SKTM). This suggests that the current resolution is sufficient to preserve fine-grained geometry.
Revision. We will add the reconstruction results to the Appendix and clarify this point in the method section.
W6. We thank the reviewer for pointing this out, and we give the precise definitions below.
Empty-ray label. We project the scan onto the 64 × 1024 range-view grid within [1.45, 80] m. A ray is labeled empty when no return of the scan falls into its pixel.
Input of the mask decoder. For each ray, the decoder features of its coarse samples are max-pooled over the samples and fed to a 4-layer MLP (256 units, with a skip connection) that outputs the empty-ray logit.
Revision. We will add the details to the Appendix.
Thank you for the authors’ response. I remain concerned about two points: 1. I would like further analysis of LSPS robustness, specifically whether the fine stage can indeed not recover from errors made at the coarse stage. 2. I agree that the labels are computed from the ground truth and are therefore exact by construction. My concern is whether the generation process itself is robust, for example, whether the threshold of 0.1 is reasonable.
Official Comment for Reviewer ii2a (1/2)
W1. We agree that triplane diffusion has been used in other fields, and we do not claim the triplane itself as our contribution. What makes it particularly suitable for LiDAR generation is that it matches both the structure of outdoor LiDAR scenes and the way a LiDAR scan is formed.
Structure of outdoor scenes. LiDAR returns lie on the surfaces of a wide outdoor scene, such as the ground, building facades and vehicles, which extends much further horizontally than vertically. The xy-plane stores the layout of the scene seen from above, and the two vertical planes store its height structure. The scene is therefore encoded compactly without spending capacity on empty space. A dense voxel grid with the same latent budget has to use a coarser resolution, and under the same pipeline the triplane gives the best overall results among the representations (Table 2).
A field that can be queried by rays. A LiDAR scan is formed by rays that return the first surface along their direction. The triplane defines a continuous 3D field that can be queried at any position, so LSPS casts the rays of a sensor through the generated scene and selects one point per ray, as a real sensor does. A range image, in contrast, is tied to the grid of one sensor. Since the triplane separates the scene from the sensor, a model trained only on KITTI-360 generates scans for the nuScenes sensor without retraining (Table 12).
Practicality. The three planes are tiled into a single 2D map and denoised with the same 2D U-Net as R2DM. With the same number of parameters, it needs fewer FLOPs per denoising step than the range image, and including decoding, LiTDM generates a scene faster than all compared methods (Table 14). The same latent also supports camera- and text-conditional generation.
Revision. We will add this explanation of why the triplane suits LiDAR generation in Sec. 1.
W2. We agree, and we thank the reviewer for this insightful point. Since the fine samples are placed only around the selected coarse bin, the fine stage cannot recover once the coarse stage selects the wrong bin. LSPS therefore relies on the coarse stage to find the correct bin. The coarse samples are directly supervised by the occupancy loss for this purpose, and using the true bin during training gives the fine stage clean supervision around the surface.
The chance of selecting a wrong bin is mainly controlled by the number of coarse samples. Denser coarse samples cover each ray more finely and make a wrong selection less likely, and Table 5 shows that increasing the number of coarse samples greatly improves the perceptual metrics. Each coarse sample, however, requires a query of the triplane and the decoder, so this comes at a higher decoding cost.
A correction step, as the reviewer suggests, would make LSPS more robust without increasing the number of coarse samples. For example, the fine samples could be used to verify the selected bin, and the next most likely coarse bin could be refined when they do not confirm a surface. We find this a valuable direction and will add it to our future work.
Revision. We will clarify the role of the coarse stage in the method section and add this discussion of the coarse sampling density, its cost, and the correction step as future work to the paper.
W3. We would like to clarify how the soft labels are defined. They are not estimated by a model or approximated in any way. They are computed in closed form from the measured LiDAR return of each ray (Eq. 3), so they are exact by construction.
What LSPS uses from the occupancy is where it peaks along each ray, not its absolute value, since the point with the highest occupancy is selected. The label reaches its maximum of 1 exactly at the measured return and decreases with the distance from it. Its peak therefore always coincides with the true surface.
The soft form is what makes this peak learnable. A hard label would mark only the measured point as occupied, but sampled points almost never coincide with it exactly, so nearly every sample would receive 0. A hard label that instead marks all samples within a fixed distance as occupied would be flat inside this distance, so it could not tell which sample is closest to the surface, and it would change abruptly at the boundary. The Gaussian kernel gives a smooth target with a single peak at the measured return, so the sample closest to the surface always receives the highest label. In addition, the surface location is supervised directly by the depth loss (Eq. 7), which compares the expected depth of the fine samples with the measured depth.
Revision. We will clarify in the method section that the labels are computed directly from the measured returns, that only their peak location is used for point selection, and why we use soft labels instead of hard ones.