acceptodds
Under review as a conference paper at ICLR 2027

LiDAR Scene Synthesis with Triplane Diffusion Models via LiDAR Simulation-based Coarse-to-Fine Point Sampling

Abstract

Realistic LiDAR scene synthesis is critical for autonomous driving perception systems, yet existing methods struggle to simultaneously capture the distinctive scanning patterns, geometric structures, and semantic properties of real-world data. To tackle these challenges, we present the first two-stage framework, called LiTDM, that leverages triplane representation as a latent space proxy for diffusion models tailored to LiDAR scene synthesis. First, a triplane-based autoencoder is trained to predict point-based occupancy and intensity with a novel ray sampling strategy. Then, we sample the diffusion model trained on triplane features to generate realistic LiDAR scenes. With triplane features as a proxy, our method achieves pattern realism, geometric accuracy, and semantic coherence. Moreover, our proposed LiDAR Simulation-based Point Sampling (LSPS) ensures the characteristic scanning patterns of real sensors. Experimental results show that our framework generates highly realistic LiDAR scenes against the baselines. Furthermore, the flexibility of LiTDM extends beyond unconditional synthesis, enabling diverse conditional generation tasks such as camera-to-LiDAR and text-to-LiDAR generation. Code will be released to support reproducibility.

Primary area:
applications to robotics, autonomy, planning
Keywords:
LiDAR Diffusion Models, Triplane, Coarse-to-fine sampling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ii2aOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

Combining a triplane representation with LiDAR physical ray sampling to form a two-stage LiDAR scene generation framework, LiTDM, is reasonable. The main experimental results are somewhat competitive. However, there are shortcomings in the incremental novelty, the evidence for conditional generation, and the consistency between training and inference.

Critical strengths
  1. Performing coarse-to-fine sampling along simulated LiDAR rays is reasonable, coarse sampling covers the ray, and fine sampling refines around the coarse point with the highest occupancy.
  2. It can be extended to camera/text control and offers good controllability.
Critical weaknesses
  1. The practicality of Triplane diffusion is limited, and it has already been applied in other fields. Why is it particularly suitable for LiDAR generation tasks?
  2. During LSPS training, the fine sampling interval is determined by the ground truth (GT), so the model always refines near the true surface; at inference, it is determined by the predicted occupancy. If the coarse sample with the highest occupancy selects the wrong bin, the fine sampling will move to the wrong region, and since points are ultimately selected only from fine samples, how can the coarse samples correct this?
  3. The occupancy labels used during training are Gaussian soft labels. How is their accuracy ensured?
  4. During LSPS training, points are selected from M_c + M_f, whereas at inference they are selected from M_f, which leads to a distribution mismatch.
  5. The triplane resolution and average pooling may create an information bottleneck, could this lead to a loss of fine-grained information?
  6. The definition of empty-ray labels and the source of the max-pool features in the mask decoder are only introduced at a general conceptual level, the specific operational definitions and sources are not clearly specified.
Questions for authors

Please refer to the weaknesses

5 comments
AuthorsOpenReview ↗

Official Comment for Reviewer ii2a (1/2)

W1. We agree that triplane diffusion has been used in other fields, and we do not claim the triplane itself as our contribution. What makes it particularly suitable for LiDAR generation is that it matches both the structure of outdoor LiDAR scenes and the way a LiDAR scan is formed.

Structure of outdoor scenes. LiDAR returns lie on the surfaces of a wide outdoor scene, such as the ground, building facades and vehicles, which extends much further horizontally than vertically. The xy-plane stores the layout of the scene seen from above, and the two vertical planes store its height structure. The scene is therefore encoded compactly without spending capacity on empty space. A dense voxel grid with the same latent budget has to use a coarser resolution, and under the same pipeline the triplane gives the best overall results among the representations (Table 2).

A field that can be queried by rays. A LiDAR scan is formed by rays that return the first surface along their direction. The triplane defines a continuous 3D field that can be queried at any position, so LSPS casts the rays of a sensor through the generated scene and selects one point per ray, as a real sensor does. A range image, in contrast, is tied to the grid of one sensor. Since the triplane separates the scene from the sensor, a model trained only on KITTI-360 generates scans for the nuScenes sensor without retraining (Table 12).

Practicality. The three planes are tiled into a single 2D map and denoised with the same 2D U-Net as R2DM. With the same number of parameters, it needs fewer FLOPs per denoising step than the range image, and including decoding, LiTDM generates a scene faster than all compared methods (Table 14). The same latent also supports camera- and text-conditional generation.

Revision. We will add this explanation of why the triplane suits LiDAR generation in Sec. 1.

W2. We agree, and we thank the reviewer for this insightful point. Since the fine samples are placed only around the selected coarse bin, the fine stage cannot recover once the coarse stage selects the wrong bin. LSPS therefore relies on the coarse stage to find the correct bin. The coarse samples are directly supervised by the occupancy loss for this purpose, and using the true bin during training gives the fine stage clean supervision around the surface.

The chance of selecting a wrong bin is mainly controlled by the number of coarse samples. Denser coarse samples cover each ray more finely and make a wrong selection less likely, and Table 5 shows that increasing the number of coarse samples greatly improves the perceptual metrics. Each coarse sample, however, requires a query of the triplane and the decoder, so this comes at a higher decoding cost.

A correction step, as the reviewer suggests, would make LSPS more robust without increasing the number of coarse samples. For example, the fine samples could be used to verify the selected bin, and the next most likely coarse bin could be refined when they do not confirm a surface. We find this a valuable direction and will add it to our future work.

Revision. We will clarify the role of the coarse stage in the method section and add this discussion of the coarse sampling density, its cost, and the correction step as future work to the paper.

W3. We would like to clarify how the soft labels are defined. They are not estimated by a model or approximated in any way. They are computed in closed form from the measured LiDAR return of each ray (Eq. 3), so they are exact by construction.

What LSPS uses from the occupancy is where it peaks along each ray, not its absolute value, since the point with the highest occupancy is selected. The label reaches its maximum of 1 exactly at the measured return and decreases with the distance from it. Its peak therefore always coincides with the true surface.

The soft form is what makes this peak learnable. A hard label would mark only the measured point as occupied, but sampled points almost never coincide with it exactly, so nearly every sample would receive 0. A hard label that instead marks all samples within a fixed distance as occupied would be flat inside this distance, so it could not tell which sample is closest to the surface, and it would change abruptly at the boundary. The Gaussian kernel gives a smooth target with a single peak at the measured return, so the sample closest to the surface always receives the highest label. In addition, the surface location is supervised directly by the depth loss (Eq. 7), which compares the expected depth of the fine samples with the measured depth.

Revision. We will clarify in the method section that the labels are computed directly from the measured returns, that only their peak location is used for point selection, and why we use soft labels instead of hard ones.

AuthorsOpenReview ↗

Official Comment for Reviewer ii2a (2/2)

W4. We thank the reviewer for the careful reading, and we would like to clarify this point. No point is selected during training. The decoder is trained with the occupancy loss on all Mc+MfM_c + M_f samples (Eq. 4), and the depth loss, which determines where the surface lies, is computed from the fine samples only (Eqs. 5–7). At inference, the point is likewise selected from the fine samples (Eq. 15), and its intensity is computed from the same fine samples as in training (Eq. 16). Training and inference therefore use the fine samples in the same way.

We believe the confusion comes from our description in Sec. 3.1, which states that the point is selected among the Mc+MfM_c + M_f candidates. This sentence is inconsistent with Eq. 15, and we apologize for the confusion.

We also verified that this choice makes no practical difference. On 4,000 generated scans, selecting among all Mc+MfM_c + M_f samples returns exactly the same point as selecting among the MfM_f fine samples on more than 99.99% of rays, and all generation metrics are identical to two decimal places. This is because the fine samples are placed densely around the coarse sample with the highest occupancy, so the highest occupancy along a ray is almost always found among them.

Revision. We will correct the description in Sec. 3.1 to state that the point is selected among the fine samples, consistent with Eq. 15.

W5. We thank the reviewer for raising this point, and we understand the concern that a 1.25 m cell and average pooling may look too coarse for fine-grained geometry. We would like to explain why the point precision is not tied to the cell size in our design.

The triplane cells store features rather than points. The decoder predicts the occupancy at any continuous 3D position from the bilinearly interpolated features of the three planes and the positional encodings, and LSPS then locates each point with fine samples along its ray. Similarly, the average pooling is applied to features that the sparse convolutional encoder extracts at a voxel size of 0.05 m (Table 6), rather than to raw coordinates.

To check this, we reconstructed test scans through the triplane. The median Chamfer distance between the reconstruction and the input is 5 cm, about 4% of the 1.25 m cell size (please refer to our respond to reviewer SKTM). This suggests that the current resolution is sufficient to preserve fine-grained geometry.

Revision. We will add the reconstruction results to the Appendix and clarify this point in the method section.

W6. We thank the reviewer for pointing this out, and we give the precise definitions below.

Empty-ray label. We project the scan onto the 64 × 1024 range-view grid within [1.45, 80] m. A ray is labeled empty when no return of the scan falls into its pixel.

Input of the mask decoder. For each ray, the decoder features [h(p),PE(p),RPE(p)][h(\mathbf{p}), \mathrm{PE}(\mathbf{p}), \mathrm{RPE}(\mathbf{p})] of its McM_c coarse samples are max-pooled over the samples and fed to a 4-layer MLP (256 units, with a skip connection) that outputs the empty-ray logit.

Revision. We will add the details to the Appendix.

Thank you for the authors’ response. I remain concerned about two points: 1. I would like further analysis of LSPS robustness, specifically whether the fine stage can indeed not recover from errors made at the coarse stage. 2. I agree that the labels are computed from the ground truth and are therefore exact by construction. My concern is whether the generation process itself is robust, for example, whether the threshold of 0.1 is reasonable.

Reviewer_jEnCOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a LiDAR generation method using triplane representation, where it first trains a Triplane VAE that compresses point cloud into 3 planes and proposes a sampling method to recover point cloud from them. A diffusion unet is then trained to generate these planes from noise and, optionally, conditioned on image or text. The paper is validated by thorough experimentation with some interesting results such as cross-sensor generation, but the novelty remains limited with some concern on quality of baselines.

Critical strengths
  • Ablation studies have good coverage for all claimed contributions, such as the LSPS and representation choice
  • Thorough experimentation that support the method, with interesting conditional and cross-sensor generations
Critical weaknesses
  • The benefit of triplane is not very clearly demonstrated. In terms of compact representation, range images are arguably better because that is how LiDAR captured in real life, and can also be processed with 2D convolution. Quality-wise, I remain skeptical due to quality concern in question 1.
  • Although the pipeline using triplane is technically sound, the novelty seems limited. LSPS uses techniques commonly used in NeRF, which can also be seen in LiDAR-related methods such as LiDAR-NeRF [1] and NFL [2]. The VAE and Diffusion follow the standard latent diffusion pipeline with little to additional insights, and is not the first work to do so in the LiDAR domain either [3,4].

[1] Tang et. al., LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields, ACM MM 2024 [2] Huang et. al., NFL : Neural LiDAR Fields for Novel View Synthesis, ICCV 2023 [3] Zyrianov et. al., Learning to Generate Realistic LiDAR Point Clouds, ECCV 2024 [4] Ran et. al., Towards Realistic Scene Generation with LiDAR Diffusion Models, CVPR 2024

Questions for authors

Questions:

  • LiDM’s (one of the SOTA range-image LiDAR generation) numbers in Table 1 seem to be much higher than reported in both its original paper and follow-up works (such as R2Flow [1]), despite using the same dataset (KITTI360). Is there a difference in experimentation settings that might explain this degradation in their performance? Table 2 also mention that the curve-based latent baseline also follows LiDM, and yet their performance is significantly better than the number cited in table 1.

Suggestion:

  • Cross-sensor generation (Table 12) is a benefit of triplane (although not exclusively, i.e., background mesh generation from LidarDM [2]), and I think it deserves to be moved to the main paper with some visuals.

[1]: Nakashima et. al., Fast LiDAR Data Generation with Rectified Flows, ICRA 2025 [2]: Zyrianov et. al., LidarDM: Generative LiDAR Simulation in a Generated World, ICRA 2025

7 comments
AuthorsOpenReview ↗

Official Comment for Reviewer jEnC (1/3)

W1. We thank the reviewer for the question.

Benefit of the triplane. We agree that the range image is the native format of a LiDAR sensor and can be processed with 2D convolutions. However, a range image describes how one sensor observes a scene, not the scene itself. The triplane keeps the advantage of 2D processing, since its three planes are tiled into a single map (192 × 192 in LiTDM) and generated by the same 2D U-Net as RV methods (e.g., R2DM), while it represents the scene in metric 3D space. This brings three benefits. (1) Efficiency. The tiled triplane has 44% fewer spatial positions than the 64 × 1024 range image. With the same denoiser, this reduces the cost per step from 229.0 G to 90.3 G FLOPs and from 27.6 to 15.4 ms (Table 14). (2) Spatial structure. Neighboring triplane cells are neighbors in 3D, whereas neighboring range-image pixels can lie on different objects at depth discontinuities. Consistent with this, the largest improvement over R2DM is in BEV MMD (1.93×10−41.93 \times 10^{-4} vs. 4.58×10−44.58 \times 10^{-4}), which compares the spatial layout of individual scans. (3) Sensor independence. Since the scan pattern is determined only at decoding, the same model can be decoded at a higher resolution (Table 3, 64 × 2048) or in the nuScenes sensor configuration without retraining (Table 12).

Regarding quality, the issue in Q1 concerns only the evaluation range of LiDM and does not affect the comparisons that isolate the representation. R2DM covers the full 1.45 ∼ 80 m range and shares our denoiser, and the triplane improves all four metrics over it (Table 1). In Table 2, all representations, including the curve-based latent of LiDM, are trained on the same split and range with the same diffusion model, and the triplane is the best on all four metrics. Within 56 m, LiTDM also outperforms the released LiDM model on three of the four metrics (please see our response to Q1).

Revision. We will state these benefits in Sec. 1 and add the spatial size and per-step cost of the range image and the tiled triplane to Sec. 4.3.

AuthorsOpenReview ↗

Official Comment for Reviewer jEnC (2/3)

W2. We thank the reviewer for finding the pipeline technically sound. We address the two novelty concerns below.

LSPS and NeRF-based LiDAR methods [1, 2]. We agree that LSPS shares the coarse-to-fine sampling principle with NeRF-based methods, and that NFL [2] is the closest prior work. It also locates a peak with coarse samples and refines within a window around it. The key difference lies in the problem being solved. [1, 2] reconstruct a single static scene from many posed sweeps through per-scene optimization, and both list generalization across scenes as a limitation. In contrast, LSPS is the decoder of a generative model, a single network shared by all scenes that decodes latents sampled by the diffusion model. This difference leads to the following design choices. (1) Supervision. Since the training data are single sweeps, LSPS supervises samples directly with soft labels relative to the measured return (Eq. 3) rather than through transmittance-based rendering, and trains the fine stage on the bin containing the true return (Alg. 1). (2) Readout. Each ray returns its argmax sample, i.e., a single point on one surface, and a per-ray emptiness head determines whether the ray returns at all. NFL instead outputs a local weighted mean and composites per-sample drop probabilities. In short, LSPS shares its sampling principle with [1, 2] but uses it to simulate the sensor on generated scenes rather than to render a fitted one, and we hope the reviewer will consider this as a methodological contribution.

Latent diffusion pipeline and prior works [3, 4]. We agree that LiTDM is not the first latent diffusion model for LiDAR, and we do not claim so. LiDARGen [3] pioneered diffusion-based LiDAR generation by applying score-based diffusion to range images, and LiDM [4] showed that latent diffusion with a curve-wise compression that preserves the scanning pattern substantially improves both realism and efficiency. LiTDM builds on these advances, and its insight lies in separating the scene from the sensor. In [3] and [4], the representation is defined on the sensor's image grid, so the scene content and the scan pattern are represented jointly, which is natural for faithfully modeling a given sensor. LiTDM instead generates a sensor-independent 3D triplane and obtains the scan pattern through LSPS at decoding time. Our experiments support this design. With the rest of the pipeline fixed, the triplane outperforms the curve-based latent (Table 2, FSVD 29.35 vs. 41.82). In addition, the same generated scenes can also be decoded for a different sensor (Table 12). We intentionally kept the diffusion stage identical to R2DM, so that the improvements over R2DM (Table 1, FSVD 34.31 to 29.35, MMD 4.58×10−44.58 \times 10^{-4} to 1.93×10−41.93 \times 10^{-4}) can be attributed to the representation and decoding, while requiring 60.6% fewer FLOPs per step (Table 14).

Revision. We will discuss [1, 2] and clarify in the related work how LiTDM builds on [3, 4].

AuthorsOpenReview ↗

Official Comment for Reviewer jEnC (3/3)

Q1. We thank the reviewer for raising this point. The difference stems from the evaluation range. Our benchmark follows the KITTI-360 setting of R2DM, in which scans cover 1.45 ∼ 80 m, and FSVD and FPVD average the features of each scan over depth partitions up to 80 m. In contrast, LiDM was evaluated within 56 m in its original paper, and its released samples also lie within this range. Since we evaluated all methods with the same 80 m protocol, the partitions beyond 56 m are empty for every LiDM sample, and this strongly increases its FSVD and FPVD.

We keep 80 m as the main protocol so that the metrics cover the entire scan that the other methods generate, including the sparse far region. To compare LiDM within its original range, we additionally evaluated both methods under a common 56 m protocol in the following table, in which all generated and real scans are cropped to 56 m and the reference statistics are recomputed within this range. Within 56 m, LiDM performs strongly and achieves a lower FSVD than LiTDM, while LiTDM is better on FPVD, JSD, and MMD. Please note that LiDM was trained on both the train and test sets.

Table 2 shows the same point from the representation side. The curve-based row uses the curve-wise latent representation of LiDM, trained on our training split over the full 1.45–80 m range and combined with our diffusion model. In this setting, it achieves lower JSD and MMD than the RV, BEV, and voxel representations, and its JSD is close to that of the triplane. This experiment shows the benefit of LiDM's representation when it is trained and evaluated over the same range, while the triplane remains better on all four metrics.

MethodProtocolFSVD ↓FPVD ↓JSD ↓ (×10−2\times 10^{-2})MMD ↓ (×10−4\times 10^{-4})
LiDM80 m217.28231.9117.8935.40
R2DM80 m34.3137.224.734.58
LiTDM (ours)80 m29.3535.234.591.93
LiDM56 m29.6435.2117.5935.90
R2DM56 m33.4332.194.834.71
LiTDM (ours)56 m31.8130.304.431.86

Revision. We will add the 56 m results to Table 1 together with the current ones and describe the evaluation range of each protocol in Sec. 4.1. We will also state in Sec. 4.3 that the curve-based row follows the representation of LiDM and shows its benefit under the full-range setting.

S1. We sincerely thank the reviewer for this suggestion. Following it, we have moved the cross-sensor results (Table 12) to the main paper and added a qualitative figure, which can be found in Fig. 3 of the revised PDF. We also agree that this property is not exclusive to the triplane, and we now cite LidarDM, which achieves it by generating the scene as a mesh.

Revision. In the revised PDF, we have moved Table 12 to the main paper with the new qualitative figure and added a citation to LidarDM.

Reviewer_SKTMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents LiTDM, a triplane-based diffusion framework for LiDAR scene generation. The method first learns a compact triplane representation from LiDAR point clouds, then performs diffusion-based generation in the learned feature space. A LiDAR simulation-based coarse-to-fine point sampling strategy (LSPS) is introduced to decode the generated triplane features into LiDAR points through occupancy, intensity, and empty-ray predictions. The experiments show improvements in generation quality, computational efficiency, and flexibility across different LiDAR sensor configurations.

Critical strengths

Although triplane diffusion and coarse-to-fine ray sampling have been explored in previous work, their adaptation to LiDAR generation is meaningful. In particular, the learned 3D representation allows LiDAR points to be generated under different sensor configurations without retraining.

The paper provides comprehensive comparisons and ablation studies. The improvements in generation quality are convincing, and the cross-sensor experiments and computational efficiency analysis further support the practical value of the proposed framework.

Critical weaknesses

The overall framework is closely related to existing triplane-based 3D scene generation methods, particularly SemCity. Similarly, coarse-to-fine ray sampling has been widely explored in neural rendering. The LiDAR-specific adaptation is useful, but the methodological advances beyond these existing techniques are relatively limited.

Table 2 compares several representations, but the voxel baseline uses dense 3D convolutions. Sparse voxel and other 3D scene-based generation approaches are not evaluated. This makes it hard to fully assess the advantage of triplanes over other 3D representations for LiDAR generation.

Questions for authors

The paper demonstrates that triplanes outperform the RV, BEV, voxel, and curve-based variants (Table 2). However, the RV row has exactly the same numbers as R2DM in Table 1, so it does not seem to be a variant trained under the same setup as claimed (L415). Could you clarify this? And how would a sparse voxel representation compare?

Table 1 does not include recent methods on the same unconditional LiDAR generation task on KITTI-360, such as TopoLiDM (IROS 2025) and OLiDM (AAAI 2025), which are both cited in the paper. Could you add these comparisons under the same evaluation protocol, and specify the number of generated samples used for each metric?

How well does the triplane autoencoder reconstruct the input scans (e.g., Chamfer distance or depth error)? With a triplane resolution of 1.25 m per cell (Table 6), this seems to set the upper bound for generation quality. Also, the model predicts intensity, but intensity is never evaluated. How good is it?

The text says (64, 8) degrades both metrics and (160, 20) gives the best overall performance (L429–431). But in the table, (64, 8) actually has the best JSD and MMD, and (128, 16) has a better FSVD than (160, 20). Could you clarify this?

3 comments
AuthorsOpenReview ↗

Official Comment for Reviewer SKTM (1/3)

W1. We thank the reviewer for acknowledging that the LiDAR-specific adaptation is useful. We respectfully clarify below how LiTDM goes beyond SemCity and beyond coarse-to-fine sampling in neural rendering.

Relation to SemCity. We agree that LiTDM shares the triplane and 2D diffusion structure with SemCity. However, the two methods encode different data and solve different decoding problems. SemCity encodes dense semantic labels of a completed voxel volume (built by superimposing many future scans, covering a forward-facing 51.2 × 51.2 × 6.4 m region) with a dense 3D CNN, and decodes them at 0.2 m voxel centers for semantic scene completion. LiTDM instead encodes a single raw sweep of about 120k points with a sparse convolutional encoder over a 200 × 200 × 40 m scene (about 95× the volume), and decodes occupancy, intensity, and ray drop at continuous positions. As a result, the latent resolution is decoupled from the output precision: our triplane cells (1.25 m) are coarser than SemCity's (0.4 m), yet points are decoded at about 2.5 cm spacing along each ray.

The choice of the triplane is also supported beyond SemCity. Under the same pipeline, it outperforms RV, BEV, voxel, and curve-based latents (Table 2). To our knowledge, existing triplane diffusion models for driving scenes (e.g., SemCity, X-Scene, DynamicCity) generate occupancy, and LiTDM is the first to generate LiDAR scans from a triplane latent.

Coarse-to-fine ray sampling in neural rendering. We agree that LSPS shares the coarse-to-fine sampling principle with neural rendering, and we do not claim this sampling strategy itself as new. We appreciate that the reviewer finds its adaptation to LiDAR generation meaningful. The key difference lies in the problem being solved. In neural rendering, it is typically applied to a single static scene reconstructed through per-scene optimization, as in NeRF. In contrast, LSPS is the decoder of a generative model, a single network shared by all scenes that decodes latents sampled by the diffusion model. This difference leads to the following design choices. (1) Supervision. Since the training data are single sweeps, LSPS supervises samples directly with soft labels relative to the measured return (Eq. 3) rather than through transmittance-based rendering, and trains the fine stage on the bin containing the true return (Alg. 1). (2) Readout. Each ray returns its argmax sample, i.e., a single point on one surface, and a per-ray emptiness head determines whether the ray returns at all. Neural rendering methods instead output a weighted mean along the ray and typically composite per-sample drop probabilities.

In short, SemCity's triplanes alone yield semantic voxels, and per-scene neural rendering alone renders a single fitted scene. LiTDM combines a triplane that represents the scene with LSPS that simulates the sensor, and we hope the reviewer will consider these differences as a methodological contribution.

Revision. We will add this comparison with SemCity and NeRF-based LiDAR methods to the related work.

AuthorsOpenReview ↗

Official Comment for Reviewer SKTM (2/3)

W2. We thank the reviewer for this valuable suggestion. We agree that a comparison with sparse voxels helps to assess the advantage of the triplane, and we conducted this comparison during the rebuttal period. Since the occupied voxels differ from scene to scene and have to be generated as well, a sparse-voxel latent cannot be evaluated by only replacing the latent in our pipeline. We therefore extended our framework into a two-stage generator. The first diffusion model generates the occupied regions of the scene, and the second diffusion model, built on a sparse 3D U-Net, generates the voxels and their latent features within these regions. The latent is obtained from a voxel autoencoder at the same 1.25 m resolution as the triplane, which follows the encoder, decoder and LSPS of LiTDM and keeps only the occupied voxels. The two diffusion models are trained with a total training cost comparable to that of LiTDM, and 2,000 samples are evaluated under the same protocol as Table 2.

RepresentationFSVD ↓FPVD ↓JSD ↓ (×10−2\times 10^{-2})MMD ↓ (×10−4\times 10^{-4})
Sparse voxel30.3832.415.504.54
LiTDM (Triplane)29.3535.234.591.93

LiTDM achieves better FSVD, JSD and MMD, with a particularly large margin in MMD, while the sparse voxel achieves a lower FPVD.

We note that this variant differs from a standalone sparse-voxel generator, since it is built within our framework and produces its scans through our decoder and LSPS. As shown in Tables 3 and 5, LSPS has a large effect on the perceptual metrics, so the result of this variant reflects our framework as well as the representation. It also shows that our framework can be extended beyond the triplane. Overall, the triplane achieves better results with a single fixed-size latent and a single diffusion model, whereas the sparse voxel requires two diffusion models applied in sequence for both training and generation.

Revision. We will add the sparse-voxel variant to Table 2 and describe its construction in the appendix.

Q1. We thank the reviewer for checking this carefully. The identical numbers are intended. Table 2 changes only the representation that the diffusion model generates, while the diffusion model itself is fixed to R2DM's denoising U-Net and DDPM formulation (Sec. 3.2). For the RV, the range image is itself the representation, so no autoencoder or LSPS is involved, and this variant of our pipeline coincides with R2DM. We therefore reported R2DM for this row, just as the triplane row is identical to LiTDM in Table 1.

Revision. We will clarify in Sec. 4.3 and the caption of Table 2 that the RV variant coincides with R2DM.

Q2. We thank the reviewer for the suggestion. We evaluated TopoLiDM with its official checkpoint and the default settings of its released sampling code, under the same protocol as Table 1. As for all methods in Table 1, every metric is computed from 2,000 generated scans. FSVD and FPVD are computed on raw points within 1.45 ∼ 80 m against all 26,367 KITTI-360 test scans, and BEV JSD and MMD on points within 3 ∼ 63 m against 10,000 randomly selected test scans. Since TopoLiDM generates points only up to 56 m, we additionally evaluate both methods with the maximum depth of every metric reduced to 56 m for generated and real scans, which removes the penalty for its missing far-range points. LiTDM outperforms TopoLiDM on all four metrics in both settings.

MethodProtocolFSVD ↓FPVD ↓JSD ↓ (×10−2\times 10^{-2})MMD ↓ (×10−4\times 10^{-4})
TopoLiDM80 m239.18254.4917.6834.10
LiTDM (ours)80 m29.3535.234.591.93
TopoLiDM56 m59.6566.1417.3634.60
LiTDM (ours)56 m31.8130.304.431.86

OLiDM. OLiDM has not released its code or checkpoints, so we could not evaluate it under the same protocol.

Revision. We will add TopoLiDM to Table 1 under both protocols and state the number of generated and reference scans for each metric in Sec. 4.1.

AuthorsOpenReview ↗

Official Comment for Reviewer SKTM (3/3)

Q3. We thank the reviewer for pointing out that the reconstruction and the intensity were not evaluated, and we now report both. The intensity error is measured on the same point pairs as the Chamfer distance, that is, between each point and its nearest neighbor in the other scan.

Reconstruction → InputInput → Reconstruction
Chamfer distance, mean (m)0.0810.093
Chamfer distance, median (m)0.0510.050
Intensity MAE0.0690.067

The reconstruction error is far below the 1.25 m cell size, with a median of 5 cm. This is because the triplane cells do not store points. They store features, and the decoder predicts occupancy at any continuous 3D position from the bilinearly interpolated features of the three planes and the positional encoding of that position. LSPS then places fine samples around the surface along each ray, so each point is located by these samples rather than by the cell grid. The cell size therefore determines how much scene detail the latent can hold, but not how precisely each point is placed. The intensity is also reconstructed well, with a mean absolute error of about 0.07.

Revision. We will add these results to the Appendix.

Q4. We thank the reviewer for pointing this out and clarify our description. In the text, "both metrics" refers to the two perceptual metrics, FSVD and FPVD.

With (64, 8), the coarse samples are too sparse to localize the surface precisely along each ray. The overall layout of the points is still preserved, so JSD and MMD, which compare statistical distributions on BEV, remain comparable to or even slightly better than the other settings. The local 3D structure, however, becomes inaccurate, and FSVD and FPVD, which capture it, degrade substantially.

We chose (160, 20) because it performs best overall rather than on every single metric. It achieves the best FPVD and an FSVD close to the best, while keeping JSD and MMD comparable to the other settings. In contrast, (64, 8) and (160, 1) reach similar BEV metrics but much worse perceptual metrics.

Revision. We will revise the text to state that "both metrics" refers to FSVD and FPVD, and that (160, 20) is chosen for its best overall balance across the four metrics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.