- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a latent factor-driven framework for unsupervised 3D scene semantic segmentation. It constructs a latent factor library to infer adaptive region-wise activations, capturing scene-dependent semantic structures. These activations are then used to measure semantic similarities between regions and guide global clustering and pseudo-label assignment through optimal transport.
Critical strengths
The overall pipeline is well-motivated and technically reasonable. BN-LFI aims to improve semantic representations through adaptive latent factor induction, while GM-OT leverages these representations for global pseudo-label assignment. The two components address complementary challenges, and the experimental results demonstrate the effectiveness of the proposed framework.
The proposed Counterfactual Calibration is a valuable component, as it attempts to identify and retain latent factors that contribute positively to semantic segmentation under unsupervised settings.
Critical weaknesses
The proposed framework appears to combine existing Bayesian latent factor modeling and optimal transport techniques without introducing a sufficiently novel mechanism that fundamentally integrates the two components. Although the overall pipeline is reasonable, the individual components largely build upon established methods, and their integration seems more sequential than jointly optimized.
Insufficient justification for the methodological complexity. The necessity of introducing Variational IBP for adaptive factor selection remains unclear. Simpler alternatives, such as PCA or sparse encoding, may provide similar representations with lower computational complexity. The paper would benefit from stronger empirical evidence demonstrating why the proposed Bayesian nonparametric formulation is essential.
Questions for authors
Can BN-LFI be replaced with simpler feature decomposition or selection methods, such as PCA or sparse encoding, without significantly degrading pseudo-label quality or downstream segmentation performance?
How much does the adaptive number of active factors contribute to the overall performance? I think it is necessary to compare BN-LFI against variants using a fixed number of factors across all regions.
2 comments
Dear Reviewer e9BX,
Thank you for the time and effort you have devoted to reviewing our paper. As the discussion period is ending very soon, we would like to kindly follow up on our detailed response to your review.
We sincerely appreciate your careful consideration of how BN-LFI and GM-OT interact and whether adaptive factor selection is necessary. Our response explains how provisional assignments inform factor calibration and how the calibrated representation guides final transport. We have also added comparisons with PCA, sparse coding, and fixed-factor-count variants to examine the value of the proposed approach.
We hope these results and clarifications address your main concerns. We would be happy to clarify any remaining point before the discussion deadline.
Best regards,
The authors
Dear Reviewer e9BX,
Thank you for asking us to clarify the interaction between BN-LFI and GM-OT and to compare against simpler alternatives. We answer each weakness and question separately below.
[W1] Coupling between BN-LFI and GM-OT.
BN-LFI and GM-OT are coupled at the mechanism level, rather than connected by a one-way feature handoff. BN-LFI’s region-wise factor posterior produces both induced features and reliability estimates; after fragment aggregation, these determine the fragment-to-slot distances, competitive margins, and reliability-dependent terms used by GM-OT. The interaction also runs in the other direction: a provisional pseudo-label assignment provides a label-free teacher-quality probe, under which candidate factors are suppressed to measure the change in teacher quality, and these effects are used to recalibrate factor activations before the final transport. Thus, factor selection shapes global allocation, while assignment-aware feedback guides factor calibration.
[W2 & Q1] Replacing BN-LFI with PCA and sparse coding.
We thank the reviewer for questioning whether variational IBP is warranted over simpler alternatives. Our goal is not merely to compress region features: pseudo-label construction requires selecting factors for each region and estimating the uncertainty of that selection. PCA uses a shared global subspace; sparse coding permits region-specific sparse coefficients but does not provide posterior factor-inclusion probabilities. Variational IBP supplies these probabilities, which determine region-specific factor usage and inform reliability-aware GM-OT assignment. To test whether simpler representations can replace BN-LFI, we conducted an additional experiment on S3DIS Area 5. We replaced BN-LFI with PCA or sparse coding while retaining the GM-OT assignment framework and segmentation backbone. The results are summarized in the table below.
As the table shows, replacing BN-LFI lowers mIoU by 3.02 points with PCA and by 2.52 points with sparse coding. BN-LFI also achieves the highest mAcc. These results support the value of posterior-based adaptive factor selection in our pseudo-label pipeline.
We have reported the results of the experiments suggested by the reviewer in the Appendix G of the revised manuscript.
[Q2] Adaptive versus fixed factor.
BN-LFI models factor activations as binary latent variables but constructs region representations using their posterior expectations, rather than thresholded binary selections. We therefore report the expected number of active factors per region, computed as ∑kϕik. On S3DIS Area 5, its median is 11.66, with an interquartile range of 8.70–15.10, showing that factor usage varies considerably across regions.
To directly evaluate the contribution of adaptive factor usage, we conduct an additional comparison on S3DIS Area 5 against fixed-size variants with K=8,12,16. For each region, these variants retain only the K largest posterior-weighted factor responses and set the remaining responses to zero. Factor identities thus remain region-dependent, but the number of retained factors is fixed across regions. The results are summarized in the table below.
As the table shows, Ours achieves 51.89% mIoU, outperforming the strongest fixed-size variant (K=16) by 1.66 percentage points. A shared factor budget may discard useful responses in regions requiring more factors or retain unnecessary responses in regions requiring fewer. The advantage over all three fixed-size variants therefore supports the benefit of adapting factor usage to individual regions.
We have reported the results of the experiments suggested by the reviewer in the Appendix G of the revised manuscript.