acceptodds
Under review as a conference paper at ICLR 2027

IFOT-SSeg: Induced Factor Optimal Transport for Unsupervised Semantic Segmentation of 3D Point Clouds

Abstract

Unsupervised 3D point cloud segmentation often relies on clustering learned region features, but mixed latent factors and unreliable fragment-to-primitive pseudo-label assignment jointly limit the quality of pseudo supervision. To address this problem, we propose a latent factor-driven global pseudo-label inference framework that first recovers semantic-relevant latent factors from neural region features and then performs globally competitive fragment-to-primitive assignment to generate reliable pseudo-labels. Specifically, Bayesian Nonparametric Latent Factor Induction (BN-LFI) models neural region features as observations from an open-ended latent factor space and combines ICA-style factor construction with variational Indian Buffet Process inference to adaptively recover active factors underlying the current scene semantics. These factors are further calibrated using unsupervised quality signals, yielding a more reliable latent representation for subsequent pseudo-label inference. Based on the induced representation, Global Margin-Aware Optimal Transport (GM-OT) formulates fragment-to-primitive-slot matching as a globally competitive transport problem, suppressing dominant primitive absorption and slot redundancy while producing more stable hard pseudo-labels. The framework operates only during pseudo-label construction, introduces no additional inference overhead, and outperforms existing unsupervised 3D point cloud segmentation methods on ScanNet, S3DIS, and nuScenes.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
ML: Bayesian Learning, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_e9BXOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a latent factor-driven framework for unsupervised 3D scene semantic segmentation. It constructs a latent factor library to infer adaptive region-wise activations, capturing scene-dependent semantic structures. These activations are then used to measure semantic similarities between regions and guide global clustering and pseudo-label assignment through optimal transport.

Critical strengths

The overall pipeline is well-motivated and technically reasonable. BN-LFI aims to improve semantic representations through adaptive latent factor induction, while GM-OT leverages these representations for global pseudo-label assignment. The two components address complementary challenges, and the experimental results demonstrate the effectiveness of the proposed framework.

The proposed Counterfactual Calibration is a valuable component, as it attempts to identify and retain latent factors that contribute positively to semantic segmentation under unsupervised settings.

Critical weaknesses

The proposed framework appears to combine existing Bayesian latent factor modeling and optimal transport techniques without introducing a sufficiently novel mechanism that fundamentally integrates the two components. Although the overall pipeline is reasonable, the individual components largely build upon established methods, and their integration seems more sequential than jointly optimized.

Insufficient justification for the methodological complexity. The necessity of introducing Variational IBP for adaptive factor selection remains unclear. Simpler alternatives, such as PCA or sparse encoding, may provide similar representations with lower computational complexity. The paper would benefit from stronger empirical evidence demonstrating why the proposed Bayesian nonparametric formulation is essential.

Questions for authors

Can BN-LFI be replaced with simpler feature decomposition or selection methods, such as PCA or sparse encoding, without significantly degrading pseudo-label quality or downstream segmentation performance?

How much does the adaptive number of active factors contribute to the overall performance? I think it is necessary to compare BN-LFI against variants using a fixed number of factors across all regions.

2 comments

Dear Reviewer e9BX,

Thank you for asking us to clarify the interaction between BN-LFI and GM-OT and to compare against simpler alternatives. We answer each weakness and question separately below.

[W1] Coupling between BN-LFI and GM-OT.

BN-LFI and GM-OT are coupled at the mechanism level, rather than connected by a one-way feature handoff. BN-LFI’s region-wise factor posterior produces both induced features and reliability estimates; after fragment aggregation, these determine the fragment-to-slot distances, competitive margins, and reliability-dependent terms used by GM-OT. The interaction also runs in the other direction: a provisional pseudo-label assignment provides a label-free teacher-quality probe, under which candidate factors are suppressed to measure the change in teacher quality, and these effects are used to recalibrate factor activations before the final transport. Thus, factor selection shapes global allocation, while assignment-aware feedback guides factor calibration.

[W2 & Q1] Replacing BN-LFI with PCA and sparse coding.

We thank the reviewer for questioning whether variational IBP is warranted over simpler alternatives. Our goal is not merely to compress region features: pseudo-label construction requires selecting factors for each region and estimating the uncertainty of that selection. PCA uses a shared global subspace; sparse coding permits region-specific sparse coefficients but does not provide posterior factor-inclusion probabilities. Variational IBP supplies these probabilities, which determine region-specific factor usage and inform reliability-aware GM-OT assignment. To test whether simpler representations can replace BN-LFI, we conducted an additional experiment on S3DIS Area 5. We replaced BN-LFI with PCA or sparse coding while retaining the GM-OT assignment framework and segmentation backbone. The results are summarized in the table below.

RepresentationOA (%)mAcc (%)mIoU (%)
PCA81.6958.8949.08
Sparse coding81.3658.4249.58
Full method81.1461.7052.10

As the table shows, replacing BN-LFI lowers mIoU by 3.02 points with PCA and by 2.52 points with sparse coding. BN-LFI also achieves the highest mAcc. These results support the value of posterior-based adaptive factor selection in our pseudo-label pipeline.

We have reported the results of the experiments suggested by the reviewer in the Appendix G of the revised manuscript.

[Q2] Adaptive versus fixed factor.

BN-LFI models factor activations as binary latent variables but constructs region representations using their posterior expectations, rather than thresholded binary selections. We therefore report the expected number of active factors per region, computed as ∑kϕik\sum_k \phi_{ik}. On S3DIS Area 5, its median is 11.66, with an interquartile range of 8.70–15.10, showing that factor usage varies considerably across regions.

To directly evaluate the contribution of adaptive factor usage, we conduct an additional comparison on S3DIS Area 5 against fixed-size variants with K=8,12,16K = 8, 12, 16. For each region, these variants retain only the KK largest posterior-weighted factor responses and set the remaining responses to zero. Factor identities thus remain region-dependent, but the number of retained factors is fixed across regions. The results are summarized in the table below.

MethodKKOA (%)mAcc (%)mIoU (%)
Fixed-size881.8959.4149.67
Fixed-size1281.7660.1350.08
Fixed-size1682.0361.1250.23
Full methodAdaptive81.2362.7651.89

As the table shows, Ours achieves 51.89% mIoU, outperforming the strongest fixed-size variant (K=16K = 16) by 1.66 percentage points. A shared factor budget may discard useful responses in regions requiring more factors or retain unnecessary responses in regions requiring fewer. The advantage over all three fixed-size variants therefore supports the benefit of adapting factor usage to individual regions.

We have reported the results of the experiments suggested by the reviewer in the Appendix G of the revised manuscript.

Dear Reviewer e9BX,

Thank you for the time and effort you have devoted to reviewing our paper. As the discussion period is ending very soon, we would like to kindly follow up on our detailed response to your review.

We sincerely appreciate your careful consideration of how BN-LFI and GM-OT interact and whether adaptive factor selection is necessary. Our response explains how provisional assignments inform factor calibration and how the calibrated representation guides final transport. We have also added comparisons with PCA, sparse coding, and fixed-factor-count variants to examine the value of the proposed approach.

We hope these results and clarifications address your main concerns. We would be happy to clarify any remaining point before the discussion deadline.

Best regards,

The authors

Reviewer_hGzcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IFOT-SSeg for unsupervised 3D point cloud semantic segmentation. The framework improves pseudo-label construction through two main components: Bayesian Nonparametric Latent Factor Induction (BN-LFI), which constructs and selects latent factors from distilled region features, and Global Margin-Aware Optimal Transport (GM-OT), which assigns overclustered fragments to primitive slots under global capacity constraints. The resulting pseudo-labels supervise a standard segmentation network. Experiments are conducted on ScanNet, S3DIS, and nuScenes.

Critical strengths
  1. The motivation for jointly improving region representations and fragment-to-primitive assignment is reasonable. These two stages address complementary sources of pseudo-label errors.

  2. The method achieves strong results on three public benchmarks, covering both indoor and outdoor scenes, and outperforms the unsupervised methods included in the comparisons.

  3. The component ablations and additional analyses of the transport cost terms provide useful evidence of their contributions. The proposed modules operate only during pseudo-label construction and introduce no additional overhead at inference.

Critical weaknesses
  1. Introducing global capacity constraints to prevent dominant clusters from absorbing ambiguous fragments is reasonable. However, both indoor and outdoor point cloud segmentation tasks exhibit a substantial class imbalance problem. How do the authors address this problem?

  2. The subsequent processing operates at the region and fragment levels, making the method potentially sensitive to errors in the initial grouping. If an initial region contains points from multiple semantic classes, can the framework correct this error, or does it propagate the mixed region into the final pseudo-labels? The robustness to imperfect initial regions requires further clarification.

  3. The framework contains numerous components, but the necessity of some specific designs is insufficiently established. Although the component ablations demonstrate improvements, they do not isolate the benefit of IBP-based factor selection over simpler alternatives, such as sparse coding or soft gating. Similarly, the additional value of intervention-based calibration over simpler factor reweighting remains unclear. Moreover, the authors do not state whether the code will be released. Given the complexity of the pipeline, reproducing the method without code would be challenging.

  4. According to the appendix, pseudo-labels are generated once before training and remain fixed thereafter. Incorrect assignments may therefore provide persistent erroneous supervision. So, how are such errors handled during training?

Questions for authors
  1. How do the global slot-capacity constraints accommodate class imbalance without forcing fragments into inappropriate clusters?

  2. What evidence supports the specific benefits of IBP-based factor selection and intervention-based calibration over simpler alternatives, such as sparse coding, soft gating, or factor reweighting?

  3. How are errors in the fixed pseudo-labels handled during segmentation training?

2 comments

Dear Reviewer hGzc,

Thank you for your questions about class imbalance, region quality, and the contributions of individual components. We address each weakness and question individually below.

[W1 & Q1] Slot capacities under class imbalance.

Regarding the substantial class imbalance in indoor and outdoor scenes, GM-OT does not require primitive slots to have equal sizes. We first tentatively group the overclustered fragments into KK slots. If h(j)h(j) denotes fragment j’s tentative slot and aja_j its normalized mass, we set

\hat b_k = \sum_{j:h(j)=k} a_j, \qquad b_k = 0.65\\,\hat b_k + \frac{0.35}{K}. \tag{1}

The initial mass b^k\hat{b}_k reflects unequal support across slots. Keeping these estimates unchanged could preserve an early shortage of support for weak slots. Giving every slot an equal share would create the opposite problem: in an imbalanced scene, a weak slot might need to receive unrelated fragments merely to fill its quota. Our blended capacity lies between these extremes: it limits the extent to which dominant slots absorb ambiguous fragments while leaving weak slots available without requiring them to reach an equal-size quota. Fragment-to-slot costs favor suitable candidates within this global allocation. In this way, the capacity design accommodates class imbalance and reduces the pressure to assign fragments to poorly matched slots.

[W2] Sensitivity to mixed initial regions.

We acknowledge that assigning a single label to a mixed region propagates errors to its minority-class points. Nevertheless, treating regions as the basic unit remains a common practice in unsupervised 3D segmentation, as also adopted by GrowSP and LogoSP, because it allows semantic evidence from neighboring points to be aggregated. We also examined whether processing smaller units would be preferable in our pipeline: when we used finer-grained units, the downstream mIoU changed from 52.10% to 51.03%. This suggests that finer granularity may introduce noise rather than useful detail in this setting, and we therefore retain the original region granularity. Addressing this limitation more fundamentally would require a finer-grained or learnable grouping mechanism that can revise the initial regions themselves, which we will pursue as future work.

We assessed sensitivity to imperfect initial regions by progressively moving boundary superpoints into spatially adjacent regions, with perturbation budgets of 5%, 10%, and 20% of points. The number of regions remained fixed, and semantic labels were not used to select transfers. For each setting, we reconstructed pseudo-labels and trained the same segmentation network. Region impurity, measured afterward as the fraction of points outside their region’s majority ground-truth class, and downstream mIoU are reported in Table R9.

Table 9

PerturbationRegion impuritymIoU
Original6.99%52.10%
5%7.59%51.63%
10%8.11%51.23%
20%8.94%50.68%

As perturbation increases, region impurity rises monotonically (6.99% → 8.94%) while mIoU decreases consistently (52.10% → 50.68%), confirming the perturbation is effective. Notably, even at 20%—where impurity increases by ∼28% relatively—mIoU drops by only ∼2.7% relatively (1.42 points absolutely). This shows our pipeline tolerates considerable region-level noise and does not rely on perfectly clean initial regions.

We have reported this sensitivity analysis in the main text of the revised manuscript Sec 4.5.

[W3 & Q2] Component controls and Evidence against simpler alternatives.

We replaced IBP-based factor selection with sparse coding and soft gating, and replaced intervention-based calibration with direct factor reweighting. Tables R10 and R11 report downstream segmentation results on S3DIS Area 5.

Table 10

MethodOA (%)mAcc (%)mIoU (%)
Sparse coding81.3658.4249.58
Soft gating82.2061.2150.09
Full method81.1461.7052.10

As shown in Table R10, Ours improves mIoU over sparse coding and soft gating by 2.52 and 2.01 points, respectively. Its IBP posterior provides region-specific factor-inclusion probabilities that shape the induced features used for fragment discovery and transport costs. The same posterior also contributes to the reliability estimates used by GM-OT.

Table 11

MethodOA (%)mAcc (%)mIoU (%)
Factor reweighting79.9162.7950.81
Full method81.1461.7052.10

As shown in Table R11, our intervention-based calibration improves downstream mIoU from 50.81% to 52.10%, compared with direct factor reweighting. While reweighting scales factor responses, our method uses changes in unsupervised teacher quality under factor interventions to refine their activation probabilities.

Together, Tables R10 and R11 show that replacing either IBP-based factor selection or intervention-based calibration with the simpler alternatives reduces downstream mIoU, providing direct empirical support for both design choices.

We will release the code and experimental configurations. In the Appendix G of the revised manuscript, we have described the experimental setups and report the corresponding results, including those for the additional experiments you suggested.

[W4 & Q3] Errors in fixed pseudo-labels during segmentation training.

In our pipeline, the initial grouping yields coarse label candidates that may contain incorrect assignments. GM-OT subsequently refines these candidates by jointly revising fragment-to-slot assignments under global transport costs and slot-capacity constraints, reducing erroneous merges and their impact on segmentation. In a paired S3DIS Area-5 experiment with identical training settings, this refinement improves downstream mIoU from 48.54% to 52.22% (+3.68 points).

Reviewer_psa5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes IFOT-SSeg for unsupervised semantic segmentation of 3D point clouds. It improves pseudo-label construction through two modules. BN-LFI uses ICA-style factor induction, truncated variational Indian Buffet Process inference, and a label-free teacher-quality proxy to select and calibrate latent factors. GM-OT assigns candidate fragments to primitive slots through capacity-constrained, margin-aware optimal transport. Both modules operate only during pseudo-label construction, without additional inference overhead.

Critical strengths

A training-only design with consistent mIoU gains. Both modules act only when pseudo-labels are built, so the deployed network and its inference cost are unchanged.

Critical weaknesses
  1. Why choose ICA for latent factor extraction? How can we verify that the latent factors extracted using this method provide benefits for subsequent unsupervised semantic segmentation? Why select an excessive number of latent factors rather than an incomplete set, which would ensure that some irrelevant factors are filtered out? If the selected latent factors are masked and pseudo-labels are regenerated using only the unselected factors, how would this affect the final segmentation results?

  2. The contribution of each module is not isolated, and the ablation baseline is unexplained.

The baseline does not match the published number. Table 6 starts at 32.65 mIoU. The row is not identified, and it is 3.15 below the 35.8 reported for LogoSP in Table 1. If it is a re-run of LogoSP, factor induction alone at 36.01 only matches the published number, and the full method gains 5.5 over LogoSP rather than 8.7.

The benefit of transport is not separated from its hand-designed terms. Table 6 reports single numbers on ScanNet validation. It has no row with GM-OT alone on the original features and no plain Sinkhorn control with distance cost and uniform marginals. Fragment Allocation is not defined. Fig. 4 removes each GM-OT cost term one at a time but never all three. Hard labels are taken by argmax over the transport plan in Eq. 59, so the capacity constraints need not hold after discretization, and slot usage after the argmax is not reported.

The BN-LFI mechanisms are not tested as described. The induced feature enters at weight 0.1. Eq. 21 scores each factor by its own projection only, so the activation update acts like a per-factor soft threshold, and the truncation is fixed at Kmax. There is no comparison with sparse coding without the IBP, fixed-size selection, or a random dictionary of equal size, and the number of active factors per region is never reported. The calibration also has a formal gap. The probe assignment is fixed during intervention, so the slot-diversity term of Q in Eq. 6 is identical before and after suppressing a factor and cancels in the factor effect. The same term cancels in the acceptance test of Eq. 36. The score Q is never checked against ground truth.

  1. The headline gains are concentrated in a few classes, and the main table does not match the seed study.

The gains come mostly from rare classes. On ScanNet validation, shower curtain, sink and bathtub give 92.4 of the 108.9 summed per-class IoU gain over LogoSP, which is 85%. The other 17 classes gain 0.97 on average, and eight of them drop, with other furniture down 16.1 and books down 8.4. The hidden test split shows the same pattern. On S3DIS Area-5, sofa and board give 95% of the gain. On nuScenes, the bus class alone gives more than the whole gain, and the other 15 classes average minus 0.3. LogoSP scores at or near 0 on shower curtain, sink, bus, sofa and board. With one-to-one Hungarian matching, a class can move from 0 to above 40 by receiving its own slot. The uniform part of the slot capacity prior pushes every slot to be used. The gains may therefore come from slot allocation rather than better features. The OA pattern fits this reading. OA is not above LogoSP on S3DIS and nuScenes, at 81.5 against 82.8, 78.9 against 79.2 and 54.7 against 54.8, while mAcc rises by 8 to 12 points.

The Area-5 numbers conflict. Table 3 gives 66.6 mAcc, which supports the claim of the highest mAcc over PointGS at 66.1. The five seeds in Table 8 range from 61.19 to 63.45 with mean 62.27, so no run reaches 66.6 and the claim fails at the seed mean. Table 3 mIoU of 52.8 matches the best seed, not the mean of 52.42. Seeds exist for this one fold only.

Hyperparameters are selected on the reporting fold.

Questions for authors

Q1. What is the baseline row of Table 6, and why is it 3.15 mIoU below the published LogoSP result? Is it a one-shot version of LogoSP without its regrouping rounds? What does Fragment Allocation denote?

Q2. Which run produced the 66.6 mAcc and 52.8 mIoU in Table 3, given the five seeds in Table 8? Are the random steps of pseudo-label construction, such as randomized SVD and random mixtures, re-run for each seed? How were eta, the lambda weights and rho chosen, and were Area-5 labels used?

Q3. With the probe assignment fixed, how does the slot-diversity term of Q enter the factor effects in Eq. 6 or the acceptance test in Eq. 36? Eq. 21 scores each factor by its own projection only. Since Kmax exceeds D, the atoms cannot be orthogonal. Are the cross-factor terms of the mean-field update dropped? How many factors are active per region in practice?

Q4. Can you provide seed-level ablations on ScanNet and S3DIS that include GM-OT alone on the original features, plain Sinkhorn with distance cost and uniform marginals, and sparse coding without the IBP? Can you also report slot occupancy after the soft transport and after the argmax, so that the capacity constraints can be checked after discretization?

Q5. Which baseline numbers are re-run and which are copied? PointDC is 63.7 and 25.7 in Table 1 but 62.4 and 26.0 in Table 11, and 54.1 and 22.6 in Table 3 but 55.5 and 23.9 in Table 17. The LogoSP per-class values in Table 19 average 47.9, not the stated 46.3. Several constants are not given, including beta, tau, the Sinkhorn regularization and iteration counts, the suppression factor, the calibration weight, the number of intervened factors and the capacity floor.

6 comments

Dear Reviewer psa5,

Thank you for your detailed feedback on the ablations, evaluation, and reproducibility. We address each weakness and your question below, with the relevant evidence and manuscript locations.

[W1] ICA-style factor construction and downstream utility.

Our ICA-style construction learns factor directions from unlabeled neural-region features, whereas an equal-size random dictionary provides directions without using the data. With the subsequent pipeline unchanged, the random dictionary obtains 80.58% OA, 58.58% mAcc, and 48.74% mIoU (Table R1). We use 512 directions as an overcomplete candidate library, not as factors retained by every region. A smaller library could filter irrelevant directions, but could also discard factors useful for less frequent structures before their contribution is assessed; posterior activation and teacher-quality calibration instead limit the influence of unhelpful candidates. To test the learned factors’ downstream value, we rank factors within each region by ∣ϕikμik∣\lvert \phi_{ik}\mu_{ik}\rvert and zero the smallest set accounting for at least 75% of its total factor-response magnitude. Keeping the remaining probabilities and original-feature residual unchanged, we regenerate pseudo-labels and retrain the segmenter. This reduces mIoU by 3.77 points relative to the full method (Table R1), showing that the emphasized factor responses contribute to unsupervised segmentation.

We have included the experiments described above in the Appendix G of the revised manuscript, along with their experimental settings and results

Table 1

SettingOA (%)mAcc (%)mIoU (%)
Equal-size random dictionary80.5858.5848.74
Mask emphasized factors80.4257.8648.33
Full method81.1461.7052.10

[W2-1] Single-pass ablation baseline.

The first row of Table 6 is our single-pass ablation baseline rather than a re-run of full LogoSP: distilled features are grouped into neural regions, clustered into K=20K = 20 primitives in the spectral representation, and used to generate pseudo-labels once. We use this shared single-pass baseline to isolate the incremental effects of BN-LFI and GM-OT in the ablation. It performs a single round only, whereas the published LogoSP additionally applies subsequent superpoint regrouping and pseudo-label regeneration rounds; this difference in iterative refinement explains why its 32.65 mIoU is below LogoSP’s published 35.8. “Fragment Allocation” refers to the variant first overclusters neural regions in spectral space into M=100M = 100 candidate fragments, then merges these fragments into KK primitive slots by nearest-prototype assignment.

We have defined it explicitly in Section 4.4 of the revised main text.

[W2-2] GM-OT cost controls and slot occupancy.

To separate GM-OT from its hand-designed cost terms, we added seed-level results on ScanNet and S3DIS for plain Sinkhorn with distance cost and uniform marginals, GM-OT on the original features, sparse coding without IBP. These results are reported in Table R2. Removing all three additional GM-OT cost terms yields 49.26% mIoU on S3DIS. To examine capacity after discretization, Table R3 reports each slot’s target share, soft-transport occupancy, and occupancy after fragment-wise argmax for both full-model seeds; all percentages are shares of total fragment source mass, not point counts. Since including all corresponding occupancy results for the ablations would make this rebuttal too long, we have provided them in the Appendix G instead (method and seed order as in Table R2). In the full-model runs, soft occupancy matches the target shares; after argmax, all 12 slots remain occupied and the fragment-mass distribution remains close to the target capacities.

We have included the experiments described above in the Appendix G of the revised manuscript, along with their experimental settings and results

Table 2

DatasetMethodSeed 3022Seed 3023
S3DISFull method52.1051.89
S3DISPlain Sinkhorn48.6248.71
S3DISGM-OT on original features49.1349.20
S3DISSparse coding without IBP49.5849.70
ScanNetPlain Sinkhorn37.8338.05
ScanNetGM-OT on original features38.2738.44
ScanNetSparse coding without IBP38.7538.83

Table 3

SeedSlotTarget (%)Soft transport (%)After argmax (%)Fragments after argmax
3022110.4610.4610.8712
3022212.4312.4314.0813
302239.639.6310.228
302247.677.677.7510
3022510.8910.8910.7711
302265.135.134.613
302276.986.987.207
302289.499.4910.187
302298.958.958.257
3022105.415.414.336
3022118.108.107.9812
3022124.864.863.764
302315.675.675.266
302329.489.489.9111
302333.943.943.476
302348.568.568.975
3023513.8713.8714.3311
302367.787.788.489
302376.756.756.518
302388.618.619.0012
302399.739.739.4811
3023108.908.909.015
3023118.048.047.768
3023128.678.677.828

[W2-3] BN-LFI factor usage and calibration.

BN-LFI constructs region representations from posterior activation probabilities rather than thresholded binary selections. We therefore report the expected number of active factors per region, ∑kϕik\sum_k \phi_{ik}. On S3DIS Area-5, the median is 11.66 with an interquartile range of 8.70–15.10, showing that factor usage varies across regions. Kmax⁡K_{\max} is the candidate-library truncation, not the number of factors activated in every region.

On Eq. 21 and orthogonality: Eq. 21 is a tractable per-factor activation-evidence approximation; it does not assume that the overcomplete dictionary is orthogonal. Pairwise cross-factor terms of the full additive likelihood are not explicitly included in this score. Factor selection nevertheless also depends on the shared stick posterior, which is iteratively updated from usage across regions, rather than on each projection alone.

We have clarified the scope of this approximation in Appendix A.3.

We also added the controls requested in W2 (Table R4, Table R5). Replacing the proposed representation with sparse coding without IBP or an equal-size random dictionary reduces mIoU from 52.10% to 49.58% and 48.74%, respectively (Table R4). In fixed-size controls, each region retains its KK largest posterior-weighted factor responses; adaptive BN-LFI reaches 51.89% mIoU, exceeding the strongest fixed-size control by 1.66 points (Table R5). These results test IBP-based adaptive selection against sparse coding, random dictionaries, and fixed-size selection.

Table 4

MethodOA (%)mAcc (%)mIoU (%)
Sparse coding81.3658.4249.58
Equal-size random dictionary80.5858.5848.74
Full method(Seed 3022)81.1461.7052.10

Table 5

MethodKKOA (%)mAcc (%)mIoU (%)
Fixed-size881.8959.4149.67
Fixed-size1281.7660.1350.08
Fixed-size1682.0361.1250.23
Full method(Seed 3023)Adaptive81.2362.7651.89

On the calibration score QQ:

“Fixed probe” refers to the intervention stage, not to probe construction. From candidate assignments Ycand\mathcal{Y}_{\mathrm{cand}} constructed using unlabeled induced features, we select

Y^=arg⁡max⁡Y∈YcandQ(Zind,Y).\hat{\mathbf{Y}} = \arg\max_{\mathbf{Y}\in\mathcal{Y}_{\mathrm{cand}}} Q(\mathbf{Z}^{\mathrm{ind}},\mathbf{Y}).

Candidate slot occupancies can differ, so HslotH_{\mathrm{slot}} contributes to this selection. We then hold Y^\hat{\mathbf{Y}} fixed during all interventions and the acceptance test. For any two representations compared under this probe,

ΔQ=ΔM−λpΔCproto,\Delta Q = \Delta M - \lambda_p \Delta C_{\mathrm{proto}},

since Hslot(Y^)H_{\mathrm{slot}}(\hat{\mathbf{Y}}) is constant, while margins and prototypes are recomputed from each representation.

We have clarified this selection-then-fix procedure and the cancellation of HslotH_{\mathrm{slot}} in Sec. 3.2 and Appendix A.5.

We also checked QQ against ground truth after complete factor calibration: for seed 3022, QQ increased from 0.1978 to 0.2044 while region-level pseudo-label mIoU increased from 50.41% to 51.88%; for seed 3023, QQ increased from 0.1975 to 0.2047 alongside a pseudo-label mIoU increase from 50.10% to 51.34%. Ground truth was used only for post-hoc evaluation.

Comments on the paper

2 comments
Area_Chair_do82OpenReview ↗

Invitation to Discuss the Authors’ Response

Dear Reviewers,

The authors have now provided their response. I encourage all reviewers to read it and participate in the discussion with the authors to clarify any remaining questions or concerns.

Thank you to both the authors and reviewers for your time and effort throughout the review process.

Dear Area Chair,

Thank you for overseeing the discussion of our submission. We have provided point-by-point responses to all three reviewers. As we have not yet received follow-up comments, we would like to summarize the main clarifications and additional results for your consideration. Unless otherwise noted, table numbers below refer to our rebuttal.

Shared concerns across reviews

[psa5 W1/W2; hGzc W3; e9BX W2/Q1/Q2] Necessity of adaptive factor inference. We compared BN-LFI with PCA, sparse coding, soft gating, a random dictionary, and fixed-factor-count variants. On S3DIS Area-5, the full method achieves 52.10% mIoU, compared with 49.08% for PCA, 49.58% for sparse coding, and 50.09% for soft gating. Adaptive factor selection also outperforms the tested fixed-count variants.

[psa5 W2/Q4; hGzc W1/Q1] Global transport and class imbalance. We added seed-level comparisons with plain Sinkhorn and GM-OT on original features (Table 2), and reported soft and post-argmax slot occupancy for full-model S3DIS runs (Table 3). GM-OT does not force equal-size primitive slots: its target capacities combine 65% of the unequal provisional mass with 35% of a uniform prior, giving weak slots a positive transport target while retaining differences in slot support.

Additional reviewer-specific points

[e9BX W1; psa5 Q3] Module interaction and calibration. We explained how provisional assignments guide factor calibration, while the calibrated features and reliability estimates guide final transport. We also clarified the fixed-probe procedure and the scope of the per-factor inference approximation.

[hGzc W2/W4] Initial-region quality and fixed pseudo-labels. We added a controlled region-perturbation study (Table 9) and clarified that GM-OT refines assignments before segmentation training. We acknowledge that a mixed initial region cannot be corrected at the point level by the current grouping.

[psa5 W3/Q1/Q2/Q5] Evaluation and reproducibility. We distinguished the single-pass ablation baseline in Table 6 of the submitted paper from full LogoSP, provided additional seed and hyperparameter results (rebuttal Tables 6–7), and addressed the concentration of per-class gains. We also identified the sources of the baseline-table inconsistencies and supplied the missing implementation constants (rebuttal Table 8).

We appreciate the reviewers' careful feedback and hope these responses assist your assessment of the submission. We remain available to clarify any remaining questions.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.