acceptodds
Under review as a conference paper at ICLR 2027

Generalized Few-Shot 3D Segmentation and More: Support-Free Novel-Class Discovery

Abstract

Existing generalized few-shot 3D semantic segmentation methods typically assume a fixed set of novel classes and rely on noisy pointwise pseudo-labels and limited support prototypes, making missed novel-class instances difficult to recover in both spatial structure and semantics. To address this issue, we focus on novel-instance recovery directly from the query scene and propose Open-World Posterior Completion (OW-PC), a nonparametric framework built upon distance-dependent Chinese Restaurant Process (ddCRP) for structure-aware novel-class discovery. Specifically, Dual-View Debiasing (DVD) integrates complementary pretrained semantic knowledge and episodic support to extract reliable novel-class evidence, while Counterfactual Base Guard (CBG) preserves confident base-class predictions and exposes ambiguous regions for further exploration. OW-PC then organizes sparse novel evidence into coherent instance structures through spatial and semantic consistency, enabling missed novel instances to be recovered both with and without available support. Unlike conventional support-conditioned refinement, its nonparametric formulation does not require the number of novel classes to be fixed in advance, allowing potential novel structures to be actively discovered from the query scene. Experiments on ScanNet, ScanNet++, and ScanNet200 demonstrate consistent improvements in novel-class recall and harmonic mean while preserving base-class segmentation accuracy.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
3D Segmentation, Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xai1OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes the DVD–CBG–OW-PC framework for generalized few-shot 3D segmentation. It leverages dual VLMs and support samples to extract reliable novel anchors, preserves high-confidence base regions, and then recovers missing structures via ddCRP and component-level posterior inference. The authors further extend the framework to partial/zero-support unknown-instance recovery and report performance improvements on four indoor 3D benchmarks.

Critical strengths
  1. Shift from point-level pseudo-label refinement to structured novel-instance recovery. OW-PC unifies local spatial/feature connectivity, base/background counterfactuals, and support-conditioned component evidence within a coherent component inference formulation. Since ddCRP is inherently suited to represent observation-dependent partitions with a variable number of components, this modeling paradigm aligns well with the goal of recovering missing instances from query structures.

  2. There is substantial positive evidence under the standard GFS setting. In Table 1, compared with GFS-VL, the proposed method achieves improved novel mIoU and HM across four datasets and two shot configurations. Table 7 further presents 1-shot results with five random seeds, demonstrating that the main standard-setting results do not strongly rely on a single random run.

Critical weaknesses
  1. The claim of support-free novel-class discovery lacks solid evidence and has evaluation issues. Under zero-support settings, the model can only detect individual unknown instances but cannot automatically group them into distinct semantic classes. This is unknown-instance discovery rather than true novel-class discovery. Besides, Table 4 uses inconsistent GT denominators across methods, and it is unclear how baseline methods run without supports. The conclusion is not fair. The authors should recompute metrics on the identical GT instance set. Either rephrase the claim as unknown-instance discovery, or show the model can separate different novel classes.
  2. Ablation results for core modules are inconsistent across tables, making module contributions hard to verify. For ScanNet 1-shot, the full model and DVD-only variant show obvious discrepancies in Tables 1/3/7/10. The paper does not clarify whether these differences come from different VLMs, random seeds, checkpoints or preprocessing. Since the paper claims individual module benefits, experiments must be standardized. Ideally, provide progressive ablation results under the same seed.
  3. LARM also studies generalized few-shot point cloud segmentation with language priors and structured matching. The authors need to explain the fundamental difference between OW-PC’s query-side structural discovery and LARM’s prototype/relation enhancement.
Questions for authors

1.Why does ScanNet200 in Table 4 have 9,740 GT instances for attMPTI/GW versus 8,029 for GFS-VL/Ours? Can you report [email protected] and mean-best-IoU for all methods on the identical GT subset using existing predictions?

2.What inputs are provided to attMPTI, GW and GFS-VL in zero-support tests? Are novel class names, text prompts, support prototypes and novel semantic logits removed for these baselines identically to our method?

3.For ScanNet 1-shot results in Tables 1, 3, 7 and 10, what seeds, registration settings, VLM pairings and checkpoints were used for each? Specifically, what causes the large gap between the two “DVD” results?

4.Is the predicted novel-category count in Table 2 calculated per scene, over the full validation set, or deduplicated across registration runs? Under Hsf={U,B,bg}\mathcal{H}_{\mathrm{sf}}=\{U,B,\mathrm{bg}\}, is there any GT-free procedure to group separate �0��0� components into distinct novel semantic classes?

6 comments

======================== PART 1/3 ========================

Thank you for carefully reading our paper and for your thoughtful comments.

W1-1 : Claim scope and unknown-instance wording

With labeled support, our method assigns named novel-class labels (Appendix Table 10 reports 57.09%57.09\% novel-class mean recall vs. 44.98%44.98\% for GFS-VL on ScanNet200 1-shot); without support, outputs remain anonymous instances, whose grouping is evaluated separately in Q4.

We have revised the title, Abstract, Sections 1 and 5 to state unknown-instance discovery.


W1-2 & Q1 : Inconsistent GT denominators in Table 4

We apologize for the coding oversight. The ScanNet200 denominator mismatch (8,0298{,}029 vs. 9,7409{,}740) stemmed from filtering semantic > 0 in the original GFS-VL and Ours runs, which excluded all 1,7111{,}711 wall instances (ID 0).

Table R1: ScanNet200 instance recovery. Recall at IoU 0.50 and mean best IoU are percentages; parenthetical counts are matched/GT.

Method[email protected]Mean best IoU
attMPTI12.04 (1,173/9,740)24.51
GW10.35 (1,008/9,740)19.61
GFS-VL20.30 (1,977/9,740)30.71
Ours20.83 (2,029/9,740)33.34

Retaining wall aligns all four methods on the identical 9,7409{,}740-instance domain across the same 312 scenes (Table R1), where GFS-VL recovers 1,9771{,}977 instances (20.30%20.30\%, 30.71%30.71\% mean best IoU) and Ours recovers 2,0292{,}029 (20.83%20.83\%, 33.34%33.34\% mean best IoU). Similarly, on ScanNet++ all rows now use the common 3,1863{,}186-instance domain (GFS-VL 17.55%17.55\%, 559/3,186559/3{,}186 vs. Ours 20.28%20.28\%, 646/3,186646/3{,}186), and Table 4 now uses a single denominator per dataset (6,2406{,}240 for ScanNet, 3,1863{,}186 for ScanNet++, and 2,3392{,}339 for S3DIS). Every baseline runs without support because its adapter receives only coordinates and frozen point features (audited in Q2).

We have corrected the support-free comparison in Section 4.5 (Table 4) and Appendix D.5, and have clarified the decoder inputs, annotation vocabularies, sampling rules, and valid instance IDs in Appendices C.1, C.3, C.4, and C.7 of the revised manuscript.


W2 & Q3 : Seeds, checkpoints and differing module configurations

The discrepancies stemmed from training seeds and module configurations. Every single-run entry (including the progressive ablation in Table 3) now uses seed 42, registration regis1, and checkpoint model_last.pth, while Table 1 and Appendix Table 8 report five-seed means (seeds 42, 43, 123, 44, and 45). Consequently, Table 3 now forms a same-seed progression whose full-model ScanNet 1-shot entry is Base 78.44, Novel 51.92, and HM 62.48 (versus the five-seed mean of 78.65, 51.18, and 62.00 in Table 1).

Regarding the VLM pairing, Appendix Table 11 compares the two full pipelines and Appendix D.6 reports single-VLM baselines and both OpenScene–Mosaic role assignments. The two "DVD" entries measure different configurations: Table 3's DVD-only row disables CBG and OW-PC on seed 42, whereas Appendix Table 11 retains both modules and compares full pipelines, combining module configuration with teacher choice.

We have standardized all experiments and clarified this convention in Appendix C.2 of the revised manuscript.


======================== PART 2/3 ========================

W3 : Conceptual comparison with LARM

LARM enhances category prototypes using LLM-generated descriptions and inter-class hyper-relations to improve point classification within a fixed, support-defined vocabulary (Li et al., ICCV 2025, Sections 3.3–3.4). In contrast, OW-PC infers query-side object formation and verification: component evidence shapes the spatial partition (Eq. 18), Eq. (19) verifies completed components against competing explanations, and the unknown hypothesis enables recovery in zero-support (Table 4) and partial-support (Table 5) regimes that LARM does not evaluate.

Because LARM's code is not yet public, we could not execute it directly; Table R2 places its published 1-shot numbers alongside our seed-42 runs (Appendix Table 8) with each protocol recorded.

Table R2: Reported 1-shot results. Base, Novel and HM are percentages.

DatasetMethod and protocolBaseNovelHM
S3DISLARM (7 base / 6 novel, Area 6 test)75.6844.5256.04
S3DISOurs (10 base / 3 novel, local split, seed 42)60.3533.8543.18
ScanNetLARM (14 base / 6 novel, 1,201/312 scenes)42.7121.5028.58
ScanNetOurs (13 base / 6 novel, GFS-VL split, seed 42)78.4451.9262.48

The protocols differ in class splits (LARM uses 7/6 on S3DIS and 14/6 on ScanNet, whereas we follow the 13/6 GFS-VL ScanNet split and our local 10/3 S3DIS split); only the ScanNet novel classes coincide across protocols. We will add a like-for-like comparison under a shared split and sampling protocol once LARM's implementation is released.


Q2 : Inputs to the zero-support baselines

Every method receives only coordinates and frozen point features (Table R3). Novel names, prompts, support prototypes, and semantic logits are excluded from discovery (GFS-VL's internal semantic outputs are discarded), GT labels enter only post-hoc evaluation, and all adapters return anonymous instance components.

Table R3: Inputs passed to discovery by each adapter.

Method/adaptationNovel names/promptsNovel supportNovel logitsFeature source
attMPTI adaptationNoNoNoattMPTI DGCNN
GW adaptationNoNoNoGW DGCNN
GFS-VL adaptationNoNoNoOpenScene SparseUNet
OursNoNoNoMosaic (DVD pair)

======================== PART 3/3 ========================

Q4 : Novel-category count and GT-free grouping

Table 2 reports a per-run, dataset-level count averaged over five runs: Kˉ=15∑rKr\bar{K}=\frac{1}{5}\sum_r K_r, where KrK_r counts final anonymous category groups pooled across validation scenes in run rr (Appendix A.5). Because groups are inferred without a supplied class count, KrK_r can exceed the GT vocabulary size (e.g., S3DIS's 3.23.2 reflects over-splitting of the three true categories). Note that Table R4 counts clusters from the shared OpenScene-feature decoder, so it is not directly comparable to Table 2.

Table R4: Support-free grouping with cached OpenScene features and the legacy feature-only decoder; cosine/average linkage at threshold 0.25, kk-means with K=99K=99. Grouping is transductive.

(a) Novel-instance recovery and semantic grouping

Category assignmentRecall$\uparrow$Coverage$\uparrow$ARI$\uparrow$NMI$\uparrow$ACC$\uparrow$
One category per instance25.5216.770.0048.417.32
One category for all instances25.5216.770.000.0042.68
K-means (K=99)25.5216.7791.7788.9891.46
Ours (OW-PC + semantic grouping)25.5216.7792.8991.8592.68

(b) Category granularity and base-class mixing

Category assignmentAll clustersNovel-touching clustersMixed base/novel clusters
One category per instance125120820
One category for all instances111
K-means (K=99)99105
Ours (OW-PC + semantic grouping)99114

Note: All eligible components enter the clustering, including base objects; a component needs at least 20 sampled points and a nonzero pooled feature to receive an assignment. Panel (a): Recall is the support-free recall of Table 4 of the manuscript on the same 6,2406{,}240 eligible instances, so it is shared across rows; Coverage is the fraction of eligible novel GT with a category-eligible match; ARI, NMI and Hungarian-matched ACC use the 8282 matched novel instances. Panel (b): novel-touching clusters contain a prediction matched to novel GT, mixed clusters also one matched to base GT. All scores are percentages.

For GT-free grouping without support, we cluster pooled component features using average linkage (following the GFS-VL feature protocol) and stop merging at a threshold calibrated on held-out base labels and frozen before novel evaluation—without using novel GT, names, prompts, prototypes, or a supplied class count. On all 312 ScanNet validation scenes (Table R4), this grouping outperforms kk-means on ARI (92.8992.89 vs. 91.7791.77), NMI (91.8591.85 vs. 88.9888.98), and ACC (92.6892.68 vs. 91.4691.46) with fewer mixed base/novel clusters (44 vs. 55), documenting category granularity while leaving semantic class separation outside its scope.

Reviewer_VHKWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Open-World Posterior Completion (OW-PC), a nonparametric framework for generalized few-shot 3D semantic segmentation that aims to recover missed novel-class instances and enable support-free novel-class discovery. The method consists of three components: (1) Dual-View Debiasing (DVD), which combines complementary vision-language models and episodic support to extract reliable novel-class anchors; (2) Counterfactual Base Guard (CBG), which preserves confident base-class predictions while exposing ambiguous regions for further exploration; and (3) OW-PC, which employs a distance-dependent Chinese Restaurant Process (ddCRP) to organize sparse novel evidence into coherent instances without requiring a predefined number of novel classes.

Experiments on ScanNet, ScanNet++, ScanNet200, and S3DIS show improved novel-class segmentation and harmonic mean performance while largely preserving base-class accuracy. Additional evaluations demonstrate the method's potential for novel-class discovery under partial- and zero-support settings.

Critical strengths

The paper addresses the meaningful problem of missed novel-class instances in generalized few-shot 3D segmentation, extending its applicability to partial- and zero-support settings. Its well-motivated framework integrates complementary modules for reliable novel-class anchoring, base-class preservation, and structured instance recovery, offering a coherent solution to this challenging problem.

Critical weaknesses
  1. Limited Methodological Novelty. The proposed framework primarily integrates existing techniques rather than introducing fundamentally new mechanisms. Specifically, DVD relies on weighted logit and likelihood-ratio fusion, CBG employs cross-view agreement and margin-based filtering, and OW-PC builds upon the established ddCRP framework. While their integration is well motivated, the paper does not sufficiently clarify the methodological innovations beyond these existing techniques or demonstrate how the proposed designs differ substantially from simpler alternatives.
  2. Insufficient Fine-Grained Ablation and Hyperparameter Analysis. Although the paper provides module-level ablations and limited sensitivity analyses, the individual contributions and necessity of several key design choices remain insufficiently validated. In DVD, the primary VLM prediction pipri(c)p_i^{\mathrm{pri}}(c), support similarity sics_i^c, and auxiliary likelihood ratio Λiaux,c\Lambda_i^{\mathrm{aux},c} are combined, but their respective contributions are not independently evaluated. In CBG, the effects of cross-view agreement, margin-based base protection, and the component-level base counterfactual used in OW-PC are not disentangled. For OW-PC, the benefits of multi-mode support modeling, ddCRP-based partition inference, hypothesis marginalization, and log-Bayes-factor acceptance are not isolated through comparisons with simpler alternatives. Furthermore, despite introducing numerous hyperparameters across the three modules, the sensitivity analysis is limited to only two thresholds, leaving the robustness of the remaining design choices insufficiently validated.
Questions for authors
  1. Clarification on Methodological Novelty: What are the key methodological innovations of DVD, CBG, and OW-PC beyond existing techniques such as weighted evidence fusion, agreement-based filtering, and ddCRP? Can the authors clarify why these specific designs are necessary and whether simpler alternatives could achieve comparable performance?

  2. Clarification on Component Contributions and Robustness: Can the authors provide fine-grained ablations to isolate the contributions of pipri(c)p_i^{\mathrm{pri}}(c), sics_i^c, and Λiaux,c\Lambda_i^{\mathrm{aux},c} in DVD; cross-view agreement, margin filtering, and base counterfactuals in CBG; and multi-modal prototypes, ddCRP, posterior marginalization, and LBF filtering in OW-PC? Additionally, how sensitive is the performance to key hyperparameters beyond τb\tau_b and δh\delta_h?

  3. Clarification on Mathematical Formulations: Several formulations in the main text require further clarification. In Eq. (2), the parameters b0b_0, λpri\lambda_{\mathrm{pri}}, and λS\lambda_S are introduced without explicitly explaining their roles or how their values are determined. In Eq. (13), how is the neighborhood N(i)\mathcal{N}(i) constructed, and how are the spatial and feature distance scales determined? In Eq. (21), how exactly are the confidence weights wicw_i^c and viv_i computed? Could the authors provide more precise definitions and implementation details to ensure reproducibility?

5 comments

======================== PART 1/3 ========================

Thank you for carefully reading our paper and for your thoughtful comments.

W1-1 & Q1 : Method novelty beyond fusion, filtering and ddCRP

Our unified framework addresses generalized few-shot segmentation and unknown-instance discovery without novel-class examples, names or prompts. DVD updates support-conditioned candidate reliability via auxiliary likelihood ratios, CBG protects reliable base predictions, and OW-PC couples spatial partitioning with component evidence and acceptance—addressing anchor reliability, base preservation, and structural recovery, respectively.

Unlike standard kk-means, which requires a prescribed cluster count and lacks an acceptance rule, OW-PC infers the component count and verifies components against base/background explanations. Supported components receive named identities, while without support the unknown hypothesis recovers anonymous instances. Tables R1, R2, and R3 provide matched comparisons against simpler alternatives.

On whether simpler designs suffice, a single VLM with one dense threshold couples base precision to novel recall, whereas dual-view prompting improves base preservation: under the same protocol, DVD raises Base mIoU from 54.7854.78 to 66.0366.03 and HM from 39.7539.75 to 41.3141.31 (with Novel mIoU at 30.0630.06 vs. 31.1631.16), while reversing the VLM roles drops HM to 35.7135.71 (Appendix D.6, Table 15). Forming the partition from the prior alone before classifying recovers comparable missed novel points (19.4919.49 vs. 19.4819.48) but increases base leakage (5.405.40 vs. 2.042.04; Table R3). The matched-coverage anchor-precision operating point for these controls remains to be measured.

We have clarified this unified framework and its methodological distinctions in Sections 3.2–3.4 and Appendix A.4 of the revised manuscript.


W2-1 – W2-4 : Module-level ablations for DVD, CBG and OW-PC

All four weaknesses share a single control design: candidates, features, support episodes, calibration protocol, and downstream evaluation are held fixed while varying only the mechanism under test.

For DVD (Section 3.2, Appendix A.1), Table R1 presents the factorial ablation over primary VLM prediction, support similarity, and auxiliary likelihood ratio by zeroing omitted coefficients and calibrating each variant on held-out base episodes; combining all three terms achieves the highest novel mIoU (16.38%16.38\%).

Table R1: DVD evidence controls: 20-scene uncalibrated ScanNet diagnostic; novel mIoU (percent). Anchors are the argmax over support classes with base points left unmasked; only within-table differences are meaningful.

EvidenceNovel
Primary VLM prediction15.55
Support similarity15.41
Auxiliary likelihood ratio12.05
Primary VLM prediction + support similarity15.70
Primary VLM prediction + auxiliary likelihood ratio15.35
Support similarity + auxiliary likelihood ratio12.97
Ours (Primary VLM prediction + support similarity + auxiliary likelihood ratio)16.38

For CBG, Table R2 crosses cross-view agreement, margin protection, and the component-level base counterfactual. The full rule reaches 99.90%99.90\% base precision, q=2.03%q=2.03\%, 60.03%60.03\% novel recall, and 3.54%3.54\% leakage (vs. 61.32%61.32\% and 5.34%5.34\% unprotected), trading 1.291.29 points of novel recall for 1.801.80 points of leakage reduction: agreement alone accounts for 1.021.02 points (5.34→4.325.34\to4.32), margin protection adds 4.32→3.614.32\to3.61 (at a 1.051.05-point recall cost) by guarding high-margin base points missed by agreement, and the counterfactual contributes the final 0.070.07. Margin-only protection sits at the aggressive end of this curve (q=21.90%q=21.90\%, novel recall 59.40%59.40\%, leakage 2.14%2.14\%).

======================== PART 2/3 ========================

Table R2: Independent CBG factorial: 20-scene uncalibrated ScanNet diagnostic, 8 posterior sweeps; scene-mean percentages. Novel R is correctly classified novel points over novel GT points; empty protection has undefined precision (n/a). Leakage uses the no-auxiliary front end, so it is not comparable with Table R3.

ProtectionBase counterfactualBase precisionqqNovel RLeakage
Noneoffn/a0.0061.325.34
Noneonn/a0.0061.325.26
Agreement onlyoff99.902.0361.094.32
Agreement onlyon99.902.0361.074.13
Margin onlyoff99.5321.9059.432.56
Margin onlyon99.5321.9059.402.14
Ours (Agreement + margin)off99.902.0360.043.61
Ours (Agreement + margin)on99.902.0360.033.54

For OW-PC (Section 3.4), support modes preserve distinct appearances, the ddCRP prior adapts component counts, Eq. (18) couples component evidence with partitioning, and Eq. (19) tests decisiveness via LBF. With sh=log⁡πh+EA(h;S)s_h=\log \pi_h+E_A(h; S) and kk feasible states,

log⁡∑hesh=max⁡hsh+log⁡∑hesh−max⁡jsj,0≤log⁡∑hesh−max⁡jsj≤log⁡k.\log \sum_h e^{s_h}=\max_h s_h+\log \sum_h e^{s_h-\max_j s_j},\qquad 0\leq\log \sum_h e^{s_h-\max_j s_j}\leq\log k.

This marginalization term vanishes for singleton anchored sets and is largest when unanchored explanations compete closely. Table R3 reports matched controls for support modes, prior-only partitioning, hard-max inference, and no-LBF verification under shared inputs and decoding settings.

Table R3: Structural controls against simpler alternatives: 20-scene uncalibrated ScanNet frozen-cache diagnostic, 8 posterior sweeps; percentages. Missed R is missed novel-point recovery (higher is more recovery); leakage is base-to-novel error. The growth row uses the legacy confidence/margin gate; the remaining rows use LBF.

Completion variantMissed RLeakage
Prototype-guided infilling26.8519.71
Spatial CC21.5110.10
Spatial–feature CC26.0510.82
Anchor-guided growth (legacy gate)24.056.80
Prior-only partition (cluster-then-classify) + LBF19.495.40
Hard-max inference + LBF13.633.08
Marginal inference, no LBF21.108.19
Marginal + LBF, single support mode19.223.08
Ours (Marginal + LBF, multi-mode)19.482.04

Beyond the two thresholds in Figure 3, Table R4 separates data-derived quantities (carrying no free values) from shared settings, and Table R5 sweeps kk, σx\sigma_x, σf\sigma_f, Mmax⁡M_{\max}, and α\alpha under seed 42 and regis1. Across all sweeps, novel mIoU stays within 0.440.44 points and leakage within 0.170.17 points of the reference.

Table R4: Sweeps vary one parameter at a time.

QuantityConstructionConsequenceSweep
DVD relation likelihoods ψu±\psi_u^\pmEstimated relation frequencies on held-out base episodesAuxiliary evidence uses estimated likelihood ratios.no free value
Support modes μcm,πcm\mu_{cm},\pi_{cm}Support centroids and weights ncm/Ncn_{cm}/N_cEach mode's weight follows its observed support count.no free value
Mode temperature TcmT_{cm}Dispersion/count rule with shared T0,γd,γnT_0,\gamma_d,\gamma_nNo independently tuned temperature for each mode.no free value
Component count ∣Π(a)∣\lvert\Pi(a)\rvertInferred from ddCRP linksNo prescribed cluster count for each scene.inferred
Neighborhood size kkSwept: 8,16,328,16,32Larger neighborhoods add candidate connections; adaptive edge filtering determines which are retained.complete
Feature scale σf\sigma_fSwept: {0.5,1,2}σf,0\{0.5,1,2\}\sigma_{f,0}Larger scales weaken the feature-distance penalty; links with similar features are less sensitive.complete
Spatial scale σx\sigma_xSwept: {0.5,1,2}σx,0\{0.5,1,2\}\sigma_{x,0}Larger scales weaken the spatial-distance penalty, so more distant connections survive.complete
Mode budget Mmax⁡M_{\max}Swept: 1,21,2Fewer support modes collapse distinct appearances into one prototype.complete
Self-link mass α\alphaSwept: 3×10−4,10−33\times10^{-4},10^{-3}Sets the prior mass on starting a new component instead of linking.complete
Evidence weights and decision rulesCalibrated coefficients, priors, τR,τB,nmin⁡(h),δh\tau_R,\tau_B,n_{\min}(h),\delta_hReused across novel evaluation instances.τB,δh\tau_B,\delta_h swept; coefficients scheduled
Posterior evidence and LBF∥z′−z∥∞≤ϵ\lVert z'-z\rVert_\infty\leq\epsilonBecause LBF=sh^−log⁡∑h≠h^esh\mathrm{LBF}=s_{\hat h}-\log\sum_{h\neq\hat h}e^{s_h}, perturbing every marginal by at most ϵ\epsilon moves the LBF by at most 2ϵ2\epsilon.derived bound

======================== PART 3/3 ========================

Table R5: Graph sensitivity: separate 4-scene uncalibrated ScanNet pilot, 8 posterior sweeps; novel mIoU and base-to-novel leakage (percent).

SettingNovelLeakage
k=16k=1617.040.63
k=8k=816.780.68
k=32k=3217.050.62
σx=0.5σx,0\sigma_x=0.5\sigma_{x,0}16.850.69
σx=2σx,0\sigma_x=2\sigma_{x,0}17.270.61
σf=0.5σf,0\sigma_f=0.5\sigma_{f,0}17.240.61
σf=2σf,0\sigma_f=2\sigma_{f,0}16.910.66
Mmax⁡=1M_{\max}=117.000.80
Mmax⁡=2M_{\max}=217.000.71
α=3×10−4\alpha=3\times10^{-4}17.060.62
α=10−3\alpha=10^{-3}16.600.56

We have specified the experimental settings in Section 4.1 and Appendix C.4, the calibration protocol in Appendices C.5–C.6, and the sensitivity-analysis protocol in Appendix D.7 of the revised manuscript; the sweeps beyond the two thresholds are reported in Table R5 of this response.


Q2 : Fine-grained ablations and sensitivity beyond two thresholds

As detailed in W2-1–2-4 above, the ten requested ablations map onto four matched controls holding all other inputs fixed: the 7-subset DVD factorial (Table R1), the 8-cell CBG factorial (Table R2), the 9-variant OW-PC structural controls (Table R3), and the two-threshold grid in Figure 3.

For hyperparameter sensitivity beyond τb\tau_b and δh\delta_h, data-derived quantities (relation likelihoods, mode weights ncm/Ncn_{cm}/N_c, shared temperature coefficients, and inferred ddCRP component counts) carry no free values (Table R4). Sweeping all remaining graph/support parameters (k∈{8,16,32}k\in\{8,16,32\}, σx,σf∈{0.5,1,2}×\sigma_x,\sigma_f\in\{0.5,1,2\}\times, Mmax⁡∈{1,2}M_{\max}\in\{1,2\}, α∈{3×10−4,10−3}\alpha\in\{3\times10^{-4},10^{-3}\}) keeps novel mIoU within 0.440.44 points and leakage within 0.170.17 points of the reference (Table R5).


Q3 : Coefficient, neighborhood and weight definitions

  • Eq. (2) coefficients (b0,λpri,λSb_0,\lambda_{\mathrm{pri}},\lambda_S): b0b_0 sets baseline candidate reliability when both input scores equal 0.50.5, while λpri\lambda_{\mathrm{pri}} and λS\lambda_S weight primary and support evidence. Consistent with Appendix C.5, we fit them on pseudo-novel episodes constructed from held-out base scenes (with disjoint support/query scenes and matching support budget).

For each point-class pair (i,c)(i,c), let picp_{ic} and sics_{ic} denote the primary VLM probability and the original RBF support score, respectively, each clipped to [δ,1−δ][\delta,1-\delta]. Define their logits as xic=log⁡(pic/(1−pic))x_{ic} = \log(p_{ic}/(1-p_{ic})) and yic=log⁡(sic/(1−sic))y_{ic} = \log(s_{ic}/(1-s_{ic})). The binary target zicz_{ic} is 1 when the GT label of point ii is class cc, and 0 otherwise.

We fit the logit tic=b+uxic+vyict_{ic} = b + u x_{ic} + v y_{ic} by minimizing the following regularized binary cross-entropy, where NeN_e is the number of point-class pairs in episode DeD_e:

L(b,u,v)=1E∑e=1E1Ne∑(i,c)∈De[log⁡(1+exp⁡(tic))−zictic]+ρ2(u2+v2).L(b,u,v) = \frac{1}{E}\sum_{e=1}^{E}\frac{1}{N_e}\sum_{(i,c)\in D_e}[\log(1+\exp(t_{ic}))-z_{ic}t_{ic}] + \frac{\rho}{2}(u^2+v^2).

We use ρ=10−4\rho = 10^{-4}, with bb unrestricted and u,v≥0u,v \geq 0. The minimizing values give b0b_0, λpri\lambda_{\mathrm{pri}}, and λS\lambda_S, respectively. The fitted triplet is shared across classes/episodes per dataset–teacher pair and frozen before novel evaluation, while τR\tau_R separately gates anchor acceptance.

  • Eq. (13) neighborhood and scales: We build spatial kkNN neighborhoods on E\mathcal{E} with k=min⁡(kref,∣E∣−1)k=\min(k_{\mathrm{ref}},|\mathcal{E}|-1) (kref=16k_{\mathrm{ref}}=16, excluding self-neighbors) and ℓ2\ell_2-normalized features. Positive configured spatial/feature scales are used directly; otherwise we use the median candidate-edge distance lower-bounded by ϵ\epsilon, making scales deterministic for fixed coordinates and features.

  • Eq. (21) confidence weights: Retained anchors (i∈Sci\in\mathcal{S}_c) use wic=qi∗(zic=1)w_i^c=q_i^*(z_i^c=1) with label c^i=arg⁡max⁡cRic\hat{c}_i=\arg\max_c R_i^c. Newly completed named points in an accepted component AA use vi=sigmoid⁡(LBF⁡A(h^A))v_i=\operatorname{sigmoid}(\operatorname{LBF}_A(\hat{h}_A)). Completion targets exclude original anchors, which retain their labels and weights even if AA is rejected; unknown components remain anonymous instance outputs.

We have specified these implementation details in Sections 3.2 and 3.5 and Appendices A.1, B, and C.5–C.6 of the revised manuscript.

Reviewer_Md14OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for generalized few-shot 3D segmentation, combining VLM-based novel-class anchor selection, base-class protection, and component-level posterior inference. Experiments on four benchmarks show improvements over existing methods, particularly GFS-VL. However, the methodological novelty, the limitations of hard base-class protection, and the independent contribution of the proposed inference mechanism remain insufficiently demonstrated.

Critical strengths

The paper evaluates several indoor point-cloud benchmarks and considers both standard few-shot segmentation and support-free instance recovery. The component-level semantic evidence in OW-PC is a potentially useful alternative to purely pointwise classification, and the results show improvements over GFS-VL in multiple dataset/shot settings.

Critical weaknesses
  1. The methodological novelty over GFS-VL and existing novel-class discovery methods is insufficiently established The closest comparison is GFS-VL (An et al., CVPR 2025), which already combines noisy 3D-VLM predictions with sparse support annotations, uses prototype-guided pseudo-label selection, and applies Adaptive Infilling to recover unlabeled regions. Thus, the general motivation of using support evidence to filter unreliable predictions and subsequently recover missing novel regions is not new. The proposed DVD similarly combines VLM predictions with support-feature compatibility and an auxiliary-model verification stage. CBG uses cross-view prediction agreement and a confidence-margin threshold to preserve selected base predictions. OW-PC then applies an established nonparametric spatial grouping mechanism, ddCRP, together with component-level semantic scores and a Bayes-factor acceptance criterion. These components are reasonably motivated, but the paper does not yet demonstrate convincingly that their combination constitutes a sufficiently distinct methodological advance over existing pseudo-label selection, adaptive infilling, graph-based propagation, and clustering-based novel-class discovery approaches. The use of a posterior formulation alone does not establish that the resulting inference is meaningfully better than simpler alternatives. The reported improvements over GFS-VL are also uneven. On ScanNet, the harmonic mean (HM) improves from 60.88 to 62.00 in the 1-shot setting and from 61.91 to 63.02 in the 5-shot setting. On S3DIS, the corresponding gains are 1.75 and 0.35 percentage points. Larger gains appear on ScanNet++ and ScanNet200, where the HM improves by 4.60/7.53 and 3.19/5.91 points, respectively. These results are positive, but they do not establish a uniformly substantial advantage over GFS-VL. Although Table 3 provides only a limited ablation on ScanNet 1-shot, Appendix Table 10 includes additional module diagnostics on ScanNet and ScanNet++. However, these experiments still do not fully isolate the contribution of OW-PC's component-level posterior inference from that of conventional spatial/feature clustering and simpler cluster-then-classify alternatives. The paper should include controlled comparisons against simpler alternatives, including single-VLM inference, simple two-VLM score fusion, prototype-guided infilling, conventional spatial/feature clustering, and a cluster-then-classify pipeline. It should also isolate the effect of marginalizing semantic explanations during partition inference. Without these experiments, the central algorithmic contribution remains insufficiently supported.

  2. The hard base-protection rule creates a blind spot, and the posterior formulation requires further clarification CBG defines the protected set using agreement between the primary and auxiliary VLMs and a sufficiently large base-class margin. Protected points are excluded from subsequent structural growth and retain their base labels. This creates a direct limitation for the paper's central motivation. Suppose a genuine novel-class instance is confidently misclassified as a base class by both VLMs. If the fused base-class margin exceeds the protection threshold, its points enter the protected set and cannot be recovered by OW-PC. This is not merely a generic limitation of VLM predictions: it follows directly from the proposed hard exclusion rule. The framework can revisit ambiguous base predictions, but it cannot recover confidently misclassified novel regions that are removed from the search domain. The paper should quantify how frequently this occurs and clarify the scope of its novel-instance recovery claims. There is also a potential mismatch between the claimed joint structural-semantic inference and the feasible hypothesis set in Equation (16). For a component containing anchors from a single novel class c, the feasible set is restricted to {c}, whereas components without anchors retain the competing hypotheses. Consequently, base and background explanations do not contribute to the component marginal Ψ(A; S) for an anchored component during partition inference. The final LBF test subsequently compares the decoded explanation against counterfactual alternatives, but it is unclear how this post-inference test interacts with the earlier restriction and how incorrect neighborhood points attached to an anchored component are handled. This raises a key question: does the partition search genuinely compare competing semantic explanations throughout inference, or does it first grow components under an effectively fixed novel-class interpretation and only evaluate alternatives afterward? The distinction is important for assessing whether OW-PC provides a substantive advantage over conventional anchor-guided region growing followed by component-level verification. Finally, the support-free instance-recovery results are highly dataset-dependent. Table 4 reports instance recall at IoU 0.50 increasing from 21.12% to 73.32% on S3DIS, while the gains are only 0.89 and 0.95 percentage points on ScanNet and ScanNet200, respectively. The paper should explain this disparity, provide per-class and per-scene breakdowns, and clarify how semantic segmentation baselines are converted into instance predictions and how instance matching is performed. The predicted novel-category count also underestimates the ground-truth count on larger-vocabulary datasets, particularly ScanNet200 (34.6 versus 45), which warrants discussion given the emphasis on nonparametric discovery.

Questions for authors
  1. Novelty and controlled comparisons How does OW-PC compare with GFS-VL's Adaptive Infilling and with simpler alternatives that combine support-conditioned candidate selection, two-VLM score fusion, and spatial/feature clustering? Could the authors provide controlled ablations that isolate the contribution of the likelihood-ratio verification, CBG, component-level semantic marginalization, and the LBF acceptance rule?

  2. Recovery of confidently misclassified novel regions What fraction of ground-truth novel-class points or instances is assigned to the protected set P_B because both VLMs confidently predict a base class? Since these points are excluded from structural growth, how can the framework recover such instances? Please provide a quantitative analysis of the trade-off between base-class preservation and novel-instance recall as the CBG threshold varies.

  3. Consistency of posterior inference and evaluation Please clarify how Equation (16), which restricts an anchored component to its anchor class during partition inference, interacts with the subsequent LBF comparison against base/background alternatives. In addition, why does support-free instance recall improve dramatically on S3DIS but only marginally on ScanNet and ScanNet200? Please report seed-level variability and, if available, per-scene or per-class breakdowns specifically for the support-free instance-recovery setting. The authors should also evaluate whether the reported recall is sensitive to allowing multiple ground-truth instances to match the same predicted component.

6 comments

======================== PART 1/4 ========================

Thank you for carefully reading our paper and for your thoughtful comments.

W1-1 : Methodological novelty beyond GFS-VL

Our central contribution is support-free discovery and segmentation of unknown instances from frozen CLIP-aligned features and query geometry, without novel-class examples, names or text prompts, integrated with generalized few-shot segmentation. Table R1 summarizes the methodological differences.

Table R1: Comparison with pointwise selection, infilling, propagation, and clustering alternatives.

ApproachInference mechanismDistinction of our method
GFS-VL: pseudo-label selectionSelects pointwise VLM predictions using support-prototype evidence.We use selected predictions as anchors and infer components over the CBG-exposed region, allowing recovery beyond the retained anchors.
GFS-VL: Adaptive InfillingAssigns unlabeled points to support prototypes according to feature similarity.We infer object membership and evaluate component-level evidence; without support, we use unknown/base/background hypotheses to recover unnamed instances.
Fixed-graph label propagationPropagates supplied labels or prototypes along a predefined graph.We infer links and the resulting partition jointly with component evidence, then apply component-level acceptance and rejection.
Spatial/feature clustering followed by classificationForms components from spatial or feature similarity, then assigns semantic labels to the fixed components.We include component evidence in partition inference through p(a∣Q,S)∝∏iϕi(ai)∏A∈Π(a)Ψ(A;S)p(a \mid Q,S)\propto\prod_i \phi_i(a_i) \prod_{A\in\Pi(a)}\Psi(A; S), so semantic evidence influences which points belong together.

The key distinction lies in what is inferred: GFS-VL infills points using support prototypes, whereas OW-PC couples point partitioning with component-level semantic evidence and accepts or rejects each decoded component via the minimum-size and LBF criteria in Eqs. (19)–(20). Without novel support, Hsf={U,B,bg}\mathcal{H}_{\mathrm{sf}}=\{\mathrm{U},\mathrm{B},\mathrm{bg}\} combines unknown, base and background evidence to recover anonymous instances. Table 1 evaluates generalized few-shot segmentation, and Table 4 evaluates anonymous instance recovery using the shared support-free OW-ddCRP decoder.

We have clarified this central contribution and its distinction from prior methods in Section 3.4 and Appendix A.4 of the revised manuscript.


W1-2 : Uneven gains across datasets and shots

We apologize for transposing the ScanNet200 GFS-VL 1-shot and 5-shot columns; with the ordering restored, the HM gains are 5.39 and 3.71 points. Table R2 explains why the gain magnitude varies: ScanNet200 and ScanNet++ have more novel categories, stronger class imbalance and more co-occurring novel objects, giving component-level inference greater scope to aggregate sparse semantic evidence, whereas ScanNet has fewer, more balanced novel classes. On S3DIS (reproduced on our local split, as GFS-VL reports no S3DIS results), novel points are dominated by the broad clutter label, so recovering additional structures within it contributes less to semantic discrimination.

Table R2: Measured class distributions and HM gains over GFS-VL, all scans; Δ\DeltaHM is in percentage points.

DatasetBase/NovelNovel pointsNovel/sceneMax/minΔ\DeltaHM (1/5)
ScanNet13/62.631.042.31.12/1.11
ScanNet20012/4522.757.16133.15.39/3.71
ScanNet++12/1813.696.10151.44.60/7.53
S3DIS (local)10/310.381.6233.11.75/0.35

We have corrected Table 1 and Section 4.2, and have clarified the dataset splits in Section 4.1 and Appendix C.1 of the revised manuscript.


======================== PART 2/4 ========================

W1-3 : Controlled comparisons and semantic marginalization

Table R3 compares a single VLM, two-VLM score averaging and DVD term removals on the same frozen inputs, while Table R4 compares prototype-guided infilling, spatial and spatial–feature clustering, a prior-only cluster-then-classify control, and the hard-max, no-LBF and single-mode variants of OW-PC (all 20-scene uncalibrated ScanNet diagnostics isolating one mechanism at a time). Because DVD is designed to emit a sparse, high-precision anchor set while OW-PC forms complete components, Table R3 measures candidate-layer quality only as a proxy; DVD's pipeline-level impact appears in Table 3, where enabling it raises Base from 67.3167.31 to 73.9573.95 and Novel from 26.2626.26 to 41.9341.93.

For a partition Π(a)\Pi(a), Eq. (18) gives

log⁡p(a∣Q,S)=∑ilog⁡ϕi(ai)+∑A∈Π(a)log⁡Ψ(A;S)+const.\log p(a \mid Q,S)=\sum_i \log \phi_i(a_i)+\sum_{A\in\Pi(a)}\log \Psi(A; S)+\mathrm{const}.

Component evidence thus alters the partition objective itself, distinguishing OW-PC from forming a fixed spatial–feature partition before classifying its components. Replacing log⁡∑hexp⁡(sh)\log \sum_h \exp(s_h) with max⁡hsh\max_h s_h, where sh=log⁡πh+EA(h;S)s_h=\log \pi_h+E_A(h; S), isolates marginalization from hard-max selection: the two coincide on singleton feasible sets, but on unanchored components with competing hypotheses their difference shifts attachment preferences and affects neighboring anchored components.

Table R3: ScanNet front-end controls: 20-scene uncalibrated frozen-cache diagnostic at the candidate layer; novel mIoU (percent). Base predictions are masked to the novel vocabulary here; only within-table differences are meaningful.

Candidate / anchor mechanismNovel
Single VLM28.47
Two-VLM score averaging29.04
DVD without support similarity29.47
DVD without auxiliary verification29.30
Ours (DVD, retained intersection gate)29.32

Table R4: Structural controls against simpler alternatives: 20-scene uncalibrated ScanNet frozen-cache diagnostic, 8 posterior sweeps; percentages. Missed R is missed novel-point recovery (higher is more recovery); leakage is base-to-novel error. The growth row uses the legacy confidence/margin gate; the remaining rows use LBF.

Completion variantMissed RLeakage
Prototype-guided infilling26.8519.71
Spatial CC21.5110.10
Spatial–feature CC26.0510.82
Anchor-guided growth (legacy gate)24.056.80
Prior-only partition (cluster-then-classify) + LBF19.495.40
Hard-max inference + LBF13.633.08
Marginal inference, no LBF21.108.19
Marginal + LBF, single mode19.223.08
Ours (Marginal + LBF, multi-mode)19.482.04

W2-1 : Protected base points and recovery scope

CBG fixes cross-view-consistent, high-margin base predictions to limit base-to-novel leakage. Our CBG factorial experiment shows that this reduces leakage from 5.34%5.34\% to 3.54%3.54\% at a 1.291.29-point novel-recall cost (61.32%→60.03%61.32\%\to60.03\%). Because points in PB\mathcal{P}_B retain their base labels, the protected fraction bounds attainable novel-point recall: for novel points NN with q=∣N∩PB∣/∣N∣q=|N\cap\mathcal{P}_B| / |N|, constrained OW-PC satisfies RN≤1−qR_N\leq1-q. For a GT novel instance GG with qG=∣G∩PB∣/∣G∣q_G=|G\cap\mathcal{P}_B| / |G| and any recovered component A⊆EA \subseteq \mathcal{E},

∣A∩G∣≤∣G∖PB∣,∣A∪G∣≥∣G∣,IoU⁡(A,G)≤1−qG.\lvert A \cap G \rvert \leq \lvert G \setminus \mathcal{P}_B \rvert, \qquad \lvert A \cup G \rvert \geq \lvert G \rvert, \qquad \operatorname{IoU}(A, G) \leq 1 - q_G.

Recovery at IoU 0.50 requires qG≤0.5q_G\leq0.5, yielding the dataset-level recall bound

Let MM be the number of GT novel instances, and let bi=1b_i = 1 if the protected fraction of instance ii is at most 0.50.5, and bi=0b_i = 0 otherwise. Then the recoverable fraction is U=1M∑i=1MbiU = \frac{1}{M}\sum_{i=1}^{M} b_i, and instance recall at IoU 0.50.5 satisfies R≤UR \leq U.

We have clarified in Sections 3.3–3.4 and 4.5 and Appendix A.2 of the revised manuscript that our recovery claim applies to the exposed domain E\mathcal{E}, while points in PB\mathcal{P}_B retain their base labels.


======================== PART 3/4 ========================

W2-2 : Partition inference and component acceptance

Equation (16) fixes an anchored component's named identity while leaving its spatial extent to partition inference. Although base/background states are excluded from an anchored component's own marginal, they remain active alternatives for neighboring unanchored regions. Specifically, when merging anchored AA and unanchored BB, Eq. (18) gives

ΔJ=Δlog⁡ϕ+log⁡Ψ(A∪B;S)−log⁡Ψ(A;S)−log⁡Ψ(B;S).\Delta \mathcal{J}=\Delta \log \phi+\log \Psi(A\cup B; S)-\log \Psi(A; S)-\log \Psi(B; S).

Strong base/background evidence on BB elevates log⁡Ψ(B;S)\log \Psi(B; S) and opposes attachment, so competing explanations directly shape the partition even though A∪BA\cup B retains the anchor class.

After partition inference, Eq. (19) evaluates the completed component against the full hypothesis set: erroneous attachments weaken support evidence or strengthen counterfactuals, triggering rejection. Rejected components supply no new completion targets, whereas the original DVD anchors and their labels remain preserved for training. Table R5 shows that combining marginal inference with LBF achieves the lowest post-acceptance attachment error.

Table R5: Partition versus acceptance: uncalibrated ScanNet 20-scene frozen-cache diagnostic, up to 8 posterior sweeps; scene-mean percentages. Attachment error is measured on non-anchor points of anchored components.

VariantError beforeError after
Anchor-guided growth (legacy gate)66.3765.87
Hard-max inference + LBF67.0671.40
Marginal inference, no LBF65.7068.91
Ours (Marginal inference + LBF)65.7064.85

W2-3 : Support-free evaluation and dataset disparity

The larger gain on S3DIS reflects its larger instance sizes and category distribution: larger instances provide stronger spatial support for grouping, whereas ScanNet and ScanNet200 contain more small instances sensitive to boundary errors (Table R6, upper). On S3DIS, gains are led by clutter, bookcase, chair and table, with recall improving in 66 of 68 scenes (Table R6, lower). All baselines are converted identically: the shared support-free OW-ddCRP decoder runs on their frozen features to emit anonymous components without semantic predictions or supports, and a GT instance counts as recovered at IoU ≥0.50\geq0.50 under post-hoc GT-majority agreement without a one-to-one constraint.

Table R6: S3DIS instance scale (upper) and per-class support-free recall (lower).

DatasetScenesAnnotated GTEvaluated GTMedian voxelsInstances <1,000<1{,}000 (%)
ScanNet3126,2406,2401,902.025.26
ScanNet2003129,7409,7401,238.542.99
ScanNet++503,1883,1861,032.048.87
S3DIS682,3462,3392,626.517.77
S3DIS classEvaluated GTGT share (%)GFS-VL recall (%)Ours recall (%)Net recovered
ceiling753.2180.0018.67-46
floor682.9167.6535.29-22
wall34314.6613.4153.94+139
beam30.130.00100.00+3
column743.166.7694.59+65
window522.2228.8573.08+23
door1275.4333.0785.83+67
table1546.5811.6967.53+86
chair25811.0339.5376.36+95
sofa110.479.0972.73+7
bookcase2169.2310.1969.44+128
board421.8066.6795.24+12
clutter91639.1611.9084.39+664
All classes2,339100.0021.1273.32+1,221

We have clarified the baseline conversion and instance-matching protocol in Appendices C.3 and C.7 of the revised manuscript.


W2-4 : Novel-category undercount on large vocabularies

Nonparametric inference adapts the number of spatial components to the observations. Table 2 reports the five-run mean of final anonymous category groups pooled across validation scenes. During grouping, visually or functionally similar categories may merge (lowering the count), while intra-class variation or fragmentation may split a category (raising it). On ScanNet200, the large vocabulary and strong class imbalance induce net merging (34.634.6 versus 4545), whereas on S3DIS over-splitting of the three true categories across runs yields 3.23.2.

We have clarified the category-count definition in Appendix A.5 of the revised manuscript.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.