acceptodds
Withdrawn from ICLR 2027

LDSGSeg: Local Distribution Encoding and Semantic Guidance for Few-Shot Point Cloud Semantic Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Few-shot 3D point cloud semantic segmentation aims to segment novel classes in query point clouds given only a few annotated support samples. However, existing prototype-based methods still encounter two limitations: i) they do not explicitly characterize the spatial distribution of neighboring points; ii) query–prototype matching relies mainly on visual representations, without fully exploiting the category-level semantic guidance provided by class names. To address these issues, we propose LDSGSeg, a novel framework that integrates local distribution encoding with category-level semantic guidance. Specifically, to solve the first issue, PCA-Guided Local Distribution Encoding derives local distribution descriptors from local covariance eigenvalues to enrich local point representations. To address the second issue, Point-to-Text Cross-Attention introduces category-level semantic information into query point features by modeling their relationships with the text representations of foreground classes. Furthermore, Text-Guided Prototype Channel Calibration employs the corresponding class text embedding to adaptively reweight the channels of each support-derived visual prototype, thereby providing category-level semantic guidance for prototype matching. Extensive experiments on the S3DIS and ScanNet datasets demonstrate that LDSGSeg consistently improves segmentation performance under various few-shot settings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Point cloud, few-shot, semantic segmentation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UsiEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LDSGSeg combines local geometric statistics and class-name information for few-shot point cloud semantic segmentation. Covariance-eigenvalue descriptors are projected into the encoder, while frozen CLIP text embeddings condition query features and support prototypes. The full model improves all eight dataset/way/shot average results over TaylorSeg-PN on S3DIS and ScanNet, without additional image inputs or task-specific 3D pretraining. The paper provides component, placement and semantic-input controls; Table 5 supports the usefulness of correctly aligned class semantics. Parameter counts and increased training time are also disclosed.

Critical strengths
  1. Two limitations are stated as concrete mechanisms.
  2. LDE injects geometry at the right place.
  3. Semantic-input control is well designed.
Critical weaknesses
  1. The advance over established descriptors and fusion operations needs clearer positioning. Linearity, planarity and sphericity from covariance eigenvalues are classical point-cloud descriptors; PTCA and PCC use established cross-attention and channel-gating operations. Table 3 already compares separate and joint conditioning. Please explain what task-specific insight or capability the joint design adds beyond these components. The already-cited GPCPR (Wei et al., 2024) offers a relevant semantic-prototype comparison; please include it under explicit pretraining and semantic-resource conditions, or explain why the protocols are not comparable.
  2. The LDE gain is not fully separated from added capacity and residual initialization. The disclosed parameter count increases from 0.27M for TaylorSeg-PN to 0.47M, approximately 74.1%. Table 5 retains the semantic architecture and supports the value of correctly aligned text, but does not isolate the extra LDE capacity. Please compare LDE with a similarly sized visual residual branch using the same initialization. The learnable zero-initialized scales are a valid optimization choice; their role should be distinguished from the information carried by the branches. A small nonzero-scale comparison with architecture and training fixed could provide an additional, optional diagnostic of the initialization effect.
  3. The background token and prompt choice need a clearer empirical scope. In Eqs. (11)-(13), the zero background text token is combined with a visual background prototype, so its gate remains input-dependent. However, foreground prototypes receive language conditioning while background does not. Please justify this asymmetry and characterize background false positives on the existing episodes. Table 5 supports correctly aligned CLIP semantics, but evaluates one prompt template; either a small paraphrase check or an explicit limitation to the tested template would clarify the scope of that result.
  4. Some geometric-branch details require clarification for reproducibility. The descriptors are projected through point-wise convolution, batch normalization and ReLU before addition, and their contribution is scaled by a learnable coefficient. Please specify the stage dimensions, relative-coordinate normalization, numerical handling of degenerate neighborhoods, and whether gradients through covariance eigendecomposition are required. These are implementation questions; the manuscript does not establish numerical collapse or feature corruption, and they are secondary to the contribution and attribution concerns above.
Questions for authors
  1. What does LDE add beyond classical descriptors? Linearity/planarity/sphericity are standard covariance features. Is the mechanism-level claim that these descriptors are newly fused into HiConv before pooling, or that the specific descriptor triplet captures something prior local aggregators miss? If the former, what does the HiConv injection enable that concatenating d_i to the point feature would not?

  2. Why condition queries and prototypes separately? PTCA and PCC both consume E, but at different stages. Does PCC's channel gate operate on information PTCA cannot reach (prototype-level category prior vs. point-level context)? Is there a mechanism reason the two cannot be merged into one text-conditioned matching step?

  3. What is the background asymmetry doing? Foreground prototypes get text conditioning; background gets a zero token. Since the gate is input-dependent, the zero token still yields a data-driven gate. Mechanistically, does this asymmetry risk making background less calibrated than foreground, and is that visible in false-positive behavior?

  4. Is the LDE gain separable from capacity? The zero-initialized α_i means LDE starts as identity. Does the gain come from the descriptor information itself, or from the extra parameters becoming active during training? A same-size visual residual branch with identical initialization would clarify what the covariance eigenvalues specifically contribute.

  5. What happens when class names are semantically close? The prompt is "a point cloud scan of a {class}." For fine-grained or ambiguous categories, does PTCA's attention distribute mass across several foreground classes rather than committing? Is there evidence the text branch helps most when visual prototypes are ambiguous?

Reviewer_FW3jOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LDSGSeg, a framework for few-shot 3D point cloud semantic segmentation that combines local geometric information with category-level semantic guidance. The method first uses PCA-based descriptors to capture the spatial distribution of neighboring points and incorporates them into the point encoder. It then uses class-name embeddings from a frozen CLIP text encoder to enhance query features through cross-attention and calibrate support-derived prototypes. The calibrated prototypes are further refined using support–query relationships before being used for segmentation.

The authors evaluate LDSGSeg on S3DIS and ScanNet under different few-shot settings. The results show consistent improvements over TaylorSeg-PN, while performance compared with the more recent EPSegFZ is mixed but generally competitive. LDSGSeg also uses considerably fewer trainable parameters than EPSegFZ. Additional ablation experiments examine the contributions of the individual components and provide evidence that correctly aligned text embeddings improve segmentation performance.

Critical strengths
  1. Integration of local geometry and semantic guidance. The combination of local geometric features and language-based guidance is a sensible choice for few-shot point cloud segmentation. The PCA-based descriptors capture the spatial structure of neighboring points, while CLIP embeddings provide additional category information for query features and support prototypes. The text embedding ablations in Table 5 are also useful. In particular, the random-token and mismatched-CLIP experiments help show that the gains depend on the semantic information in the embeddings.

  2. Competitive results with fewer trainable parameters. The method is evaluated on both S3DIS and ScanNet across several few-shot settings and consistently improves over TaylorSeg-PN. Although LDSGSeg does not outperform EPSegFZ in every setting, it achieves competitive performance with substantially fewer trainable parameters (0.47M versus 2.02M). Achieving comparable performance with a considerably smaller parameter count is a notable advantage.

Critical weaknesses
  1. I would like a clearer justification for the way LDE is incorporated into the encoder. In Eq. (5), the same distribution feature is added to every neighbor of a center point. Under the pooling operation described after Eq. (6), this is equivalent to adding that feature after taking the neighborhood maximum and before applying the activation. The descriptor therefore provides additional geometric information, but the reason for injecting it into each neighbor is unclear. Tables 3–5 provide useful component, placement, and semantic-input ablations. A comparison with a simple alternative, such as concatenating the encoded descriptor with the max-pooled feature and applying a projection, would help establish the value of the chosen fusion design.

  2. The final prototype refinement step needs more detail. Eq. (14) introduces R, but the paper does not explain how the support–query correlations are computed or how they update the prototypes. Figure 2(c) shows the connections, while Appendix A mainly covers training and encoder settings. I could not determine whether this refinement is inherited from the baseline or whether it is kept identical across the ablations.

  3. The smaller improvements also need evidence of stability across training runs. On ScanNet 3-way 1-shot, the full model improves over TaylorSeg-PN by 0.61 mIoU points. In Table 3, adding LDE to PTCA+PCC increases the average from 66.06 to 66.19. The paper reports results across episodes and class folds, but gives no standard deviations or confidence intervals across independent training runs. This makes it difficult to assess how much weight to place on these small differences.

Questions for authors
  1. How exactly does R in Eq. (14) compute the support–query correlations and update the prototypes? Is it taken from an existing method, and is the same refinement used in the baseline and every row of Table 3? If it differs, could you include a comparison that isolates this change?

  2. Could you repeat the ScanNet 3-way 1-shot experiments with several training seeds for TaylorSeg-PN, PTCA+PCC, and the full model, and report the mean and standard deviation? Please also clarify whether the methods use the same evaluation episodes. This would help assess both the 0.61-point improvement over the baseline and the 0.13-point gain from adding LDE.

  3. Does the LDE implementation follow Eqs. (5)–(6) directly, with no additional operation between the descriptor addition and max pooling? If so, what motivated adding the descriptor to every neighbor? How does this design compare with concatenating the encoded descriptor with the max-pooled HiConv feature and projecting the result before activation, using a comparable number of parameters?

Reviewer_ubLCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LDSGSeg for few-shot 3D point cloud semantic segmentation. At a high level, the framework brings together three components: PCA-derived local distribution descriptors incorporated into the point encoder, point-to-text cross-attention that injects foreground class semantics into query features, and text-conditioned channel calibration of support-derived prototypes. The calibrated prototypes are then refined for query classification. The evaluation covers S3DIS and ScanNet under 2-way/3-way and 1-shot/5-shot settings, together with ablations on component combinations, descriptor placement, and alternative semantic inputs. Overall, the reported results show consistent improvements over TaylorSeg-PN and competitive performance against EPSegFZ with fewer trainable parameters.

Critical strengths

(1)The method shows useful improvements across a range of evaluation settings with a compact trainable model. Tables 1–2 report higher average mIoU than TaylorSeg-PN in all eight dataset–setting combinations, with gains of 1.29–3.56 percentage points on S3DIS and 0.61–1.73 points on ScanNet. With 0.47M trainable parameters, LDSGSeg also appears to offer a competitive accuracy–parameter trade-off relative to EPSegFZ. In my reading, the breadth of these improvements supports the practical value of the proposed integration. (2)The semantic-input controls provide meaningful evidence for the role of class semantics. Table 5 keeps the semantic architecture intact while independently training variants with correct CLIP embeddings, mismatched embeddings, random tokens, and zero tokens. Correct CLIP embeddings perform best in all four evaluated settings, exceeding random tokens by 1.66–4.53 mIoU points. These controls make it more plausible that the correspondence between class labels and text representations contributes to the gains, and I find this evidence fairly persuasive.

Critical weaknesses

(1)A consequential part of the prediction pipeline is insufficiently specified. Equation (14) represents prototype refinement only as P∗=R(Pc,Fs,F~q)P^*=\mathcal{R}(P^c,F^s,\widetilde F^q), without describing the internal computation of R\mathcal{R} or clearly identifying an inherited implementation. Neither Figure 2 nor the appendix seems to resolve this omission. Since this operation directly determines the prototypes used for prediction, its specification is necessary both for reproducing the method and for establishing whether the baseline and ablations differ only in the proposed components. This substantially limits my confidence in the attribution of the reported improvements. (2)The robustness of the smaller gains, and of the claimed component complementarity, remains insufficiently established. The paper does not report variability across independent training runs. In Table 3, for example, the complete model achieves 66.19 on ScanNet 3-way 1-shot, compared with 66.12 using LDE alone. Although the complete configuration obtains the highest average scores, this 0.07-point difference provides only limited evidence for the incremental benefit of the semantic components in that setting. Given that the contribution is primarily empirical, uncertainty about whether these gains persist across training runs materially affects my assessment.

Questions for authors

(1)Method clarification: What is the complete computation performed by R\mathcal{R} in Equation (14), and how does it relate to the prototype refinement used in TaylorSeg-PN? In particular, are the refinement operation and associated projection layers identical across the direct baseline and all component ablations? (2)Validation of the reported gains: Have the principal configurations been evaluated across multiple independent training runs using a common set of test episodes? I would appreciate means and variability for the baseline, LDE-only, PTCA+PCC, and complete model, particularly for the ScanNet 3-way 1-shot comparison of 66.19 versus 66.12. Evidence of this kind would materially affect my assessment of the claimed complementarity. (3)Evaluation-protocol clarification: How are the validation categories and scenes used for checkpoint selection separated from episodic training and final novel-class testing? Which baseline results in Tables 1–2 were reproduced versus taken from prior publications, and are their model-selection and evaluation protocols aligned?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.