acceptodds
Under review as a conference paper at ICLR 2027

Coherence-Preserving Distribution Matching for 3D Point Cloud Dataset Distillation

Abstract

Dataset distillation compresses a large training set into a compact synthetic set while preserving its training utility. For 3D point clouds, distribution matching must additionally account for arbitrary point indexing. Existing Semantically Aligned Distribution Matching (SADM) addresses this issue by independently sorting pointwise feature responses within each channel. However, such channel-wise sorting discards the cross-channel associations among responses originating from the same point, introducing an unnecessary invariance to independent channel permutations. We propose Coherence-Preserving Distribution Matching (CPDM), which complements channel-wise matching with a shared point ordering derived from aggregated normalized feature responses. Applying this shared ordering to complete pointwise feature vectors preserves their cross-channel coherence while remaining invariant to point re-indexing. CPDM further employs trained expert checkpoints for feature extraction and auxiliary classification supervision. Theoretically, we show that, under distinct point scores, the full shared-order representation is invariant to shared point permutations and identifiable up to such permutations, avoiding the additional ambiguity induced by independent channel-wise sorting. Experiments on five 3D classification benchmarks demonstrate consistent improvements across a wide range of compression budgets. In the non-parameterized setting, CPDM achieves the highest mean accuracy among the compared methods in 12 of 15 dataset–budget configurations, while its parameterized variant outperforms 3DDP in 14 of 15 configurations. Cross-architecture evaluations and ablation studies further support the transferability and effectiveness of coherence-preserving matching.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Dataset Distillation, 3D Point Clouds, Distribution Matching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9kyrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a CPDM method for distilling 3D point cloud datasets. Building upon channel-level distribution matching, the method introduces a shared point ordering mechanism to preserve cross-channel feature correlations. Synthetic point clouds are optimized by leveraging pre-trained expert features and auxiliary classification supervision. Experiments cover five classification datasets, various compression budgets, and cross-architecture evaluations.

Critical strengths
  1. Clear research motivation: Independent per-channel ranking may discard correlations within individual point feature vectors, whereas the proposed shared ranking directly addresses this issue.
  2. The reported results are encouraging, and component ablation experiments help distinguish the respective contributions of shared ranking, expert features, and classification supervision.
Critical weaknesses
  1. Theoretical analysis demonstrates information retention under various scoring conditions but does not verify the stability of the matched representations. Minor feature perturbations may alter point rankings, and the practical implications of these discontinuities have not been fully investigated.
  2. There is currently no quantitative description of the computational cost of the complete distillation process. We recommend reporting the actual time taken for both the offline expert preparation and the distillation process separately, and comparing these with SADM under identical hardware and representative settings to assess the trade-off between accuracy and computational cost. The paper has reported the mean and standard deviation of 10 independent initialisations for evaluating the network; please clarify further whether these repetitions also included independent distillation processes.
  3. This paper primarily employs PointNet for distillation, and the cross-architecture evaluation focuses mainly on earlier models; consequently, its practical value when applied to more powerful backbones or pre-training setups remains unclear. It is recommended to compare it with existing distillation methods on newer backbones, such as PointNext, using the same compression budget, and to report the extent of performance retention when trained on relatively comprehensive datasets. If supplementary experiments cannot be provided at this stage, please explain, based on existing theory and experiments, why the issues identified have general relevance beyond these specific architectures, and what substantive new insights this paper offers compared to existing distribution-matching methods. A convincing explanation would also assist me in reassessing the paper’s contributions.
  4. The figures in the paper require improvement, particularly Figure 2.
Questions for authors
  1. The counterexample to this theory involves arbitrary feature matrices. Is it possible that the features from independent permutations in this counterexample actually originate from the same ‘frozen’ extractor processing the same point cloud? Is there any evidence to suggest that SADM’s additional invariance leads to ambiguity between point cloud features?

  2. Preserving information does not necessarily yield useful matching distances. When the scores of two points are nearly equal but their feature vectors differ significantly, even a very small perturbation can cause their rankings to swap and result in a drastic change in the Gaussian kernel distance. How frequently does this occur? Why is the resulting distance suitable for distribution matching?

  3. The recognisability argument appears to apply to any deterministic reordering that results in different scores (such as permutations). Specifically, what justification is there for claiming that the proposed average response score is superior to simpler shared ordering rules? Can its advantages be distinguished from the general benefits of retaining complete point feature vectors?

Reviewer_7YzmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how distribution matching can lose important information in 3D dataset distillation. The authors point out that SADM independently sorts feature channels to remove sensitivity to point ordering, but in doing so, it loses the relationships between features from the same point. To address this, they propose CPDM, which uses a shared sorting order to preserve these pointwise relationships. The method also relies on trained expert checkpoints and auxiliary classification supervision. The authors further provide theoretical results showing permutation invariance and feature-map identifiability under certain assumptions.

Critical strengths
  • A correct observation: The product-group argument (Eq. 10) and the 2×3 counterexample (A.1) make precise why independent sorting loses cross-channel coherence. Figure 2(b) communicates it well.

  • Simple and efficient solution: The method uses a single shared permutation for each point cloud, introduces no additional parameters, and can be easily integrated into existing pipelines, including both direct and 3DDP-parameterized settings.

  • Extensive evaluation: The authors test their method across multiple datasets, budgets, and architectures, with additional backbone studies, hyperparameter analysis, and results averaged over 10 runs.

Critical weaknesses
  • Unclear source of the main improvements: CPDM introduces three changes over SADM, but it is unclear which one drives the largest gains. Table 4 shows only modest improvements from the coherence term (+0.5 to +2.9 points), while the largest gains (up to +20.8 points) have no ablation. Since the other two components already appear in TGDD (Ran et al., 2026), it is difficult to isolate CPDM's actual contribution. Figure 3 does not resolve this, as SADM's zero discrepancy is expected by construction.

  • For me, it's unclear why DD3D (Bo & Wang, ICML 2025) was not included as a baseline.

  • (Minor) The text in Figure 1 is too small and difficult to read. I suggest making it readable.

Questions for authors
  • Does the specific score of Eq. 5 matter? Please compare against (a) a shared ordering by a fixed coordinate, by one channel, and by a random fixed projection, and (b) the ordering-free alternative: MMD over the set of complete pointwise feature vectors (Eq. 20) or per-object mean embeddings. If trivial orderings match Eq. 5 the theory should be re-framed; if the ordering-free variant matches or beats CPDM the mechanism claim changes.

  • How sensitive is the method to τ and K? How are ties handled, and how frequently do rankings change during training? Also, could the authors clarify the normalization used in Figure 3 and explain why the discrepancy reaches 1.28 with only 10% channel corruption?

Reviewer_xUf8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the limitations of the existing 3D point cloud dataset distillation method of SADM, and suggests an incremental work by changing channel-wise feature sorting into shared point ordering across channels. The authors compared the performance of the proposed method to those of the existing methods reported in a recent paper.

Critical strengths

Dataset distillation for 3D point clouds remains relatively underexplored, making research in this area welcomed.

Critical weaknesses
  • Limited technical novelty: The proposed method primarily differs from the existing method of SADM by replacing channel-wise feature sorting with channel-invariant sorting. This modification is relatively incremental and does not constitute a substantial technical contribution.

  • Insufficient justification of problem: The benefits of obtaining a shared-order representation instead of performing channel-wise feature sorting, as well as the underlying motivation, are not sufficiently justified. For dataset distillation of 3D point clouds, we need to have an efficient descriptor of geometric features instead of explicitly synchronizing point orders across channels. For example, each channel may capture specific geometric characteristics of 3D point clouds, which can serve as invariant descriptors to compare different 3D point cloud models. Therefore, the sorted point orders may exhibit similar patterns within each corresponding channel between two 3D models with similar geometric shapes, even without synchronizing the point orders across channels. The authors should provide a clearer justification for why cross-channel order synchronization is necessary and how it benefits 3D shape recognition.

  • Unclear experimental setup: The authors compared the performance of the proposed method with that of SADM by taking the performance scores reported in the 3DPP paper. However, it is not clear whether the proposed method was compared under identical implementation settings, with the sorting scheme being the only difference.

Questions for authors

The authors compared the performance scores of the existing methods reported in the 3DPP paper. Did the authors compare the proposed method with SADM under identical implementation settings, with the sorting scheme being the only difference in Tables 1 and 4?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.