acceptodds
Under review as a conference paper at ICLR 2027

A Non-Parametric Framework for Dynamic Point Cloud Classification Using Optimal Transport-Based Embeddings

Abstract

Dynamic point clouds, time-varying collections of unordered 3D points, exhibit complex variability arising from two distinct sources: spatial deformation of the permutation-invariant point configuration at each instant, and temporal variation of trajectories across time. Existing approaches to dynamic point cloud recognition predominantly rely on deep learning architectures that capture these variations through large numbers of learnable parameters and GPU-dependent optimization, posing practical challenges in resource-constrained settings. In this work, we propose a completely non-parametric framework for dynamic point cloud classification grounded in optimal transport theory. Linear Optimal Transport (LOT) resolves spatial permutation ambiguity by mapping each unordered point cloud to a canonical Euclidean representation, while the Cumulative Distribution Transform (CDT) encodes per-joint temporal signals into a representation where temporal variability becomes more tractable. Together, these embeddings capture the full dynamic structure in a space suitable to linear analysis, where each class is modeled as a low-dimensional subspace and classification is performed by nearest subspace assignment, requiring no gradient-based optimization, no learnable parameters, and no GPU at either training or inference time. Evaluated on KIMORE, UI-PRMD, MSR-Action3D, and NTU RGB+D against eight competitive parametric baselines, the proposed framework achieves competitive classification performance while requiring no learnable parameters or GPU-based optimization. It also maintains substantially lower floating point operations (FLOPs) and very low CPU inference latency, making it a strong candidate for resource-constrained deployment.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Dynamic Point Cloud Classification, Optimal Transport, Non-Parametric Learning, Efficient 3D Recognition

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_P8mWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a fully parameter-free dynamic point cloud classification method: it uses linear optimal transport (LOT) to eliminate the permutation ambiguity of points, employs cumulative distribution transform (CDT) to encode temporal trajectories, and then performs classification using nearest subspaces. Experimental results also demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The problem setting targeted by the paper is of considerable value, and it is the first work of its kind. It enables fast dynamic point cloud recognition without gradients or GPUs, which is an important requirement for rehabilitation monitoring and edge-device recognition, and it also offers stronger interpretability.

2.The combination of methods in the paper is relatively novel and is supported by sufficient theoretical analysis, which provides a solid foundation for the interpretability of the method.

Critical weaknesses

1.The paper claims the advantages of its method for dynamic point cloud processing, but in fact its experimental datasets are all joint-related. Although the conclusion states in future work that dense point clouds or LiDAR sequences will be processed later, the difference between the two lies not only in data distribution but also in the number of points. I am concerned that dense point clouds may cause the computational complexity of the method to far exceed that of existing learnable neural networks, causing the advantages of the paper's method to be lost.

  1. There are some problems with the experimental setup, which reduce its credibility: (a) The FLOPs reported in the appendix indicate that for the other methods the computational cost is measured over multiple forward passes during training, whereas the proposed method reports only a single forward pass. This unfair comparison can only demonstrate the advantage of requiring no training, while the computational cost at inference time remains unknown. (b) The other methods should all have been retrained on the joint data, and Table 10 in the appendix shows a case where PointNet4D achieves lower performance with 90% of the training data than with 10% of the training data, which raises concerns about whether the training of these baselines is reasonable or sufficient. (c) The experiments are not sufficiently comprehensive; for example, ablation studies on techniques such as LOT, CDT, and joint shuffling are missing.
Questions for authors
  1. The techniques proposed in the paper are claimed to be for dynamic point clouds, but I am concerned that for dense point clouds or LiDAR sequences, in addition to the difference in data distribution, the difference in the number of points may cause the complexity of the proposed method to increase substantially, so that its computational cost may become worse than that of learnable neural models. Therefore, has the author considered changing the task target, or further demonstrating the generality of the method?

2.The methods compared in the experiments should all need to be retrained on the dataset. Could the corresponding training settings be provided? This is because some strange results currently appear in the paper (the discrepancy between the two performances of PointNet4D in Table 10 of the appendix), which may raise concerns about whether the compared methods were trained sufficiently or reasonably.

3.Why not consider reporting the GFLOPs of a single forward pass at inference time for each method? This would be fairer.

4.The current experiments are not yet complete, especially the ablation studies on the various techniques in the model, including LOT, CDT, etc.

3 comments

We are grateful for the reviewer’s helpful suggestions, which have improved the clarity of our paper.

(W1 & Question 1)

We use “dynamic point clouds” as a general descriptor to characterize the spatiotemporal structure of these datasets, where 3D joint coordinates (X,Y,Z) evolve over time (t), providing the basis for developing our method.

However, our research scope is exclusively focused on skeleton-based data, as clarified in the Introduction (lines 38–42, 44, 52) and Literature Review (line 84). The Method section explicitly states, “In geometric settings such as skeleton-based human pose representations...” (line 121) and further explains the alignment of skeleton data with Brenier’s theorem (line 169). Additionally, the Dataset Details section explicitly identifies our scope as a “skeleton-based dynamic motion classification benchmark dataset” (line 303).

Clarifying and Narrowing the Described Scope of the Target Task in the revised Manuscript

Nevertheless, we agree with the reviewer’s concern and, to better communicate the scope of this research, will narrow down and revise the paper’s title and relevant descriptions, where necessary, to more accurately represent the target task as skeleton-based sparse dynamic point-cloud classification rather than dense dynamic point-cloud classification.

(W2) (a & Question 3)

We report total training FLOPs for all baselines including our proposed method, not for a single forward pass. Deep models need many epochs of forward and backward passes to reach good accuracy, while ours is a non-iterative, closed-form fit: one single pass over the training data (alignment and CDT) plus an SVD. A single forward pass over an untrained neural network gives chance-level accuracy, whereas one pass of our method already yields its final, mature performance. Therefore, a single-pass comparison would not reflect the cost of reaching comparable accuracy, so total training FLOPs is the fair basis. We will state this explicitly in the revised manuscript.

(b & Question 2)

We re-verified the end-to-end training pipeline and confirmed that retraining the baseline methods reproduces the reported results.

We attribute the near-chance performance of PointRNN and PointNet4D on UI-PRMD, which remains largely unchanged as training data increases from 10% to 90%, to potential limitations of these architectures in capturing the dataset's discriminative motion patterns. This limitation appears model-specific, as other models perform well on the same dataset. In contrast, both PointRNN and PointNet4D perform competitively on KIMORE, NTU, and MSR, with performance improving as training data increases from 10% to 90%.

We also warmly welcome the reviewer to consult the provided complete codebase for independent verification and reproducibility of our results.

(C & Question 4)

We report component-wise ablation results for our proposed method across five random seeds, presenting mean ± standard deviation for robust evaluation.

Componentwise Ablation Study (reporting accuracy)

DatasetTrain data%BaselineAlignment OnlyCDT OnlyAlignment+CDT
KIMORE10%25.96 ± 1.0084.24 ± 0.8526.09 ± 0.5684.40 ± 0.70
90%33.38 ± 4.3489.56 ± 1.3538.82 ± 1.9790.00 ± 2.18
UI-PRMD10%11.51 ± 0.9766.16 ± 4.5511.53 ± 0.9069.07 ± 4.92
90%27.20 ± 3.3596.87 ± 0.5526.20 ± 1.3099.60 ± 0.55
NTU10%27.75 ± 0.6239.86 ± 1.9129.13 ± 0.5341.18 ± 1.87
90%16.68 ± 0.5152.69 ± 1.2115.94 ± 0.2455.41 ± 1.58
MSR10%8.89±0.8753.70±5.789.40±0.9568.03±6.61
90%13.84±4.9791.92±4.1618.84±4.5994.23±1.35

Additionally, we have added benchmark comparisons with non-parametric static point-cloud classification models, whose poor performance on dynamic data highlights their limitations in capturing underlying spatiotemporal patterns. Please see our responses to Reviewer BFkX (W2 & Question 2) for details.

Could we analyze the comparison of computational costs between other methods and your method when the number of point clouds increases? Based on my previous experience, when the number of point clouds increases significantly, the growth rate of computational costs for the parameter-free method is higher than that of other trainable methods.

We sincerely thank the reviewer for taking the time to raise this thoughtful and constructive comment. We appreciate the opportunity to address this point and further clarify our work.

To address the reviewer’s concern regarding the computational cost associated with an increased number of points in the data, we conducted an additional experiment on the dense PCVD (Point Cloud Video for Dynamic Human Interaction) dataset [1]. For a head to head comparison, we use 10% of the training data for both the deep learning-based trainable models and our proposed method. Our method outperforms the deep learning-based baselines in terms of both computational efficiency and predictive performance.

These results demonstrate that increasing the number of point-cloud samples does not adversely affect the computational efficiency or predictive performance of our method.

ModelAccuracy (%)Precision (%)Recall (%)F1 Score (%)FLOPs (Single Forward Pass)Latency (ms)
P4Transformer55.3859.0555.3853.2774.59 MFLOPs12.5190
PointLSTM37.3138.7537.3135.141.39 GFLOPs49.1811
PSTNet21.1526.6021.1518.142.44 GFLOPs155.6451
SequentialPointNet18.4620.2318.4617.86818.82 MFLOPs13.4058
Proposed_KIMORE56.1564.4156.1555.233.32 MFLOPs0.5000

However, we would like to reiterate that the primary scope of this work is skeleton-based and sparse point-cloud-sequence human motion classification, as consistently stated in the Introduction (lines 38–42, 44, 52), Literature Review (line 84), Method (lines 121, 169), and Dataset Details (line 303). We also explicitly note in the Future Work section that dense point cloud analysis lies beyond the scope of the present study and is left for future investigation.

For this reason, the above experiment is provided solely to address the reviewer’s concern. We do not intend to include results on dense dynamic point clouds in the final version of the paper.

Instead, we will carefully refine the wording throughout the manuscript (particularly the title and abstract) to more precisely align our scopes & contributions with prior works [2, 3, 4, 5] that similarly focus on sparse point-cloud-sequence-based human motion classification.

References

[1] Li, Jie, et al. "Pcvd: A dataset of point cloud video for dynamic human interaction." Proceedings of the 16th ACM Multimedia Systems Conference. 2025.

[2] Fan, Hehe and Yu, Xin and Ding, Yuhang and Yang, Yi and Kankanhalli, Mohan. "Pstnet: Point spatio-temporal convolution on point cloud sequences." The International Conference on Learning Representations (ICLR). 2021.

[3] Fan, Hehe and Yang, Yi and Kankanhalli, Mohan. "Point 4d transformer networks for spatio-temporal modeling in point cloud videos." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). 2021.

[4] Li, Xing and Huang, Qian and Wang, Zhijian and Yang, Tianjin and Hou, Zhenjie and Miao, Zhuang. "Real-time 3-d human action recognition based on hyperpoint sequence." IEEE Transactions on Industrial Informatics. 2022.

[5] Min, Yuecong and Zhang, Yanxiao and Chai, Xiujuan and Chen, Xilin. "An efficient pointlstm for point clouds based gesture recognition." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.

Reviewer_BFkXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a non-parametric method for dynamic point cloud classification. Each frame is aligned to a reference with LOT (Hungarian matching), each joint trajectory is transformed with CDT, and classification is done by nearest subspace using SVD. The method is evaluated on four skeleton datasets with shuffled joint order and compared with eight parametric baselines.

Critical strengths
  1. The method has no learnable parameters and shows low inference latency on a single CPU core, measured under the same setting as the baselines. This is useful for resource-limited settings.

  2. The method works relatively well with limited training data (e.g., best accuracy on UI-PRMD at 10:90), and the code is provided.

Critical weaknesses
  1. Misleading FLOPs comparison. According to App. E, baseline FLOPs include full training, while the FLOPs of the proposed method include only a single forward pass. The latency comparison is fair, but the FLOPs reductions highlighted in Sec. 4.4.1 (2.3K×–2.1M×) compare different quantities and may mislead readers.

  2. Missing non-parametric baselines and component ablation. The method mainly combines existing LOT- and CDT-based subspace classifiers, but all baselines are parametric deep models. There is no comparison with simple non-parametric methods (e.g., DTW with kNN, or LOT-only and CDT-only subspace classifiers), and the only ablation is the SVD threshold (Table 6). It is therefore unclear what each component contributes.

  3. Inconsistent alignment design. The joint permutation is fixed across all frames of a sample (App. D), but each frame is aligned to the reference independently (App. E). As a result, frame-to-frame agreement drops to 60–87% (Table 8), so a channel may switch between different joints over time. Aligning only the first frame and applying the same order to all frames seems to avoid this, but this option is not discussed.

  4. Some baseline results are questionable. PointRNN and PointNet4D perform at chance level on UI-PRMD even with 90% training data, while several other baselines reach 98–100%. This suggests these two baselines may not be properly trained on this data.

  5. Limited evaluation protocol. The experiments use skeleton data with shuffled joints rather than real dynamic point clouds, and ratio-based splits are used instead of the standard protocols of these datasets (e.g., cross-subject). It is also not stated whether splits are subject-wise, and no repeated runs are reported. This makes the results hard to compare with prior work.

Questions for authors
  1. Since the joint order is fixed within each sample, why is each frame aligned independently? How does the method perform if only the first frame is aligned?

  2. How does the method compare with LOT-only, CDT-only, and simple non-parametric baselines such as DTW with kNN?

  3. Are the splits subject-wise? How does the method perform under the standard cross-subject protocols?

1 comment

We thank the reviewer for the constructive comments, which have helped improve our manuscript.

(W1)

We report total training FLOPs for baselines and our proposed method. for clarification Please see our response to Reviewer P8mW, (W2)(a & Question 3), for details.

(W2 & Question 2)

We provide comprehensive comparisons with DWT+kNN and recent non-parametric static point-cloud baselines. We conducted component-wise ablations across five random seeds and inclue the mean performance in this comparison table. Please see our response to Reviewer P8mW, Comment W2 (C & Question 4), for details.

We will also incorporate the multi-seed results into the main tables; notably, they do not affect any of the claims made in the paper.

Our method outperforms all baselines, while SOTA static non-parametric models struggle to capture dynamic point-cloud patterns due to their representational limitations.

DatasetRatioModelAccPrecRecF1FLOPsLatency(ms)
10:90DWT+kNN28.4520.7125.1923.680.000005G10.2
PointNN27.255.4520.008.570.28T226.87
PointGN27.255.4520.008.570.29T67.51
PointLN27.255.4520.008.5743.17T489.00
NPMFF-Net27.255.4520.008.570.40T22.18
KIMOREOurs84.4087.4282.5183.900.33G0.85
90:10DWT+kNN35.2025.6630.2528.850.00005G10.8
PointNN27.215.4420.008.552.51T225.22
PointGN27.215.4420.008.552.61T83.59
PointLN27.215.4420.008.55387.86T527.28
NPMFF-Net27.215.4420.008.553.58T26.75
Ours90.0093.3087.9689.235.03G0.90
---------
10:90DWT+kNN35.2025.6630.2528.850.000002G10.2
PointNN51.4458.7851.4447.770.10T134.91
PointGN59.8970.7159.8959.570.10T61.62
PointLN62.8967.7862.8962.7220.11T419.74
NPMFF-Net55.4458.4555.4454.320.13T26.21
UIOurs69.0773.3969.0769.380.12G0.50
90:10DWT+kNN40.2530.1835.6533.420.00002G11.83
PointNN45.0054.0845.0040.340.91T134.76
PointGN70.0082.4970.0070.520.87T62.67
PointLN72.0079.2872.0069.18180.96T429.05
NPMFF-Net51.0069.4351.0049.481.14T31.47
Ours99.6099.6499.6099.601.39G0.47
---------
10:90DWT+kNN31.221.427.624.30.00025 G24.7
PointNN4.233.594.212.1111.62T198.75
PointGN12.256.4912.206.6112.07T69.62
PointLN8.327.518.294.101796.78T425.85
NPMFF-Net9.027.378.984.2716.59T25.99
NTUOurs41.1847.3041.2140.1016.32G2.77
90:10DWT+kNN47.2331.5141.8936.120.0025G26.10
PointNN3.071.303.050.68104.49T156.43
PointGN13.1014.3213.048.33108.59T62.92
PointLN8.289.638.234.9516159.92T336.00
NPMFF-Net10.957.4810.906.38149.17T21.92
Ours55.4160.6455.4354.62434.78G7.30
---------
10:90DWT+kNN47.3431.7842.5639.670.01G60
PointNN15.3514.9314.649.950.08T138.40
PointGN41.5446.0440.1236.540.09T61.75
PointLN41.9347.7540.7437.1914.99T330.67
NPMFF-Net27.5634.3726.7725.050.11T20.81
MSROurs68.0376.1768.3768.670.07G1.06
90:10DWT+kNN57.4341.1947.5845.260.1G63.1
PointNN16.9511.7716.6711.400.73T136.96
PointGN38.9824.8638.3327.910.73T61.07
PointLN38.9824.8138.3327.69129.05T329.27
NPMFF-Net32.2031.5231.6727.600.98T20.78
Ours94.2395.4293.8393.490.70G1.02

(W3 & Question 1)

The per-frame alignment is standard practice in LOT [1,2], each frame is embedded independently against a common reference, so channel i denotes the point matched to reference point i, not a fixed anatomical joint. We nevertheless tested the suggested first-frame design against per-frame design, and our per-frame alignment outperforms the first-frame design.

DatasetAlignmentAccuracyPrecisionRecallF1 Score
KIMOREPer-frame(ours)89.7193.3488.0589.80
KIMOREFirst-frame only88.9792.2886.8588.61
UIPer-frame(ours)86.0091.9286.0084.95
UIFirst-frame only76.0085.9776.0073.35
MSRPer-frame(ours)92.3190.8390.8389.90
MSRFirst-frame only88.4691.3386.6786.82

(W4)

We attribute this to model-specific limitations in capturing the dataset’s discriminative motion patterns. details in response to Reviewer P8mW, (W2) (b & Question 2)

(W5 & Question 3)

Our scope remains strictly limited to skeleton-based data.

We repeat all experiments over five seeds and report mean ± std for every dataset and split. details in response to Reviewer P8mW, Comment W2 (C & Question 4)

We also report the accuracy for cross-subject protocols below-

KIMOREUIMSRNTU
84.8260.274.9151.3

References:

[1] Wang et al., "A Linear Optimal Transportation Framework for Quantifying and Visualizing Variations in Sets of Images." IJCV.

[2] Shifat-E-Rabbi M, Rohde GK. "Linear optimal transport subspaces for point set classification." JMIV

Reviewer_JegAOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary
  • Abstract:
  • Ln.23: 'per-joint' what is a joint? I thought we were classifying point clouds not human poses.
  • Ln.24: 'temporal variability becomes more tractable' : what does it mean for a variability to become tractable? Tractability is a computational property.
  • Ln. 28: 'no learnable parameters': if you don't have learnable parameters, what do you mean by 'training'?

Abstract can make it clear what is being classified here. Is it the motion or the shape? Do we have same shape appearing with different dynamics or different objects with different dynamics? The latter is an easier setting I suppose.

  • Introduction:
  • Why is it bad to depend on learned neural weights? I think this is a strength given that almost all deployments of CV pipelines use neural nets now.
  • Optimal transport is also very costly - especially when the number of classes (and samples per class) become large.
  • "Linear Optimal Transport (LOT) maps unordered point configurations into a consistent Euclidean representation" => but optimal transport doesn't map point configurations to representations. It's a map between things .
  • Ln 73: 'first fully non-parametric frameworks' => first fully non-parametric framework
  • In general, introduction does a really bad job of .
  • The literature review is really weak. Both point cloud classification and optimal transport are heavily studied methods and the paper should dedicate a strong related work section for these.

  • Method

  • Ln 198: what's a cost-matrix tie?
  • I fail to understand what LOT brings here. Why are we not simply solving the frame-to-frame OT? Isn't it the same cost as solving to a reference measure (whose sampling might also have a negative impact in the representation of points)?
  • Ln. 247: how does the paper normalize \hat{P} into s? one is high dimensional, whereas the other is 1D. CDT is defined for 1D only.
  • Are we only limited to translations, isotropic scalings, or their near-affine perturbations? This limits the dynamics and makes it a completely different problem.
  • Ln 276: what is h'?
  • What is a generative model here really? How does it capture data distribution? How do you sample from this? Also, I thought we were interested in classification. Why do we have a generative model?
Critical strengths
  • Experiments seem thorough although on small datasets.
  • The paper made an effort to control the use of AI. I appreciate that. Though, traces remain.
  • The idea is sensible - yet some claims are not justified.
Critical weaknesses
  • Abstract:
  • Ln.23: 'per-joint' what is a joint? I thought we were classifying point clouds not human poses.
  • Ln.24: 'temporal variability becomes more tractable' : what does it mean for a variability to become tractable? Tractability is a computational property.
  • Ln. 28: 'no learnable parameters': if you don't have learnable parameters, what do you mean by 'training'?

Abstract can make it clear what is being classified here. Is it the motion or the shape? Do we have same shape appearing with different dynamics or different objects with different dynamics? The latter is an easier setting I suppose.

  • Introduction:
  • Why is it bad to depend on learned neural weights? I think this is a strength given that almost all deployments of CV pipelines use neural nets now.
  • Optimal transport is also very costly - especially when the number of classes (and samples per class) become large.
  • "Linear Optimal Transport (LOT) maps unordered point configurations into a consistent Euclidean representation" => but optimal transport doesn't map point configurations to representations. It's a map between things .
  • Ln 73: 'first fully non-parametric frameworks' => first fully non-parametric framework
  • In general, introduction does a really bad job of .
  • The literature review is really weak. Both point cloud classification and optimal transport are heavily studied methods and the paper should dedicate a strong related work section for these.

  • Method

  • Could the paper please include an algorithm? This would make things much clearer.
  • Ln 198: what's a cost-matrix tie?
  • I fail to understand what LOT brings here. Why are we not simply solving the frame-to-frame OT? Isn't it the same cost as solving to a reference measure (whose sampling might also have a negative impact in the representation of points)?
  • Ln. 247: how does the paper normalize \hat{P} into s? one is high dimensional, whereas the other is 1D. CDT is defined for 1D only.
  • Are we only limited to translations, isotropic scalings, or their near-affine perturbations? This limits the dynamics and makes it a completely different problem.
  • Ln 276: what is h'?
  • Do we have only 1 ?
  • What is a generative model here really? How does it capture data distribution? How do you sample from this? Also, I thought we were interested in classification. Why do we have a generative model?
Questions for authors

Critically, I did not quite understand how a high-dimensional problem is transformed into a single dimensional one where CDT can be used to solve OT. In general the way LOT and CDT-type matching interact is very unclear. Also why is there a generative model? (I did not get if that's a generative model). What does it mean to apply 'CDT'? This is just a transform. What does the paper use this for?

I guess most of these confusions are there because of AI edited text that contains covered-meaninglessness rather than unpolished but meaningful text.

3 comments

We thank the reviewer for raising these concerns.

Abstract

  1. Each joint represents a skeletal keypoint (e.g., shoulder, elbow, hip, or knee). Our scope is strictly skeleton-based human motion classification, as clarified in the Introduction (ll. 38–42, 44, 52), Literature Review (l. 84), Method (ll. 121, 169), and Dataset Details (l. 303).

  2. Here, “tractable” refers to the analytical treatment of temporal variability, not computational complexity. To avoid ambiguity, we will replace it with “more amenable to linear analysis.”

  3. In our method, training refers to a one-shot, non-iterative fitting stage (alignment, CDT, and per-class SVD) rather than gradient-based parameter optimization.

Introduction

  1. Deep learning methods require GPUs for fast inference and model updates, mostly unavailable on resource-constrained devices; our method enables GPU-free CPU training and inference with faster fitting and inference.

  2. Head-to-head evaluations on large-scale NTU RGB+D (60 classes, 56,880 samples) and small-scale KIMORE (5 classes, 2,516 samples) demonstrate significantly lower inference latency in both cases, showing that computational cost remains stable as the number of classes and samples increases (Table 9,11).

  3. We agree that the Monge map transports probability measures; LOT uses its reference-dependent displacement from the identity as a representation in a common tangent space [4], yielding the finite-dimensional representation s∘σs∗∈RN×3s\circ\sigma_s^*\in\mathbb{R}^{N\times3} under our discrete equal-mass implementation.

  4. To our knowledge, ours is the first non-parametric framework for dynamic sparse point-cloud classification, precluding direct apples-to-apples comparisons. We therefore review closely related areas and their limitations. We additionally compare our method with recent non-parametric static point-cloud models (please see Reviewer BFkX, Comment W2).

Method

  1. A cost-matrix tie occurs when multiple point correspondences have identical costs, potentially yielding distinct assignments with the same minimum total cost (multiple optimal solutions).

  2. Unlike frame-to-frame OT’s local correspondences, LOT maps every frame to a fixed reference, yielding a shared Euclidean representation without temporal correspondence composition, as established in LOT-based point-set [2] and SCDT/R-CDT subspace classification [1,3].

  3. We do not reduce P^\hat{P} directly to a single dimension; instead, we normalize each coordinate-wise marginal independently into a 1D probability density ss, to which the CDT is applied separately.

  4. Our framework processes full motion sequences, retaining observed dynamics; translations, isotropic scalings, and near-affine perturbations define the conventional transformation model for theoretical analysis in LOT/R-CDT subspace formulations [2,3].

  5. Here, h′h' denotes the derivative of the one-dimensional temporal deformation h(t)h(t) with respect to time; it is scalar-valued because the temporal variable is one-dimensional.

We will add an algorithm and correct typos in the revised manuscript.

Questions

We do not reduce a high-dimensional OT problem to 1D, and CDT is not used to solve the 3D OT. LOT handles spatial variation and CDT handles temporal variation. LOT solves the 3D OT per frame to give a consistent point ordering. After that ordering, each point’s coordinate (x, y or z) over time is a 1D function of time, obtained by indexing, not by projecting or slicing the 3D problem. CDT is then applied to each of these functions as a separate 1D OT problem along the time axis, where the OT map has a closed form. CDT is a fixed, invertible transform with no learned parameters that turns temporal warps (variation in the time domain) into a form where each class can be well approximated by a linear subspace.

The term “generative model” is a deformation data model in Optimal Transport (OT)[1,2,3], not a learned one (unlike GANs or diffusion models). The generative model in OT characterizes each class as a template undergoing spatial and temporal deformations, establishing the theoretical basis for class-wise subspace modeling in LOT–CDT space.

We respectfully disagree with this “AI-edited meaninglessness text” characterization and believe it is now addressed by our explanations; We welcome any further questions for clarification.

References

[1] Rubaiyat AH, Li S, Rohde GK. "End-to-end signal classification in signed cumulative distribution transform space." IEEE TPAMI.

[2] Shifat-E-Rabbi M, Rohde GK. "Linear optimal transport subspaces for point set classification." JMIV

[3] Shifat-E-Rabbi M, Yin X, Nichols JM, Rohde GK. "Radon cumulative distribution transform subspace modeling for image classification." JMIV

[4] Wang et al., "A Linear Optimal Transportation Framework for Quantifying and Visualizing Variations in Sets of Images." IJCV.

I thank the authors for clarifications.

I double-checked the paper, and I don't think the paper is focusing strictly on the skeleton-based human motion classification. It is full of statements like "this work, we propose a completely non-parametric framework for dynamic point cloud classification grounded in optimal transport theory.", that lack the notion of human skeletons (which would in fact have a particular skeletal topology that can be leveraged as well), which desire to be a general dynamic point cloud network, but it's not. In human cases, we also know the point correspondences by the way -- if we use 25 body joints in 3D space, this means it's like a skeleton, and the fact that we scramble joints to simulate realistic settings doesn't help. Given the current presentation, I unfortunately cannot recommend accepting this work. Also, I would not call this the first non-parametric work that can do dynamic point cloud classification, there are many 4D point cloud works from early days, such as EigenJoints [https://ieeexplore.ieee.org/document/6239232] and similarly Hayden et al. CVPR 2020 [https://sli.csail.mit.edu/spotlight/paper-hayden2020cvpr/] that can solve a related problem. I would also suggest CaSPR (https://geometry.stanford.edu/projects/caspr/) which include components like normalizing flows for density estimation of dynamic point clouds.

Also, much like other preceding works, the term “non-parametric” requires a restriction. OT does not involve learned feature-extraction weights, but the classifier still estimates fixed, class-specific bases from training data. I don't see this as a problem, but if we are accepting that, then there may be many other non-parametric dynamic PC classifiers as well.

We thank the reviewer for raising these questions, which provide an scope to clarify and better explain our research work.

Skeleton-based motion classification data fulfill all the requirements to be termed dynamic point clouds.

Our experiments are conducted exclusively on skeleton-based human action recognition datasets. Because each frame of skeleton-based motion data comprises an unordered set of 3D joint coordinates (X,Y,Z) that change over time (t), these data fulfil the properties required to be considered dynamic point clouds. We therefore adopt the term “dynamic point clouds” to better characterize their spatiotemporal structure.

Nevertheless, we explicitly clarify that our work focuses on skeleton-based motion classification.

This is consistently established throughout the manuscript: the Introduction defines each frame of skeleton-based motion data as a 3D joint set (Lines 38–42); Literature Review started with dynamic point-cloud and skeleton-based motion analysis (line 84); the Method explicitly formulates each pose as an unordered collection of 3D skeletal joint positions and describes the corresponding discrete implementation (Lines 121, 169–179); the Dataset Details explicitly identify all four evaluation benchmarks as skeleton-based (Line 303); and the Appendix specifies that the input sequences for each benchmark comprise 3D skeletal joint positions. Furthermore, the Limitations and Future Work section explicitly identifies evaluation on dense dynamic point clouds and LiDAR sequences as future work (Lines 668–670).

Although beyond our scope, we demonstrate that our pipeline also effectively handles dense dynamic point clouds encountered in more realistic settings.

Please see our response to the official comment by Reviewer P8mW.

The suggested non-parametric methods failed to address our problem setting, generalizing spatiotemporal variations, whereas our proposed method successfully does so.

We partially agree with the reviewer that our claim of being the first non-parametric model requires more precise wording, and we will refine this claim in the final manuscript.

Nevertheless, we carefully examined all the suggested papers, EigenJoints [1], CaSPR [2], and NPP [3], and found that these methods failed substantially under spatiotemporal variations. Although CaSPR [2] achieved competitive performance, its results remained considerably below those of our proposed method.

Thus, our proposed method appears to be among the first non-parametric approaches capable of generalizing across spatiotemporal variations.

DatasetMethodAcc(%)Prec(%)Rec(%)F1(%)Train FLOPs (G)Inference Latency (ms)
EigenJoints20.0519.8820.0519.660.080628.34
KIMORECaSPR77.3577.8876.0176.360.08069.64
NPP20.0519.8420.0419.610.080628.78
Ours90.0093.3087.9689.235.03G0.90
--------
EigenJoints10.5610.0410.5910.240.02167.05
UI-PRMDCaSPR73.7873.6574.0073.950.02165.54
NPP8.1110.967.907.780.02167.70
Ours99.6099.6499.6099.601.39G0.47
--------
EigenJoints2.206.961.672.081.8115640.79
NTU RGB+DCaSPR39.2556.4132.0137.731.8115426.55
NPP1.958.841.051.741.8115664.84
Ours55.4160.6455.4354.62434.78G7.30
-------
EigenJoints6.695.106.342.910.00212.25
MSR-Action3DCaSPR49.4155.6149.0449.940.00212.07
NPP7.688.307.273.570.00212.42
Ours94.2395.4293.8393.490.70G1.02

Clarification on non-parametric method

Our entire pipeline is non-parametric: LOT (Hungarian assignment to a fixed reference), CDT (closed-form 1D transport), and class subspaces (one-shot SVD) involve no learned weights, loss functions, backpropagation, or GPU computation. In our response to Reviewer BFkX (W2, Q2), we provide additional non-parametric comparisons on static data. However, these methods do not address spatiotemporal variations, whereas ours jointly resolves spatial permutations (LOT) and temporal deformations (CDT) within a unified OT-based framework. To the best of our knowledge, ours is the first backpropagation-free framework of its kind to address this problem.

References

  1. Yang, Xiaodong, and Ying Li Tian. "Eigenjoints-based action recognition using naive-bayes-nearest-neighbor." 2012 IEEE computer society conference on computer vision and pattern recognition workshops. IEEE, 2012.

  2. Rempe, Davis, et al. "Caspr: Learning canonical spatiotemporal point cloud representations." Advances in neural information processing systems 33 (2020): 13688-13701.

  3. Hayden, David S., Jason Pacheco, and W. Fisher John. "Nonparametric object and parts modeling with lie group dynamics." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.