acceptodds
Under review as a conference paper at ICLR 2027

Geometry-Guided Conservative Soft Ordering for Point Cloud Understanding

Abstract

3D point clouds are inherently unordered, whereas state space models (SSMs) process ordered sequences, making token ordering critical for point cloud understanding. Existing methods derive fixed hard orders from geometry that cannot be optimized by the task loss, forcing SSMs to accommodate suboptimal ordering and potentially weakening the learned representations. To address this limitation, we propose Geometry-guided Conservative Soft Ordering (GCSO), which replaces one-to-one hard assignment with trainable conservative soft ordering based on geometry prior. Specifically, GCSO uses current token features and coordinates to predict a local offset for each scan position in a geometric base order. Then the offset centers the weights over neighboring tokens, whose weighted feature mixture forms the soft ordering. Since the weights vary continuously, the task loss can optimize the offset and the resulting feature-adaptive soft ordering end to end. Moreover, GCSO organizes the weights into a nonnegative doubly stochastic routing matrix, ensuring that each scan position receives one unit of assignment while each source token contributes one unit of participation. Experiments on shape classification and part segmentation support the effectiveness of GCSO for point cloud understanding.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
point cloud understanding, state space models, mamba, differentiable ordering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_36vZOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper studies the token ordering problem in point-cloud Mamba models. Existing approaches typically rely on fixed geometric orders, which cannot adapt to the task during training. To address this limitation, the authors propose GCSO, which introduces trainable local soft routing around a geometric base order while keeping token participation balanced. Experiments on point-cloud classification, segmentation, and few-shot learning show consistent improvements over existing Mamba-based baselines, with particularly strong results on ScanObjectNN.

Critical strengths
  1. The motivation is clear and easy to follow. The paper identifies a meaningful limitation of fixed geometric ordering in point-cloud SSMs and provides a straightforward way to make the routing trainable.
  2. The experimental results are strong, especially on ScanObjectNN. GCSO consistently outperforms several recent Mamba-based baselines, and the ablation studies show that the proposed components contribute to the final gains.
Critical weaknesses

The “soft ordering” interpretation is not fully convincing. GCSO does not actually reorder the original tokens into a new sequence. Instead, it computes H~=PΠH\tilde{H}=P\Pi H, where each output slot is a weighted mixture of several neighboring tokens, which looks more like an adaptive token-mixing layer than an ordering operation. The paper therefore needs to clarify whether the gain really comes from learning a better order, or simply from adding this extra feature mixing before Mamba.

Questions for authors
  1. Could the authors clarify in what sense GCSO actually performs “ordering”? Since each output slot mixes several neighboring tokens, it seems closer to a token mixer than an actual reordering operation.
  2. If GCSO does learn a meaningful new order, a visualization of the original and learned orders would help demonstrate this. Otherwise, a comparison with a similar local token-mixing baseline would be useful.
  3. Could the authors show what kind of ordering GCSO actually learns, either quantitatively or with a simple visualization?

My score is currently borderline, but I would consider raising it to 3 if these concerns are addressed.

7 comments
AuthorsOpenReview ↗

Response to 36vZ-W1 (Part 1/2)

Thank you for pointing out that H~=PΠH\widetilde H=P\Pi H acts as an adaptive token-mixing layer. We agree: token mixing describes the forward computation, while soft ordering describes the constrained assignment of source features to ordered scan slots. Below we explain this assignment and use fixed-mixing controls to assess whether the gain goes beyond extra feature mixing before Mamba. To make the terminology self-contained, HH is the patch-feature matrix in original patch-index order, with each row representing one patch token; Π\Pi arranges these tokens by a geometric Morton order; and PP determines how sources in that base sequence contribute to the scan slots Mamba reads in order. We call this “soft ordering” because the essential effect of an order for Mamba is not to output a list of indices, but to determine which features occupy its successive input positions.

Consider first a hard order. Let AA be a binary permutation matrix: Aij=1A_{ij}=1 means scan slot ii reads source token jj in the geometric base sequence, and each row and column has exactly one 1. The hard assignment selects one source per slot, and AΠHA\Pi H then presents the slot-ordered feature sequence to Mamba. For this illustration, label the three rows of ΠH\Pi H as h1,h2,h3h_1,h_2,h_3. If AA swaps the first two positions, the operation is

AΠH=[010100001][h1\h2\h3]=[h2\h1\h3].A\Pi H=\begin{bmatrix}0&1&0\\\\1&0&0\\\\0&0&1\end{bmatrix}\begin{bmatrix}h_1\\\h_2\\\h_3\end{bmatrix}=\begin{bmatrix}h_2\\\h_1\\\h_3\end{bmatrix}.

Mamba then reads h2,h1,h3h_2,h_1,h_3 at slots 1, 2, and 3. Assigning tokens to positions and forming a feature sequence in those positions are thus two linked parts of hard ordering.

GCSO makes both parts continuous. Current features and geometry predict local sequence-index offsets, which produce Gaussian reading affinities between scan slots and nearby source tokens; these affinities are then turned into a nonnegative conservative matrix PP. Here PijP_{ij} is the weight with which source jj enters slot ii, and H~i=∑jPij(ΠH)j\widetilde H_i=\sum_jP_{ij}(\Pi H)_j is the feature Mamba actually reads at that slot. Alongside the hard swap above, consider this illustrative soft assignment:

PΠH=[0.70.300.30.40.300.30.7][h1\h2\h3]=[0.7h1+0.3h20.3h1+0.4h2+0.3h30.3h2+0.7h3].P\Pi H=\begin{bmatrix}0.7&0.3&0\\\\0.3&0.4&0.3\\\\0&0.3&0.7\end{bmatrix}\begin{bmatrix}h_1\\\h_2\\\h_3\end{bmatrix}=\begin{bmatrix}0.7h_1+0.3h_2\\\\0.3h_1+0.4h_2+0.3h_3\\\\0.3h_2+0.7h_3\end{bmatrix}.

The same PP can be derived from three named hard orders. Let II keep the original sequence, S12=[010100001]S_{12}=\begin{bmatrix}0&1&0\\\\1&0&0\\\\0&0&1\end{bmatrix} swap positions 1 and 2, and S23=[100001010]S_{23}=\begin{bmatrix}1&0&0\\\\0&0&1\\\\0&1&0\end{bmatrix} swap positions 2 and 3. Entrywise addition then gives

0.4I+0.3S12+0.3S23=[0.4+0.30.300.30.40.300.30.4+0.3]=P.0.4I+0.3S_{12}+0.3S_{23}=\begin{bmatrix}0.4+0.3&0.3&0\\\\0.3&0.4&0.3\\\\0&0.3&0.4+0.3\end{bmatrix}=P.

AuthorsOpenReview ↗

Response to 36vZ-W1 (Part 2/2)

The Birkhoff–von Neumann theorem guarantees that a doubly stochastic PP can be expressed as a convex combination of hard permutation matrices. Here the off-diagonal entries P12=P21=0.3P_{12}=P_{21}=0.3 give the coefficient of S12S_{12}, and P23=P32=0.3P_{23}=P_{32}=0.3 give that of S23S_{23}; the remaining coefficient is 1−0.3−0.3=0.41-0.3-0.3=0.4 for II. Thus, at the scan-input level, our soft ordering is a linear combination of hard orderings: PΠH=0.4IΠH+0.3S12ΠH+0.3S23ΠHP\Pi H=0.4I\Pi H+0.3S_{12}\Pi H+0.3S_{23}\Pi H. Each hard matrix selects one source per slot; for example, S12S_{12} makes Mamba read h2,h1,h3h_2,h_1,h_3. Their weighted combination instead gives the three mixed features above. Relative to the original order, h1h_1 retains 70% of its reading weight at slot 1 and contributes 30% to slot 2; h2h_2 contributes 30%, 40%, and 30% to slots 1, 2, and 3; and h3h_3 contributes 30% to slot 2 and 70% to slot 3. Thus Mamba's second slot reads all three nearby sources rather than only h2h_2. Hard ordering exchanges whole tokens; the soft assignment changes how each source contributes to the ordered input.

This continuous formulation has two direct benefits. First, a binary hard assignment cannot be adjusted smoothly by ordinary task gradients; gradients can instead pass through PP to the local offset predictor, changing Gaussian affinities and fractional reading weights. Second, P1=P⊤1=1P\mathbf1=P^\top\mathbf1=\mathbf1 retains both participation budgets of a hard permutation: every slot reads one unit of feature weight and every source contributes one unit in total. This avoids excessive repeated reading or complete omission of a source while the scan input changes. Soft ordering can also change non-dominant weights without swapping the largest-weight source of any slot: in the example above, the dominant sources at slots 1, 2, and 3 remain h1,h2,h3h_1,h_2,h_3, yet the fractional weights make Mamba read a feature sequence different from (h1,h2,h3)(h_1,h_2,h_3). A new hard permutation is therefore not required for soft ordering.

The forward soft assignment PΠHP\Pi H is both a continuous allocation to ordered scan slots and an adaptive local token-mixing operation; in GCSO these describe the same computation. We further compare it with fixed local mixing under the same routing interface. On PB_T50_RS/ModelNet40, full GCSO reaches 94.20%/95.70%, compared with 93.27%/95.30% for fixed Gaussian mixing and 93.58%/95.02% for frozen offsets. These comparisons support the benefit of learned, input-dependent soft assignment over fixed mixing in the tested architecture.

The three-slot example is on p. 16, and the fixed-mixing controls are in Table 6 on p. 18 of the revised manuscript.

AuthorsOpenReview ↗

Response to 36vZ-Q1, Q2, and Q3

36vZ-Q1

We appreciate this constructive comment. We define soft ordering as a continuous assignment of source tokens to ordered scan slots, and W1's three-slot example shows it is a linear combination of hard orderings at the scan-input level. We use “ordering” in the sense of an assignment matrix from source tokens to ordered scan slots. A hard order is a binary permutation matrix AA: each source receives one integer slot, and each slot receives one source. GCSO relaxes the binary entries to a nonnegative continuous matrix PP while retaining P1=P⊤1=1P\mathbf1=P^\top\mathbf1=\mathbf1. Column jj of PP therefore assigns source jj fractional participation in nearby scan ranks rather than one unique hard rank; row ii specifies the source weights read at rank ii. The doubly stochastic constraint gives the mathematical link to hard ordering: PP lies in the convex hull of hard permutation matrices and retains the hard order's two unit assignment budgets. Learned local sequence offsets make this soft-order assignment responsive to the current point cloud.

As derived in W1, consider a Morton base sequence with features h1,h2,h3h_1,h_2,h_3. The illustrative P=[0.70.300.30.40.300.30.7]P=\begin{bmatrix}0.7&0.3&0\\\\0.3&0.4&0.3\\\\0&0.3&0.7\end{bmatrix} shows soft ordering as a linear combination of three hard orders: 0.4 times the original order, 0.3 times a swap of positions 1 and 2, and 0.3 times a swap of positions 2 and 3. Reading its columns, h1h_1 participates with weights 70% and 30% at ranks 1 and 2; h2h_2 has weights 30%, 40%, and 30% across ranks 1, 2, and 3; h3h_3 has weights 30% and 70% at ranks 2 and 3. A hard order places a whole source at an integer rank; this soft order distributes its participation continuously across nearby ranks.

The ordering still has to supply a sequence model: reading the rows of PP, Mamba receives 0.7h1+0.3h20.7h_1+0.3h_2, 0.3h1+0.4h2+0.3h30.3h_1+0.4h_2+0.3h_3, and 0.3h2+0.7h30.3h_2+0.7h_3 at slots 1, 2, and 3, and propagates state along those ordered slots. Even if the largest-weight sources remain h1,h2,h3h_1,h_2,h_3, soft ordering changes their participation across scan ranks and changes Mamba's ordered input. Thus “token mixer” describes the forward computation, while “soft ordering” describes the constrained continuous assignment it implements; both refer to the same operation. The fixed-Gaussian and frozen-offset controls in W1 test whether learning this assignment adds value beyond fixed local mixing.

The corresponding three-slot illustration appears on p. 16 of the revised manuscript.

36vZ-Q2

We provide a visualization of the original scan and the learned GCSO reading targets in the revised manuscript. Figure 4 shows all 128 slots of the first Morton path in layer 12 on a real ModelNet40 test airplane: the left panel connects the original patch centers in the original scan order, and the right panel (GCSO) connects the interpolated targets c(i+δi)c(i+\delta_i) derived from learned sequence-index offsets. At the scale of the whole object, individual local changes are hard to trace. Figure 5 therefore shows three disjoint ten-slot windows from early, middle, and late parts of the same scan: A (13–22), B (76–85), and C (109–118). In each local comparison, gray dots are sampled airplane points; colored circles in the left panel are original patch centers cic_i, and those in the right panel, titled GCSO, are interpolated targets c(i+δi)c(i+\delta_i). The panels share a projection, point-cloud background, and axis limits. Solid lines join consecutive slots within each window, while the faint dashed line in the GCSO panel shows the original local path. In A, a long excursion between original centers is drawn into a more compact group of targets; in B, the lower branch shifts inward while a longer connection remains; in C, the selected zigzag becomes narrower.

The corresponding full-path and local visualizations appear as Figures 4 and 5 on p. 18 of the revised manuscript.

36vZ-Q3

We provide the whole-path context in Figure 4, the three local comparisons in Figure 5, and a quantitative path-length measurement across all 581 OBJ_ONLY test shapes. The selected A, B, and C windows show distinct local changes in the candidate reading targets. As separate quantitative evidence, we measured the final weighted-center paths across all 581 OBJ_ONLY test shapes: the mean ratio of successive-center distance after versus before routing is 0.906 for the first Morton path across 12 layers and 0.903 for the complementary path in layers 4, 8, and 12. Thus the routed paths retain 90.6% and 90.3% of their original lengths, respectively. Together, the visualization and path-length ratios characterize local, position-dependent soft assignment with moderate geometric smoothing.

The visualizations appear as Figures 4 and 5 on p. 18 of the revised manuscript.

Finally, we sincerely thank the reviewer for the time and effort devoted to evaluating our work.

Reviewer_abaFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Geometry-guided Conservative Soft Ordering (GCSO) for point cloud understanding with state space models. The method uses current token features and coordinates to predict local offsets around a geometric base order. These offsets determine the weights used to mix neighboring token features. Sinkhorn normalization, symmetric diagonal completion, and identity blending produce a doubly stochastic routing matrix. The matrix mixes features before the Mamba scan, and its transpose redistributes the scan outputs to the original patch indices. The method also uses complementary geometric paths and continuity and entropy regularization. Experiments report classification improvements on ModelNet40 and ScanObjectNN and evaluate part segmentation and few-shot classification.

Critical strengths
  1. The conservative routing construction is clearly defined and mathematically supported. Equation (6) produces a nonnegative, symmetric, doubly stochastic routing matrix with local support. The diagonal completion ensures the final marginal constraints even when the preceding Sinkhorn iterations have not fully converged. Appendix A also describes back-projection as participation-weighted aggregation rather than an inverse transformation. These properties give the routing operation a clear mathematical basis.
  2. The reported classification improvements are supported by component ablations. Under the same pre-training and fine-tuning protocol, Table 4 improves PB_T50_RS accuracy from 92.37% to 94.20% and ModelNet40 accuracy from 94.7% to 95.7%. Table 5 further examines complementary paths, path fusion, and the two routing regularizers. In Table 1, GCSO improves PB_T50_RS accuracy by 1.45 percentage points over the strongest listed baseline. These results provide evidence for the effectiveness of the overall design.
Critical weaknesses
  1. The distinction from closely related differentiable reordering is not sufficiently explained. DM3D already uses a geometric base sequence, learned sequential offsets, and Gaussian-weighted aggregation within a local index window for differentiable reordering. This is directly related to Equation (5). Section 2.2 cites DM3D but describes its spatial resampling without discussing the relevant reordering component. The experiments also do not report a direct comparison. The additional contribution appears to lie mainly in the conservative routing matrix and its use for routing and back-projection. A clearer method-level comparison and experiments under the same backbone and training settings are needed to establish the value of these additions.
  2. The ablation studies do not fully separate the benefit of learning local offsets from that of local feature mixing. Table 4 compares a geometric hard order, coordinate-driven local offsets, and feature conditioning, providing some support for the overall design. However, the comparison between the hard order and trainable soft routing introduces both soft feature mixing in the scan branch and learned offsets. Their effects are not directly separated. Equation (6) can also be written as P=I−(ρ/κ)LBP=I-(\rho/\kappa)L_B, where LB=Diag⁡(B1)−BL_B=\operatorname{Diag}(B\mathbf{1})-B. The routing therefore also admits an interpretation as an input-dependent local graph diffusion operator. In addition, the continuity term in Equation (9) measures distances between mixed coordinates. It may decrease through coordinate averaging without improving a permutation of the original tokens. A comparison with fixed Gaussian mixing or frozen offsets, using the same backbone and number of scan paths while retaining the other routing operations, would help clarify the additional benefit of learning offsets.
Questions for authors
  1. Please explain the method-level distinction between Equation (5) and Gaussian-based Differentiable Reordering in DM3D. Can the authors compare the reordering in DM3D with GCSO under the same backbone, number of scan paths, and training protocol to clarify the contribution of the conservative construction and back-projection?
  2. What evidence distinguishes the learned routing from an effective local smoothing operator? A comparison with fixed Gaussian mixing or frozen offsets, using the same number of scan paths, would be helpful. Please also clarify whether the reduction in the continuity loss reflects better geometric transitions or mainly contraction of the mixed coordinates.
8 comments
AuthorsOpenReview ↗

Response to abaF-W1 (Part 1/2)

Thank you for identifying the close relationship to DM3D and the missing direct comparison. We agree that Section 2.2 underexplains DM3D's Gaussian-based Differentiable Reordering (GDR) and that our added contribution lies in the conservative routing matrix and its paired use for routing and back-projection. A hard ordering assigns each source to exactly one scan slot and has unit row and column sums. GCSO retains these two constraints in a continuous local assignment; GDR's per-slot normalization retains the row sums but does not guarantee the source-column sums. We clarify this hard-order connection and report same-backbone, same-training controls for the matrix construction and paired return. Both methods begin with a geometric base sequence and use learned sequential offsets for Gaussian-weighted aggregation within a local index window; the learned-offset idea also has earlier precedents in visual sampling, such as Deformable ConvNets (Dai et al., ICCV 2017). Thus both our Eq. (5) and GDR draw on earlier offset-based visual sampling, and Eq. (5) is closely related to GDR's local Gaussian read. In GCSO, however, Eq. (5) provides affinities for a subsequent constrained assignment rather than the final reading matrix.

The distinction begins with what an ordered scan must assign: each scan slot reads from source tokens, and a hard permutation gives every slot exactly one source and uses every source exactly once. GDR normalizes the Gaussian weights separately for each slot, so its reading matrix GG satisfies G1=1G\mathbf1=\mathbf1, but the total weight assigned to a source need not be one. To illustrate what the row-sum constraint alone permits, consider the three-slot matrix G0=[100100001]G_0=\begin{bmatrix}1&0&0\\\\1&0&0\\\\0&0&1\end{bmatrix}: its column totals are (2,0,1)(2,0,1), so the first source is read twice and the second is omitted.

GCSO instead converts the local affinities into a nonnegative matrix PP with P1=P⊤1=1P\mathbf1=P^\top\mathbf1=\mathbf1 through Sinkhorn normalization, symmetric diagonal completion, and an identity blend. A three-slot example is P=[0.70.300.30.40.300.30.7]P=\begin{bmatrix}0.7&0.3&0\\\\0.3&0.4&0.3\\\\0&0.3&0.7\end{bmatrix}. Here the middle scan slot reads 0.3h1+0.4h2+0.3h30.3h_1+0.4h_2+0.3h_3, while source h2h_2 contributes 0.3+0.4+0.3=10.3+0.4+0.3=1 across all slots. Let II retain the three source positions, S12=[010100001]S_{12}=\begin{bmatrix}0&1&0\\\\1&0&0\\\\0&0&1\end{bmatrix} exchange positions 1 and 2, and S23=[100001010]S_{23}=\begin{bmatrix}1&0&0\\\\0&0&1\\\\0&1&0\end{bmatrix} exchange positions 2 and 3. Direct addition gives P=0.4I+0.3S12+0.3S23P=0.4I+0.3S_{12}+0.3S_{23}. Thus it continuously combines three valid hard assignments while preserving both participation budgets; in particular, 1⊤PH=1⊤H\mathbf1^\top PH=\mathbf1^\top H. This is the additional constraint beyond GDR's shared Gaussian reading step. The original Table 4 supports it within our model: adding conservation to the offset-and-feature variant (C→D) raises PB_T50_RS from 93.23% to 93.82% and ModelNet40 from 95.1% to 95.4%. Under the same backbone, the GDR-style per-slot-normalized operator also scores below GCSO on both datasets; this is consistent with extending hard ordering while retaining its two participation budgets.

Further, to test the matrix-construction difference directly, we replaced GCSO's conservative assignment with a row-normalized GDR-style read while retaining the same backbone, number of scan paths (two), training settings, and transpose-return interface. On PB_T50_RS and ModelNet40, the control reaches 93.33% and 95.06%, versus 94.20% and 95.7% for full GCSO. Specifically, Table 4's C→D control adds conservation to the offset-and-feature model; this new comparison instead replaces the full model's conservative matrix with per-slot normalization, so the two controls test the same construction at different stages of the ablation. Both favor conservation. The revised Related Work clarifies DM3D's local sequence-index offsets and the conservation distinction; the expanded routing-control table appears in Appendix B.1.

AuthorsOpenReview ↗

Response to abaF-W1 (Part 2/2)

For hard permutations such as the inverse-serialization returns in [1] and [2], restoring source indices keeps subsequent residual and spatial operations aligned. For a hard permutation, A⊤A^\top exactly reverses the assignment after Mamba processes AΠHA\Pi H, and Π⊤\Pi^\top restores the original patch order. GCSO follows the same scan-and-return layout with PΠHP\Pi H and Π⊤P⊤U\Pi^\top P^\top U, but P⊤P^\top is a weighted redistribution, not an inverse permutation: output at scan slot ii contributes PijUiP_{ij}U_i to source patch jj. Each patch receives total return weight one because PP also has unit column sums. This keeps residual updates and the two complementary Morton paths aligned at the same patches for fusion. DM3D instead keeps the Hilbert slot identities after Gaussian reading. Its row-normalized GDR matrix can be transposed, but the transpose alone does not guarantee unit return weight at each source patch; obtaining the same conservative paired return would require an additional constraint or normalization. Consequently, directly using that transpose with a residual branch indexed by original patches could give source-dependent return weights and alter the balance of the residual update. GCSO reuses its conservative forward matrix for the return, without a separate spatial resampling module or an extra SSM branch. To isolate the paired return in GCSO, we retain the forward scan PΠHP\Pi H, two paths, and Π⊤\Pi^\top restoration, but omit P⊤P^\top. This control reaches 93.89% on PB_T50_RS and 95.38% on ModelNet40, compared with 94.20% and 95.70% for full GCSO: 0.31 and 0.32 percentage points in favor of the paired return.

References:

[1] Y. Chen, R. Sun, W. Li, H. Mai, S. Chen, Z. Li, Z. Cheng, and T. Zhang, “BeyondMix: Leveraging Structural Priors and Long-Range Dependencies for Domain-Invariant LiDAR Segmentation,” in Advances in Neural Information Processing Systems 38 (NeurIPS), 2025. Paper.

[2] A. Bahri, M. Yazdanpanah, M. Noori, S. Dastani, M. Cheraghalikhani, G. A. V. Hakim, D. Osowiechi, F. Beizaee, I. Ben Ayed, and C. Desrosiers, “Spectral Informed Mamba for Robust Point Cloud Processing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11799–11809, 2025. Paper.

The related-work clarification is on p. 3, the three-slot illustration on p. 16, and the back-projection discussion and routing controls on pp. 17–18 of the revised manuscript.

AuthorsOpenReview ↗

Response to abaF-W2

We appreciate the helpful comments on the ablations and continuity term. We added fixed-Gaussian and frozen-offset controls, accept the input-dependent local graph diffusion operator interpretation, and measure how much coordinate averaging contracts the path. To separate learning local offsets from local feature mixing, we extended the original Table 4 on its two datasets, PB_T50_RS and ModelNet40, while keeping the Mamba3D backbone and two scan paths. Table 4 already separates coordinate offsets, feature conditioning, conservation, and two-path fusion (C: 93.23/95.1; D: 93.82/95.4; E: 94.20/95.7). The new controls specify which operation changes: fixed centered Gaussian supplies local mixing without learned offsets; freezing the learned-offset predictor prevents offset adaptation during fine-tuning; DM3D-style row-normalized GDR Gaussian replaces the conservative assignment while keeping learned offsets and transpose return. The full GCSO row uses the original Table 4 E result:

Routing variantPB_T50_RS OA (%)ModelNet40 OA (%)
Fixed centered Gaussian93.2795.30
Frozen learned-offset predictor93.5895.02
Row-normalized GDR-style Gaussian93.3395.06
Full GCSO (original Table 4, E)94.2095.70

The fixed Gaussian mixing control supplies local feature mixing without learned offsets, while the frozen offsets control keeps the offset predictor fixed during fine-tuning; both retain the two-path routing interface and fall below full GCSO on both datasets. This separates the benefit of adapting offsets from the presence of soft mixing itself. The GDR-style control keeps learned Gaussian reading and the same transpose-return interface, replacing the conservative matrix with per-slot normalization. Its lower accuracy on both datasets agrees with Table 4's C→D improvement when conservation is introduced. The resulting conservative PP also admits the input-dependent local graph diffusion operator interpretation in Equation (6): for symmetric nonnegative local BB, P=I−(ρ/κ)LBP=I-(\rho/\kappa)L_B, where LB=Diag⁡(B1)−BL_B=\operatorname{Diag}(B\mathbf1)-B.

Beyond these operator controls, we examined whether the reduction in the continuity loss could arise through coordinate averaging. Because the continuity term C\mathcal C in Equation (9) measures distances between mixed coordinates, namely weighted patch centers, averaging nearby coordinates can shorten the path. We therefore optimize C\mathcal C together with a row-entropy term H\mathcal H; each routed path contributes C+ηH\mathcal C+\eta\mathcal H, with η>0\eta>0. The entropy term is

H=−1Glog⁡K∑i=1G∑j=1GPijlog⁡max⁡(Pij,ε),K=max⁡(2,min⁡(G,2w+1)).\mathcal H=-\frac{1}{G\log K}\sum_{i=1}^{G}\sum_{j=1}^{G}P_{ij}\log\max(P_{ij},\varepsilon),\qquad K=\max(2,\min(G,2w+1)).

Here PijP_{ij} is the weight with which scan slot ii reads source jj, GG is the number of slots, and ww is the local window radius. Thus KK is the number of local candidate positions used to normalize the entropy, capped by GG and kept at least 2 so that log⁡K\log K is defined; η\eta, rather than KK, controls the entropy term's weight in the loss. In our experiments w=4w=4 and G=128G=128, giving K=9K=9; ε\varepsilon only prevents taking the logarithm of zero. A concentrated one-source read has entropy 0, whereas an interior slot reading its nine candidates equally has normalized entropy 1. Because we minimize H\mathcal H with a positive coefficient, nearly uniform reads are penalized while selective local mixing remains possible. In the original regularizer ablation, moving from C\mathcal C alone to C+ηH\mathcal C+\eta\mathcal H raises PB_T50_RS from 93.68% to 94.20% and ModelNet40 from 95.3% to 95.7%.

We then measured how much the trained coordinates actually contract on all 581 OBJ_ONLY test shapes. For each shape and layer, we divide the mean distance between successive weighted centers c~i=∑jPijcj\tilde c_i=\sum_jP_{ij}c_j by that between successive original centers cic_i, then average these ratios. A ratio near zero would indicate that the routed centers had nearly collapsed to one location; a ratio near one indicates little overall shortening. The mean ratio is 0.906 along the Morton path used in all 12 layers and 0.903 along the complementary path used in layers 4, 8, and 12. Thus the intended smoothing shortens these paths by about 9.4% and 9.7%, still well above the zero associated with complete collapse. The normalized row entropies average 0.160/0.166. For comparison, a uniform read over nine local candidates at an interior slot has normalized entropy 1. Moreover, each slot retains mean weight 0.922/0.919 on its original source. Together, the path-length and weight measurements show local smoothing while preserving most source weight and geometric path length.

The method clarification is on p. 6, with the regularizer comparison on p. 17 and the routing controls in Table 6 on p. 18 of the revised manuscript.

Reviewer_EEvAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose GCSO, a point cloud understanding method based on state space models. The method consists of two core parts. The first is Geometry-guided Conservative Soft Ordering (GCSO). This ordering method takes the Morton order as a geometric prior and uses the current features and coordinates to predict a local offset for each scan position. Continuous weights are generated from the relative distances between the shifted position and neighboring tokens, and then processed by Sinkhorn normalization, symmetric completion, and identity blending to construct a nonnegative, symmetric, doubly stochastic routing matrix, so that the ordering can be optimized end to end. The second is complementary path fusion: this module introduces two Morton paths (xyz and yzx) in some layers and fuses them with weights based on path geometric continuity. In addition, continuity and entropy regularization terms are added during training. Finally, the authors conduct experiments on ModelNet40, ScanObjectNN, ShapeNetPart, and few-shot classification tasks.

Critical strengths
  1. The method proposes a Geometry-guided Conservative Soft Ordering (GCSO) module. This module replaces discrete hard ordering with continuous soft assignment and predicts local offsets from features and coordinates, so that the scan order can be optimized end to end through the task loss. This alleviates the mismatch between fixed geometric orders and downstream tasks.

  2. The authors design a conservative routing matrix that prevents tokens from being excessively reused or omitted, while a learnable coefficient ρ\rho controls the degree of deviation from the geometric base order.

Critical weaknesses
  1. The novelty of this manuscript is limited. Differentiable sorting methods such as Sinkhorn-based soft sorting and Gumbel-Sinkhorn are already well established, and DM3D has already proposed offset-based differentiable scanning. Moreover, the number of parameters and FLOPs of the model are exactly the same as those of Mamba3D, so the overall method looks more like an incremental improvement over Mamba3D.

  2. The experiments in this manuscript are insufficient. Evaluation is conducted only on small object-level datasets such as ModelNet40 and ScanObjectNN, where each sample contains only 64 patches, so the impact of ordering is inherently limited. Experiments on scene-level datasets such as ScanNet and S3DIS are missing, although long-sequence scenarios would better demonstrate the effect of ordering.

  3. The compared methods are insufficient. The paper cites ordering-related methods such as DM3D, SI-Mamba, PointMamba++, and PCM, but does not compare with them in Table 1. The training-from-scratch results in Table 6 also omit strong baselines such as Mamba3D and StruMamba3D, so the 4.48-point improvement on PB_T50_RS lacks a credible reference.

  4. The analysis of experimental reliability and efficiency is insufficient. Results on ScanObjectNN have large variance, yet the mean and variance over multiple runs are not reported. Part segmentation performance is on par with StruMamba3D, and the few-shot improvements fall within the standard deviation. In addition, after adding the offset predictor, 7 Sinkhorn iterations, and two paths, the number of parameters and FLOPs remain the same as those of Mamba3D, and inference latency is not reported.

  5. There are problems with method details and writing. Key hyperparameters such as the offset magnitude aa, window radius ww, σ\sigma, τ\tau, λ\lambda, η\eta, and the initial value of ρ\rho are not provided, and sensitivity analysis is missing. The paragraphs on shape classification in Sections 4.2 and 4.3 are almost entirely duplicated.

Questions for authors

The novelty of the method in this manuscript is limited, whether the proposed soft ordering actually achieves reordering is not sufficiently validated, its advantage over state-of-the-art methods is not convincing, and the experiments are insufficient. I suggest that the authors provide explanations for the weaknesses I have raised.

7 comments
AuthorsOpenReview ↗

Response to EEVA-W1 (Part 1/2)

Thank you for your valuable comments. We agree that the constituent sorting and offset mechanisms are established; we clarify below that our contribution is a conservative soft assignment to ordered scan slots with paired routing and back-projection, a Sinkhorn-inspired adaptation to point-cloud scanning that relaxes hard permutations into trainable local assignments while retaining unit row and column sums and, unlike the per-slot normalization of DM3D's Gaussian-based Differentiable Reordering (GDR), guarantees a unit participation budget for each source token, and we report its measured cost.

A point-cloud SSM first needs to place unordered patch tokens into ordered scan slots. A hard order is a permutation matrix A∈0,1n×nA\in\\{0,1\\}^{n\times n} with A1=A⊤1=1A\mathbf1=A^\top\mathbf1=\mathbf1: every slot reads one token and every token participates once. We wanted to make this assignment trainable while retaining both participation budgets. Inspired by learned offsets and differentiable sampling in vision, exemplified by Deformable ConvNets (Dai et al., ICCV 2017), we predict local sequence-index offsets around a geometric Morton order and convert them to continuous Gaussian reading weights. Established work on differentiable sorting, offset-based scanning, Gaussian kernels, and Sinkhorn normalization informs our approach; our contribution is the conservative constraint and paired scan/back-projection in this setting.

To carry the one-to-one constraints of hard-order matrix AA into a trainable continuous assignment, we retain A1=A⊤1=1A\mathbf1=A^\top\mathbf1=\mathbf1, relax the binary entries to nonnegative fractional weights, and additionally require locality and symmetry. This yields P≥0P\geq0 and P1=P⊤1=1P\mathbf1=P^\top\mathbf1=\mathbf1. In a three-slot example, II leaves the order unchanged; S12=[010100001]S_{12}=\begin{bmatrix}0&1&0\\\\1&0&0\\\\0&0&1\end{bmatrix} swaps the tokens in slots 1 and 2; and S23=[100001010]S_{23}=\begin{bmatrix}1&0&0\\\\0&0&1\\\\0&1&0\end{bmatrix} swaps slots 2 and 3. All three are hard-order matrices of the form AA. Their weighted combination is

P=[0.70.300.30.40.300.30.7]=0.4I+0.3S12+0.3S23.P=\begin{bmatrix}0.7&0.3&0\\\\0.3&0.4&0.3\\\\0&0.3&0.7\end{bmatrix}=0.4I+0.3S_{12}+0.3S_{23}.

This example shows that soft assignment PP retains the row and column budgets of hard order AA while admitting continuous values. More generally, the Birkhoff–von Neumann theorem states that every doubly stochastic PP is a convex combination of permutation matrices AkA_k: P=∑kαkAkP=\sum_k\alpha_k A_k, where αk≥0\alpha_k\geq0 and ∑kαk=1\sum_k\alpha_k=1. Here H=[h1,h2,h3]⊤H=[h_1,h_2,h_3]^\top denotes the source-token feature matrix in geometric base order for the three-slot example. Thus the linear routed input PH=∑kαkAkHPH=\sum_k\alpha_kA_kH is a weighted combination of valid hard-ordered inputs.

This construction makes the comparison with DM3D more precise. DM3D's Gaussian-based Differentiable Reordering (GDR) already shares the geometric base sequence, learned offsets, and local Gaussian read. GDR's per-slot normalization enforces the row budget, while our construction additionally imposes a source-column budget. To see the difference in allowed assignments, the idealized row-normalized matrix G0=[100100001]G_0=\begin{bmatrix}1&0&0\\\\1&0&0\\\\0&0&1\end{bmatrix} has row sums one but column sums (2,0,1)(2,0,1): one source is read twice while another has zero weight. This example illustrates why we additionally retain the column-marginal condition of hard-order matrix AA, giving each source a unit participation budget before state propagation. Unlike GDR's per-slot normalization, our construction in Fig. 1 of the original manuscript makes the assignment a conservative doubly stochastic matrix: both each slot's reading total and each source token's participation total remain one. GCSO thus retains the row and column marginal constraints of hard ordering while extending its binary assignment to differentiable continuous weights and adaptive local interaction. For any feature matrix HH, the routing step also satisfies 1⊤PH=1⊤H\mathbf1^\top PH=\mathbf1^\top H. The original Table 4 provides within-model evidence: adding conservation to the offset-and-feature variant (C→D) raises PB_T50_RS from 93.23% to 93.82% and ModelNet40 from 95.1% to 95.4%.

AuthorsOpenReview ↗

Response to EEVA-W1 (Part 2/2)

GCSO also differs in how scan outputs return to patches. It uses the same assignment for the forward scan PΠHP\Pi H and the patch-aligned return Π⊤P⊤U\Pi^\top P^\top U. The return restores the original patch indices for residual processing and fusion of the complementary Morton paths. Since PP is doubly stochastic, P⊤P^\top preserves unit return weight for every source patch without further marginal normalization. DM3D's Gaussian-based Differentiable Reordering (GDR) updates features at Hilbert-ordered slots and does not pair its Gaussian read with a transpose return to original patch indices. Although a row-normalized GDR read can be transposed, its source-column sums need not equal one. If such a transpose were directly combined with a residual branch indexed by original patches, source-dependent return weights could change the balance of the residual update; an additional constraint or normalization would be needed for a conservative return. DM3D also combines three SSM paths through Tri-Path Fusion (TPF), adjusts state updates at geometric jumps through Continuity-Aware State Update (CASU), and reads displaced 3D positions through Deformable Spatial Resampling (DSR). GCSO instead uses two complementary Morton scans with shared SSM parameters and constrains continuity in the input routing. Our claim is the conservative local soft assignment and its paired use for scan and return.

We test the two distinctions with the same Mamba3D backbone, two scan paths, and training protocol. The GDR-style control retains learned offsets, a local Gaussian read, and the transpose-return interface, but replaces GCSO's conservative matrix with per-slot row normalization. The no-return control retains GCSO's forward PΠHP\Pi H, both scan paths, and Π⊤\Pi^\top restoration of the original patch order, but omits the paired P⊤P^\top redistribution. Full GCSO retains both conservation and paired return:

Routing variantPB_T50_RS OA (%)ModelNet40 OA (%)
Row-normalized GDR-style Gaussian93.3395.06
Without paired P⊤P^\top return; retain Π⊤\Pi^\top93.8995.38
Full GCSO (original Table 4, E)94.2095.70

Full GCSO achieves higher scores than the row-normalized control on both datasets, consistent with the original Table 4 C→D ablation, which adds conservation within an earlier offset-and-feature variant. The paired P⊤P^\top return also contributes 0.31 and 0.32 percentage points on PB_T50_RS and ModelNet40, respectively, relative to the control without it. These controls isolate the conservative construction and paired return within our architecture.

In addition, under the same profiling protocol, Mamba3D has 16.93M parameters, while GCSO has 16.97M parameters, indicating that their parameter counts are nearly identical. This is because GCSO only introduces a lightweight, trainable Geometry-Guided Conservative Soft Ordering module before Mamba3D, resulting in a negligible increase in model parameters and comparable FLOPs. We have updated the reported computational costs accordingly. The corresponding revisions appear on pp. 3, 7, and 16–18 of the revised manuscript. We sincerely thank the reviewer again for drawing our attention to this point.

AuthorsOpenReview ↗

Response to EEVA-W2 and W3

EEVA-W2

Thank you for suggesting scene-level evaluation. Given the limited time available, we selected S3DIS from the suggested scene-level datasets and evaluated semantic segmentation on Area 5, one of the proposed scene-level datasets. GCSO obtains 71.0% mAcc and 63.4% mIoU. The comparison is shown below.

MethodmAcc (%)mIoU (%)
PointNet++ (Qi et al., 2017b)67.153.5
Point-BERT (Yu et al., 2022)70.360.8
Point-MAE (Pang et al., 2022)69.960.8
PointGPT-L (Chen et al., 2023)70.662.2
Point-PQAE (Zhang et al., 2025b)70.661.4
PCM (Zhang et al., 2025a)–63.4
Point-MaDi (Xiao et al., 2025)71.061.2
PointGST-M (Liang et al., 2025)68.458.6
RI-MAE (Su et al., 2025)–60.3
GCSO (ours)71.063.4

GCSO matches the highest listed mAcc (Point-MaDi, 71.0%) and mIoU (PCM, 63.4%) simultaneously. The two leading values are reported by different comparison methods, while GCSO attains both on S3DIS Area 5, demonstrating competitive scene-level semantic segmentation performance across both metrics. We plan to extend this evaluation to ScanNet in future work. We thank the reviewer again for this valuable suggestion.

The S3DIS results and comparison are in Table 8 on p. 19 of the revised manuscript.

EEVA-W3

Thank you for identifying the missing ordering-related baselines and the training-from-scratch comparison. We present the additional fine-tuning baselines and a direct Mamba3D training-from-scratch comparison below.

(1). Finetune Experiments

We now provide the DM3D and PointMamba++ results alongside the methods already in Table 1 in the revised manuscript. The reported results below use self-supervised pre-training followed by classification fine-tuning; ModelNet40 numbers are without voting. SI-Mamba's Surface-Aware Spectral Traversing (SAST) is already listed as SAST in our original Table 1. In the revised manuscript, Related Work names SI-Mamba and Table 1 lists its SAST result alongside the added DM3D and PointMamba++ rows.

MethodOBJ_BGOBJ_ONLYPB_T50_RSModelNet40
SAST (SI-Mamba; already in Table 1)94.3291.9189.1093.4
DM3D94.7193.8393.3094.9
PointMamba++93.8092.9487.9693.7
GCSO95.5294.8494.2095.7

These reported comparisons place GCSO above the listed methods on all four classification settings. On PB_T50_RS, GCSO exceeds DM3D, the strongest baseline in this table, by 0.90 percentage points. These results concern self-supervised pre-training followed by classification fine-tuning.

(2). Training-from-Scratch Experiments

To assess the separate supervised training-from-scratch claim, we use our Mamba3D run as a direct reference for GCSO without voting. We place the published PCM results in the training-from-scratch comparison, alongside our Mamba3D and GCSO runs:

Training-from-scratch methodOBJ_BG OA (%)OBJ_ONLY OA (%)PB_T50_RS OA (%)ModelNet40 OA (%)
PCM (published; 1,024 input points)——88.1 ± 0.393.4 ± 0.2
Mamba3D (our run)93.2992.6090.8793.35
GCSO (our run)93.6393.6392.7894.0

Our direct PB_T50_RS difference against Mamba3D is 1.91 points. The revised supervised comparison uses this direct reference in place of the previous 4.48-point strongest-listed-baseline claim. StruMamba3D is included in the pre-trained comparison, while its code is not yet publicly available for a matched from-scratch reproduction. GCSO also has the highest listed accuracy in all four columns of the scratch-trained comparison, including 92.78% on PB_T50_RS and 94.0% on ModelNet40.

The added pre-trained baselines are in Table 1 on p. 7, and the direct scratch-trained Mamba3D comparison is in Table 7 on p. 19.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.