acceptodds
Under review as a conference paper at ICLR 2027

Training-Free Spatially Grounded 3D Object Encoding

Abstract

We present X3DEnc, a training-free framework for spatially grounded 3D objects encoding. Our central insight is that such a 3D object is characterized not only by its geometry, but also by its appearance and spatial pose. Different downstream tasks may require different relative emphasis on these components. X3DEnc decomposes a 3D object into normalized geometry and color fields within the unit ball and a pose transformation. It then lifts the finite-dimensional pose vector to a harmonic field and projects geometry, color, and pose onto a shared 3D Zernike basis. This common spectral coordinate system supports separate or joint encoding. The geometry-color-pose joint encoding allow emphasis-adjustable encoding. X3DEnc accommodates voxel, signed-tetrahedral, and surface-triangle representations, covering volumetric solids, watertight meshes, hollow objects, and open surfaces. Theoretical analysis and various experiments demonstrate that X3DEnc can redirect similarity toward geometry, color, or pose while preserving a single encoding formulation. Rather than competing with neural encoders, X3DEnc provides an alternative when training data are limited or when interpretable, jointly controllable multi-component encoding is required.

Primary area:
learning theory
Keywords:
3D Object Encoding, Train-Free, Geometry-Color-Pose Encoding, Emphasis-Adjustable Encoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LRWqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents X3DEnc, a training-free framework for encoding spatially grounded 3D objects. The method decomposes each object into normalized geometry, color, and spatial pose, lifts the pose vector into a harmonic field, and projects all three components onto a shared 3D Zernlike basis, enabling both separate and jointly controllable encodings without retraining. It supports voxel, signed-tetrahedral, and surface-triangle representations and produces fixed-length descriptors through frequency propagation and structured projection. Experiments on attribute-controlled retrieval, inter-object spatial relation classification, cross-domain generalization, and object classification demonstrate its potential as an interpretable and dataset-independent complement to learned 3D encoders.

Critical strengths
  1. Unified spectral encoding of geometry, appearance, and pose. The paper maps normalized geometry, color fields, and a harmonically lifted pose vector into a shared 3D Zernlike coordinate system. This provides a technically coherent way to represent intrinsic object properties and extrinsic spatial information within the same descriptor space.

  2. Explicit and controllable multi-component feature fusion. X3DEnc introduces independent geometry, color and pose gains, together with energy normalization and coefficient-level magnitude/phase fusion. This design allows the same descriptor construction to be redirected toward different attributes at inference time without retraining an additional fusion network.

  3. Support for heterogeneous 3D representations within one formulation. The method provides voxel-, signed-tetrahedral-, and surface-triangle-based integrations, covering volumetric solids, watertight meshes, hollow objects, and open surfaces. The three formulations also expose clear trade-offs between geometric measure, input requirements, approximation fidelity, and computational complexity.

Critical weaknesses
  1. Essential ablations and statistical analyses are missing. The paper does not isolate the contributions of the pose lifting, frequency propagation, nonlinear fusion, structured projection or representation choice, and it reports neither repeated runs nor variance; the mixed-emphasis results are also strongly color-dominated.

  2. The paper lacks empirical efficiency evaluation. It provides only asymptotic complexity and qualitative efficiency discussions, without reporting preprocessing time, encoding latency, memory usage, throughput, scalability, or runtime comparisons with competing descriptors.

  3. The main comparisons use fundamentally unfair inputs. In Table I, the baselines must encoder a merged object-plus-large-plane point cloud, while X3DEnc receives isolated canonical geometry, color and explicit pose; in the relation experiment, X3DEnc also receives the two object instances separately while the baselines receive only their merged point cloud. The reported gains may therefore come from privileged segmentation and metadata rather than the proposed descriptor.

Questions for authors

The current experiments focus on object retrieval and object-level or relation-level classification, while no semantic or instance segmentation results are reported. Since X3DEnc produces a global object descriptor and assumes that the input object has already been isolated from the background, how can it be adapted to generate point or voxel-level segmentation predictions in cluttered real-world scenes? If the authors can address all of the concerns raised above, I would be willing to reconsider my assessment and potentially my rating.

5 comments
AuthorsOpenReview ↗

Author Feedback 1

We thank the reviewer for providing such a critical and insightful comments. We provide detailed feedback regarding your concerns.

Q1: Essential ablations and statistical analyses are missing. The paper does not isolate the contributions of the pose lifting, frequency propagation, nonlinear fusion, structured projection or representation choice, and it reports neither repeated runs nor variance; the mixed-emphasis results are also strongly color-dominated.

A1: Thanks for pointing this out. Part of the questions have been partially addressed in the paper, we provide one-by-one feedback.

  1. Representation Choice. Table 1 reports voxel, tetrahedra and surface on the geometry-color-pose emphasis based retrieval task, and they agree closely. Table 3 and Appendix A.8 give the cost and applicability trade-off comparison.

  2. Structured Projection. In Sec. A.9 of Appendix, we have discussed the motivation and necessity of the structured projection, accompanied by Fig. 6 showing cluster survive from 4096-d to 512-d projection. Per your suggestion, we run an ablation to test w/ and w/o structured projection on geometry-color-pose emphasis retrieval task, the result is shown in the table below (mAP ↑\uparrow): | Method | Geometry Emphasis | Color Emphasis | Pose Emphasis | |--------|----------|-------|------| | X3DEnc (w/o Structured Projection, direct 512 encode length from 3D Zernike bases) | 0.177 | 0.301 | 0.506 | | X3DEnc (w/ Structured Projection, 4096 →\rightarrow 512, surface representation) | 0.324 | 0.956 | 1.000 |

From this table, we can observe that without structured projection has observed performance drop sharply.

  1. Pose Lift: We provided the proof of the pose lift invertibility in A.7 of Appendix; Pose-emphasis retrieval mAP is 1.000 in Table 1. Per your suggestion, we tested an alternative pose encoding in Transformer architecture (Attention is All You Need) on geometry-color-pose emphasis retrieval task, the result is shown in the table below (mAP ↑\uparrow): | Method | Geometry Emphasis | Color Emphasis | Pose Emphasis | |--------|-------------------|----------------|---------------| | SinCos based Pose Lift | 0.203 | 0.472 | 0.370 | | X3DEnc based Pose Lift | 0.324 | 0.956 | 1.000 |

We can learn from table that using Sin/Cos pose encoding leads performance drop. It is worth noting that Sin/Cos based pose lift loses invertibility.

  1. Fusion Controllability: The other two reviewers asked similar question (Q4 in R#449f, Q2 in R#g75G). We ablated two fusion strategies: encode each component separately, then use elementwise add concatenation to fuse them together. The experiment result on geometry-color-pose emphasis retrieval task is shown for our feedback in A4 for R#449f, from which we have observed sharp performance drop when using Add/Concate fusing strategy. If you have any other fusing strategy that you want us to test, please let us know.

  2. FreqProp: Thanks for pointing it out. We inherit the idea of FreqProp from XShapeEnc (arxiv 2604.07522) but extend it to 3D space. Per your suggestion, we run experiment w/ and w/o FreqProp on inter-object topological relation classification tasks (Accuracy Rate ↑\uparrow). | Method | Linear Probe | 3 layer MLP Probe | |--------|----------|------| | X3DEnc (surface, w/o FreqProp) | 0.78 | 0.81 | | X3DEnc (surface, w/ FreqProp) | 0.84 | 0.86 |

From this table we can observe that removing FreqProb reduces the classification accuracy a lot.

  1. Variance: X3DEnc encoding process is deterministic, randomness enters only through the projection seed. All the comparing baselines in Table 1 (geometry-color-pose emphasis retrieval task) are deterministic without variance incurred. We thus report the variance of X3DEnc by differing the structured projection seed 100 times independently. The result is shown in the table below: | Method | Geometry Emphasis | Color Emphasis | Pose Emphasis | |--------|----------|-------|------| | X3DEnc~(surface) | 0.324 ±\pm 0.002 | 0.956 ±\pm 0.001 | 1.000 ±\pm 0.003 |

From this table, we can observe that variance is small, which shows X3DEnc is relatively invariant to the nuisance randomness of the structured projection.

AuthorsOpenReview ↗

Author Feedback 2

  1. Variance: continued result: In inter-object topological relation classification in Table 2, we independently train all methods 5 times with seeds from 0 to 4, same training recipe. The new results with variance is reported in the table below (Accuracy Rate, ↑\uparrow):

    MethodLinear ProbeMLP Probe
    SPH0.81 $\pm$0.0020.82$\pm$0.003
    D20.71$\pm$0.0100.76$\pm$0.004
    ESF0.79$\pm$0.0030.83$\pm$0.002
    SH0.59$\pm$0.0040.60$\pm$0.003
    LFD0.80$\pm$0.0020.80$\pm$0.005
    PlainZernike0.72$\pm$0.0010.77$\pm$0.003
    PointNet0.65$\pm$0.0020.68$\pm$0.002
    PointNet++0.64$\pm$0.0050.64$\pm$0.004
    DGCNN0.83$\pm$0.0030.88$\pm$0.004
    Mamba3D-lite0.81$\pm$0.0050.80$\pm$0.004
    X3DEnc (surface)0.84$\pm$0.0030.86$\pm$0.002

    From this table, we can observe that the variance is relatively small across all methods (maximum variance is 1% for D2 baseline).

    1. Color Dominance: We agree this is visible in Table 4 and offer a tentative explanation. The benchmark assigns each object one of five uniform colors, so color has zero within-class spread, whereas geometry is a 40-way (ModelNet40 has 40 classes) attribute with large intra-class variation. Fusing the two at matched energy tends to yield a lexicographic ranking: same-color objects first, ordered by geometry within.

Q2: The paper lacks empirical efficiency evaluation. It provides only asymptotic complexity and qualitative efficiency discussions, without reporting preprocessing time, encoding latency, memory usage, throughput, scalability, or runtime comparisons with competing descriptors.

A2: Thanks for point out. We added the experiment and report both FLOPS and inference time here.

  1. Experimental setting 1: 3D object: ModelNet40/airplane/test/airplane_0627.off, pose vector is 13-value vector (3D position, scale, rotation). The FLOPS is reported in the table below: | Baseline | FLOPS | |:---|:---:| | SH | 0.1 M | | D2 | 0.2 M | | SPH | 1.7 M | | ESF | 2.7 M | | LFD | 8.7 M | | PlainZernike | 20.7 M | | X3DEnc | 20.7 M | | PointNet++ | 1,763.4 M | | PointMAE | 4,866.3 M | | Mamba3D | 8,475.4 M | | Sonata | 18,648.3 M | | DGCNN | 99,743.6 M | | Uni3D | 124,315.5 M |

Conclusion: We can observe that X3DEnc consumes much smaller FLOPS than neural network based baselines. Comparing with other training-free baselines, X3DEnc consumes relatively more FLOPS, we find 40% of the FLOPS is used for 3D Zernike basis construction, which is just one-time cost when encoding multiple spatially grounded 3D objects because the basis can be constructed beforehand.

  1. Experimental setting 2: we use the same 3D object and spatial position and compare the inference encoding time on 1 NVIDIA A100 80GB PCIe between X3DEnc and other neural network based baselines, 50 times indepedent tests. The averaging inference encoding time comparison is given in the table below: | Methods | Inference Time (ms) | |:---|:---:| | X3DEnc | 5.532 | | PointMAE | 12.206 | | Mamba3D | 29.437 | | DGCNN | 30.942 | | Uni3D | 53.428 | | Sonata | 76.634 | | PointNet++ | 141.812 |

Conclusion: From this table, we can see that X3DEnc takes much less inference time on GPUs than neural network based methods, which shows the effectiveness of X3DEnc.

Q3: The main comparisons use fundamentally unfair inputs. In Table I, the baselines must encoder a merged object-plus-large-plane point cloud, while X3DEnc receives isolated canonical geometry, color and explicit pose; in the relation experiment, X3DEnc also receives the two object instances separately while the baselines receive only their merged point cloud.

A3: Thanks for pointing this out. We kindly suggest you refer to Reviewer #449f Q4/A4 for our detailed explaination and experimental result as the reviewer raised the same question. Summary: we encode object-plus-large-plane point cloud in order to fit for the baseline 3D-data-only input requirement. Per your suggestion, we split the pose and 3D object encoding as we did for X3DEnc, new experimental results show improved performance than object-plus-large-plane encoding, but the performance is still inferior to X3DEnc.

AuthorsOpenReview ↗

Author Feedback 3

Q4: The current experiments focus on object retrieval and object-level or relation-level classification, while no semantic or instance segmentation results are reported. Since X3DEnc produces a global object descriptor and assumes that the input object has already been isolated from the background, how can it be adapted to generate point or voxel-level segmentation predictions in cluttered real-world scenes?

A4: In the main paper, we just reported object-level encoding by X3DEnc. However, X3DEnc can be extended to provide per-point encoding.

  1. How to achieve it: we take a ball of radius r around each point, translate it to the origin, scale it to the unit ball, and apply the unchanged pipeline: local occupancy and RGB fields projected onto the 3D Zernike basis, reduced to yaw-invariant block norms so vertical structure is retained, and energy-matched across geometry and color. Three radii from 0.1 to 0.6 m give local surface type plus part-level contex) provide surface-level and part-level context. The result encoding is per-point descriptor computed once per scene, with no object isolation required; We remove per-point position in the encoding.

  2. Experimental Setup: We test whether a training-free X3DEnc per-point descriptor adds information that learned segmentation features lack. We use ScanNet v2 (1,513 indoor scans, 20 classes), training on the 1,201 train scenes and reporting mIoU on the 312 validation scenes. We train PointNet++ (xyz + RGB input) and DGCNN (xyz input) from scratch with the unchanged public recipes, and compare against the same networks trained from scratch with the X3DEnc geometry–color descriptor appended to each point’s input channels. Results are reported as mean ± std over three seeds.

  3. Experimental Result:

    • The result on PointNet++ is shown in the Table below: | Method | Input | Epochs | Seeds | ScanNet v2 val mIoU (%) | |---|---|---|---|---| | PointNet++ (single-scale grouping), mmdetection3d report | xyz + rgb | 200 | 1 | 54.44 | | PointNet++ (single-scale grouping) + X3DEnc | xyz + rgb + X3DEnc Per-Point Enc | 200 | 3 | 58.90 ±\pm 0.13 |

    • The result on DGCNN is shown in the Table below: | Method | Input | Epochs | Seeds | ScanNet v2 val mIoU (%) | |---|---|---|---|---| | DGCNN, antao97/dgcnn.pytorch report | xyz + normalized xyz | 200 | 1 | 49.6 | | DGCNN + X3DEnc | xyz + normalized xyz + X3DEnc Per-Point Enc | 200 | 3 | 54.01 ±\pm 0.17 |

Conclusion: we both observed performance gain by PointNet++ and DGCNN, an average of 4 point gain. it thus shows the feasibility to extend X3DEnc to solve segmentation task in the real-world dataset. We are working to extend X3DEnc to more various tasks to shows its applicability.

We hope we have addressed your concerns. If you have any further question, feel free to let us know, we are more than happy to provide more feedback.

Reviewer_g75GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents X3DEnc, a training-free framework for encoding 3D object geometry, color, and spatial pose using a shared 3D Zernike basis. It supports voxel, signed-tetrahedral, and surface-triangle representations, and introduces an emphasis-adjustable joint encoding mechanism. The authors evaluate the framework through attribute-conditioned retrieval, inter-object spatial relation classification, cross-domain generalization, and 3D object classification.

Critical strengths
  1. Unified training-free representation. The paper presents an interpretable framework for encoding geometry, color, and spatial pose without training a neural backbone. This provides a potentially useful alternative when training data are limited.

  2. Flexible geometric representations. Supporting voxel, signed-tetrahedral, and surface-triangle representations broadens the applicability of the method. The experiments also show results on attribute-conditioned retrieval and spatial relation classification.

Critical weaknesses
  1. Limited novelty over XShapeEnc[1]. The core ideas of harmonic pose encoding, frequency propagation, and controllable joint encoding were already introduced in XShapeEnc. While X3DEnc extends the framework to 3D and incorporates color and multiple geometric representations, the technical significance of these extensions is not sufficiently established. In particular, the contribution of the proposed joint fusion over simpler alternatives remains unclear.

  2. Potentially unfair baseline comparisons. In Section 4.3, the baselines process object-plus-context point clouds, whereas X3DEnc separately encodes object geometry and pose. Similarly, in Section 4.4, X3DEnc processes two objects independently while the baselines receive merged point clouds. These differences make it difficult to attribute the improvements solely to the proposed encoding method and weaken the experimental conclusions.

[1] He, Y. (2026). Training-free Spatially Grounded Geometric Shape Encoding. arXiv:2604.07522.

Questions for authors

Q1. What are the key technical contributions beyond XShapeEnc? Can the authors provide ablations to isolate the benefits of the newly introduced components?

Q2. Why are the baseline methods evaluated on object-plus-context or merged point clouds, while X3DEnc uses separately encoded objects? Can the authors provide comparisons under equivalent input conditions?

Q3. How does the proposed joint encoding compare against simpler fusion strategies, such as weighted concatenation of geometry, color, and pose descriptors? Can the authors demonstrate effective simultaneous control over multiple attributes?

4 comments
AuthorsOpenReview ↗

Author Feedback 1

We sincerely thank the reviewer spending time providing the insightful and constructive comments.

Q1: Limited novelty over XShapeEnc.

A1: We thank the reviewer for identifying this close lineage. We agree that harmonic pose lifting, FreqProp, and emphasis-adjustable geometry-color-pose joint encoding build on top of XShapeEnc. In the paper, we acknowledged it: Sec. 3.4: "Following the principle in XShapeEnc"; Sec. 3.6: "Motivated by XShapeEnc". However, X3DEnc isn't "just replacing 2D basis with a 3D one". Extending from 2D to 3D is nontrivial, we have substantial added new contributions to X3DEnc.

  1. New 3D Data Representation: while XShapeEnc uses binary mask to represent a 2D shape,X3DEnc flexibly supports voxel, tetrahedra, and surface triangle based presentation, accommodating water tight/not tight, solid/hollow, voxelized representation that commonly exist for 3D data.

  2. New Feature Mapping: X3DEnc required basis number grow cubically as the encoding length increases. So a small number of basis results in high-dimension encoding, where detail inevitably gets discarded. X3DEnc chooses to compute with a much larger number of basis than required to get initial coefficient and further maps it to the target length with one shared, seeded, data independent Krahmer-Ward projection. XShapeEnc doesn't have such component.

  3. New Fusion Strategy:. The proposed fusion strategy in Sec. 3.5 is newly designed, it supports geometry-color-pose independent emphasis (emphasize just one or two components). In summary, we propose to fuse geometry and color first, then continue to fuse pose.

In summary, X3DEnc isn't a just lift-up of XShapeEnc from 2D to 3D. We introduced new technical novelties to make X3DEnc to accommodate the main characteristics of the spatially grounded 3D object.

Q2: Why are the baseline methods evaluated on object-plus-context or merged point clouds, while X3DEnc uses separately encoded objects? Can the authors provide comparisons under equivalent input conditions?

A2: Thanks for pointing this out. Reviewer#449f raised the same question (See Q4 and Q5). The underlying reason we evaluate baselines on merged point clouds is that all these baselines are designed to only encode 3D data, they cannot directly encode spatial position by default, we add the 100 m context plane to the 3D object so that its spatial position is implicitly involved in the 3D data, and the merged data can be directly encoded by baselines without any modification.

We followed your advice to provide comparison under equivalent input conditions. We use X3DEnc pose vector encoding to encode spatial position, while baselines to just encode 3D object. Then, we either elementwise add or concatenate them to form the final encoding.

First, for the geometry-color-pose joint encoding retrieval task (Sec. 4.3 in the main paper), we report both the Add the Concate based comparison. The result is reported in the table below,

MethodGeometry EmphasisColor EmphasisPose Emphasis
SPH (PointEnc/Concat/Add)0.034/0.072/0.071-/-/-0.400/0.198/0.197
PlainZernike (PointEnc/Concat/Add)0.038/0.077/0.075-/-/-0.921/0.232/0.231
PointNet++ (PointEnc/Concat/Add)0.034/0.224/0.223-/-/-0.900/0.423/0.421
DGCNN (PointEnc/Concat/Add)0.034/0.161/0.162-/-/-0.911/0.481/0.483
Point-MAE (PointEnc/Concat/Add)0.034/0.154/0.153-/-/-0.858/0.428/0.426
Uni3D-B (PointEnc/Concat/Add)0.034/0.300/0.3000.650/0.744/0.7370.552/0.219/0.218
Sonata (PointEnc/Concat/Add)0.035/0.178/0.1790.201/0.348/0.3460.505/0.161/0.161
X3DEnc (voxel/tetrahedra/surface)0.300/0.323/0.3240.968/0.957/0.9561.000/1.000/1.000

We can see that:

  1. Separating 3D object encoding from its pose encoding substantially improves the performance. Addition and Concatenation produce similar performance.

  2. X3DEnc still maintains as the best-performing method, significantly outperforming all baselines variants.

Second, we test on inter-object topological relation classification. The result is in the table below (AccuRate ↑\uparrow) (Separate Enc means Concatenate):

MethodLinear ProbeMLP Probe
PlainZernike (Joint Enc)0.720.77
PlainZernike (Separate Enc)0.610.62
PointNet++ (Joint Enc)0.630.64
PointNet++ (Separate Enc)0.540.55
DGCNN (Joint Enc)0.830.88
DGCNN (Separate Enc)0.800.84
Mamba3D-lite (Joint Enc)0.800.80
Mamba3D-lite (Separate Enc)0.820.84
X3DEnc (surface)0.840.86

We can see that:

  1. Separating object and position encoding has encoder-dependent effects rather than yielding a uniform gain.

  2. X3DEnc exceeds every separate-encoding baseline on both probes.

In Summary, equivalent input yields improved performance, but still below X3DEnc.

AuthorsOpenReview ↗

Author Feedback 2

Q3: How does the proposed joint encoding compare against simpler fusion strategies, such as weighted concatenation of geometry, color, and pose descriptors? Can the authors demonstrate effective simultaneous control over multiple attributes?

A3: Thanks for mentioning this alternative joint encoding. Although simple weighted concatenation is theoretically viable, it has limitations.

  1. Setting appropriate weights for geometry, color, and pose to achieve a predefined level of controllability is challenging. Since their encodings vary with the inputs, fixed weights may not consistently reflect the intended relative emphasis. Achieving meaningful control therefore requires careful, input-dependent weight tuning.

  2. Weighted concatenation combines the three representations without explicitly modeling their cross-attribute dependencies. Under a standard Euclidean similarity measure, concatenation yields only a weighted sum of separate feature distances, with no cross-attribute interaction terms. Therefore, adjusting the concatenation weights alone cannot capture these dependencies or ensure a coherent joint representation.

We further experimentally show the advantage of our proposed joint encoding. Following Sec. 4.3, we run Concatenation based Geometry, Color and Pose controlled retrieval experiment. We use weight 1 and 2 for neutral-emphasis and emphasis, respectively. The result is shown in the table below (mAP ↑\uparrow):

MethodGeometryColorPose
X3DEnc (weighted concatenation)0.1700.3780.377
X3DEnc (ours)0.3240.9561.000

From this table, we can clearly observe that weighted concatenation reduce the performance significantly, which intern shows the advantage of using our proposed joint encoding to achieve controllability.

We hope we have addressed your concern. Feel free to let us know if you have any other concern. Thanks for spending time reviewing our paper.

Thank you for the detailed response and the additional experiments. I appreciate the effort, but the new results do not resolve my main concerns.

1. Equivalent-input comparison (Q2)

  • Geometry. Once the context plane is removed, baseline geometry mAP rises from 0.034 to as high as 0.300 (Uni3D-B), against 0.300 to 0.324 for X3DEnc. It seems that most of the gap in Table 1 came from the input difference.
  • Relation classification. With separate encoding, DGCNN and Mamba3D-lite reach 0.80 to 0.84, against 0.84/0.86 for X3DEnc. On 1,000 test samples, a 2-point gap is within sampling error.

2. Fusion vs. weighted concatenation (Q3)

The concatenation baseline uses weights 2 and 1, whereas the X3DEnc numbers use pure emphasis such as (0,0,2), so the two are not compared under matched weights.

3. Novelty over XShapeEnc (Q1)

Thank you for clarifying the lineage. The requested ablations isolating the new components (fusion rule, frequency propagation, structured projection) were not provided.

For these reasons I maintain my current rating. I also note that the revised PDF has not yet been uploaded.

AuthorsOpenReview ↗

Further Feedback Regarding Reviewer's Comment

We appreciate the reviewer's further comments. Regarding your comments, we'd like to provide more clarification.

Equivalent-input comparison: Under equivalent inputs, however, X3DEnc still leads every baseline on all three axes: geometry 0.324 vs 0.300 for Uni3D-B, a model pretrained on roughly one million objects with 6,000× the FLOPs, while the other six baselines reach at most 0.224; color 0.968 vs 0.744; pose 1.000 vs 0.483, although the equivalent-input baselines borrow X3DEnc’s pose encoder. For relation classification we agree the margin over DGCNN (0.84 ± 0.003 vs 0.83 ± 0.003, five seeds) is within sampling error and will state parity: a zero-training descriptor matches a trained encoder. We provide the FLOPS cost for all the baselines (3D object: ModelNet40/airplane/test/airplane_0627.off, pose vector is 13-value vector (3D position, scale, rotation).):

BaselineFLOPS
SH0.1 M
D20.2 M
SPH1.7 M
ESF2.7 M
LFD8.7 M
PlainZernike20.7 M
X3DEnc20.7 M
PointNet++1,763.4 M
PointMAE4,866.3 M
Mamba3D8,475.4 M
Sonata18,648.3 M
DGCNN99,743.6 M
Uni3D124,315.5 M

Fusion vs. weighted concatenation (Q3): Thanks for pointing this out. We didn't report the same weight as X3DEnc is because we found it lead to worse performance than 1/2 choose. For example, in the table below:

MethodGeometry EmphasisColor EmphasisPose Emphasis
Sonata (1/2 weight)0.1780.3480.161
Sonata (0/2 weight)0.1100.3180.147

We find the selecting appropriate weights is a tricky thing, there is no universal guidance and requires case-by-case adjustment. On the contrary, X3DEnc weighting strategy is carefully designed, it doesn't need tricky weight strategy design.

Novelty over XShapeEnc and Requested Ablations: The requested ablations have now been run and are posted in our reply to Reviewer LRWq, visible to all reviewers; we summarise them here. Structured projection: removing it drops geometry/color/pose mAP from 0.324/0.956/1.000 to 0.177/0.301/0.506. Frequency propagation: removing it drops relation-classification accuracy from 0.84/0.86 to 0.78/0.81 (linear/MLP probe). Harmonic pose lifting vs. sinusoidal encoding: 0.324/0.956/1.000 vs 0.203/0.472/0.370. Fusion rule vs. add/concat: table in A4 to Reviewer 449f, with the matched-weight version above to follow. Variance over 100 projection seeds is at most 0.003.

We will upload the newly modified PDF by Nov. 18.

We hope we have addressed your remaining questions. If so, it would be great if you can adjust your score accordingly. Really thanks for spending time reviewing our paper.

Reviewer_449fOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes X3DEnc, a training-free framework for encoding a spatially grounded 3D object (geometry, color, and pose) into a fixed-length vector. Each object is decomposed into geometry, a color field, and a pose vector, which are projected onto the same 3D Zernike basis. Three user-set weights (w_g, w_c, w_p) then combine the components coefficient-wise into a joint encoding whose emphasis can be adjusted without retraining. The claimed contributions are:

  1. a general-purpose, training-free encoder that places geometry, color, and pose in a common Zernike coefficient space;
  2. support for multiple 3D representations;
  3. coefficient-level interpretability;
  4. emphasis-adjustable joint encoding. The authors position X3DEnc as a complement to learned encoders, intended for settings with limited data or where interpretable, controllable multi-component encoding is needed.
Critical strengths

1 The proposed encoder needs no training data, which suits data-scarce settings and avoids dependence on any training distribution. 2 This work encodes geometry, color, and pose with the same 3D Zernike basis. Each coefficient has a clear frequency meaning, and the three parts can be used separately or combined. 3 The paper is theoretical grounding. It gives proofs of orthogonality, rotation equivariance and invariance, pose invertibility, and distance preservation of the projection.

Critical weaknesses

The two experiments are unfair compared. In Section 4.3, the baselines encode the object plus a 100 m context plane, while X3DEnc encodes the pre-separated object. And in Section 4.4, X3DEnc encodes the two objects separately while the baselines encode the merged point cloud.

Questions for authors

[Section 3.1, 3.4] What exactly does the pose vector contain (dimension, rotation parameterization, how it is extracted), and which components are used in each experiment? [Section 3.5, Appendix A.7] Section 3.5 encodes the pose "across all harmonic channels". What is the formula, and is the encoding still invertible? [Section 3.5 Eqs. (11)–(12), App. A.10] Why not use a simpler weighted concatenation followed by the same projection, which is also training-free? [Section 4.3 Table 1, App. A.14] The baselines encode the object plus a 100 m context plane, while X3DEnc encodes the pre-separated object. We consider this comparison unfair, since the 8.5× geometry gain may mainly come from the input difference. Could the baselines be evaluated on the object alone? [Section 4.4 Table 2] X3DEnc encodes the two objects separately while the baselines encode the merged point cloud, which is also unfair. Could baselines such as PlainZernike also be encoded per object?

2 comments
AuthorsOpenReview ↗

Author Feedback 1

We sincerely thank the reviewers for the insightful comments and positive review. We provide one-to-one feedback to address your concern:

Q1: [Section 3.1, 3.4] What exactly does the pose vector contain (dimension, rotation parameterization, how it is extracted), and which components are used in each experiment?

A1: The pose vector is designed to contain any information indicating a 3D object spatial cues, including centroid coordinate [x,y,z][x, y, z], scale ss and 3×33\times 3 rotation matrix AA~(flattened into a one-dimensional vector). They are concatenated to give a final pose vector, like [x,y,z,s,flatten(A)][x, y, z, s, \text{flatten}(A)]. In our experiments, we used all of them. We treat the rotation as identity rotation matrix diag(1,1,1)diag(1, 1, 1) as we keep the 3D object's original rotation intact while obtaining its spatial position.

Q2: [Section 3.5, Appendix A.7] Section 3.5 encodes the pose "across all harmonic channels". What is the formula, and is the encoding still invertible?

A2: "Across all harmonic channels" means we define harmonic pose field for each (l,m)(l,m) channel, separately, then fP(r,θ,ϕ∣p,l,m)=(∑k=1Kpkwk(r))Ylm(θ,ϕ)f_\mathcal{P}(r,\theta,\phi|\mathbf{p},l, m) =\left(\sum_{k=1}^{K}p_k w_k(r)\right)Y_{l}^{m}(\theta,\phi). For all (l,m)(l,m) defined in line 193 in the main paper. The reason for ``across all harmonic channels'' is that we want to associate each 3D object geometry encoding coefficient (in Eqn. (6)) with a corresponding pose encoding coefficient, so that they can be further combined in the emphasis-adjustable encoding.

Invertibility: Yes, the pose is invertible, from each harmonic channel.

Q3: [Section 3.5 Eqs. (11)–(12), App. A.10] Why not use a simpler weighted concatenation followed by the same projection, which is also training-free?

A3: Thanks for mentioning this alternative joint encoding. Although simple weighted concatenation is theoretically viable, it has limitations.

  1. Setting appropriate weights for geometry, color, and pose to achieve a predefined level of controllability is challenging. Since their encodings vary with the inputs, fixed weights may not consistently reflect the intended relative emphasis. Achieving meaningful control therefore requires careful, input-dependent weight tuning.

  2. Weighted concatenation combines the three representations without explicitly modeling their cross-attribute dependencies. Under a standard Euclidean similarity measure, concatenation yields only a weighted sum of separate feature distances, with no cross-attribute interaction terms. Therefore, adjusting the concatenation weights alone cannot capture these dependencies or ensure a coherent joint representation.

We further experimentally show the advantage of our proposed joint encoding. Following Sec. 4.3, we run Concatenation based Geometry, Color and Pose controlled retrieval experiment. We use weight 1 and 2 for neutral-emphasis and emphasis, respectively. The result is shown in the table below (mAP ↑\uparrow):

MethodGeometryColorPose
X3DEnc (weighted concatenation)0.1700.3780.377
X3DEnc (ours)0.3240.9561.000

From this table, we can clearly see that weighted concatenation reduce the performance sharply, which intern shows the advantage of using our proposed joint encoding to achieve controllability.

AuthorsOpenReview ↗

Author Feedback 2

Q4: [Section 4.3 Table 1, App. A.14] The baselines encode the object plus a 100 m context plane, while X3DEnc encodes the pre-separated object. We consider this comparison unfair, since the 8.5x geometry gain may mainly come from the input difference. Could the baselines be evaluated on the object alone?

A4: The reason why the baselines encode object plus 100 m context plane is that these baselines can only encode 3D data (e.g., point cloud). As these baselines can not encode spatial position by design, we add the 100 m context plane to the 3D object so that its spatial position is implicitly involved in the 3D data. With this process, we can directly run these baselines without any modification.

Per your suggestion, we have tried two other modified baselines: use the baselines to purely encode 3D object, our position vector encoder (Sec. 3.4 in the main paper) to encode its spatial position, separately, then either concatenate or elementwise add them together to give the final encoding. The result is shown in the table below (mAP ↑\uparrow) where PointEnc means the method reported in the main paper (encode 3D object and extended context as a whole 3D point):

MethodGeometry EmphasisColor EmphasisPose Emphasis
SPH (PointEnc/Concat/Add)0.034/0.072/0.071-/-/-0.400/0.198/0.197
PlainZernike (PointEnc/Concat/Add)0.038/0.077/0.075-/-/-0.921/0.232/0.231
PointNet++ (PointEnc/Concat/Add)0.034/0.224/0.223-/-/-0.900/0.423/0.421
DGCNN (PointEnc/Concat/Add)0.034/0.161/0.162-/-/-0.911/0.481/0.483
Point-MAE (PointEnc/Concat/Add)0.034/0.154/0.153-/-/-0.858/0.428/0.426
Uni3D-B (PointEnc/Concat/Add)0.034/0.300/0.3000.650/0.744/0.7370.552/0.219/0.218
Sonata (PointEnc/Concat/Add)0.035/0.178/0.1790.201/0.348/0.3460.505/0.161/0.161
X3DEnc (voxel/tetrahedra/surface)0.300/0.323/0.3240.968/0.957/0.9561.000/1.000/1.000

From this table, we can learn that:

  1. Separating object geometry from pose substantially improves the performance. Addition and concatenation yield close performance.

  2. X3DEnc still maintains the best performance, significantly outperforming all baselines.

Q5: [Section 4.4 Table 2] X3DEnc encodes the two objects separately while the baselines encode the merged point cloud, which is also unfair. Could baselines such as PlainZernike also be encoded per object?

A5:: The reason why we use baselines to encode the merged point cloud that the baselines can only encode 3D data by design. Per your suggestion, we follow the setting in Q4/A4 to encode per object and its spatial position separately. The result is shown in table below (classification accuracy rate ↑\uparrow):

MethodLinear ProbeMLP Probe
PlainZernike (Joint Encode)0.720.77
PlainZernike (Separate Encode)0.610.62
PointNet++ (Joint Encode)0.630.64
PointNet++ (Separate Encode)0.540.55
DGCNN (Joint Encode)0.830.88
DGCNN (Separate Encode)0.800.84
Mamba3D-lite (Joint Encode)0.800.80
Mamba3D-lite (Separate Encode)0.820.84
X3DEnc (surface)0.840.86

From this table, we can learn that:

  1. Separating object and position encoding has encoder-dependent effects rather than yielding a uniform gain: it lowers accuracy for PlainZernike (0.72/0.77→0.61/0.620.72/0.77 \rightarrow 0.61/0.62), PointNet++ (0.63/0.64→0.54/0.550.63/0.64 \rightarrow 0.54/0.55), while improving Mamba3D-lite (0.80/0.80→0.82/0.840.80/0.80 \rightarrow 0.82/0.84). Thus, simply separating the inputs does not reliably preserve or improve relational performance.

  2. X3DEnc achieves 0.840.84 linear-probe accuracy, the highest in the table (compared with 0.830.83 for DGCNN), and 0.860.86 with the MLP probe, just 0.020.02 below the best result (0.880.88 for DGCNN). It also exceeds every separate-encoding baseline on both probes, supporting X3DEnc as a competitive alternative without implying that it wins in every setting.

Summary:

We truly hope we have addressed your concern. We also incorporated your suggestion in our revised version. If you have any further concern or question, feel free to let us know, we are more than happy to provide more details.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Feedback too All Reviewers and AC

General Response to All Reviewers and the Area Chair

We sincerely thank all reviewers for spending time providing insightful and constructive comments, and the Area Chair for handling our submission. During the discussion we ran every additional experiment the reviewers suggested and posted the results in the individual threads. Here we put the common concerns together, summarize how we addressed each one, and clarify two points where we feel our framework may have been partially misunderstood (we updated the PDF and all revised texts are shown in blue color).

1. Common concerns and how we addressed them

Concern (raised by)What we didWhat we found
Unequal inputs in Tables 1–2 (449f, g75G, LRWq)Re-ran all 7 baselines on object-only input plus our pose encoder (concat/add); re-ran Table 2 with per-object encodingBaseline geometry mAP rises to at most 0.300 (Uni3D-B) vs 0.324 for X3DEnc; color 0.744 vs 0.968; pose at most 0.483 vs 1.000. We will replace Table 1 with this comparison and remove the "8.5×" claim
Fusion vs. simpler alternatives (449f, g75G, LRWq)Add / concat / weighted-concat ablationsAdd and concat lose pose control (mAP ≤ 0.48). Per Reviewer g75G's note, a matched-gain comparison over all 7 settings of Table 4 is running and will be posted whichever way it comes out
Component ablations (g75G, LRWq)w/o structured projection; w/o FreqProp; sinusoidal vs. harmonic pose lift0.324/0.956/1.000 → 0.177/0.301/0.506; relation accuracy 0.84/0.86 → 0.78/0.81; pose mAP 1.000 → 0.370
Variance (LRWq, g75G)100 projection seeds; 5 training seeds for Table 2, all methodsX3DEnc std ≤ 0.003; Table 2: 0.84 ± 0.003 vs DGCNN 0.83 ± 0.003. We will state parity rather than superiority
Efficiency (LRWq)FLOPs and A100 latency20.7 MFLOPs, 5.5 ms vs 1.8–124 GFLOPs, 12–142 ms for neural encoders
Segmentation in cluttered scenes (LRWq)Per-point X3DEnc appended to PointNet++ / DGCNN, ScanNet v2 val, 3 seeds+4.5 and +4.4 mIoU over the public baselines, with no object isolation required

2. Two points we would like to clarify

"Privileged inputs." X3DEnc is designed for a spatially grounded 3D object: geometry, color and pose are its input specification, not a preprocessing advantage. Point-cloud encoders cannot take a pose vector by design, which is why we originally embedded the position in a context plane. In the equivalent-input experiment the baselines receive the same object crops, point sampling and normalization, plus our own pose encoder, so the remaining margin comes from the descriptor itself. We agree that our original framing invited the "8.5×" reading, and we have corrected it.

"Competing with neural encoders." We do not claim that X3DEnc beats trained encoders. Our claim is that a zero-parameter, deterministic encoding matches them (parity in Table 2; 85.8% on ModelNet40 without training) at a small fraction of the cost, while offering what they do not provide: a single encoding whose emphasis on geometry, color or pose is steered by three gains without re-encoding, exact pose recovery, and coefficient-level interpretability.

3. What the discussion has established

After the fairness corrections requested by all three reviewers, we believe the following claims stand on the strengthened evidence:

  • Controllability is unique to X3DEnc. No baseline, even when given our pose encoder, recovers pose retrieval above 0.48 mAP; X3DEnc reaches 1.000 from the same encoding by changing one gain, with the pose exactly invertible (App. A.7).
  • Every new component carries measurable weight. Removing structured projection, FreqProp or harmonic pose lifting costs between 6 and 63 points on the corresponding task, so the extension beyond XShapeEnc is not nominal.
  • Zero training matches trained encoders. Parity with DGCNN on relation classification over five seeds, 85.8% on ModelNet40, and 12.2% vs 11.6% cross-domain, all without a learned parameter.
  • The descriptor transfers to scenes. The same encoding, applied per point, adds 4.4 to 4.5 mIoU to two standard segmentation networks on ScanNet v2 without any object isolation.
  • It is deterministic and cheap. Std at most 0.003 across 100 projection seeds; 20.7 MFLOPs and 5.5 ms per object on an A100, three orders of magnitude fewer FLOPs than the neural baselines.

We truly hope the added experiments and clarifications have addressed the reviewers' concerns. We are more than happy to provide further details or run additional experiments if any question remains, and we thank the reviewers and Area Chair again for their time and careful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.