- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents X3DEnc, a training-free framework for encoding spatially grounded 3D objects. The method decomposes each object into normalized geometry, color, and spatial pose, lifts the pose vector into a harmonic field, and projects all three components onto a shared 3D Zernlike basis, enabling both separate and jointly controllable encodings without retraining. It supports voxel, signed-tetrahedral, and surface-triangle representations and produces fixed-length descriptors through frequency propagation and structured projection. Experiments on attribute-controlled retrieval, inter-object spatial relation classification, cross-domain generalization, and object classification demonstrate its potential as an interpretable and dataset-independent complement to learned 3D encoders.
Critical strengths
-
Unified spectral encoding of geometry, appearance, and pose. The paper maps normalized geometry, color fields, and a harmonically lifted pose vector into a shared 3D Zernlike coordinate system. This provides a technically coherent way to represent intrinsic object properties and extrinsic spatial information within the same descriptor space.
-
Explicit and controllable multi-component feature fusion. X3DEnc introduces independent geometry, color and pose gains, together with energy normalization and coefficient-level magnitude/phase fusion. This design allows the same descriptor construction to be redirected toward different attributes at inference time without retraining an additional fusion network.
-
Support for heterogeneous 3D representations within one formulation. The method provides voxel-, signed-tetrahedral-, and surface-triangle-based integrations, covering volumetric solids, watertight meshes, hollow objects, and open surfaces. The three formulations also expose clear trade-offs between geometric measure, input requirements, approximation fidelity, and computational complexity.
Critical weaknesses
-
Essential ablations and statistical analyses are missing. The paper does not isolate the contributions of the pose lifting, frequency propagation, nonlinear fusion, structured projection or representation choice, and it reports neither repeated runs nor variance; the mixed-emphasis results are also strongly color-dominated.
-
The paper lacks empirical efficiency evaluation. It provides only asymptotic complexity and qualitative efficiency discussions, without reporting preprocessing time, encoding latency, memory usage, throughput, scalability, or runtime comparisons with competing descriptors.
-
The main comparisons use fundamentally unfair inputs. In Table I, the baselines must encoder a merged object-plus-large-plane point cloud, while X3DEnc receives isolated canonical geometry, color and explicit pose; in the relation experiment, X3DEnc also receives the two object instances separately while the baselines receive only their merged point cloud. The reported gains may therefore come from privileged segmentation and metadata rather than the proposed descriptor.
Questions for authors
The current experiments focus on object retrieval and object-level or relation-level classification, while no semantic or instance segmentation results are reported. Since X3DEnc produces a global object descriptor and assumes that the input object has already been isolated from the background, how can it be adapted to generate point or voxel-level segmentation predictions in cluttered real-world scenes? If the authors can address all of the concerns raised above, I would be willing to reconsider my assessment and potentially my rating.
5 comments
Author Feedback 2
-
Variance: continued result: In inter-object topological relation classification in Table 2, we independently train all methods 5 times with seeds from 0 to 4, same training recipe. The new results with variance is reported in the table below (Accuracy Rate, ):
Method Linear Probe MLP Probe SPH 0.81 $\pm$0.002 0.82$\pm$0.003 D2 0.71$\pm$0.010 0.76$\pm$0.004 ESF 0.79$\pm$0.003 0.83$\pm$0.002 SH 0.59$\pm$0.004 0.60$\pm$0.003 LFD 0.80$\pm$0.002 0.80$\pm$0.005 PlainZernike 0.72$\pm$0.001 0.77$\pm$0.003 PointNet 0.65$\pm$0.002 0.68$\pm$0.002 PointNet++ 0.64$\pm$0.005 0.64$\pm$0.004 DGCNN 0.83$\pm$0.003 0.88$\pm$0.004 Mamba3D-lite 0.81$\pm$0.005 0.80$\pm$0.004 X3DEnc (surface) 0.84$\pm$0.003 0.86$\pm$0.002 From this table, we can observe that the variance is relatively small across all methods (maximum variance is 1% for D2 baseline).
- Color Dominance: We agree this is visible in Table 4 and offer a tentative explanation. The benchmark assigns each object one of five uniform colors, so color has zero within-class spread, whereas geometry is a 40-way (ModelNet40 has 40 classes) attribute with large intra-class variation. Fusing the two at matched energy tends to yield a lexicographic ranking: same-color objects first, ordered by geometry within.
Q2: The paper lacks empirical efficiency evaluation. It provides only asymptotic complexity and qualitative efficiency discussions, without reporting preprocessing time, encoding latency, memory usage, throughput, scalability, or runtime comparisons with competing descriptors.
A2: Thanks for point out. We added the experiment and report both FLOPS and inference time here.
- Experimental setting 1: 3D object: ModelNet40/airplane/test/airplane_0627.off, pose vector is 13-value vector (3D position, scale, rotation). The FLOPS is reported in the table below: | Baseline | FLOPS | |:---|:---:| | SH | 0.1 M | | D2 | 0.2 M | | SPH | 1.7 M | | ESF | 2.7 M | | LFD | 8.7 M | | PlainZernike | 20.7 M | | X3DEnc | 20.7 M | | PointNet++ | 1,763.4 M | | PointMAE | 4,866.3 M | | Mamba3D | 8,475.4 M | | Sonata | 18,648.3 M | | DGCNN | 99,743.6 M | | Uni3D | 124,315.5 M |
Conclusion: We can observe that X3DEnc consumes much smaller FLOPS than neural network based baselines. Comparing with other training-free baselines, X3DEnc consumes relatively more FLOPS, we find 40% of the FLOPS is used for 3D Zernike basis construction, which is just one-time cost when encoding multiple spatially grounded 3D objects because the basis can be constructed beforehand.
- Experimental setting 2: we use the same 3D object and spatial position and compare the inference encoding time on 1 NVIDIA A100 80GB PCIe between X3DEnc and other neural network based baselines, 50 times indepedent tests. The averaging inference encoding time comparison is given in the table below: | Methods | Inference Time (ms) | |:---|:---:| | X3DEnc | 5.532 | | PointMAE | 12.206 | | Mamba3D | 29.437 | | DGCNN | 30.942 | | Uni3D | 53.428 | | Sonata | 76.634 | | PointNet++ | 141.812 |
Conclusion: From this table, we can see that X3DEnc takes much less inference time on GPUs than neural network based methods, which shows the effectiveness of X3DEnc.
Q3: The main comparisons use fundamentally unfair inputs. In Table I, the baselines must encoder a merged object-plus-large-plane point cloud, while X3DEnc receives isolated canonical geometry, color and explicit pose; in the relation experiment, X3DEnc also receives the two object instances separately while the baselines receive only their merged point cloud.
A3: Thanks for pointing this out. We kindly suggest you refer to Reviewer #449f Q4/A4 for our detailed explaination and experimental result as the reviewer raised the same question. Summary: we encode object-plus-large-plane point cloud in order to fit for the baseline 3D-data-only input requirement. Per your suggestion, we split the pose and 3D object encoding as we did for X3DEnc, new experimental results show improved performance than object-plus-large-plane encoding, but the performance is still inferior to X3DEnc.
Author Feedback 3
Q4: The current experiments focus on object retrieval and object-level or relation-level classification, while no semantic or instance segmentation results are reported. Since X3DEnc produces a global object descriptor and assumes that the input object has already been isolated from the background, how can it be adapted to generate point or voxel-level segmentation predictions in cluttered real-world scenes?
A4: In the main paper, we just reported object-level encoding by X3DEnc. However, X3DEnc can be extended to provide per-point encoding.
-
How to achieve it: we take a ball of radius r around each point, translate it to the origin, scale it to the unit ball, and apply the unchanged pipeline: local occupancy and RGB fields projected onto the 3D Zernike basis, reduced to yaw-invariant block norms so vertical structure is retained, and energy-matched across geometry and color. Three radii from 0.1 to 0.6 m give local surface type plus part-level contex) provide surface-level and part-level context. The result encoding is per-point descriptor computed once per scene, with no object isolation required; We remove per-point position in the encoding.
-
Experimental Setup: We test whether a training-free X3DEnc per-point descriptor adds information that learned segmentation features lack. We use ScanNet v2 (1,513 indoor scans, 20 classes), training on the 1,201 train scenes and reporting mIoU on the 312 validation scenes. We train PointNet++ (xyz + RGB input) and DGCNN (xyz input) from scratch with the unchanged public recipes, and compare against the same networks trained from scratch with the X3DEnc geometry–color descriptor appended to each point’s input channels. Results are reported as mean ± std over three seeds.
-
Experimental Result:
-
The result on PointNet++ is shown in the Table below: | Method | Input | Epochs | Seeds | ScanNet v2 val mIoU (%) | |---|---|---|---|---| | PointNet++ (single-scale grouping), mmdetection3d report | xyz + rgb | 200 | 1 | 54.44 | | PointNet++ (single-scale grouping) + X3DEnc | xyz + rgb + X3DEnc Per-Point Enc | 200 | 3 | 58.90 0.13 |
-
The result on DGCNN is shown in the Table below: | Method | Input | Epochs | Seeds | ScanNet v2 val mIoU (%) | |---|---|---|---|---| | DGCNN, antao97/dgcnn.pytorch report | xyz + normalized xyz | 200 | 1 | 49.6 | | DGCNN + X3DEnc | xyz + normalized xyz + X3DEnc Per-Point Enc | 200 | 3 | 54.01 0.17 |
-
Conclusion: we both observed performance gain by PointNet++ and DGCNN, an average of 4 point gain. it thus shows the feasibility to extend X3DEnc to solve segmentation task in the real-world dataset. We are working to extend X3DEnc to more various tasks to shows its applicability.
We hope we have addressed your concerns. If you have any further question, feel free to let us know, we are more than happy to provide more feedback.
Author Feedback 1
We thank the reviewer for providing such a critical and insightful comments. We provide detailed feedback regarding your concerns.
Q1: Essential ablations and statistical analyses are missing. The paper does not isolate the contributions of the pose lifting, frequency propagation, nonlinear fusion, structured projection or representation choice, and it reports neither repeated runs nor variance; the mixed-emphasis results are also strongly color-dominated.
A1: Thanks for pointing this out. Part of the questions have been partially addressed in the paper, we provide one-by-one feedback.
Representation Choice. Table 1 reports voxel, tetrahedra and surface on the geometry-color-pose emphasis based retrieval task, and they agree closely. Table 3 and Appendix A.8 give the cost and applicability trade-off comparison.
Structured Projection. In Sec. A.9 of Appendix, we have discussed the motivation and necessity of the structured projection, accompanied by Fig. 6 showing cluster survive from 4096-d to 512-d projection. Per your suggestion, we run an ablation to test w/ and w/o structured projection on geometry-color-pose emphasis retrieval task, the result is shown in the table below (mAP ↑): | Method | Geometry Emphasis | Color Emphasis | Pose Emphasis | |--------|----------|-------|------| | X3DEnc (w/o Structured Projection, direct 512 encode length from 3D Zernike bases) | 0.177 | 0.301 | 0.506 | | X3DEnc (w/ Structured Projection, 4096 → 512, surface representation) | 0.324 | 0.956 | 1.000 |
From this table, we can observe that without structured projection has observed performance drop sharply.
We can learn from table that using Sin/Cos pose encoding leads performance drop. It is worth noting that Sin/Cos based pose lift loses invertibility.
Fusion Controllability: The other two reviewers asked similar question (Q4 in R#449f, Q2 in R#g75G). We ablated two fusion strategies: encode each component separately, then use elementwise add concatenation to fuse them together. The experiment result on geometry-color-pose emphasis retrieval task is shown for our feedback in A4 for R#449f, from which we have observed sharp performance drop when using Add/Concate fusing strategy. If you have any other fusing strategy that you want us to test, please let us know.
FreqProp: Thanks for pointing it out. We inherit the idea of FreqProp from XShapeEnc (arxiv 2604.07522) but extend it to 3D space. Per your suggestion, we run experiment w/ and w/o FreqProp on inter-object topological relation classification tasks (Accuracy Rate ↑). | Method | Linear Probe | 3 layer MLP Probe | |--------|----------|------| | X3DEnc (surface, w/o FreqProp) | 0.78 | 0.81 | | X3DEnc (surface, w/ FreqProp) | 0.84 | 0.86 |
From this table we can observe that removing FreqProb reduces the classification accuracy a lot.
From this table, we can observe that variance is small, which shows X3DEnc is relatively invariant to the nuisance randomness of the structured projection.