acceptodds
Under review as a conference paper at ICLR 2027

Mind the Gap Between Train and Test Grids: Resolution Collapse in Voxel-Indexed Point Networks Is Mostly a Units Problem

Abstract

For point cloud semantic segmentation, the leading backbones voxelize the scene and read it along a space-filling curve, which keeps their cost low on large scenes. Evaluated on a grid coarser than the one they trained on, they lose around 50 mIoU points on standard indoor and outdoor benchmarks. The usual explanation is that a coarser grid destroys geometry. In this work, we show that most of that collapse reverses without restoring any geometry. Sparse convolution, pooling, serialization and rotary position encoding all read the integer index , so changing the voxel size changes the unit of every operator, and nothing in the network signals it. We propose relabelling: it recomputes that index from the physical coordinate at the training grid, without retraining. On six architecture-dataset pairs, it recovers 70 to 90% of the collapse on grids three times coarser than the training one. On dense indoor scenes, it cuts peak memory by 3.4 to 4.4× and inference time by up to 46%. Most of what a coarser grid appears to cost is a unit mismatch: even on grids finer than the training one, which lose no geometry, the network still collapses, and our method recovers all of it. This new training-free relabelling step offers a generic speedup for any voxel-indexed point network.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
point cloud semantic segmentation, 3D deep learning, voxel-based networks, sparse convolution, resolution robustness, training-free adaptation, discretization invariance, point transformers, rotary position encoding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_hiJMOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript investigates the substantial performance degradation of voxel-indexed point cloud networks when the voxel resolution differs between training and inference. The authors attribute this degradation primarily to a mismatch in spatial indexing units rather than the loss of geometric information. To address this issue, they introduce a training-free relabelling method that recomputes voxel indices using the training resolution while retaining the points selected at the test resolution. Experiments across three network architectures and two benchmark datasets show that the method recovers 70–90% of the performance loss at three times the training voxel size. The results also demonstrate reduced memory consumption and inference time on dense indoor point clouds. Overall, the study offers a useful perspective on resolution sensitivity in voxel-based point cloud networks.

Critical strengths
  1. The paper provides a useful explanation for the performance degradation of voxel-indexed point cloud networks under mismatched training and inference resolutions. By separating voxel sampling from spatial indexing, it shows that much of the degradation arises from indexing-unit mismatch rather than geometric information loss alone. This finding improves our understanding of resolution sensitivity in voxel-based architectures.
  2. The proposed relabelling method is simple, requires no retraining, and achieves substantial performance recovery across three architectures and two datasets. The use of frozen checkpoints, negative controls, and operator-level ablations strengthens the experimental evidence and supports the practical relevance of the approach.
Critical weaknesses
  1. The proposed explanation does not fully disentangle indexing-unit mismatch from the changes in neighborhood connectivity, pooling structure, and serialization induced by relabelling. Although the ablation studies provide supporting evidence, the extent to which unit mismatch alone accounts for the observed performance degradation remains uncertain.
  2. The method's generalizability is not fully established. Table 15 shows a noticeable performance drop when voxel centroids replace sampled points, indicating sensitivity to the voxelization strategy. Moreover, the proposed relabelling operation is closely related to existing scale normalization concepts, leaving its methodological novelty and broader applicability insufficiently demonstrated.
Questions for authors
  1. To what extent can the performance recovery be attributed specifically to correcting the indexing-unit mismatch, rather than to the accompanying changes in neighborhood connectivity, pooling, and serialization? Do the existing ablations sufficiently distinguish these effects?
  2. Table 15 shows that replacing sampled points with voxel centroids reduces the relabelled mIoU from 62.21 to 56.76, while the stock performance remains unchanged. What accounts for this sensitivity, and how does it affect the claimed applicability of the method to different voxelization pipelines?
  3. Given the close relationship between relabelling and existing scale normalization approaches, what is the substantive methodological distinction beyond applying the training voxel size to recompute spatial indices at inference time?
2 comments

Thank you for the review. The revised paper answers each question; the passages are in blue.

W1 / Q1. Unit mismatch, or the changes in neighbourhood, pooling and serialization. The ablations now separate them (Figure 6, Figure R3, Section 4.5).

  • Serialization: kept on the stock order while everything else is relabelled, it costs at most 0.26 mIoU point, although many tokens change window.
  • Neighbourhood: relabelling does not restore it. At the first stage, a relabelled ScanNet token has ten times fewer occupied neighbours than in training, yet it recovers.
  • Pooling: it is not a separate effect. It computes the index of every later stage, so it is how the unit reaches them. Relabelling the convolution or the serialization without it recovers nothing, and keeping the first two stages on the stock index loses a fifth to half of the gain.

The gain needs the training unit in the pooling and, on LitePT, in the rotary phase; it does not need the training neighbourhood or the relabelled window order. The operators act in series, so we report which ones are needed rather than a share for each.

W2 / Q2. Sensitivity to voxel centroids (Table 15). A centroid stays inside its coarse cell, so the stock index does not change. Relabelled, its fine index is what matters, and the centroid is pulled towards the middle of the coarse cell: it loses where the points were. The fix is to keep the centroid as the token but read the index on one of the cell's points, which recovers the score of the random-point voxelizer (Section 3.2, Appendix D.1). Appendix D.1 now counts this without a model, at 3τ: on ScanNet the centroid's fine index differs from the kept point's in three cells out of four, and a quarter of the centroids fall in the middle fine cell, against 3 % of the kept points. Outdoors the centroid costs nothing, because two thirds of the coarse cells of a nuScenes sweep hold a single point, which is its own centroid. A centroid pipeline must therefore also keep one point per cell and read the index on it. Table 15 shows it in an added row, 61.76, from a later evaluation run in which the random point also gives 61.76.

W2 / Q3. Difference from scale normalisation. Section 3.3 now spells out the two scale fixes and why they fail (Figure R1). Scene shrinking divides the coordinates by k, as Utonia does in pretraining, but then the index is exactly the stock one, so the unit is not repaired. Index rescaling multiplies the stock index by k: the step length is right, but every token sits on a multiple of k, so neighbours end up k cells apart. Relabelling gets both right. Measured, index rescaling stays 17 to 37 mIoU points below relabelling at 3τ on all six pairs, and scene shrinking scores within 0.17 mIoU point of stock on the three ScanNet backbones (Section 4.5, Table 16). The released Utonia model behaves the same way: dividing its input scale by k does not help, while relabelling recovers most of its drop (Table 4).

Thanks to the author for the reply. The revised manuscript added operator-level ablation experiments, explained the impact of different voxelization strategies on the results, and further clarified the difference from existing scale normalization methods. The new experiments and analysis addressed the previous questions well, and my main concerns about the paper have been resolved, so I’m keeping my original rating of Weak Acceptance (3 points).

Reviewer_TQtkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the performance degradation of voxel-indexed point cloud networks when the training and testing grid resolutions do not match. The authors separate point cloud sampling from integer indexing, retain the points selected by the testing grid, and recalculate the indices using the training grid size, without the need for retraining. Experiments on three backbone networks and two datasets demonstrate that this approach can recover approximately 70%–90% of the performance loss incurred when the test grid is three times coarser, while also reducing computational overhead in dense indoor scenes.

Critical strengths
  1. The problem is practically significant, and the solution is simple and consistent with the motivation. It mitigates performance degradation caused by changes in test resolution without modifying model parameters, making it valuable for deployment. The controlled experiments are persuasive. By retaining the same sampling points and freezing model weights, simply changing the index restores a significant portion of performance, providing valuable evidence to distinguish between sampling loss and index-scale mismatch.
  2. The controlled experiments are compelling. By changing only the index while keeping the sampling points and freezing the model weights, we were able to recover a significant amount of performance, providing valuable evidence to distinguish between sampling loss and index-scale mismatch.
Critical weaknesses
  1. The new insights relative to prior work still need to be further clarified. While the paper discusses Utonia and Invaria, the understanding that grid size affects physical scale has already been established in previous research. The main contribution of this paper lies in separating sampling from indexing and performing training-free corrections; the scope of this contribution should be more clearly defined. Additionally, the scene scaling experiments in Table 16 test only a single operation and cannot substitute for an evaluation of the relevant complete methods.
  2. The scope of practical efficiency gains is limited. Significant time and video memory gains are primarily observed in dense indoor scenes, while outdoor gains are modest; after reindexing, a noticeable drop in accuracy persists on coarse grids. Therefore, the current results are insufficient to support the broad, general acceleration claims made in the abstract, and the trade-off between accuracy and computational cost needs to be presented more clearly.
Questions for authors
  1. Compared to Utonia and Invaria, which conclusions in this paper represent new insights into mechanisms, and which constitute experimental validation or implementation of existing knowledge? Which controlled experiments clearly support this distinction?
  2. Based on the existing experiments, what practical advantages does reindexing offer over conventional grid settings and multi-resolution training at comparable accuracy or computational budgets? Under what scenarios do these advantages no longer hold? At this point, I am leaning toward a mild rejection. I recognize the experimental value of this paper, but I still have reservations about its incremental contribution. I will reassess the paper in light of the authors’ responses.
1 comment

Thank you for the review. We revised the paper on both points; the passages are in blue.

W1 / Q1. What is new relative to Utonia and Invaria. Your question led us to evaluate the released Utonia model, and the result answers it. Although Utonia is pretrained at a common scale, it loses about a third of its ScanNet score at three times its grid-cell size, and relabelling recovers 88 % of that drop without retraining, while dividing its input scale by k scores below stock (Table 4, Section 4.6). Utonia still reads its index in the unit of its grid. We agree that the link between grid size and physical scale is known; Utonia builds on it. What was not known is how much of the collapse comes from the index alone. Invaria trains for invariance, and Utonia rescales each domain during pretraining. Both change the selected points and the index at once, so neither isolates the index (Introduction). Three results are new, each from a controlled experiment:

  1. On frozen weights, with the coarse points held fixed, changing only the index recovers 70 to 90 % of the drop at 3τ and all of it on finer grids (Tables 2 and 3). On finer grids at least as many points survive, so lost points cannot explain that drop.
  2. The unit is needed in the pooling and, on LitePT, in the rotary phase. The gain needs neither the relabelled window order nor the training neighbourhood (Figure 6, Figure R3, Section 4.5).
  3. The two obvious scale fixes fail: scene shrinking gives back the stock index, and index rescaling puts neighbours k cells apart (Section 3.3, Figure R1, Table 16).

On complete methods: Table 16 indeed tests a single operation, hence the evaluation of the complete released Utonia model above, with its released linear head (Table 4). Invaria has released no code or evaluation protocol, so we can only set its reported drops beside ours (Appendix F.1).

W2. Scope of the efficiency claim. We agree, and we narrowed it. The abstract now ends: “With this training-free step, a coarser grid saves memory and time on dense scenes, where it removes many points.” The introduction and the conclusion say the same: the saving is large in a dense room and small in a lidar sweep. Section 4.7 now states the trade-off in a new paragraph, “When relabelling does not help” (see Q2), beside Figure 7, which plots accuracy against memory and time. It also gives the price: on the same coarse grid, relabelling is 13 to 38 % slower than the stock pipeline, because its deep stages keep more tokens.

Q2. Advantages over a conventional grid and multi-resolution training, and when they no longer hold. Section 4.7 now answers this. If the model can be retrained, the simpler option is to train it on the grid it will run on: on ScanNet, LitePT-S trained at 3τ scores higher at that grid than the frozen model relabelled, with several times fewer operations (Tables 4 and 20). A model trained on several grids also scores higher, and relabelling still adds to it (Table 4). Relabelling is meant for the other case: weights that are already trained, including published ones, or a single model that must run on several grids. Its advantage also fades on very coarse grids: at 7τ outdoors it recovers only about half of the collapse (Table 8).

Reviewer_H352OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the severe accuracy degradation that occurs when voxel-indexed point networks are evaluated at a different grid resolution from training. It argues that much of this degradation comes from a mismatch in coordinate units, rather than geometric information loss alone. The proposed correction retains the coarse token set but recomputes its indices in training-grid units, recovering substantial accuracy without retraining. Experiments cover several backbones and both indoor and outdoor datasets.

Critical strengths
  1. The diagnosis is useful and experimentally testable. Holding model weights and the coarse token set fixed while changing the coordinate representation provides a compelling intervention on the proposed mechanism. The substantial recovery suggests that resolution collapse is not simply an unavoidable consequence of discarding geometric detail.
  2. The correction has immediate practical value. It can be applied to existing checkpoints, and evaluation across attention-based and sparse-convolution backbones suggests that the issue extends beyond a particular architectural design. The paper also reports settings where the correction offers limited benefits, which helps establish its scope.

I lean toward weak acceptance. The paper identifies a consequential implementation issue and provides a simple correction with convincing empirical benefits. More precise attribution and a clearer account of the outdoor results would strengthen the claims, but the central contribution is already useful.

Critical weaknesses
  1. The experiment establishes that representation matters, but does not fully isolate which aspect matters. Relabelling may affect coordinate scale, neighborhood construction, or traversal/serialization order. The remaining 11–16 mIoU gap at 3× grid size also leaves substantial degradation unexplained. Controls that hold ordering fixed, together with stage-wise corrections, would make the causal attribution more precise. This matters particularly for the claim that the collapse is “mostly” a units problem.
  2. The outdoor results narrow the deployment argument. At matched token counts, random sampling outperforms the relabelled grid approach. This does not refute the unit-mismatch diagnosis, but it raises a separate question: when is corrected grid-based token selection preferable to a simpler sampling strategy? An analysis of spatial density and token allocation would help explain the indoor–outdoor difference and guide practitioners.
  3. Compatibility with token merging remains a plausible extension rather than a demonstrated result. A joint experiment would strengthen the practical case, although I consider this secondary to the mechanism and sampling controls above.
Questions for authors

Can you disentangle coordinate-unit correction from changes in serialization order and neighborhood structure? What explains the outdoor disadvantage relative to random sampling, and under what conditions does relabelled grid selection provide a better accuracy–cost trade-off?

1 comment

Thank you for the careful review. Each point below is answered in the revised paper; the passages are in blue.

W1 / Q1. Unit, neighbourhood and serialization order. Relabelling changes three things at once: the length of an index step, which tokens are kernel neighbours, and the serialization order that cuts the attention windows. Figure 6 now relabels them alone and in groups (Section 4.5).

  • Order held fixed. With every operator relabelled except the serialization, which keeps the stock order, the score drops by at most 0.26 mIoU point, although many tokens change window (Figure R3a). SpUNet, which has no windows, recovers as well.
  • Neighbourhood. Relabelling does not restore the training neighbourhood: on ScanNet, at the first stage, a relabelled token has ten times fewer occupied neighbours than in training (Figure R3b).
  • Stage-wise. The unit matters in the pooling, which computes the index of every later stage. Keeping the first two stages on the stock index loses a fifth to half of the gain. On LitePT, the rotary phase also needs it: kept stock, it loses most of the gain.

The operators act in series, so the figure shows which ones are needed rather than a share for each. The fixed-order control covers all four model and dataset pairs.

W1. “Mostly” and the remaining 11–16 points. “Mostly” means 70 to 90 % of the drop at 3τ, and the whole drop on finer grids, where at least as many points survive (Table 3). For the rest, Section 4.6 now compares the frozen model with retrained ones on the same points. Relabelled, the frozen LitePT-S loses about 16 mIoU points at 3τ. A model trained on several grids and relabelled loses fewer than 3 mIoU points (one seed per training, Table 4). The missing points therefore cost a model trained on them at most about 3 mIoU points; most of the remaining drop is not geometry the grid removed.

W2 / Q2. Random sampling outdoors, and when to prefer the grid. The difference is token allocation. Table R2 gives mIoU and point density by distance to the sensor at 3τ. A random draw keeps tokens where the training grid has most of them, near the sensor: 53 % of its tokens lie within 10 m, against 36 % for the grid. It wins within 10 m and loses beyond. The grid is the better choice where density is even, as indoors (Table 12), and for distant objects (Section 4.4).

Q3. When the relabelled grid gives a better accuracy–cost trade-off. Section 4.7 now states it. On dense scenes with frozen weights, relabelling at 3τ divides peak memory by 3.4 to 4.4 and recovers most of the collapse; on the same coarse grid it is 13 to 38 % slower than the stock pipeline, which loses far more. On lidar sweeps a coarser grid removes few points, so there is little to save. If the model can be retrained, a model trained on the target grid is both cheaper and more accurate (Tables 4 and 20).

W3. Token merging. We agree that it is not demonstrated. We removed the sentence that suggested it, and the conclusion now gives it only as future work: a coarse grid at the input combined with token selection or merging inside attention.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision

We thank the three reviewers. Each question pointed at something the paper did not explain or measure well enough, so we answered it in the paper itself. The revised PDF accompanies this response. Every new or changed passage is in blue. New figures and tables are numbered R1 to R3 in reading order, so the numbers the reviews cite still refer to the same tables and figures. In each reply, W and Q number the weaknesses and questions of that review in its own order. The changes below follow the order of the paper.

Main changes:

  1. Centroid voxelizers (hiJM). Section 3.2 and Appendix D.1 explain the centroid result, count it without a model, and give the fix. Table 15 gains one row: the centroid kept as the token, with the index read on one of the cell's points, which gives back the score of the random point.
  2. Scale normalisation (hiJM). Section 3.3 defines scene shrinking and index rescaling, and Figure R1 shows why neither restores the index.
  3. Grid against random sampling outdoors (H352). Table R2 gives mIoU and point density by distance to the sensor, and Section 4.4 explains it by where each sampler puts its tokens.
  4. Which part of relabelling carries the gain (H352, hiJM). Figure 6 now relabels the operators alone and in groups, with the window order kept stock and the first two stages kept stock. Table 16 gives the values. Figure R3 shows, on a real ScanNet scene, what the neighbourhood and the attention windows become. Section 4.5 reads them: the gain needs the training unit in the pooling and the rotary phase. It does not need the training neighbourhood, and the window order can stay stock at a cost of at most 0.26 mIoU point.
  5. What relabelling leaves (H352). Section 4.6 bounds the remaining indoor drop with the retrained models of Table 4.
  6. Complete methods (TQtk). The released Utonia model is evaluated stock, relabelled, and with its input scale divided by k (Table 4). Section 4.6 sets Invaria's reported drops beside ours.
  7. Scope of the efficiency claim (TQtk). The last sentence of the abstract, the introduction and the conclusion now limit the saving to dense scenes. Section 4.7 adds when relabelling does not help.
  8. Corrections to the noise tables. Tables 5 and 6 and the noise threshold of Section 4.1 now state precisely which trainings differ in seed only and which also in batch size. The values of Table 6 are corrected accordingly. Its grid-origin row for LitePT-S ScanNet is now measured on the published checkpoint, so the sub-cell phase bound of Section 4.4 becomes 0.74 point instead of 0.57. Figure 8 is removed, since Figure R1 now shows the same three indices in Section 3.3. Section 4.5 is rewritten, and the paragraph on the model trained at the coarse grid is replaced by “What relabelling leaves” (Section 4.6); the numbers they no longer quote are in Tables 4 and 16.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.