1. The paper overstates what prior work is missing.
A lot of the motivation in Secs. 1–2 rests on claims about prior work that don't hold up.
- Padding-free sparse window attention already exists. Secs. 1, 2.1 and 3.2 describe existing sparse transformers as relying on zero-padding, and Sec. 2.1 concludes this leaves "operator-level purely sparse attention open". But Stratified Transformer [R1] (Sec. 4 and Fig. 7) already does essentially what Secs. 3.2.1–3.2.2 describe: it precomputes all valid query–key index pairs, computes the dot products by index, applies a per-query (scatter) softmax, and sums the indexed values back per query. Each step is a single CUDA kernel, and there's no dummy-token padding. The main difference I can see is the table layout: DeS3 keeps per-query offsets (Item Count, which is basically a CSR row pointer) plus a group map, instead of plain index pairs and scatter ops. That could still be a useful engineering contribution, but then the paper should say that's the contribution and show that it matters.
- In-backbone pruning and sparse peak selection already exist too. The paper says existing fully sparse detectors process all voxels without screening (abstract, Secs. 1 and 3.3), and that they lack local peak selection and rely on global top-k plus NMS (Secs. 2.3 and 3.4; Sec. 4.4 describes the VoxelNeXt head the same way). But VoxelNeXt [R2], which is in the paper's own comparison, already prunes voxels inside the backbone based on feature magnitude (Sec. 3.1, Fig. 6). It also replaces NMS with a sparse max pooling that only runs on non-empty voxels (Sec. 3.2, Fig. 7), and reports that this works on par with NMS (its Table 10). So calling fixed-window max pooling "inapplicable" on sparse voxels (Sec. 3.4) is too strong. DeS3 is still different (a supervised pruning rule instead of a magnitude rule, and predicted-box windows instead of fixed pooling kernels), but those narrower differences are what the paper should claim and test.
- End-to-end learned pruning exists too. Sec. 2.2 says GSP's "decoupled design limits joint optimization with the backbone". But GSP [R3] learns its pruning end-to-end from the task loss plus a sparsity regularizer; "decoupled" in that paper just means it's a plug-and-play layer. So this isn't really a limitation of GSP.
None of this makes DeS3 the same as any prior method. But it narrows the new part down to three things: the specific kernel/indexing design, the cross-stage loss weighting, and the predicted-box query selection. The paper needs direct evidence for each of them, which brings me to the next point.
2. The ablations don't isolate what's new.
- Pruning. Tables 3 and 4 vary the loss terms and the number of stages, but there's no "same backbone, no pruning" row, and none of these ablations report latency, MACs or memory, so we can't see how much compute the pruning actually saves. There's also no SFP-only row (CA and ESP both off), which is basically a plain foreground-weighted focal loss and the natural simple baseline. Since all variants use the same threshold (R=0.2), turning off SFP (P2, β=1) moves background removal from 83.10% to 95.71%, so P1 vs. P2 compares two very different operating points, not just two loss designs. And the two terms that are specific to this paper have small effects: removing CA costs 0.5 mAP / 0.3 NDS, and removing ESP costs 0.2 mAP with no change in NDS. That could be within run-to-run noise on nuScenes val. To show the proposed weighting really helps, I'd want to see it beat a simpler pruning loss with a tuned threshold on an accuracy-vs-cost curve, ideally over a few seeds.
- Head. The VoxelNeXt head is single-stage (Sec. 4.4 says so), while the proposed head adds a KNN cross-attention refinement stage. So the +0.7 mAP / +0.2 NDS in Table 5 could come from the extra refinement as much as from the box-based pooling. And the transplanted TransFusion head not converging only shows that this particular port didn't work; it doesn't show that dynamic pooling is the fix.
- Operator. Figure 8 only compares against SST and DSVT, which both work on padded or fixed-size groups. Since the claimed contribution is a padding-free kernel, the baseline that matters is an existing padding-free implementation (e.g., Stratified Transformer's) on the same inputs.
3. The efficiency claims don't match the measurements.
- Sec. 4.3 says DeS3 has "the lowest peak memory" (469.2 MB), but Figure 7 puts VoxelNeXt at about 250 MB. As written, that sentence isn't correct.
- The same figure shows DeS3 is the slowest of the five detectors: 173.04 ms vs. roughly 75–105 ms for the others. That's about 1.6× the latency of the next slowest one and more than 2× that of VoxelNeXt, so "slightly higher latency" undersells it. To be fair, the conclusion does admit "a certain reduction in inference speed", but the abstract ("an improved accuracy-efficiency trade-off") and Sec. 4.3 paint a rosier picture than Figure 7 does. What the data show is better accuracy and lower memory than most baselines (not VoxelNeXt), at a clearly higher latency. That's a legitimate trade-off, but it should be presented as one.
- Two more claims have no measurements behind them. The argument that compute scales with the number of valid points rather than the perception range (Sec. 5, App. B) has no end-to-end experiment; Figure 8 is operator-level and varies the number of sweeps, not the range. And "half the training epochs" (abstract, Sec. 4.2) doesn't by itself mean lower training cost, since each iteration may well be slower (inference already is). GPU-hours, or iteration counts plus per-iteration time, would settle this.
Minor. A few references list the wrong venue (all given as CVPR): [19] GSP is ICRA 2025, [32] AFDetV2 is AAAI 2022, and [36] PillarNet is ECCV 2022.
References
[R1] X. Lai et al. Stratified Transformer for 3D Point Cloud Segmentation. CVPR 2022.
[R2] Y. Chen, J. Liu, X. Zhang, X. Qi, J. Jia. VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking. CVPR 2023.
[R3] J. Li et al. Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning. ICRA 2025.