acceptodds
Withdrawn from ICLR 2027

VWS-PFE: Modeling Vertically Weakly Ordered Structures for Pillar-Based 3D Object Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pillar-based LiDAR detectors compress the points within each vertical column into a compact feature for efficient bird's-eye-view detection. Conventional Pillar Feature Network (PFN) encoders apply shared point-wise mappings and symmetric pooling, leaving cross-point vertical relations implicit. Gravity, ground constraints, object geometry, and LiDAR sampling provide a natural organizing direction, while the observed returns remain sparse and irregularly spaced. We introduce the Vertical Weakly-Ordered Structure-Aware Pillar Feature Encoder (VWS-PFE), which realizes height-induced weak ordering through a sorted traversal and relative height-rank descriptors. Context extracted from these features is restored to each source point and reused for feature recalibration before the shared PFN projection and aggregation after projection. This connects how points are represented with how they contribute to the pillar feature, without height bins or auxiliary intra-pillar grids. With downstream detectors fixed, VWS-PFE improves mean moderate 3D AP on KITTI from 58.66 to 61.86. On nuScenes, it improves PointPillars by 4.27 NDS and 5.22 mAP, and CenterPoint-Pillar by 0.90 NDS and 1.40 mAP. KITTI ablations show benefits from contextual encoding and a further 1.43-point gain from ascending-height over random traversal. Conv1D, Transformer, and Mamba provide different accuracy–latency trade-offs within the same encoder design.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Object Detection, LiDAR, Pillar Feature Encoding, Point Cloud Representation, Weak Ordering, Inductive Bias

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_M5KvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VWS-PFE, a pillar feature encoder that models vertical relationships through height sorting and relative height-rank descriptors. The resulting context guides both feature recalibration and weighted aggregation. Experiments on KITTI and nuScenes show improvements over the original encoders in PointPillars and CenterPoint-Pillar, while ablations examine context operators, traversal order, and pooling strategies.

Critical strengths
  1. The motivation is interesting and intuitive: vertical relationships provide a meaningful structural cue for pillar feature encoding.

  2. The paper is clearly written and easy to follow.

Critical weaknesses
  1. The comparison results primarily demonstrate improvements over the original PFN. While these controlled comparisons validate the effectiveness of the modules, they are insufficient to establish the method’s competitiveness relative to existing state-of-the-art work; the current version does not meet the ICLR benchmark: First, the paper lacks direct comparisons with relevant methods such as PillarHist and FG-PFE, and its advantages over existing highly perceptual encoders remain unclear. Second, while the significant improvements over the baseline method are exciting, the paper currently lacks direct comparisons with SOTA methods in terms of both speed and accuracy.

  2. The trade-off between accuracy and latency lacks sufficient justification. On the KITTI dataset, the Mamba variant increased the average AP for medium-difficulty tasks from 58.66 to 61.86, but latency increased from 5.68 milliseconds to 13.37 milliseconds, and no comparison was made under similar computational budget conditions.

  3. The figures lack readability. The labels and equations in Figure 2 are too small, and in Figure 3, the point cloud has low contrast, the bounding boxes are small, and there are no close-ups, which hinders the assessment of the method’s details and qualitative improvements.

Questions for authors

1 How does VWS-PFE compare with PillarHist or FG-PFE under the same downstream detector and training protocol?

2 Do the improvements persist against simpler PFN enhancements with comparable inference latency?

Reviewer_HsGmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes VWS-PFE, which establishes a weakly-ordered traversal along the vertical dimension within each pillar. By utilizing relative height, rank, and interval descriptors to generate point-level context, the method applies this context both for feature recalibration prior to PFN projection and for weighted aggregation following it. This approach effectively addresses the issue of insufficient utilization of vertical structures during the pillar compression stage in conventional PFNs.

Critical strengths

(1) Addressing the long-standing bottleneck in the PointPillars family (the premature compression of points into pillars and the insufficient utilization of vertical relationships) is important.

(2) The authors keep the preprocessing, pillarization, BEV backbone, detection head, and training pipeline constant, replacing only the PFN. This setup allows for a clear attribution of performance changes to the encoder.

Critical weaknesses

(1) The lack of a comparison with the most direct PFE-related competitors makes it impossible to determine the relative contribution. This paper explicitly discusses PillarHist, FG-PFE, SVP, SwiftPillars, AA-Pillars, and GeoFormer. However, Tables 1–2 compare only the vanilla PFN against the three context operators proposed in this paper.

(2) The paper claims that "continuous relative heights and discrete ranks are complementary" and that "adjacency gaps can characterize irregular sampling." However, the experiments only include a full "Without RHRE" ablation. This paper does not validate individual variants such as only ranks, relative heights, gaps, spans, or raw heights. Consequently, the current ablation study fails to demonstrate that each of the five descriptors is necessary, nor does it prove that ranks and continuous gaps are truly complementary.

(3) The main results and ablation studies report only Moderate 3D AP_{R40}. This paper does not include results for Easy/Hard categories, BEV AP, or results of the KITTI test-server.

(4) The performance still lags significantly behind current SOTA methods. Authors need to provide more experiments with more advanced methods (e.g., PillarNeXt [1], HEDNet [2])

(5) The visualization presents only a single object within a specific scene, which is insufficient to verify the effectiveness of the proposed method.

[1] PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Clouds

[2] HEDNet: A Hierarchical Encoder-Decoder Network for 3D Object Detection in Point Clouds

Questions for authors

[1] The authors should conduct a thorough comparison against other pillar-based operators, perform ablation studies, and include experimental comparisons with more advanced methods.

Reviewer_khyEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VWS-PFE, a replacement for the pillar feature encoder in LiDAR-based 3D object detection. The method sorts points within each pillar by height and augments them with descriptors of relative height, rank, neighboring height gaps, and occupied height span. A contextual operator produces point-aligned features that are reused for input-channel recalibration before the shared PFN projection and for weighted aggregation after projection. The weighted branch is combined with max pooling. The paper evaluates Conv1D, Transformer, and Mamba implementations and reports improvements on KITTI PointPillars and nuScenes PointPillars/CenterPoint-Pillar, together with component, traversal, and pooling ablations.

Critical strengths
  1. A focused intervention with improvements across multiple detector configurations. According to Section 4.1, preprocessing, downstream architecture, training schedule, and post-processing are held fixed within each baseline–replacement pair. The reported gains are meaningful: +3.20 mean moderate APR40_{R40} on KITTI, +5.22 mAP/+4.27 NDS for nuScenes PointPillars, and +1.40 mAP/+0.90 NDS for CenterPoint-Pillar. These comparisons provide useful evidence that improving point-to-pillar encoding can benefit different detector configurations.

  2. Informative ablations beyond the overall baseline comparison. Table 3 compares traversal orders while retaining height descriptors and separately examines pooling-weight sources. Ascending-height traversal improves over random traversal by 1.43 mean moderate APR40_{R40} points. Pooling weights derived from the shared context improve over max-only pooling by 1.54 points and over raw-input-derived weights by 0.97 points. These experiments support the usefulness of height-organized traversal and contextual aggregation within the evaluated KITTI configuration.

Critical weaknesses
  1. The empirical contribution relative to closely related encoders is insufficiently established. Tables 1 and 2 compare VWS-PFE with the original PFN, but not with existing height-aware or attention-based encoder alternatives. PillarHist and FG-PFE are discussed in the related work, yet neither is included in the experimental comparisons. PVTransformer, which studies attention-based point-to-voxel aggregation, is also relevant but is not discussed. The current experiments therefore establish improvement over PFN without showing how the proposed design compares with representative alternatives addressing similar limitations. This matters particularly for the accuracy–latency trade-off: on KITTI, the Mamba variant increases detector latency from 5.68 to 13.37 ms, approximately 2.35×, despite adding only about 0.72% to the detector’s parameters. The paper acknowledges this overhead, but comparisons under a common detector, training recipe, and timing protocol are needed to assess whether the trade-off is competitive.

  2. The ablations support the use of context at both stages, but do not isolate the advantage of sharing that context. Removing SGR or SRP shows that each operation contributes within the proposed architecture, and the pooling-weight study supports deriving aggregation weights from contextual features. However, these experiments do not establish whether reusing one context offers an advantage over separately learned contexts with a comparable parameter or computation budget. This distinction matters because the paper presents the coupling of feature recalibration and aggregation through a common context as a central design contribution. The other ablations also have limits: removing VCS changes both cross-point interaction and capacity, while removing RHRE evaluates the five descriptors as a group. These are useful component tests, but they do not fully separate generic contextual capacity from the specific structural design. Finally, the mechanistic ablations are confined to KITTI, and the absence of repeated-run statistics makes the stability of smaller differences difficult to assess.

Questions for authors
  1. Could the authors provide controlled comparisons with representative height-aware and attention-based encoders, such as PillarHist or FG-PFE and a PVTransformer-style aggregation module, using the same detector, training recipe, and timing protocol?

  2. How does the shared-context design compare with separately learned contexts for recalibration and aggregation under a comparable parameter or computation budget?

  3. In every traversal variant, are RHRE descriptors first computed in height order and then transported with their source points? Is the random permutation fixed or resampled during training and evaluation?

  4. Appendix C specifies the local training and checkpoint procedures, but Section 4.1 describes CenterPoint-Pillar through its corresponding baseline protocol. Could the authors identify the exact configuration and clarify whether both models were retrained under that protocol? Could they also report encoder-only latency and explain the different total-latency increments of 5.64 ms for PointPillars and 1.66 ms for CenterPoint-Pillar on nuScenes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.