acceptodds
Under review as a conference paper at ICLR 2027

Beyond Point Anchors: Patch-Aware Rotary Encoding for Masked Point Cloud Representation Learning

Abstract

Point cloud patch tokens summarize local neighborhoods, yet anchor-only rotary encoding derives positional phases solely from normalized anchor coordinates. We introduce Patch-Aware Rotary Encoding (PaRE), which incorporates local patch geometry into rotary positional interactions through additive phase residuals derived from measured anchor-to-centroid offsets. The resulting relative phases combine anchor displacement with differences in centroid offsets, complementing the geometry encoded in token content. PaRE preserves the original anchors, fixed frequencies, and anchor-based normalization, and uses shared, bounded, frequency-wise residual coefficients that add only 32 learnable parameters in our configuration. With point-only pre-training on ShapeNet in a masked latent prediction framework, PaRE achieves strong performance across diverse downstream classification and segmentation tasks, including 90.77% overall accuracy on ScanObjectNN's PB_T50_RS setting. These results demonstrate the effectiveness of patch-aware rotary interactions for learning transferable representations through masked point cloud pre-training. Code is provided in the appendix.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
point cloud representation learning, self-supervised learning, masked point modeling, rotary positional encoding, patch-aware positional encoding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6YW1OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Patch-Aware Rotary Encoding (PaRE) for masked point cloud representation learning. The main idea is simple: a point cloud token summarizes a local patch, but standard rotary positional encoding only depends on the patch anchor and does not directly reflect the geometry within that patch. PaRE therefore adds the anchor-to-centroid offset as a residual to the rotary phase, allowing attention to capture both the relative positions of patch anchors and differences in local patch geometry. The design is lightweight, with only a small number of additional learnable parameters. Experiments on several classification and segmentation benchmarks show consistent gains over the anchor-only baseline and competitive results compared with recent methods.

Critical strengths
  1. The motivation is clear and well grounded. The paper identifies an intuitive mismatch between patch-level token representations and anchor-only positional encoding, and the proposed patch-aware formulation directly addresses this limitation.
  2. The solution is simple, technically clean, and effective. PaRE augments the standard anchor-based rotary phase with a residual derived from the anchor-to-centroid offset, allowing pairwise attention to capture both anchor displacement and differences in local patch geometry. This introduces only a very small number of additional parameters, while consistently improving over the anchor-only baseline and achieving competitive results across multiple downstream tasks.
Critical weaknesses

The role of the masked latent prediction framework is not well motivated. It is neither a novel component nor a design that directly follows from the motivation of patch-aware rotary encoding. Similar latent prediction objectives have already been widely explored, and it remains unclear why this particular framework is needed for PaRE or what additional contribution it brings to the proposed method.

Questions for authors

Could the authors clarify whether the masked latent prediction framework is a necessary component of PaRE? If not, please provide results using a standard masked point modeling setup, such as Point-MAE-style coordinate reconstruction, to enable a fairer comparison with Point-MAE.

2 comments
AuthorsOpenReview ↗

Could the authors clarify whether the masked latent prediction framework is a necessary component of PaRE? ...

We thank the reviewer for this comment. The masked latent prediction framework is not a necessary component of RaPE; rather, we adopt it as an effective pre-training framework that provides supervision for representation learning. RaPE operates on positional encoding within attention, serving a different function, and we do not claim the framework itself as a novel contribution. Similar approaches have been explored: point2vec[1] predicts teacher latent features of masked patches to learn semantic point cloud representations using global context, while Point-JEPA[2] predicts latent representations of masked regions to learn structural dependencies between regions. Both demonstrate the effectiveness of masked latent prediction for point cloud representation learning and downstream transfer.

To further validate RaPE, we integrate it into other frameworks, including Point-MAE and Point-M2AE, while fully retaining their original reconstruction objectives. Tab.1 and Tab.2 show that RaPE’s centroid-offset encoding substantially improves both frameworks under these established reconstruction objectives.

Table 1

Pre-trainingFine-tuningOBJ_BGOBJ_ONLYPB_T50_RS
Point-MAEPoint-MAE90.0288.2985.18
Point-MAE + PaREPoint-MAE+ PaRE93.8092.0889.31

Table 2

Pre-trainingFine-tuningOBJ_BGOBJ_ONLYPB_T50_RS
Point-M2AEPoint-M2AE91.2288.8186.43
Point-M2AE+ No PaREPoint-MA2AE+ No PaRE90.8889.5086.26
Point-M2AE + PaREPoint-M2AE+ PaRE91.9190.0286.81

[1] point2vec Self-Supervised Representation Learning on Point Clouds. GCPR 2023.

[2] Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud. WACV 2025.

Thank you for the clarification and the additional experiments. The results on Point-MAE and Point-M2AE help clarify the role of RaPE and its applicability to other pre-training frameworks. I appreciate the authors’ response and will maintain my original score.

Reviewer_LfVqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PaRE augments anchor-based rotary positional encoding with patch centroid-offset phase residuals, controlled by 32 shared, learnable coefficients. Evaluated within a masked latent prediction framework, it improves ScanObjectNN classification over controlled positional baselines and achieves competitive results on additional classification and segmentation tasks.

Critical strengths
  1. PaRE is simple and interpretable, introducing only 32 learnable parameters. The relative-phase formulation clearly explains how local centroid offsets complement anchor displacement.
  2. The controlled ScanObjectNN ablations hold the pre-training objective fixed and examine matched versus shuffled offsets, alternative positional parameterizations, and coefficient transfer. These provide useful evidence supporting the proposed design.
Critical weaknesses
  1. The broader transfer benefits are not sufficiently isolated. Anchor-only and No-RoPE comparisons are restricted to ScanObjectNN. Without these controls on ModelNet40, ShapeNetPart, and S3DIS, their results cannot be attributed specifically to PaRE. Moreover, the PB_T50_RS gain over No RoPE is 0.35 percentage points, and Tables 4–5 provide no variability estimates, leaving the robustness of smaller improvements unclear.

  2. The significance and scope of the proposed geometric correction need stronger justification. The observation that anchor-only phases ignore neighborhood changes is valid, but token content already encodes local geometry; this observation alone does not establish an important representational bottleneck. Anchor-only also underperforms No RoPE in all three ScanObjectNN settings. Although the paper acknowledges that centroid offsets capture only first-order information, it does not establish when this statistic is particularly useful or insufficient, or its trade-off against richer local geometric statistics.

Questions for authors
  1. Can you provide Anchor-only and No-RoPE results on the other transfer tasks under matched training settings? These would clarify whether PaRE’s benefits generalize beyond ScanObjectNN.
  2. Are the improvements in Tables 4–5 consistent across independent runs? Please report variability, particularly for the smaller gains on PB_T50_RS.
  3. What evidence identifies situations where centroid-offset phase residuals provide information that token content does not adequately exploit? A targeted analysis of patch asymmetry or sampling density would help clarify the mechanism and its limitations.
3 comments
AuthorsOpenReview ↗

Q1: Can you provide Anchor-only and No-RoPE results on the other transfer tasks under matched training settings? ...

We have provided comparisons of Anchor-only, No-RoPE, and RaPE on additional datasets (S3DIS and ModelNet40). As shown in Tab. 1, RaPE clearly outperforms both variants, demonstrating the effectiveness of its centroid offsets for understanding point cloud structure.

Table 1

PretrainingFinetuningModelNet40 OAShapeNetPart Cls. mIoUShapeNetPart Inst. mIoUS3DIS Area-5 mAccS3DIS Area-5 mIoU
No RoPEPaRE93.4 ± 0.1384.3 ± 0.0985.9 ± 0.0968.8 ± 0.5659.5 ± 0.34
Anchor-onlyPaRE93.4 ± 0.0684.5 ± 0.0986.1 ± 0.0669.6 ± 0.1860.6 ± 0.30
PaREPaR93.7 ± 0.0884.7 ± 0.1186.1 ± 0.0671.1 ± 0.2760.6 ± 0.27
AuthorsOpenReview ↗

Q2: Are the improvements in Tables 4–5 consistent across independent runs? Please report variability, particularly for the smaller gains on PB_T50_RS.

We thank the reviewer for this suggestion. In Tab. 1, Tab. 2 and Tab. 3, we report the variance of the results is more stable than No-RoPE and Anchor-only. This benefits from the centroid prior introduced by PaRE, which consistently incorporates local geometric information into relative positional encoding, allowing attention to consider both global positional relationships and local geometric differences. These results indicate that PaRE achieves more stable downstream performance while incorporating local geometric information.

Table 1

VariantSettingCoeff.OBJ_BGOBJ_ONLYPB_T50_RS
No RoPENone094.6 ± 0.1692.9 ± 0.3789.8 ± 0.50
Anchor-onlyNone094.6 ± 0.2192.7 ± 0.2190.0 ± 0.10
Fixed offsetFixed 0.25094.2 ± 0.3293.3 ± 0.3790.1 ± 0.17
Centroid rotaryFixed 1.0094.1 ± 0.2193.2 ± 0.3289.3 ± 0.13
Shuffled offsetShuffled3294.7 ± 0.2193.2 ± 0.3590.0 ± 0.09
PaRE (ours)Matched3295.1 ± 0.4193.7 ± 0.0890.7 ± 0.04

Table 2

VariantComponentCoeff.OBJ_BGOBJ_ONLYPB_T50_RS
Scalar additiveAdditive PE194.4 ± 0.1692.9 ± 0.3790.0 ± 0.37
Scalar rotaryLocal residual194.1 ± 0.2892.9 ± 0.4990.1 ± 0.28
Learned frequencyFull phase3294.3 ± 0.5793.0 ± 0.1690.4 ± 0.14
PaRE (ours)Local residual3295.1 ± 0.4193.7 ± 0.0890.7 ± 0.04

Table 3

IDPre-trainingFine-tuningInit.OBJ_BGOBJ_ONLYPB_T50_RS
C0Anchor-onlyAnchor-only–94.6 ± 0.2192.7 ± 0.2190.0 ± 0.10
C1Anchor-onlyPaREInit 0.2594.4 ± 0.2192.8 ± 0.1489.8 ± 0.16
C2Fixed offsetPaREInit 0.2594.4 ± 0.2193.1 ± 0.4989.8 ± 0.34
C3PaREPaREReset 0.2594.2 ± 0.1693.3 ± 0.2889.8 ± 0.14
C4PaREPaRECarry95.1 ± 0.4193.7 ± 0.0890.7 ± 0.04
AuthorsOpenReview ↗

Q3: What evidence identifies situations where centroid-offset phase residuals provide information that token content does not adequately exploit?...

Centroid offsets describe deviations of local point distributions from their anchors, allowing attention interactions to account for both anchor positions and local geometric support. Although token content may implicitly encode this information, anchor-only phases cannot reflect such changes when the anchors remain fixed. In contrast, PaRE explicitly incorporates offset differences into relative phases through learnable phase residuals to modulate attention, increasing its sensitivity to local geometric variations. This enables the model to adjust information aggregation between patches according to differences in local point distributions, thereby improving its representation of local geometric structures. In Tab. 4(a), PaRE outperforms Anchor-only by 1.38%/1.03%/0.76% on ScanObjectNN, while shuffling the correspondence between patches and offsets reduces accuracy by 1.21%/0.51%/0.90%. These results support the benefit of explicitly incorporating correctly matched patch centroid offsets into relative phases, helping the model exploit local geometric information more effectively.

Additionally, in Tab. 1, we conduct experiments on ScanObjectNN with different numbers of sampled points. RaPE consistently achieves substantial improvements in classification over Anchor-only across these settings. This suggests that centroid offsets provide effective local geometric cues across different sampling densities, with benefits extending beyond a particular point-count setting.

Table 1

Pre-trainingFine-tuningpoint num per patchOBJ-BGOBJ-ONLY
Anchor-onlyAnchor-only1694.3292.08
PaREPaRE1694.6692.43
Anchor-onlyAnchor-only32 (paper)94.3292.77
PaREPaRE32 (paper)95.7093.80
Reviewer_NxcvOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PaRE, which is a patch-aware rotary PE for point cloud representation learning. PaRE augments anchor-based rotary phases with centroid-offset residuals weighted by learnable frequency-wise coefficients.

Critical strengths

1. Well-motivated formulation. The method addresses the mismatch between neighborhood-based patch tokens and anchor-only rotary encoding by incorporating local patch geometry into the positional representation.

2. Superior performance in selected comparisons. PaRE outperforms both Anchor-only and No RoPE across all three evaluated ScanObjectNN settings, supporting the empirical benefit of the proposed encoding within the tested framework.

Critical weaknesses

1. Omission of a directly relevant baseline. Table 2(a) omits Point-JEPA [1], although the paper cites it elsewhere and reports identical results for several baselines shared by the two papers. Point-JEPA reports higher mean accuracy than PaRE in all four corresponding ModelNet40 few-shot settings. The authors should include these published results or explain any protocol differences that preclude a fair comparison, providing a more complete assessment of PaRE’s competitiveness.

2. Incremental contribution and insufficient justification of design choices.

  • PaRE introduces a relatively incremental modification of anchor-based RoPE through a centroid-offset residual and learned frequency-wise coefficients.
  • Table 4 supports the learned formulation, but fixed-coefficient variants do not consistently outperform No RoPE, suggesting that the benefit depends on how the offsets are weighted. The paper provides limited insight into why its particular parameterization is appropriate, especially the choice of coefficient bound and initialization.

[1] Ayumu Saito, Prachi Kudeshia, and Jiju Poovvancheri. Point-JEPA: A joint embedding predictive architecture for self-supervised learning on point cloud. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 7348–7357, 2025.

Questions for authors
  1. (Completeness of the baseline comparison) Could you include Point-JEPA’s published results in Table 2(a) and clarify whether its few-shot evaluation protocol is directly comparable to yours? If protocol differences prevent a fair comparison, could you identify these differences and explain their implications? Please also check for other directly relevant baselines with comparable published results and include them where appropriate, using existing results without additional training.

  2. (Analysis of the learned coefficient) Could you analyze the role of the learned λk\lambda_k in model behavior before and after fine-tuning, and help explain their advantage over fixed offset weighting? In particular, does learning mainly adjust the overall scale of the local-offset contribution, or does it produce more complex, frequency-dependent effects? As this component appears central to the reported performance gains, such an analysis would strengthen the paper and clarify why the proposed parameterization is effective.

  3. In Figure 3, qualitative comparison of other downstream task (e.g., segmentation, not classification) would be more helpful, not just showcase the examples of output or classification input cases. Optionally, if there are some cases of qualitative comparison that would be helpful to understand of the effect of PaRE, can authors provide the such comparisons?

13 comments
AuthorsOpenReview ↗

Q1: Completeness of the baseline comparison

We carefully reviewed the Point-JEPA paper and confirmed that it uses the same pre-training setup in terms of the 3D backbone (a 12-layer Transformer) and dataset (ShapeNet). We acknowledge that our few-shot results on ModelNet40 are slightly lower than those of Point-JEPA (see Tab. 1). However, our method substantially outperforms Point-JEPA on ScanObjectNN (see Tab. 2). This may be because occlusions, missing regions, and non-uniform sampling in real scans produce more irregular local geometric support. Centroid offsets explicitly capture these variations in local point distributions relative to the anchors, providing additional geometric cues for attention. In contrast, synthetic point clouds in ModelNet40 are more regularly sampled, so the additional benefits of centroid offsets may be relatively limited. We thank the reviewer for this valuable suggestion. In future work, we will further improve the effectiveness of centroid offsets in RaPE on regularly sampled point clouds.

Table 1

Method5-way5-way10-way10-way
10-shot20-shot10-shot20-shot
Point-JEPA97.4±2.299.2±0.895.0±3.696.4±2.7
PaRE (ours)97.0 ± 2.098.9 ± 0.593.6 ± 3.695.6 ± 2.6

Table 2

MethodOBJ-BGOBJ-ONLYPB-T50-RS
Point-JEPA93.291.987.6
PaRE (ours)95.7093.8090.77
AuthorsOpenReview ↗

Q2: Analysis of the learned coefficient

λk\lambda_k is a learnable coefficient that controls the influence of local geometry on positional encoding in the k-th rotary channel. Compared with a fixed coefficient λk=1\lambda_k=1, learnable λk\lambda_k allows different rotary channels to adaptively adjust their sensitivity to local geometry, enabling attention to capture both global positional relationships and local geometric differences between patches. However, excessively large coefficients amplify geometry-induced phase changes, potentially causing different local geometries to produce similar encodings and weakening attention’s ability to distinguish them. Therefore, we use moderate initialization and a bounded constraint to control the magnitude of local phase corrections (0.25, 0.5).

As shown in Tab.1, repeating pre-training and downstream fine-tuning with λk=1\lambda_k=1 leads to a slight performance drop. This suggests that learnable λk=1\lambda_k=1 helps different rotary channels adaptively regulate the contribution of local geometry, better coordinating the representation of global positional relationships and local geometric differences.

Table 1

Pre-trainingFine-tuningInitOBJ_BG OA(%)OBJ_ONLY OA(%)PB_T50_RS(HARDEST)OA(%)
Fixed λₖ=0.25Fixed λₖ=0.250.2594.6693.8089.94
Fixed λₖ=1Fixed λₖ=1193.8093.4689.45
Fixed λₖ=0.25Learning λₖ0.2594.1592.7789.35
Fixed λₖ=1Learning λₖ0.2594.4992.7789.31
Learning λₖLearning λₖ0.2595.7093.8090.77
AuthorsOpenReview ↗

Q3: Aualitative comparison of other downstream task

We provide more detailed qualitative comparisons in the main paper, using t-SNE plots to visualize classification results and semantic color maps to visualize segmentation results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.