acceptodds
Withdrawn from ICLR 2027

SV-ROPE: TOKEN PROFILES OF SPATIAL VARIATION FOR REMOTE SENSING FOUNDATION MODELS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Remote-sensing foundation models (RSFMs) acquire transferable spatial priors from large-scale Earth observation (EO) imagery, which are commonly inherited during downstream parameter-efficient fine-tuning (PEFT). Yet, for Transformer-based RSFMs, the spatial meaning of the same index-space token relation can vary substantially across observations, leaving inherited priors poorly matched to target-specific spatial variation. This raises a central question: how can inherited priors be made responsive to target-specific spatial variation within a PEFT adaptation framework? We therefore introduce a spatial variation-aware view of RSFM adaptation and propose SV-RoPE (Spatial-Variation-Aware Rotary Position Embedding), a PEFT method that associates each visual token with a spatial-variation profile (SV-Profile). Each profile summarizes how the token's feature agreement with neighboring tokens changes across increasing spatial extents. Leveraging the coupled displacement–frequency structure of RoPE, SV-RoPE encodes spatial variation directly into a unified rotary phase, jointly parameterized by Relation Coordinate Calibration (RCC) for initial ground-extent-calibrated displacement and Relation Frequency Calibration (RFC) for SV-Profile-conditioned frequency modulation. SV-RoPE is realized through a lightweight attention-parallel branch whose output is added to the attention of the frozen backbone. Across diverse RSFMs, sensors, resolutions, and downstream tasks, SV-RoPE consistently improves adaptation performance, demonstrating the value of explicitly modeling spatial variation in downstream EO adaptation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
remote sensing, adaptation, position embedding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_xLxGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The proposed method (SV-RoPE) is an approach to finetune vision transformers in remote sensing by providing a context-aware positional encoding that adapts to the image's true resolution, and how heterogeneous or homogeneous different parts of the image are. The method claims to be more parameter-efficient and provide better adaptation compared to baselines.

Critical strengths
  1. The paper is well motivated. It clearly distinguishes physical scale from local structure: even at the same resolution, a uniform forest and a dense urban area present different spatial relationships. Adapting positional encoding to these differences is a reasonable direction.
  2. The evaluation is broad and the results are encouraging. Across three pretrained backbones and all downstream datasets, SV-RoPE achieves the highest overall average for each backbone. The component ablations also provide useful results that both physical-scale calibration and local-profile conditioning contribute meaningfully.
Critical weaknesses

I identify two main weaknesses in the work relating to the proposed methodology. Additionally, I cannot find any appendix in the paper either as additional pages in the main text or a separate supplementary material attachment. Appendix sections, however, are referenced multiple times in the main text.

My primary concern is a potential mismatch between the motivation and the implemented method. The authors propose a positional encoding that adapts to the local structure within an image, but their formulation also changes the interaction depending on where that structure exists within the image crop. For example, an image with a house next to a forest receives different positional treatment when moved from one part of the crop to another. The authors acknowledge that their formulation is not translation-invariant, but I do not think acknowledging this explains why the additional dependence is desirable. Satellite imagery is translation invariant, and I believe a positional encoding designed for satellite imagery should take this into account. Testing consistency across crop placements would directly address my concern.

Lastly, I find the writing to be very complicated and difficult to understand on a first read. For instance, the abstract's mention of the proposed method is incredibly dense.

Leveraging the coupled displacement–frequency structure of RoPE, SV-RoPE encodes spatial variation directly into a unified rotary phase, jointly parameterized by Relation Coordinate Calibration (RCC) for initial ground-extent-calibrated displacement and Relation Frequency Calibration (RFC) for SV-Profile-conditioned frequency modulation.

Some simplification of writing would help the readability of the paper.

I also find comparisons to baselines difficult to interpret in the paper. If the proposed adaptation techniques needs an attention-parallel branch to every block of the transformer, I believe the additional parameter and flop count, as well as the parameter and flop count of baselines could be reported. This will help clarify is the proposed SV-RoPE method is indeed parameter efficient compared to baselines.

My inclination is a weak reject given the current draft. However, if the authors can challenge and respond to my critiques surrounding the methodology, and provide a clean appendix with the experimental details, computational cost comparisons, and simplify the writing, I would be willing to increase my score.

Questions for authors
  1. I find the methods comparing with RoPE and RCC to be helpful and interesting. Could the authors comment on the relationship between the LoRA + RoPE ablation and the attention-parallel architecture? What is held constant here?
  2. Would performance improvements hold at similar parameter and computational budgets?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xLxG

Dear Reviewer xLxG,

We are grateful for your careful reading and constructive feedback. We address the methodology in Q1–Q4 below with new experiments, and Q5 lists the concrete contents of the appendix and the specific writing changes we will make.

1. Translation invariance and crop-placement consistency (Q1).

We agree that dependence on crop placement is undesirable and unintended. Our goal is to assign distinct positional sensitivities to tokens with heterogeneous local spatial variation; origin dependence is merely a side effect of token-wise phase scaling, which our design minimizes rather than exploits.

In the reviewer's example, the scales κ\kappa for both the house and forest tokens are computed over symmetric, content-tracking neighborhoods and thus remain invariant as the pair shifts within the crop. Only their coupling with absolute coordinates changes: a spatial shift cc alters the relative phase by θmc(κv,m−κu,m)\theta_m c (\kappa_{v,m} - \kappa_{u,m}), proportional to the scale difference between the two tokens. Because neighborhood averaging (Eq. 14) keeps adjacent scales close, this residual term stays small for the local token pairs.

Moreover, since the frozen DOFA backbone relies on absolute position embeddings, the LoRA baseline is not translation-invariant either; the key question is whether SV-RoPE introduces additional sensitivity. We further evaluated the spatial sensitivity in three settings:

  • Coordinate shift. Shift all token coordinates by 1–8 patches with the image fixed.
  • Crop placement. Evaluate ten 448 × 448 crops at different offsets per 512 × 512 image.
  • Tiled inference. Predict four 256 × 256 tiles per segmentation image and stitch the predictions.

As shown in Table 1, coordinate shifts leave both scores essentially unchanged. Under crop placement, SV-RoPE exhibits sensitivity comparable to LoRA without degrading accuracy or mIoU. Tiling drops several mIoU points on both methods, so that degradation is a tiling effect rather than a phase effect. We will state the limitation and this analysis in Section 3.3.3.

Table 1. Spatial consistency. Parentheses show the score change from the original setting.

SettingMethodCLSSEG
OriginalLoRA82.9363.76
SV-RoPE84.3366.43
Coordinate shiftLoRA——
SV-RoPE84.48 (+0.15%)66.80 (+0.37%)
Crop placementLoRA81.68 (−1.25%)62.02 (−1.74%)
SV-RoPE84.43 (+0.10%)65.86 (−0.57%)
Tiled inferenceLoRA—57.41 (−6.35%)
SV-RoPE—59.26 (−7.17%)

2. Baseline comparability and efficiency (Q2).

SV-RoPE follows the same downstream PEFT setting as the baselines. We freeze the pretrained backbone and train a small set of additional parameters. SV-RoPE merely adapts positional interactions to ground scale and local spatial variation.

(i) Parameters. The branch adds 0.30M low-rank parameters over rank-8 LoRA; the frequency mapper adds <0.01M.

(ii) Compute. On RTX A6000, training FLOPs rise by about 10% for classification and 30% for segmentation and stay below SLR. Peak memory and inference latency are higher than both baselines because the branch computes a second attention map in each block. SV-RoPE is therefore parameter efficient; its cost is in compute and memory, and we will state this in the revision (Table 2).

Table 2. Trainable parameters and computation costs.

MethodAdapter params (M)CLS FLOPs/stepCLS memory (GiB)CLS latency (ms)SEG FLOPs/stepSEG memory (GiB)SEG latency (ms)
LoRA (rank 8)1.032.342.9026.18.687.9130.8
SLR1.165.035.7928.615.5714.5132.4
SV-RoPE1.332.577.3036.211.2519.0041.0
AuthorsOpenReview ↗

Response to Reviewer xLxG


3. Ablation setup (Q3).

All ablation variants in manuscript Table 2 share the same training schedule, differ only in the components listed in Table 3. Every variant below the LoRA row adds the same attention-parallel branch, and every variant from the RCC row onward keeps RCC. The three conditioned variants (Global scaling, DAPE, and SV-RoPE) share the same frequency mapper and differ only in its input. Each row therefore isolates one change at constant capacity.

We will revise manuscript Table 2 to make these settings explicit.

Table 3. Configuration of the ablation variants in manuscript Tab. 2.

VariantParallel branchRCCConditioning inputMapper outputTrainable params (M)
LoRA––––1.03
+ RoPE branch✓–––1.33
+ RCC✓✓––1.33
Global scaling✓✓SV-Profile averaged over tokensone factor per band, shared across the image1.33
DAPE [1]✓✓token embeddingone factor per band per token1.34
SV-RoPE✓✓SV-Profile per tokenone factor per band per token1.33

[1] DAPE: Data-Adaptive Positional Encoding for Length Extrapolation. In NeurIPS 2024.


4. Adapter parameter budget (Q4).

Doubling the LoRA rank from 8 to 16 doubles its parameters but barely moves its scores, and rank 16 still trails SV-RoPE with fewer parameters (Table 4). The gap is therefore not a matter of adapter capacity.

Table 4. Performance at different adapter parameter budgets.

MethodTrainable adapter parametersCLSSEG
LoRA (rank 8)1.0382.9363.76
LoRA (rank 9)1.16M83.2865.38
LoRA (rank 16)2.06M83.3565.58
SV-RoPE1.33M84.3366.43

5. Writing clarity and the missing appendix (Q5).

(i) Appendix. Thank you for flagging that it was missing. The revised appendix will contain:

  • Dataset descriptions: source, resolution, split sizes, and GSD for each of the 12 datasets.
  • Implementation details: parallel-branch design, SV-Profile scales, optimizer, and schedule.
  • Computation cost comparison (as already presented in Q2 Table 2)

(ii) Writing. We will make three specific changes. First, we will refine the presentation to make it more readable; Second, section 3 will open with a one-paragraph overview that states the two problems (scale mismatch across GSDs and uniform positional frequencies across heterogeneous content) and maps RCC and RFC to them before any equations; Third, we will correct Eq. (8), unify the notation for the modulation factors.


We thank you again for your feedback. We would be happy to provide any further clarification during the discussion period.

Best regards,

The Authors

Reviewer_xLxGOpenReview ↗

Response by Reviewer xLxG to Authors

Thank you for the additional experiments and clarifications. I remain at weak reject given the memory and latency costs relative to the performance gains. More importantly, the authors' rebuttal revisions still do not contain an appendix a month after the submission deadline despite references made to it in the main text. Creation of an appendix is a substantial revision. The paper is thus incomplete, and I encourage the authors to finish the manuscript and resubmit.

Reviewer_6dS9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SV-RoPE adapts frozen remote-sensing foundation models using rotary phases conditioned on ground sampling distance and token-wise neighborhood-similarity profiles. An attention-parallel adapter implements the calibration, producing consistent average improvements across three backbones and twelve GeoBench tasks.

Critical strengths
  1. Combining physical scale with local feature variation gives positional adaptation a clear remote-sensing motivation.
  2. Evaluation spans classification, segmentation, and backbones with different native positional encodings.
  3. Component ablations and resolution-shift experiments provide useful evidence for the proposed conditioning signals.
Critical weaknesses
  1. Equation (8) makes relative phases depend on the coordinate origin when token modulations differ, but translation and crop robustness are not evaluated.
  2. Fixing LoRA-based baselines to rank eight does not establish fairness without matched trainable-parameter and computation budgets.
  3. Reported improvements lack repeated-run uncertainty estimates, making small task-level differences difficult to assess.
  4. The related-work survey insufficiently covers visual and remote-sensing PEFT [1-10].

[1] TEA: A Training-Efficient Adapting Framework for Tuning Foundation Models in Remote Sensing. IEEE Transactions on Geoscience and Remote Sensing. 2024.

[2] Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Model. ICLR. 2024.

[3] Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition. CVPR. 2025.

[4] 5%>100%: Breaking Performance Shackles of Full Fine-Tuning on Visual Recognition Tasks. CVPR. 2025.

[5] Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategy. ICCV. 2025.

[6] AiRs: Adapter in Remote Sensing for Parameter-Efficient Transfer Learning. IEEE Transactions on Geoscience and Remote Sensing. 2024.

[7] Stronger Fewer & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation. CVPR. 2024.

[8] Sensitivity-Aware Efficient Fine-Tuning via Compact Dynamic-Rank Adaptation. CVPR. 2025.

[9] Parameter-efficient is not Sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions. ACM Multimedia. 2024.

[10] V-PETL Bench: A Unified Visual Parameter-Efficient Transfer Learning Benchmark. NeurIPS. 2024.

Questions for authors
2 comments
AuthorsOpenReview ↗

Response to Reviewer 6dS9

Dear Reviewer 6dS9,

We are grateful for your constructive feedback. We address each point below and will incorporate all additions into the revision.


1. Equation (8) and translation robustness (Q1).

We will correct Eq. (8) to use ψv−ψu\psi _v - \psi _u, matching Eq. (4). The corrected form still depends on the coordinate origin when two tokens have different scales, as the reviewer notes: a uniform coordinate shift changes the relative phase in proportion to the scale difference between the two tokens.
However, the symmetric neighborhood averaging in Eq. (14) keeps the scales of nearby tokens close, so this change stays small for the local token pairs.

We measured the remaining sensitivity on DOFA in three settings:

  • Coordinate shift. Shift all token coordinates by 1–8 patches with the image fixed.
  • Crop placement. Evaluate ten 448 × 448 crops at different offsets per 512 × 512 image.
  • Tiled inference. Predict four 256 × 256 tiles per segmentation image and stitch the predictions.

As shown in Table 1, coordinate shifts leave both scores essentially unchanged. Under crop placement, SV-RoPE exhibits sensitivity comparable to LoRA without degrading accuracy or mIoU. Tiling costs both methods several mIoU points, so this degradation is mainly a tiling effect rather than a phase effect. We will add this analysis in revision.

Table 1. Spatial consistency. Parentheses show the score change from the original setting.

SettingMethodCLSSEG
OriginalLoRA82.9363.76
SV-RoPE84.3366.43
Coordinate shiftLoRA——
SV-RoPE84.48 (+0.15%)66.80 (+0.37%)
Crop placementLoRA81.68 (−1.25%)62.02 (−1.74%)
SV-RoPE84.43 (+0.10%)65.86 (−0.57%)
Tiled inferenceLoRA—57.41 (−6.35%)
SV-RoPE—59.26 (−7.17%)

2. Parameter budget and efficiency (Q2).

(i) Matched parameters. Raising the LoRA rank from 8 to 16 roughly doubles its adapter parameters but barely moves its scores, and rank 16 still trails SV-RoPE with fewer parameters (Table 2). The gap is therefore not a matter of adapter capacity.

Table 2. Performance at different adapter parameter budgets.

MethodTrainable adapter parametersCLSSEG
LoRA (rank 8)1.0382.9363.76
LoRA (rank 9)1.16M83.2865.38
LoRA (rank 16)2.06M83.3565.58
SV-RoPE1.33M84.3366.43

(ii) Resource costs. On an RTX A6000, SV-RoPE adds 0.30M adapter parameters compared with LoRA. Training FLOPs rise by about 10% for classification and 30% for segmentation and remain below SLR in both settings. Peak memory and latency are higher than for both baselines.

Table 3. Training and inference costs.

MethodAdapter params (M)CLS FLOPs/stepCLS memory (GiB)CLS latency (ms)SEG FLOPs/stepSEG memory (GiB)SEG latency (ms)
LoRA (rank 8)1.032.342.9026.18.687.9130.8
SLR1.165.035.7928.615.5714.5132.4
SV-RoPE1.332.577.3036.211.2519.0041.0
AuthorsOpenReview ↗

Response to Reviewer 6dS9

3. Uncertainty across seeds (Q3).

We repeated LoRA and SV-RoPE with three seeds on DOFA (Table 4). On both averages, the worst SV-RoPE seed is above the best LoRA seed.

Table 4. DOFA results across three seeds.

MethodSeed / summaryBEBKESFNPVSSCWNTCTCPPSSCAvg. CLSAvg. SEG
LoRA088.2398.3098.1354.7997.9060.2460.6755.4975.1460.8694.6435.3782.9363.76
LoRA188.4098.5098.4155.0998.1062.5867.9556.7775.5660.8594.6233.4183.5164.86
LoRA288.8797.9098.2251.9597.9061.5664.4756.2375.6661.5694.8734.1482.7364.49
LoRAMean ± SD88.50 ± 0.3398.23 ± 0.3198.25 ± 0.1453.94 ± 1.7397.97 ± 0.1261.46 ± 1.1764.36 ± 3.6456.16 ± 0.6475.45 ± 0.2861.09 ± 0.4194.71 ± 0.1434.31 ± 0.9983.06 ± 0.4164.35 ± 0.60
SV-RoPE089.6898.5098.3157.1998.8063.4668.5760.3475.6463.6795.0135.6884.3266.49
SV-RoPE188.8898.2098.2056.2998.0063.5868.1960.6775.6065.3695.0035.8483.8666.78
SV-RoPE289.4698.4098.0454.6498.6062.9868.3860.8375.6665.5994.9835.8383.6966.88
SV-RoPEMean ± SD89.34 ± 0.4198.37 ± 0.1598.18 ± 0.1456.04 ± 1.2998.47 ± 0.4263.34 ± 0.3268.38 ± 0.1960.61 ± 0.2575.63 ± 0.0364.87 ± 1.0595.00 ± 0.0235.78 ± 0.0983.96 ± 0.3366.71 ± 0.20

4. Coverage of recent PEFT methods (Q4).

Thank you for the references; We will expand the related work to cover recent visual and remote-sensing PEFT methods and explain how they relate to SV-RoPE.


Best regards, Authors

Reviewer_TBbbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SV-RoPE is a parameter-efficient fine-tuning method for remote sensing foundation models that adapts positional encoding to ground sampling distance (GSD) and local spatial variations. It constructs a multi-scale SV-Profile based on feature similarities between neighboring tokens, uses Relation Coordinate Calibration (RCC) to adjust spatial positions according to ground-scale information, and applies Relation Frequency Calibration (RFC) to adapt positional encoding frequencies for individual tokens. The method introduces an additional attention branch while keeping the pretrained backbone frozen. Experiments on three remote sensing foundation models and twelve downstream datasets show consistent performance improvements over existing PEFT baselines, particularly in semantic segmentation. Ablation studies further suggest that token-wise frequency modulation is beneficial, although additional experiments are needed to fully validate the proposed mechanism and its computational efficiency.

Critical strengths

(i) The research problem is well-defined, and the method is purposefully designed. The authors introduce RCC for ground-scale calibration and RFC for local spatial-variation-aware frequency modulation, providing a clear technical rationale.

(ii) The experiments cover multiple models and downstream tasks. Evaluations across three remote sensing foundation models and twelve datasets, together with Global scaling ablations and frequency-response analyses, provide useful evidence for the proposed method.

(iii) The method shows potential transferability across backbones. SV-RoPE improves performance across models with different positional encoding mechanisms, suggesting that it may be applicable beyond a single architecture.

Critical weaknesses

(i) The contribution of the additional attention branch is not sufficiently isolated. It is unclear whether the performance gains come from the proposed spatial-variation-aware mechanism or from the additional parameters and representational capacity of the new attention branch. The authors should clarify this distinction through controlled comparisons with structurally and parameter-wise matched baselines.

(ii) Token-wise frequency modulation may break translation invariance. According to the formulation, when different tokens have different modulation coefficients, shifting the coordinate origin can change their relative phases even when the image content and relative displacement remain unchanged. The authors should explain whether this behavior is intentional and investigate its impact on cropping consistency and tiled inference.

(iii) It is unclear whether the SV-Profile genuinely captures spatial variation. The descriptor is constructed from feature cosine similarities across different neighborhoods rather than directly measuring geometric structure or spatial variation. The authors should explain why this similarity-based descriptor is a valid proxy and evaluate it against simpler local-similarity measures or alternative spatial descriptors.

(Ⅳ) The learned behavior of RFC has not been sufficiently validated. The frequency-sweep experiments suggest that different SV groups may prefer different frequencies, but they do not establish that the trained model actually learns this correspondence. The authors should visualize the learned modulation coefficients across SV groups and conduct interventions, such as shuffling SV-Profiles or replacing token-wise modulation with global modulation, to verify the mechanism.

(Ⅴ) The independent contribution of RCC and its cross-scale generalization remain unclear. Image resampling can simultaneously alter image details and feature distributions, making it difficult to isolate the effect of scale calibration. The authors should clarify how GSD, image dimensions, and token stride are coordinated and provide more controlled experiments to establish RCC's actual contribution.

Questions for authors

the same question wrote in Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer TBbb

Dear Reviewer TBbb,

We are grateful for your detailed and constructive review. We address each point below and will incorporate all additions into the revision.


1. Adapter capacity and fair comparison (Q1).

We address the matched parameter capacity and architecture issues separately. Table 1 lists all variants with their parameter counts; the branch rows are taken from manuscript Table 2.

(i) Parameter capacity. We compared SV-RoPE with LoRA at higher ranks (9 and 16). SV-RoPE outperforms both, including rank-16 LoRA, which uses more parameters.

(ii) Parallel branch. We also applied SV modulation directly inside the attention blocks and trained without the parallel branch. This disrupts the pretrained positional prior: the variant uses more parameters than SV-RoPE yet trails it by 1.12 accuracy points and 0.63 mIoU points.

Table 1. Performance at different adapter parameter budgets.

MethodParallel branchSV modulationTrainable params (M)CLSSEG
LoRA (rank 8)––1.0382.9363.76
LoRA (rank 9)––1.1683.2865.38
LoRA (rank 16)––2.0683.3565.58
SV-RoPE w/o branch–✓1.4683.2165.80
SV-RoPE✓✓1.3384.3366.43

2. Sensitivity to crop placement (Q2).

The coordinate dependence is a side effect, not a design goal. Token-specific modulation is introduced so that tokens with heterogeneous spatial variation receive different positional sensitivity. Since each token's phase is scaled separately, a uniform coordinate shift changes the relative phase in proportion to the scale difference between the two tokens.
However, the symmetric neighborhood averaging in Eq. (14) keeps the scales of nearby tokens close, so this change stays small for the local token pairs.

We measured the remaining sensitivity on DOFA in three settings:

  • Coordinate shift. Shift all token coordinates by 1–8 patches with the image fixed.
  • Crop placement. Evaluate ten 448 × 448 crops at different offsets per 512 × 512 image.
  • Tiled inference. Predict four 256 × 256 tiles per segmentation image and stitch the predictions.

As shown in Table 2, coordinate shifts leave both scores essentially unchanged. Under crop placement, SV-RoPE exhibits sensitivity comparable to LoRA without degrading accuracy or mIoU. Tiling costs both methods several mIoU points, so this degradation is mainly a tiling effect rather than a phase effect. We will state this trade-off explicitly in Section 3.3.3.

Table 2. Spatial consistency. Parentheses show the score change from the original setting.

SettingMethodCLSSEG
OriginalLoRA82.9363.76
SV-RoPE84.3366.43
Coordinate shiftLoRA——
SV-RoPE84.48 (+0.15%)66.80 (+0.37%)
Crop placementLoRA81.68 (−1.25%)62.02 (−1.74%)
SV-RoPE84.43 (+0.10%)65.86 (−0.57%)
Tiled inferenceLoRA—57.41 (−6.35%)
SV-RoPE—59.26 (−7.17%)

3. Effectiveness of the SV-Profile and further ablations (Q3).

(i) Effectiveness of the SV-Profile. The SV-Profile records how quickly a token's feature agreement with its surroundings decays with distance: slow decay marks a homogeneous region, fast decay marks a boundary or fine structure. On Chesapeake (the setting of Figure 4c), this matches real spatial variation in two ways. The share of ground-truth boundary pixels inside a token rises from 0.5% in the low-SV group to 3.5% and 11.1% in the medium and high groups, so high-SV tokens are the ones that contain label boundaries. The SV score also correlates with image gradient energy, a direct measure of local geometric variation, at r = 0.70. On these groups, Figure 4(c) shows that SV-RoPE improves all three and improves the low-SV group most.

(ii) Comparison with alternative descriptors. To evalute the descriptor, we replaced the SV-Profile with single-scale similarity and feature variance, while keeping the RFC architecture and training configuration fixed. The multiscale SV-Profile scored highest on both tasks (Table 3), suggesting that feature agreement across spatial extents provides useful conditioning information that a single scale or a variance statistic does not.

Table 3. Ablations of the descriptor and its token-level conditioning.

VariantCLSSEG
Single-scale similarity83.0965.77
Feature variance83.2866.15
Image-wise RFC83.7565.41
Shuffled SV-Profiles83.5064.64
SV-RoPE (SV-Profile)84.3366.43
AuthorsOpenReview ↗

Response to Reviewer TBbb

4. Validation of learned frequency modulation (Q4).

(i) Learned coefficients across SV groups. Following Fig. 4(b), we grouped the 12 datasets by their frequency preference and split their tokens into low-, medium-, and high-SV groups, then averaging the learned scale factor α\alpha over layers for the 4 lowest and 4 highest frequency bands (Table 4). Each dataset group scales up its preferred bands and scales down the others, and the modulation grows stronger as token SV increases, which confirms that it is applied token by token rather than as one scale per dataset.

Table 4. Mean modulation coefficients on lowest bands / highest bands.

Token SV groupLow-frequency-preferring datasetsHigh-frequency-preferring datasets
Low SV1.090 / 0.9280.948 / 1.101
Medium SV1.103 / 0.9180.944 / 1.109
High SV1.112 / 0.9110.939 / 1.120

(ii) Interventions. Shuffling the SV-Profiles across tokens within each image keeps the image-level statistics and breaks only the match between a token and its own descriptor. This reduced accuracy by 0.83 points and mIoU by 1.79 (Table 3). Replacing token-wise modulation with a single image-wise factor per band (Image-wise RFC, labeled Global scaling in manuscript Table 2) also lowers both scores. Both interventions remove the token-level correspondence, and both hurt, so the gain comes from modulating each token according to its own spatial variation.


5. Cross-scale generalization and the contribution of RCC (Q5).

(i) Physical scale and resampling. Images are resampled to the model input size, so RCC uses the effective GSD after resampling: GSD_eff = GSD_native ×\times (native side / input side). The ground distance per token step is GSD_eff ×\times token stride (Eq. 3). And spatial resampling is a common practice to harmonize imagery across sensor resolutions [1].

(ii) RCC ablation. We retrained the RCC module with the GSD replaced by a constant, keeping architecture unchanged. Scores fall back to the level of the model without RCC (Table 2 in manuscript), so the gain comes from the correct physical scale rather than from the additional input.

[1] D. Hong, J. Hu, J. Yao, J. Chanussot, and X. X. Zhu. “Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model.” ISPRS Journal of Photogrammetry and Remote Sensing, 178, 68–80, 2021.


We thank you again for your valuable feedback. We would be happy to provide any further clarification during the discussion period.

Best regards, The Authors

Reviewer_tnTyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SV-RoPE proposes a parameter-efficient adaptation method for remote-sensing foundation models by conditioning rotary positional interactions on physical scale and local spatial variation. It constructs token-wise Spatial Variation Profiles (SV-Profiles) from average feature cosine similarities across multiscale neighbourhoods. Relation Coordinate Calibration (RCC) rescales positional offsets using token-step ground distance relative to a reference scale, while Relation Frequency Calibration (RFC) converts the profiles into modulation factors for individual frequency bands. A trainable attention-parallel branch implements these adaptations while keeping the pretrained backbone frozen.

Evaluation covers six classification and six segmentation tasks in GEO-Bench using DOFA, Scale-MAE and DINOv3-SAT. The paper reports improved average performance over representative PEFT baselines, particularly for segmentation. Further validation includes component ablations, frequency-preference analyses across spatial-variation groups, and image-resampling experiments assessing robustness to changes in ground sampling distance.

Critical strengths
  1. A coherent spatial inductive bias for remote-sensing adaptation. The method combines token-step ground distance with multiscale neighbourhood affinities to condition rotary positional interactions. This addresses both physical scale and local scene structure within a unified adaptation mechanism. I consider this a meaningful contribution because it provides a concrete, task-relevant way to incorporate local spatial statistics into the adaptation of pretrained representations.

  2. Broad empirical support across different pretrained models. The evaluation covers three backbones with different native positional encodings and 12 GEO-Bench classification and segmentation tasks. Table 1 reports average improvements over SLR of 1.56, 2.00 and 0.69 points for DOFA, Scale-MAE and DINOv3-SAT, respectively. This breadth supports the method's applicability across different pretrained models and provides encouraging evidence of its practical value for dense remote-sensing prediction.

Critical weaknesses
  1. The specific contribution of SV-Profiles is not sufficiently isolated. Table 2 lacks matched comparisons with direct token-feature conditioning and learnable static frequencies. These controls would help determine whether the proposed multiscale affinity descriptor provides additional value beyond generic content conditioning or frequency adaptation. Their absence limits the evidence supporting the paper's central methodological contribution.

  2. The comparability of the ablation baselines requires clarification. The presented description does not establish whether the variants use identical attention branches and comparable parameter budgets. In particular, it is unclear whether Global scaling produces a single scalar or a shared vector of frequency-wise factors. If it uses a scalar, the comparison changes both token specificity and frequency flexibility, weakening the attribution of the improvement to token-wise conditioning.

  3. The coordinate dependence of the phase formulation is insufficiently validated. Under Eq. (8), uniformly shifting coordinates by cc adds θmc(κu,m−κv,m)\theta_m c(\kappa_{u,m}-\kappa_{v,m}) to the relative phase when features and modulation factors are held fixed. Although Section 3.3.3 acknowledges the lack of translation invariance, the reported analyses do not evaluate its effects on crop placement or overlapping-tile consistency.

Questions for authors
  1. Targeted validation of the contribution of SV-Profiles. What evidence distinguishes the value of the proposed multiscale affinity profiles from generic content conditioning or frequency adaptation? Are comparisons with direct token-feature conditioning and learnable static frequencies available under matched settings?

  2. Clarification of the ablation setup. Which variants in Table 2 share the same attention-parallel branch, and which retain RCC? Please clarify their trainable parameter counts and training budgets. Does Global scaling produce a single scalar or a vector containing a separate factor for each frequency band, shared across tokens?

  3. Clarification and targeted validation of the phase formulation. When all modulation factors equal one, Eq. (8) appears to have the opposite phase sign to Eq. (4). Which convention is used in the implementation, and does it recover standard RoPE in this setting? Given the acknowledged lack of translation invariance, what evidence characterizes sensitivity to the coordinate origin or crop placement?

3 comments
AuthorsOpenReview ↗

Response to Reviewer tnTy

Dear Reviewer tnTy,

We are grateful for your careful reading and constructive feedback. We address each point below and will incorporate all additions into the revision.


1. Effectiveness of the SV-Profile (W1, Q1).

To isolate the contribution of the SV-Profile, we evaluated two controls on DOFA with the matched training setup and model architecture,each control changes only what drives the frequency modulation:

  • Learned static frequencies. Replace the frequency mapper with a learned vector per layer, shared across images and tokens and initialized to 1 (standard RoPE). This control keeps learnable frequencies but removes spatial conditioning.
  • Token-conditioned mapping. Replace the SV-Profile with a learned projection directly on token embeddings and retain the MLP frequency mapper. This control keeps token-wise conditioning but removes the multiscale spatial-variation descriptor.

SV-RoPE achieved the highest average scores on both tasks (Table 1). The learned static frequencies, which rescale every band without spatial conditioning, still fall short of SV-RoPE, suggesting that spatial conditioning guides the learning of distinct frequencies. The token-conditioned mapping sees the full token embedding yet also trails on both tasks, supporting multiscale spatial conditioning in this setup. We will add these comparisons to the revision.

Table 1. DOFA frequency modulation ablations.

VariantBEBKESFNPVSSCWNTCTCPPSSCAvg. CLSAvg. SEGOverall avg.
Learned static frequencies88.6398.4098.3354.3498.3063.6967.4159.2675.3161.6294.7733.2883.6265.2874.45
Token-conditioned mapping88.6798.2098.2254.0498.2062.3768.0058.7375.2964.0594.7633.9983.2865.8074.54
SV-RoPE89.6898.5098.3157.1998.8063.4968.5760.3475.6463.3695.0135.6884.3366.4375.38
AuthorsOpenReview ↗

Response to Reviewer tnTy

2. Ablation setup and image-wise frequency modulation (W2, Q2).

(i) Ablation setup. All ablation variants in manuscript Table 2 share the same training schedule, differ only in the components listed in Table 2. Every variant below the LoRA row adds the same attention-parallel branch, and every variant from the RCC row onward keeps RCC. The three conditioned variants (Global scaling, DAPE, and SV-RoPE) share the same frequency mapper and differ only in its input.

Table 2. Configuration of the ablation variants in manuscript Tab. 2.

VariantParallel branchRCCConditioning inputMapper outputTrainable params (M)
LoRA––––1.03
+ RoPE branch✓–––1.33
+ RCC✓✓––1.33
Global scaling✓✓SV-Profile averaged over tokensone factor per band, shared across image1.33
DAPE [1]✓✓token embeddingone factor per band per token1.34
SV-RoPE✓✓SV-Profile per tokenone factor per band per token1.33

We will revise manuscript Table 2 to make these settings explicit.

(ii) Image-wise frequency modulation. The variant labeled Global scaling averages the SV-Profiles over all spatial tokens and then applies the same frequency mapper used by token-wise RFC, producing one scale per frequency band for the whole image. Token-wise RFC applies the mapper separately to each token, and thus have same mapper capacity and per-band flexibility, differ only in conditioning granularity. We will rename this variant Image-wise RFC in manuscript Table 2.

[1] DAPE: Data-Adaptive Positional Encoding for Length Extrapolation, NeurIPS 2024


3. Phase convention, RoPE recovery, and spatial consistency (W3, Q3).

(i) Phase convention and RoPE recovery. Eq. (8) has a sign typo. The implementation uses the phase difference from Eq. (4): (qu′)⊤kv′=qu⊤R(ψv−ψu) kv.(q'_u)^{\top} k'_v = q_u^{\top} R(\psi_v - \psi_u)\, k_v.

We will correct Eq. (8) to: ϕu,v,mSV=ψv,mSV−ψu,mSV=θm(pvκv,m−puκu,m).\phi^{\mathrm{SV}} _{u,v,m} = \psi^{\mathrm{SV}} _{v,m} - \psi^{\mathrm{SV}} _{u,m} = \theta _m \left( p _v \kappa _{v,m} - p _u \kappa _{u,m} \right).

When every modulation factor equals one (κu,m=1κ_{u,m} = 1, i.e., ρ=1ρ = 1 and α=1α = 1 in Section 3.3.1), this reduces to θm(pv−pu)θ_m (p_v - p_u), which is standard axial RoPE.

(ii) Sensitivity to the coordinate origin and crop placement. As the reviewer notes, a uniform coordinate shift by cc adds θmc(κv,m−κu,m)θ_m c (κ_{v,m} - κ_{u,m}) to the relative phase, so the sensitivity is proportional to the scale difference between the two tokens. However, we conduct symmetric neighborhood averaging in Eq. (14) keeps the scales of nearby tokens close, so this term stays small for the local token pairs that dominate attention.

We measured the remaining sensitivity on DOFA in three settings:

  • Coordinate shift. Shift all token coordinates by 1–8 patches with the image fixed.
  • Crop placement. Evaluate ten 448 × 448 crops at different offsets per 512 × 512 image.
  • Tiled inference. Predict four 256 × 256 tiles per segmentation image and stitch the predictions.

As shown in Table 3, coordinate shifts leave both scores essentially unchanged. Under crop placement, SV-RoPE changes performances at a rate similar to LoRA but does not lose accuracy or mIoU. Tiling costs both methods several mIoU points, so this degradation is mainly a tiling effect rather than a phase effect. We will add this analysis in revision.

Table 3. Spatial consistency. Parentheses show the score change from the original setting.

SettingMethodCLSSEG
OriginalLoRA82.9363.76
SV-RoPE84.3366.43
Coordinate shiftLoRA——
SV-RoPE84.48 (+0.15%)66.80 (+0.37%)
Crop placementLoRA81.68 (−1.25%)62.02 (−1.74%)
SV-RoPE84.43 (+0.10%)65.86 (−0.57%)
Tiled inferenceLoRA—57.41 (−6.35%)
SV-RoPE—59.26 (−7.17%)

We thank you again for your valuable feedback. The new controls, the clarified ablation configuration, and the corrected phase formulation will all be included in the revised manuscript, and we believe they make the contribution of the SV-Profile considerably clearer. We would be happy to provide any further clarification during the discussion period.

Best regards,

The Authors

Thank you for addressing my questions. However, the main text refers to supplementary materials and appendices several times, but I could not find the corresponding pages. Could the authors please provide the complete manuscript, including the supplementary materials?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.