acceptodds
Under review as a conference paper at ICLR 2027

TRAPE: Compact Axial Position Tapes for Cross-Resolution Vision Transformers

Abstract

Vision transformers trained at one resolution often degrade at larger resolutions or other aspect ratios. We propose TRAPE (Tabulated Relative Axial Positional Encoding), a relative attention bias trained as a continuous function and deployed as compact tables. Along each axis, TRAPE adds a local correction table to a continuous function of log-spaced offsets with a fixed reference; a shared learnable amplitude bounds the sum of both axis responses. The fixed reference makes a smaller grid's table an exact centre slice of a larger one. Fine-tuning only positional parameters at a few square resolutions yields a resolution tape of per-axis tables that height and width slice and blend independently, serving unseen resolutions and aspect ratios with one batched lookup and no positional network. On ImageNet-1k, TRAPE-DeiT-S trained at retains 70.2% top-1 at without fine-tuning, against 36.4% for resized absolute embeddings and 12.1% for 2D RoPE-Mixed. As a zero-initialized branch, TRAPE raises five frozen public ViTs by 18–56 points at with under 0.1 ImageNet epoch of position-only adaptation. On one A800 GPU, assembling tables for a new shape takes about 0.1 ms. Fused attention avoids materializing the bias matrix, enabling over the batch size of dense attention at . Hierarchical TRAPE backbones with axial attention improve ADE20K segmentation by 3.0–3.1 mIoU over Swin with fewer FLOPs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision transformers, positional encoding, relative position bias, resolution generalization, parameter-efficient adaptation, axial attention, semantic segmentation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GEoQOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces TRAPE, a relative positional bias for vision transformers. It combines continuous axial functions, local correction tables, and a shared bounded output function. TRAPE supports different input resolutions and improves performance on several benchmarks.

Critical strengths

The experiments cover a broad range of settings, including resolution extrapolation without fine-tuning, position-only adaptation, rectangular inputs, semantic segmentation, and inference efficiency. Some experiments use multiple random seeds. The paper also provides detailed experimental settings and clearly states several limitations.

Critical weaknesses

1.The paper does not discuss or compare with LookHere [1]. LookHere directly studies resolution extrapolation in plain ViTs, including training at 2242224^2 and testing at 102421024^2. Its directional attention masks and distance biases are closely related to TRAPE’s evaluation without fine-tuning. A comparison with this method would help establish TRAPE’s contribution.

2.The method uses established ideas, including log-spaced coordinates and continuous bias functions [2], as well as axial factorization [3]. These ideas have already been explored in prior work. TRAPE therefore appears to combine existing techniques, and its main methodological novelty remains unclear.

[1] LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate. NIPS2024

[2] Swin Transformer V2: Scaling Up Capacity and Resolution. CVPR2022

[3] Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution. NIPS2023

Questions for authors

Please see weakness. Since I am not an expert on positional encoding or Transformer architectures, I will consult other reviewers' comments.

2 comments

Thank you for these comments and for highlighting LookHere [1]. We agree that it directly addresses resolution extrapolation in plain ViTs and should be included. We add the comparison and distinguish the mechanisms and evaluated adaptation settings below. Our table and section numbers refer to the original submission.

1. Resolution extrapolation without fine-tuning.

We compare LookHere's ViT-B/16 variants (Fuller et al., NeurIPS 2024) with TRAPE-DeiT-B/16. All are trained at 2242224^2 and evaluated on the original 50,000-image ImageNet-1k validation set without high-resolution gradient fine-tuning. Values are top-1 (%).

Test sizeLH-180LH-90LH-45TRAPE-B
2242224^281.3181.0281.0682.09
3202320^282.6582.2282.4582.90
3842384^282.3882.0882.3882.89
4482448^281.8681.9182.1982.47
5122512^281.1181.1981.6281.89
7682768^276.5977.1278.1378.70
102421024^269.2471.5873.1573.30

LookHere values are from its Table 1 (main text; Best, 2242224^2) and Table 6 (Appendix A.2; larger sizes), using the Val top-1 column. TRAPE-B uses one unchanged checkpoint at every size: 102421024^2 quotes Table 1 of our main text; the remaining values are reported from evaluations of that same checkpoint under App. B.1, including additional test resolutions.

The training and evaluation protocols differ. LookHere trains for 150 epochs on 99% of ImageNet and uses aspect-preserving resizing followed by centre cropping. TRAPE uses the 300-epoch DeiT recipe and direct resizing above 2242224^2 (App. B.1). LookHere's reported extrapolation results select a global distance slope on training-held-out minival at each target resolution. This is parameter selection rather than gradient fine-tuning; LookHere also reports effective extrapolation without slope tuning. TRAPE-B uses no target-resolution parameter selection in this comparison.

At 102421024^2, TRAPE-B and LH-45 report 73.30% and 73.15%, respectively. These results under their respective protocols show competitive reported accuracy at the same model scale and patch size. For controlled evidence, Table 1 in our main text holds the DeiT-Ti recipe fixed: at 102421024^2, axial Log-CPB, its variant with the shared stanh⁡s\tanh output, and full TRAPE reach 52.0%, 62.9%, and 63.4%, respectively.

2. Attention masks, distance biases, and position-only adaptation.

LookHere assigns fixed directional fields of view to selected heads, with Euclidean-distance penalties and additional undirected heads. TRAPE learns block/head-specific relative-offset responses, permits asymmetric and non-monotonic patterns, and adds finite bounded biases without directional exclusion in plain ViTs.

LookHere uses fixed masks and distance penalties without learned positional embeddings or an online positional MLP. TRAPE's compactness concerns the storage and reuse of its learned relative-bias responses.

TRAPE also provides an evaluated position-only adaptation route for existing checkpoints. A zero-output branch retains native APE/RoPE and preserves initial predictions, while content weights remain frozen. On five public classification ViTs, it updates under 0.13% of parameters using 0.048-0.096 ImageNet epochs, measured by training image presentations (Sec. 4.4; Appendix B, Table 6). The fitted responses export to tables reused across resolutions and aspect ratios. This adaptation setting is separate from the unchanged-checkpoint comparison above.

We will add LookHere to the related work and comparison, and clarify the differences in spatial inductive bias and adaptation mechanisms.

[1] Fuller et al. LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate. NeurIPS 2024.

Thank you for the opportunity to clarify TRAPE's novelty. Our contribution is a relative positional bias designed jointly with a resolution tape and position-only adaptation of frozen ViTs.

We start from Swin V2-style Log-CPB [2, Sec. 3.3], with log-spaced coordinates, joint 2D continuous bias and its official 16σ16\sigma output (Table 1(a)); Swin V2 permits precomputation. Row (b) applies established axial factorization, used by NaViT [3, Sec. 2.1] for absolute/fractional input embeddings, to relative bias responses. We add a shared stanh⁡s\tanh output and central tables. Beyond these components, the resolution tape enables compact table-only deployment and independent height/width reuse; position-only adaptation grafts this bias onto frozen ViTs, retaining native encoding.

1. Relative positional bias and component ablations.

In our original main-text Sec. 3.1, Eq. (3) defines the raw axis response and Eq. (4) the attention bias (block/head indices omitted):

Ua(Δ)=fa(c(Δ;G0))+ea(Δ),a∈h,w,U_a(\Delta)=f_a(c(\Delta;G_0))+e_a(\Delta),\quad a\in\\{h,w\\},

b(Δh,Δw)=stanh⁡(Uh(Δh)+Uw(Δw)).b(\Delta_h,\Delta_w)=s\tanh\left(U_h(\Delta_h)+U_w(\Delta_w)\right).

Here cc is the fixed-reference log coordinate in our main-text Eq. (2), and eae_a the local correction. Starting from (b), row (c) adds the shared stanh⁡s\tanh output, and row (d) adds the central table, completing TRAPE. The learned ss is shared across blocks and heads; local corrections merge into exported tables.

Main-text Table 1 uses the same DeiT-Ti architecture and recipe, trained at 2242224^2 and evaluated unchanged on ImageNet-1k validation (top-1, %):

Positional encoding2242224^2102421024^2320×960320\times960
(a) Log-CPB, 16σ16\sigma72.252.664.3
(b) (a) + axial factorization72.152.061.9
(c) (b) + shared stanh⁡s\tanh output72.862.966.0
(d) (c) + central table (TRAPE)72.863.468.1

Relative to (a)/(b), full TRAPE gains 0.6/0.7 percentage point (pp) at 2242224^2 and 10.8/11.4 pp at 102421024^2. Factorization alone does not yield these gains. Adding the shared output raises 102421024^2 by 10.9 pp; the central table adds 0.5 pp there and 2.1 pp at 320×960320\times960. Thus both training-resolution accuracy and generalization improve. Row (c) changes amplitude and input scaling jointly (main-text Sec. 3.2).

2. Resolution tape and table-only deployment.

Building on this bias, the resolution tape stores raw axis responses before the nonlinearity for compact deployment and reuse across resolutions and aspect ratios. Our original main-text Sec. 3.3 gives the exact center-slice identity for fixed parameters in Eq. (5) and the empirical cross-anchor interpolation rule in Eq. (6). During position-only adaptation, ss is frozen. Inference slices and blends each axis independently and applies the shared nonlinearity after composition. Mixing final squashed biases generally gives a different function (App. A.2).

For 320×960320\times960, height uses anchors 224/336 and width 704/1024, without rectangular fitting or positional MLP evaluation within the exported range. Fixed-reference slicing also applies to a joint 2D bias; factorization enables independent anchor pairs and O(H+W)O(H+W) rather than O(HW)O(HW) stored responses per anchor/block/head. Plain-ViT attention remains full.

Appendix F/Table 16 compares fp32 positional storage for ViT-S, four anchors 224,336,480,704\\{224,336,480,704\\}, and image sides 224-704:

Stored representationSize
TRAPE tape154 KiB
Log-CPB MLP weights864 KiB
Joint 2D anchor tables5.6 MiB

Training checkpoints and expanded attention matrices are excluded. Joint 2D biases can also be cached; Table 16 notes that joint table blending need not reproduce MLP-weight blending. TRAPE retains independent axis routing.

3. Position-only adaptation of frozen ViTs.

Position-only adaptation makes this bias usable with existing frozen ViTs (main-text Sec. 3.4).

Original-submission results use position-only fine-tuning: 160 steps/anchor, global batch 128. Six square anchors 144/224/336/480/704/1024 start independently from zero branches on the public 224 checkpoint; Table 13 refits only 704. Appendix E.1/Table 13 tests the same RoPE-Mixed ViT-S with matched data and steps at fitted 7042704^2, where each method uses its own fitted state without interpolation:

MethodTop-1, mean ±\pm sample SD (%)
TRAPE79.9±0.0579.9\pm0.05
Log-CPB75.3±0.0675.3\pm0.06
2D-Tanh77.8±0.1277.8\pm0.12

TRAPE's 4.6/2.1 pp gains over Log-CPB/2D-Tanh persist without cross-anchor interpolation. These are three-seed comparisons under matched data/step budgets; optimization is parameterization-specific (App. B.5). We will clarify the joint design and this controlled evidence in the revision.

[2] Liu et al. Swin Transformer V2: Scaling Up Capacity and Resolution. CVPR 2022.

[3] Dehghani et al. Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution. NeurIPS 2023.

Reviewer_CRWSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper deals with degradation in vision transformer performance when the image resolutions and aspect ratios in testing are different from those in training. To address this issue, it proposes Tabulated Relative Axial Positional Encoding (TRAPE) which represents relative position along the height axis and width axis separately using continuous functions plus learnable local correction tables. For inference, these positional responses are exported as lookup tables. Different resolutions are supported by interpolation between different anchor sizes, while independent handling of height and width allows support of arbitrary aspect ratios.

This approach is validated with cross-resolution experiments on ImageNet-1k classification and ADE20K semantic segmentation, with comparisons to interpolated absolute position embeddings and a 2D RoPE-Mixed baseline. It is also shown that TRAPE can be added to existing frozen ViTs and adapted by training on just a small number of positional parameters.

Critical strengths

One critical strength is the well-motivated and practical design of TRAPE. Modeling relative position separately along the height and width axes makes it possible to adjust the learned positional information independently for the two dimensions, and thus makes it flexible for test images with different resolutions or aspect ratios. The fixed coordinate system allows positional information for a smaller resolution to be directly obtained as part of a larger table, without resizing or recomputation. This compact representation can be stored as lookup tables for efficient inference. Overall, I find this design to be sensible, elegant and practically useful.

Another critical strength is the strong performance across resolutions and aspect ratios. When the test resolution is significantly larger than the training resolution, the advantage of TRAPE is especially clear, with considerable gains over the comparison baselines. Strong performance is demonstrated on both square and rectangular images. In addition, the experiments show that TRAPE is not only effective when training models from scratch, but also can be added to frozen public ViTs with parameter-efficient adaptation to substantially improve their large-resolution performance.

Critical weaknesses

The segmentation experiments are not as convincing as the classification experiments. They don't clearly show how much improvement comes specifically from TRAPE because several components are changed at the same time. In comparison to Swin, TRAPE-Swin changes the attention mechanism (from shifted windows to single-axis global), positional encoding, and adds a depthwise convolution to the value features. As a result, it is hard to know how much of the observed improvement can be attributed to TRAPE. This comparison would be a lot better if only the positional encoding was changed.

Questions for authors

Can segmentation results be provided that more clearly isolate the contribution of TRAPE, such as by changing only the positional encoding?

3 comments

Thank you for recognizing TRAPE's practical design and strong performance across resolutions and aspect ratios. We agree that the Swin/TRAPE-Swin comparison in Table 4 changes attention, positional encoding (PE), and the depthwise value branch together, so its full improvement reflects the combined design. Comparisons changing only PE are already reported in Appendix I, Table 21 (p. 33) and summarized in Sec. 4.5 (p. 9, lines 452-456). We summarize these original results and add a new ablation removing only the central tables.

1. Changing only the positional encoding.

The three variants retain the same Tiny backbone, alternating height/width global attention, 3×33\times3 depthwise value branch, and UPerNet decoder. They use the same ImageNet-1k pre-training recipe and ADE20K training/evaluation protocol, with 512×512512\times512 training crops and 160k iterations. Only the PE configuration differs across pre-training and segmentation training; no additional position-only adaptation is applied.

PE on the same Tiny axial backboneIN-1k top-1 (%)SS mIoU (%)
Swin V2-style axial continuous bias81.446.6
2D RoPE-Mixed81.647.0
TRAPE81.547.6

TRAPE improves single-scale (SS) mIoU by 1.0 point over the Swin V2-style bias and 0.6 point over 2D RoPE-Mixed, with ImageNet top-1 within 0.1 point of either control. Attention connectivity and the depthwise branch are held fixed, so these comparisons isolate the PE configuration within this Tiny axial backbone.

What the Swin V2-style control changes. This is a bias replacement inside TRAPE-Swin-T, not the full Swin V2 architecture. It keeps the same offset coordinates and 1D axis-MLP widths. In a block attending along axis aa in stage ii, the TRAPE bias is sitanh⁡(fa(c)+ea)s_i\tanh(f_a(c)+e_a); the control uses 16σ(fa(c))16\sigma(f_a(c)). Thus it removes the central correction tables and replaces the learned stage amplitude and tanh output with the fixed Swin V2-style output map. It does not restore shifted-window attention or use Swin V2's joint 2D MLP. The 1.0-point gap therefore compares these PE configurations together and does not isolate the central table alone.

The RoPE control replaces the TRAPE bias with query/key rotations while retaining the same attention and value branch.

2. Central table and depthwise convolution.

New ablation of the central table. We remove only the central tables EaE_a, i.e. the local correction parameters, from TRAPE-Swin-T. The continuous axis functions, learned stage amplitudes, attention, depthwise branch and decoder are retained, with the same classification and segmentation protocols described above. Unlike the Swin V2-style control, this variant retains the sitanh⁡s_i\tanh output and changes the bias only to sitanh⁡(fa(c))s_i\tanh(f_a(c)).

TRAPE-Swin-T configurationIN-1k top-1 (%)SS mIoU (%)
Full TRAPE (original submission)81.547.6
Without central tables (new)81.547.2

Removing the central tables reduces SS mIoU by 0.4 point, while ImageNet top-1 remains 81.5% at the reported precision. This directly supports the local correction's contribution to segmentation under this fixed backbone and output map. This new result is separate from the original Table 21 controls.

Depthwise convolution controls from the submission. Removing the 3×33\times3 branch from full TRAPE-Swin-T reduces SS mIoU from 47.6 to 47.1 (-0.5). Adding it to Swin-T increases SS mIoU from 44.5 to 45.8 (+1.3). The branch's contribution depends on the backbone; these effects should be interpreted within their respective controlled comparisons, rather than added to partition the full Swin/TRAPE-Swin improvement.

3. Contribution of TRAPE to semantic segmentation.

We will bring the three-row PE comparison into the main text and include the new central-table ablation. We will distinguish the combined design improvement over Swin-T/S/B in main-text Table 4 (+3.0-3.1 SS mIoU points) from the improvements from changing only PE on the same Tiny axial backbone in Appendix I/Table 21 (+0.6 and +1.0 points) and the gain from the central tables in the new ablation (+0.4 point). This will make the encoding's contribution and the scope of its evidence clearer.

Thank you for the detailed response to my concerns. They are satisfactorily addressed. Making the evidence clearer on the encoding's contribution in semantic segmentation is definitely helpful.

Thank you for your constructive review and positive follow-up. We appreciate your confirmation that our response satisfactorily addresses your concerns. Your comments helped us make the contribution of positional encoding to semantic segmentation clearer. In the revised manuscript, Sec. 4.5 (pp. 9-10) and Table 4a (p. 9) now present the three PE controls on the same Tiny axial backbone together with the new ablation removing only central tables (47.2 versus 47.6 SS mIoU for full TRAPE). Appendix I (pp. 34-36), especially Table 21 (p. 35), details the fixed architecture and training protocol and clarifies the Swin V2-style bias replacement. We distinguish these PE and component comparisons from the combined backbone improvement over Swin. Thank you again for helping us improve the clarity and presentation of this evidence.

Reviewer_9JzTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes TRAPE, a compact axial relative positional encoding for cross-resolution vision transformers. It combines continuous positional functions, local correction tables, and a resolution-tape mechanism to support efficient adaptation to unseen resolutions and aspect ratios. Experiments show strong high-resolution classification performance, improvements on pretrained ViTs, and promising segmentation and inference-efficiency results. The approach is practically useful, although its interpolation strategy and generalization beyond the evaluated range require further investigation.

Critical strengths

(1) Strong empirical performance. TRAPE substantially improves high-resolution extrapolation, particularly at 1024×1024, and generalizes well to unseen rectangular resolutions.

(2) Effective and practical design. The fixed-reference coordinate system enables exact center slicing, while compact table export and independent axis-wise interpolation support efficient deployment without training separate models for every resolution.

(3) Useful ablations and efficiency analysis. The paper examines key design choices and reports inference improvements, providing evidence for both the encoding design and its practical applicability.

Critical weaknesses

(1) Heuristic cross-anchor interpolation. The center-slicing property holds for a fixed parameter state, but independently adapted anchors need not define consistent positional functions. Interpolating their tables is therefore empirical, with limited analysis of sensitivity to anchor placement and generalization reliability.

(2) Limited extrapolation scope. The method is primarily validated within a predefined resolution range. Behavior beyond the stored range requires additional offline processing, and the trade-off among anchor count, adaptation cost, and generalization is insufficiently characterized.

(3) Outdated baselines. The comparisons do not sufficiently cover recent positional encoding methods. Including stronger and more recent baselines is important to establish the method's relevance relative to the current state of the art, such as: Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane, ICML 2026; nD-RoPE: A Generalized RoPE for n-Dimensional Position Embedding, ICML 2026; Compass-RoPE: Isotropic Rotary Position Embeddings for Vision Transformers, ICML 2026.

Questions for authors

Could TRAPE be extended to image generation tasks, such as diffusion transformers (DiTs), where resolution extrapolation is also important?

4 comments
AuthorsOpenReview ↗

Heuristic cross-anchor interpolation

Thank you for recognizing TRAPE's high-resolution performance, generalization to unseen rectangles, and practical design. We clarify the scope of center slicing and provide evidence on cross-anchor interpolation, sensitivity to anchor placement, and generalization reliability. Section/table references are to the original submission.

1. Center slicing and cross-anchor interpolation.

We agree. Sec. 3.3 and App. A.1 establish Eq. (5) only for fixed positional parameters, including raw tables averaged over the same states at both sizes. They explicitly state that this identity does not imply agreement between differently adapted anchors. Eq. (6) is an empirical interpolation rule, without an accuracy guarantee.

Fixed-reference coordinates align table entries by relative offset. Inner-neighbour warm starts, frozen amplitude and function-preserving capacity growth keep the same representation during TRAPE-DeiT adaptation (App. B.3), but do not prove consistency between anchors. App. A.2 bounds bias errors by ∣b^−b∣≤∣s∣(ϵh+ϵw)|\hat{b}-b|\leq |s|(\epsilon_h+\epsilon_w) for raw-axis errors; this does not bound accuracy.

2. Sensitivity to anchor placement.

Our additional comparisons evaluate TRAPE-DeiT-S on all 50,000 ImageNet-1k validation images at each of 14 common non-anchor targets: squares 176,192,256,384,560,608,800,896,9602\\{176,192,256,384,560,608,800,896,960\\}^2 and rectangles 384×512384\times512, 512×384512\times384, 448×672448\times672, 672×448672\times448, 320×960320\times960.

We exclude 2882288^2, fitted in the placement variant 144,224,288,416,624,1024\\{144,224,288,416,624,1024\\}, and 4002400^2, fitted in the eight-anchor count variant 144,224,336,400,480,704,864,1024\\{144,224,336,400,480,704,864,1024\\}, to keep these targets unfitted in both experiment families. The 4162416^2 size was outside the original candidate target set.

The original plan 144,224,336,480,704,1024\\{144,224,336,480,704,1024\\} gives 80.429% mean top-1 using archived seed-0 records underlying App. D/Table 9 and App. G/Table 19. Alternative plans 144,224,432,624,832,1024\\{144,224,432,624,832,1024\\} and 144,224,288,416,624,1024\\{144,224,288,416,624,1024\\} give seed-0 means of 80.474% and 80.485%. The three tested plans span 0.056 percentage point (pp). The first alternative gives 80.473±0.01580.473\pm0.015% across three adaptation seeds (sample SD).

3. Generalization reliability.

The original submission reports the following evidence:

  • App. G/Table 19 compares neighbouring-anchor rules on 48 non-anchor shape-model cases across DeiT-Ti/S/B. At the reported precision, Blend is strictly best in 39 and tied for best in eight; one favours Lower. Table 18 finds Blend best in all 12 capacity-scale combinations. Routing was selected on a 10,000-image subset included in the final 50,000-image evaluation.
  • App. D/Table 10 repeats the complete DeiT-S adaptation with three seeds. Per-shape sample SD is at most 0.18 pp and below 0.1 pp for 19/22 shapes.
  • App. G finds that using linear side length instead of log-spaced coordinates changes the probe mean by at most 0.01 pp.
  • App. B.4 fits each public-ViT anchor independently from its zero branch. At unseen 9602960^2, RoPE-Mixed ViT-S improves from 23.3% to 74.4% with TRAPE (main Table 3).

Additionally, for the first alternative plan, seed 0, we remove the 432/624/832 anchor tables one at a time at inference without retraining. Each removed anchor's resolution is then evaluated using its neighbours. Direct fit uses the original fitted table. All entries are top-1 (%) on the same 50,000 images per row.

Removed anchor / targetLowerUpperBlendDirect fitBlend - better single (pp)
4322432^281.68281.83282.13882.028+0.306
6242624^281.01281.08081.49881.502+0.418
8322832^279.15679.12679.70279.628+0.546

Blend exceeds both single-anchor predictions in all three tests and differs from direct fit by at most 0.110 pp. This tests interpolation at inference while retaining the sequential training history of later anchors. These results support reliability under the tested anchor plans and resolutions; we will add them and their protocols to the revision.

AuthorsOpenReview ↗

Limited extrapolation scope

Thank you for raising the questions about extrapolation beyond the stored range and adaptation cost. We quantify the effect of anchor count, then report the offline export cost and accuracy beyond the original range. Inference within the stored range uses table lookup (Sec. 3.3; App. A.3).

1. Anchor count, adaptation cost, and generalization.

TRAPE-DeiT-S is evaluated on all 50,000 ImageNet-1K validation images per shape. Placement and count comparisons share 14 non-anchor targets: squares 176,192,256,384,560,608,800,896,9602\\{176,192,256,384,560,608,800,896,960\\}^2 and rectangles 384×512384\times512, 512×384512\times384, 448×672448\times672, 672×448672\times448, 320×960320\times960. We exclude 2882288^2, fitted in the placement plan 144,224,288,416,624,1024\\{144,224,288,416,624,1024\\}, and 4002400^2, fitted in the count plan 144,224,336,400,480,704,864,1024\\{144,224,336,400,480,704,864,1024\\}. This keeps the evaluation set common to both experiment families.

Counts include the unadapted 224 anchor. Rows use seed 0; the original six-anchor row is from archived records, and the others are new fits. Storage counts fp32 axis tensors only.

AnchorsStepsImage presentationsAxis KiBMean top-1 (%)
349396,896158.0679.917
4642127,904191.2580.400
6, original888188,400263.2580.429
81,110219,664351.0080.436

Four anchors 144,224,480,1024\\{144,224,480,1024\\} retain the same advertised range while reducing steps/presentations/storage by 27.7%/32.1%/27.4%, with a 0.029 pp difference in seed-0 mean accuracy. Their three-seed mean is 80.407±0.02080.407\pm0.020% (sample SD). Three anchors lose 0.512 pp on average and 0.974 pp at worst versus the archived six-anchor run. Four-anchor controls matched by steps (888) or presentations (188,384) reach 80.469%/80.471%; these controls match the stated budgets separately. For serving only sides at least 224, App. D describes omitting the 144 anchor, saving 169 steps and 86,528 presentations, with 248.1 KiB storage.

2. Offline export beyond the stored range.

Extending image sides from 1024 to 1536 changes token sides from 64 to 96. Full generation evaluates 191 offsets per axis; incremental append computes 64 new offsets and retains 127. Both are forward evaluations requiring no optimization.

OperationRecorded component time
Full generation, median of 10 runs4.446 ms
Incremental append, median of 10 runs3.427 ms
Apply retained positional state27.13 ms
Save full/deploy artifacts, two files29.80 ms

Generation includes CPU transfer/concatenation. Model construction and backbone loading are outside this timing; state application and saving are measured separately. State application uses parameters already in memory. Once exported, the enlarged tape supports table-only inference.

3. Generalization beyond the stored range.

Top-1 (%) on 50,000 images per shape; No-FT is pretrained TRAPE-DeiT-S without position-only adaptation. Full/Append extend the original adapted tape without further training; +1280 additionally fits a 128021280^2 anchor.

ShapeNo-FTFullAppendFull +1280Append +1280
115221152^264.77674.63474.62674.89474.880
128021280^258.53071.50671.38271.75071.750
153621536^246.16262.57862.60465.40465.404
1536×7681536\times76863.41072.55672.68073.15073.188
768×1280768\times128070.89277.03277.02676.93676.924
768×1536768\times153666.21474.51074.49274.64274.614

Full export regenerates the 1024-anchor table from its final state; the original tape averages late-training responses (App. B.3). Correspondingly, the 102421024^2 control is 76.954% versus archived 76.912%. Append keeps the old averaged responses and evaluates the final generator only at new offsets. All 112 old-range raw-axis checks remain bitwise unchanged, including after adding 1280. With an identical fixed generator, full/append outputs agree exactly in our control.

Adding 1280 costs 202 steps and 3,232 presentations: 4.30 min optimization plus 4.77 min holdout evaluation/selection on eight A800 GPUs, measured separately from setup, saving and final 50K evaluation. Targets above 1280 use that anchor's exported continuation. Axis storage is 299.25 KiB for export alone and 388.69 KiB with 1280, excluding retained generators/metadata.

App. J/Table 23 also evaluates segmentation at Long-2048 beyond its largest fitted crop, 1344, using retained positional parameters. These measurements characterize the accuracy, adaptation and storage trade-offs and distinguish lightweight range export from optional position-only fitting. We will include them in the revision.

AuthorsOpenReview ↗

Outdated baselines

Thank you for highlighting Spiral RoPE [1], nD-RoPE [2], and Compass-RoPE [3]. We agree that these recent methods are directly relevant. We provide additional comparisons below and will incorporate them into the revision and related work.

1. Comparisons with recent positional encoding methods.

All models below are trained at 2242224^2 and evaluated on ImageNet-1k validation without high-resolution gradient updates (top-1, %). nD-RoPE and its stronger +YaRN variant are from Table 1 in the main text of [2]. Pretrained checkpoints for nD-RoPE and Compass-RoPE were not publicly available when we conducted these comparisons, so we use their published results. Spiral values are our evaluation of a public 2242224^2 checkpoint. TRAPE-S uses the unchanged checkpoint reported in Table 1 (main text) and Table 8 (Appendix C) of our original submission, with additional test sizes and without anchor adaptation.

These Small-scale comparisons retain the source recipes: TRAPE uses 300-epoch DeiT; nD-RoPE uses 400-epoch AdamW, and Compass uses 400-epoch DeiT III with LAMB. Our Spiral evaluation uses centre cropping throughout; TRAPE directly resizes above 2242224^2 (App. B.1). Spiral's 79.86% at 2242224^2 is our checkpoint/protocol result; its published result under the source recipe is 80.39% (Table 2 in the main text of [1]).

Test sizenD-RoPEnD + YaRNSpiral (our eval.)TRAPE-S
1602160^277.0977.0975.4077.19
1922192^279.4679.4678.3979.40
2242224^281.0781.0779.8680.64
2562256^281.5481.6580.4180.86
3202320^282.0382.1580.3281.60
3842384^281.6282.0779.5281.84
4482448^280.7881.7778.0381.63
5122512^279.4681.3476.2881.08
6402640^274.6879.6972.3979.78
7682768^266.0677.2267.9277.63
8962896^252.2673.5662.8674.57
102421024^235.5168.4657.7070.20

For Compass-RoPE, we use Table 1 in the main text of [3] at all test sizes shared with our available TRAPE evaluations:

Test sizeCompass-RoPETRAPE-S
1442144^274.5075.10
1922192^279.6079.40
2242224^281.0080.64
3202320^282.5081.60
4482448^281.5081.63
5122512^280.3081.08
6402640^275.4079.78
7202720^270.2078.51

TRAPE-S retains 70.20% at 102421024^2 and has higher reported accuracy at larger test resolutions: +1.01/+1.74 percentage points over nD-RoPE + YaRN at 8962896^2/102421024^2, and +8.31 over Compass at 7202720^2. The tables also show the recent baselines' strong performance at moderate resolutions. Because recipes, model details and preprocessing differ, we compare results under each source's training and evaluation protocol. They establish TRAPE's relevance alongside recent methods, while our ablations using the same training recipe (Table 1 in the main text; Appendix C) provide controlled evidence for the parameterization's components.

2. Relative positional bias and rotary position embeddings.

Spiral expands rotary directional coverage; nD-RoPE couples position and frequency vectors using regular-simplex wave vectors; Compass uses polar frequency parameterization and DFT angular mixing. All three introduce positional interactions through query/key rotations.

TRAPE learns a query-independent, bounded relative logit bias: b(Δh,Δw)=stanh⁡(Uh(Δh)+Uw(Δw)),b(\Delta_h,\Delta_w)=s\tanh\left(U_h(\Delta_h)+U_w(\Delta_w)\right), where each UaU_a combines a continuous function in fixed-reference coordinates with a learnable local correction. The positional representation is factorized; our plain ViT retains full spatial attention.

For fixed parameters, the learned raw responses export exactly into compact axis tables, and smaller tables are exact centre slices of larger ones (Sec. 3.3). Inference within the exported range requires no positional MLP. Optional position-only adaptation freezes the backbone and classifier, and independently routes height/width tables across anchors to serve unseen rectangles. When grafted onto a pretrained ViT, it retains the native APE/RoPE (Sec. 3.4). This adaptation is not used in the tables above. TRAPE's contribution combines the bounded axial relative-bias representation, exact table export, and position-only adaptation with independent per-axis serving.

[1] Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane. ICML 2026.

[2] nD-RoPE: A Generalized RoPE for n-Dimensional Position Embedding. ICML 2026.

[3] Compass-RoPE: Isotropic Rotary Position Embeddings for Vision Transformers. ICML 2026.

AuthorsOpenReview ↗

Extension to Diffusion Transformers

Thank you for suggesting diffusion transformers. We extend TRAPE to ImageNet-1k class-conditional generation with public DiT-XL/2-256, evaluating position-only adaptation and transfer to unseen resolutions and aspect ratios.

1. Position-only adaptation and evaluation protocol.

All variants retain native sinusoidal APE, bicubically resized at new grids, and freeze original DiT/VAE parameters. TRAPE adds initially zero relative biases to all 28 attention blocks, training axis MLPs and local tables with fixed shared s=10s=10 and reference side G0=16G_0=16. APE-FT learns an absolute-position residual; Log-CPB trains a joint 2D relative-bias MLP.

Adaptation uses ImageNet-1K train, the DiT loss, aspect-preserving resizing, centre crops and random horizontal flips; AdamW, zero weight decay, global batch 128 and BF16. All three methods receive 6,000 steps at 5122512^2; Log-CPB/TRAPE each receive 600 at 102421024^2. Anchors start independently from public 256, with matched step/data budgets at each anchor.

Sampling uses final EMA weights, the frozen sd-vae-ft-ema VAE, DDIM-50 (η=0\eta=0), CFG 1.5 on three epsilon channels, and BF16. Per shape, 5,000 balanced samples (five/class) are compared with 10,000 validation references (ten/class, seed 0), aspect-preserving resized and centre-cropped to target. References, labels and sampling settings match across methods. Metrics use custom Clean-FID/KID, clean-mode 2048-dimensional Inception features and 50 KID subsets; adaptation/sampling seed is 0. This custom 5K protocol differs from DiT's ADM FID-50K benchmark; comparisons are within a shape. Entries are FID / KID ×1000\times1000, lower is better.

2. Results at fitted and unseen resolutions.

Adapted rows use the fixed 512/6000 adapters; both 384 targets are unfitted. Native APE is public 256 without updates, with bicubic positional resizing.

Method3842384^2384×448384\times4485122512^2 (fitted)
Native APE, no updates166.26 / 112.33212.80 / 164.19243.08 / 191.96
APE-FT122.61 / 73.50156.79 / 103.84199.81 / 126.43
Log-CPB14.08 / 1.8514.21 / 1.4220.34 / 3.30
TRAPE13.87 / 1.8113.83 / 1.3219.54 / 2.97

TRAPE improves both metrics over native APE, APE-FT and Log-CPB at all three shapes. At 512, trained counts (TRAPE/Log-CPB/APE-FT) are 91,280/272,384/1,179,648: TRAPE uses 66.49%/92.26% fewer parameters.

Method102421024^2 (fitted)Trained position parameters
Log-CPB100.53 / 63.62272,384
TRAPE79.15 / 44.42184,464

At 1024, TRAPE lowers observed FID by 21.38 using 32.28% fewer positional parameters. This measures direct adaptation at the fitted resolution.

3. Cross-anchor interpolation and rectangular inputs.

"512" uses only the fitted 512 adapter; "1024" transfers its fixed adapter to smaller unseen shapes. Blend combines the independently fitted anchors without target-shape updates. TRAPE exports raw axis responses over the required offsets, centre-slices and blends before the shared nonlinearity:

U^a=(1−αa)Ca(Ta,512)+αaCa(Ta,1024).\widehat{U}_a=(1-\alpha_a)C_a(T_{a,512})+\alpha_aC_a(T_{a,1024}).

b=stanh⁡(U^h+U^w).b=s\tanh(\widehat{U}_h+\widehat{U}_w).

Weights follow Eq. (6) independently per axis. Log-CPB follows App. B.5, mixing MLP parameters with one weight from the geometric-mean token side. Squares use the same weight in both methods; TRAPE routes rectangular axes independently. No rectangular anchor is fitted.

MethodAnchor use7682768^28962896^2
Log-CPB51289.84 / 39.13153.20 / 83.17
Log-CPB102458.76 / 33.2579.47 / 47.05
Log-CPBBlend61.00 / 30.7081.26 / 46.90
TRAPE51286.14 / 37.51145.57 / 79.01
TRAPE102452.12 / 30.3164.26 / 35.78
TRAPEBlend48.30 / 22.4262.62 / 32.53
MethodAnchor use640×768640\times768768×640768\times640
Log-CPB51263.38 / 22.1062.39 / 22.61
Log-CPB102449.27 / 26.2052.23 / 29.85
Log-CPBBlend47.44 / 19.5250.60 / 22.19
TRAPE51261.08 / 21.9661.22 / 21.47
TRAPE102447.05 / 27.4550.73 / 33.22
TRAPEBlend38.73 / 14.1742.88 / 19.51

TRAPE Blend lowers both metrics versus either TRAPE single anchor at all four targets and has the lowest observed scores among these six variants. This compares the positional encoding together with its anchor interpolation rule. Both axis and joint 2D biases can be cached; TRAPE combines compact axis storage with independent routing and requires no positional MLP evaluation during denoising. We will include these results and the evaluation protocol in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.