acceptodds
Withdrawn from ICLR 2027

Beyond Positional Adaptation: Calibrating Attention for Image Resolution Extrapolation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion transformers have rapidly advanced image generation, yet training them at higher resolutions remains expensive because dense attention scales quadratically with the number of image tokens. A prominent approach to training-free resolution extrapolation adapts rotary positional embeddings to spatial coordinates beyond the training range. However, higher resolution also expands the pool of tokens competing within each softmax, which can dilute attention to informative tokens even after positional adaptation. We introduce Cardinality-Calibrated Attention (CCA), a plug-and-play correction that compensates for this change in token count. CCA rescales attention logits by , where and are the current and native-reference key counts of each attention pool. The rule retains the chosen positional mapping and requires neither retraining nor fitted calibration parameters. It exactly recovers the native attention operator at the reference key count. Experiments on FLUX and Qwen-Image across multiple benchmarks demonstrate that CCA improves image quality in training-free resolution extrapolation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Generation, Diffusion Transformers, Rsolution Extrapolation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fLKWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work looks at another problem for diffusion transformers when generating higher‑resolution images beyond positional embedding mismatches. As more image tokens get introduced, a larger number of key values compete inside the softmax operation. This reduces how much focus important tokens receive. The authors propose CCA. It scales attention logits by the factor log⁡K/log⁡K0\log K/\log K_0, calculated from the current key count K and the key count K0K_0 used at the model’s original training resolution. This approach keeps positional embeddings unchanged. It needs no extra training and no manually‑tuned parameters. Tests are run on FLUX and Qwen‑Image. Results show CCA delivers better generation quality and sharper local details in most high‑resolution test scenarios.

Critical strengths
  1. Simple method that is easy to put into practice. CCA adjusts logit scaling purely based on key counts from the current input and the reference resolution. It requires no training and leaves the original positional mapping untouched. One particularly careful design choice: the method does not treat all attention groups the same way. Instead it applies different handling for image‑only, mixed image‑text, and fixed‑text‑key groups based on their real key counts.
  2. More complete theoretical analysis compared to common temperature‑based quick fixes. The authors show that keeping targeted token focus under given conditions requires scaling values that grow at least logarithmically together with K. They also work out the exact adjustment under a two‑group token setup, and analyse gaps between CCA’s output and perfect focus‑preserving results for finite‑size inputs. The paper fairly avoids presenting this simple formula as a perfectly optimal solution
Critical weaknesses

Test‑set improvements are inconsistent, and there is no statistical significance analysis. CCA improves results in many cases but can also hurt output quality, for example in certain rectangular‑resolution settings and some low‑resolution outputs from Qwen‑Image. The paper repeatedly notes that reported numbers are just single measured estimates. For many differences sitting between 0.01‑0.05, looking only at average scores cannot confirm whether improvements hold reliably.

Questions for authors

Could you add paired bootstrap tests, confidence intervals or significance testing? Many observed improvements are small, and CCA sometimes produces worse results. For this zero‑training calibration method, knowing whether gains hold reliably on average matters more than reporting isolated single‑run measurements.

Reviewer_v24fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Cardinality-Calibrated Attention (CCA) for training-free resolution extrapolation in diffusion transformers. CCA multiplies attention logits by log K / log K₀ using each attention pool's visible key count and a declared native-resolution reference. It preserves the chosen positional mapping, replaces inherited temperature corrections, and leaves unchanged key pools at unit scale. The analysis derives native operator recovery within a chosen logarithmic family and quantifies mass-preservation error in a two-population model. Experiments include controlled FLUX comparisons, fitted-temperature alternatives, Qwen-Image transfer, attention diagnostics, and human preference evaluation.

Critical strengths
  1. The calibration recipe is simple, precise, and theoretically well scoped. It specifies visible key counts, joint image–text pools, unchanged text-key pools, and removal of inherited temperature factors, and can be implemented through query scaling. The paper carefully distinguishes exact native operator recovery from approximate mass preservation and explicitly discusses failure cases. This makes it a usable and interpretable inference modification.
  2. Controlled comparisons and transfer provide useful empirical evidence. Table 2 compares original mscale, a square-root-log rule, a fitted constant, and a fitted count-dependent slope. CCA improves ImageReward over original mscale by 0.06/0.17 at 4K and 0.25/0.28 within the 6K cohort. Qwen transfer and original-pixel human preference results provide complementary evidence; the reported local-detail preference scores are 85% and 82%. These quantitative findings are a substantive strength, although the qualitative presentation needs broader and more directly comparable examples.
Critical weaknesses
  1. Insufficient qualitative examples. Despite focusing on high-resolution image quality and local detail, the paper provides only two qualitative generation comparisons, both at 4K. There are no corresponding visual examples for Qwen transfer or the 6K setting, and no systematic presentation of failure cases. This limited coverage makes it difficult to assess whether the reported improvements are visually consistent across prompts, resolutions, and backbones.
  2. Incomplete display of a key comparison figure. In the PDF view used for review, the left and right full-image panels of Figure 4 are blank; only the central detail crops, region markers, and connecting lines remain visible. This prevents inspection of the global results and the source context of the crops. It is a substantive presentation defect affecting the verification of visual evidence. The authors should provide a corrected, reliably viewable PDF and complete full-resolution images. An independent rendering displays the panels, so the precise cause of the display discrepancy remains unresolved.
  3. Limited mathematical novelty. The cited Scalable-Softmax already uses softmax((c log K)s) and considers replacement after pretraining. Within that family, c=1/log K₀ follows directly from native identity. The contribution therefore rests mainly on DiT-specific adaptation and empirical validation, making the limited and partly inaccessible visual evidence particularly consequential.
  4. Uncertain significance of small comparative gains. Several Table 2 differences relative to fitted alternatives are only 0.01–0.03, without paired per-prompt uncertainty estimates. The human-preference results also lack uncertainty estimates accounting for the three ratings clustered within each prompt. The results are promising, but the reliability of the smaller advantages is not established.
  5. Incomplete reproducibility details. The Qwen numerical native-grid anchor and complete inference settings are not specified, while the fixed-state attention diagnostic lacks a complete sampling protocol. These omissions make the cross-backbone transfer and mechanistic evidence harder to reproduce independently.
Questions for authors
  1. Qualitative coverage: Can you provide broader paired full-image and original-pixel comparisons covering diverse prompts, Qwen transfer, 6K outputs, and failure cases? Please clarify the pairing/seed and crop-selection protocol.
  2. Figure completeness: Figure 4's left and right full-image panels are blank in my PDF view, although the central crops remain visible. Could you provide a corrected PDF with reliably rendered panels and the original full-resolution comparison images?
  3. Analysis of existing results: Can you provide paired intervals for the primary 4K/6K quality contrasts and prompt-clustered intervals for human preference? This would clarify the larger baseline gains and near-ties with fitted alternatives.
  4. Implementation clarification: What are the exact Qwen native grid/token-count reference, packing/text-length conventions, and sampler configuration? Please also specify the layer/head/query and reference-pool selection protocol for Figure 2.
  5. Scope clarification: Does the joint-pool/query-placement advantage in Table 8 also hold in the already evaluated Qwen setting, or should that architectural conclusion remain explicitly limited to FLUX at 4K?
Reviewer_xLELOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper presents Cardinality-Calibrated Attention (CCA), a plug-and-play attention scaling for high-resolution image generation using pretrained diffusion transformers. CCA rescales attention logits according to the ratio of logarithmic key counts at the target and native resolutions. The method is evaluated on FLUX and Qwen-Image, demonstrating improvements over several RoPE positional extrapolation baselines.

Critical strengths
  1. Simplicity and applicability. CCA is a training-free, plug-and-play modification that can be combined with existing positional extrapolation methods.

  2. Experimental evaluation. The paper evaluates the method across multiple resolutions, diffusion backbones, and several positional extrapolation methods. Additionally, the paper includes comparisons with alternative temperature scaling strategies.

Critical weaknesses
  1. Limited novelty. Logarithmic attention scaling has already been studied in Nakanishi (2025), and a related approach was proposed for diffusion models by Jin et al. (2023). In particular, CCA essentially squares the scaling factor of Jin et al. (2023), i.e. (gCCA=gJin2)(g_{\mathrm{CCA}}=g_{\mathrm{Jin}}^2). While its adaptation to DiT is useful, the methodological contribution appears incremental.

  2. Clarity. The method section is difficult to follow, with extensive notation and mathematical derivations that make the proposed approach seem more complicated than it actually is. I believe the paper would benefit from a more concise and simple presentation of the method.

Questions for authors
  1. Could the authors clarify the conceptual and technical novelty of CCA compared with Jin et al. (2023) and Scalable-Softmax (Nakanishi, 2025)?

  2. Have the authors evaluated CCA on content-adaptive RoPE positional extrapolation methods such as SEGA (Rajabi et al. 2026)? If not, are there any limitations preventing such a combination?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.