- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work looks at another problem for diffusion transformers when generating higher‑resolution images beyond positional embedding mismatches. As more image tokens get introduced, a larger number of key values compete inside the softmax operation. This reduces how much focus important tokens receive. The authors propose CCA. It scales attention logits by the factor , calculated from the current key count K and the key count used at the model’s original training resolution. This approach keeps positional embeddings unchanged. It needs no extra training and no manually‑tuned parameters. Tests are run on FLUX and Qwen‑Image. Results show CCA delivers better generation quality and sharper local details in most high‑resolution test scenarios.
Critical strengths
- Simple method that is easy to put into practice. CCA adjusts logit scaling purely based on key counts from the current input and the reference resolution. It requires no training and leaves the original positional mapping untouched. One particularly careful design choice: the method does not treat all attention groups the same way. Instead it applies different handling for image‑only, mixed image‑text, and fixed‑text‑key groups based on their real key counts.
- More complete theoretical analysis compared to common temperature‑based quick fixes. The authors show that keeping targeted token focus under given conditions requires scaling values that grow at least logarithmically together with K. They also work out the exact adjustment under a two‑group token setup, and analyse gaps between CCA’s output and perfect focus‑preserving results for finite‑size inputs. The paper fairly avoids presenting this simple formula as a perfectly optimal solution
Critical weaknesses
Test‑set improvements are inconsistent, and there is no statistical significance analysis. CCA improves results in many cases but can also hurt output quality, for example in certain rectangular‑resolution settings and some low‑resolution outputs from Qwen‑Image. The paper repeatedly notes that reported numbers are just single measured estimates. For many differences sitting between 0.01‑0.05, looking only at average scores cannot confirm whether improvements hold reliably.
Questions for authors
Could you add paired bootstrap tests, confidence intervals or significance testing? Many observed improvements are small, and CCA sometimes produces worse results. For this zero‑training calibration method, knowing whether gains hold reliably on average matters more than reporting isolated single‑run measurements.