- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces a new type of Image transformers called AceiT. The problem with the diffusion generation is that UNets are not scalable while providing high quality results, while Image transformers are computationally expensive. This paper uses adaptive attention residuals to make it computationally feasible to use pixel level image transformers. While increasing scalability with chunk downsampling.
Critical strengths
1- The paper is very well written, it motivates the problem at hand and considers the challenges that is associated with it.
2- The adaptive attention residuals to overcome pixel space inefficiencies and limiting the scalability of conventional skip connections.
3- Query sharing and hybrid residuals improves generation quality while reducing computation and memory.
4- Chunk downsampling and adaptive attentions allows for arbitrary compression ratios, hence ensuring the scalability of the model.
Critical weaknesses
(Minor)
1- How would the method works for different resolutions and it is required for a method that is scalable to show performance in different resolutions.
2- While FID improves, it is a very broad measure and GenEval is not broken down to its subtask, so it is not clear how the proposed solution will affect the downstream tasks.
Questions for authors
1- In table 5 DeCo-XXL/16 has a quite similar performance to the proposed model but with half the parameters. Is the throughput has considered the fact that due to less memory, two of that model can be used parallelly on the same resource as AceiT.
2- How would performance of AceiT differ for different resolutions?
3- How AceiT affect downstream tasks, and how would latent space models in Table 5 compare based on FID and IS to the method?
Suggestions:
1- SDXL and Flux 1 are not very recent and currently SD3.5 and Flux2 are the most recent versions. I believe since AceiT throughput is much better than those models while having much less parameters, it would be good to also include them, since I believe the GenEval performance for those models still is less than the AceiT
2- The figures and tables are usually very far from their explanation in the text, that makes it quite hard for reader to go 2 pages back to see the explanation, while at the same time reader becomes confused when they see a figure which its explanation starts 2 pages later, so it would be great if these can be fixed.