- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PixelDDT, which makes two main improvements over DiP. First, it connects Transformer features from different depths to different stages of the U-Net decoder. Second, it introduces Window Attention to enable direct feature interaction between neighboring patches.
Critical strengths
-
The motivation is reasonable. DiP separates global modeling from local decoding, but its decoder mainly relies on the final Transformer features. PixelDDT improves this information flow with only a small increase in parameters and computational cost.
-
The paper includes useful ablation studies. Depth Routing reduces FID from 2.74 to 2.31. The authors also test broadcasting the final-layer features to all decoder stages, showing that the improvement is not simply due to adding more connections. In addition, feature probes and intervention experiments provide insights into how the internal representations change.
Critical weaknesses
-
The novelty is somewhat limited. Connecting intermediate Transformer features to a multiscale decoder has already been explored in prior work such as DPT, and Shifted Window Attention is also a well-established technique. PixelDDT mainly combines these ideas within the DiP framework.
-
The paper does not clearly explain why deeper Transformer features should be routed to coarser decoder stages and shallower features to finer stages. Without comparisons to reversed routing or other depth assignments, it is unclear whether the proposed routing strategy is necessary.
-
The contribution of Cross-Patch Attention is unclear. Adding this module only reduces FID from 2.31 to 2.27. It is unclear how much of this improvement comes from cross-patch interaction itself, rather than the additional feature processing introduced by the attention module.
Questions for authors
-
How important is the Depth Routing order? What happens if the routing order is reversed, or if all decoder stages use features from the REPA-aligned layer? These comparisons would help separate the benefits of using multiple feature depths from those of the specific routing strategy.
-
How much of the improvement from Cross-Patch Attention comes from cross-patch interaction itself? Could the authors include a Patch-Only Attention baseline, using the same attention module but restricting interactions to within each patch? This would help distinguish the benefits of cross-patch communication from those of additional feature processing, and better justify the need for Cross-Patch Attention.