acceptodds
Withdrawn from ICLR 2027

PatchBridge: Decoupling Patch Roles for Pixel-Space Diffusion Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

14 views

Abstract

Pixel-space diffusion Transformers denoise RGB images directly, avoiding the fixed autoencoder compression interface common in latent diffusion. Yet many process one patch grid throughout the backbone, asking it to support global context modeling, full-depth denoising, and late local refinement. These roles differ in their spatial and computational requirements; we call their forced sharing of one grid patch-role coupling. We introduce PatchBridge, which assigns role-specific patch sizes and lifetimes to three coordinated streams. A compact semantic stream maintains global context through sparse updates, a structural stream carries the principal denoising state through the full backbone, and a finer detail stream activates in later layers for local refinement. This allocation preserves full-depth denoising without processing the finest grid at every layer. Parent-to-child spatial modulation supplies aligned context to finer streams, while child-to-parent pooled feedback returns updated evidence to coarser streams, keeping their representations coordinated as views of one evolving denoising state. At the output, an overlapping clean-image readout decouples detail-token stride from prediction support, allowing neighboring tokens to jointly estimate shared pixels. PatchBridge achieves FIDs of 1.44 and 1.54 on class-conditional ImageNet at 256x256 and 512x512, respectively. In text-to-image generation, it scores 0.87 on GenEval and 84.1 on DPG-Bench. These results support role-specific patch allocation for direct pixel-space generation.

Primary area:
generative models
Keywords:
Image generation, Diffusion model, Pixel-level diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fVDTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PatchBridge, a pixel-space diffusion Transformer that explicitly decouples three roles traditionally served by a single patch grid: global semantic context, persistent denoising state, and late-stage fine-detail refinement. The model implements this design through three coordinated streams, each with a patch size and lifespan tailored to its role: a semantic stream that uses coarse patches and is updated at only a few stages; a structural stream that uses medium-sized patches and persists throughout the network; and a detail stream that uses fine patches and is activated only in the final K blocks. Adjacent streams communicate through bidirectional cross-scale interactions, including parent-to-child spatial modulation and child-to-parent pooled feedback. In addition, an overlapping clean-image readout allows neighboring fine-grained tokens to jointly predict pixels in overlapping regions. PatchBridge achieves strong results on ImageNet class-conditional generation at resolutions of 256 and 512, as well as in a text-to-image setting with relatively limited training data. Ablation studies also demonstrate the contributions of individual components.

Critical strengths
  1. The paper proposes a clear and practically meaningful approach to jointly allocating spatial granularity and processing depth. Decoupling patch roles through three streams operating at different scales and with different lifespans is a simple and well-motivated way to address the competing demands of global context, persistent denoising, and local detail in pixel-space DiTs.

  2. The experiments cover multiple model sizes, image resolutions, and conditional generation tasks, demonstrating broad applicability.

Critical weaknesses
  1. The experimental comparisons are not sufficiently clear or complete. In Table 1, the number of function evaluations (NFE) should be used to compare sampling costs fairly. Under this accounting, JiT and PixelU should both be reported as 100 × 2 due to classifier-free guidance (CFG). The paper therefore lacks a clearly established comparison under equivalent sampling budgets. Meanwhile, the Precision and Recall for papers such as PixelU are missing in the table. In addition, it lacks a strictly comparable result at approximately 800M parameters against the strong PixelDiT-XL/16 baseline, which achieves an FID of 1.54 with 800M parameters after 800 training epochs, as reported in its appendix.

  2. The advantages of the proposed network architecture are not sufficiently demonstrated. More comprehensive comparisons with other representative architectures, such as Deco[1],HyperDiT[2], and PixelDiT[3], should be included under comparable computational budgets, rather than only comparing against JIT. In particular, HyperDiT is highly relevant as it also explores the decoupling of semantic information and fine-grained details, employing collaborative designs with different patch sizes to enhance representation capability. Furthermore, FLOPs should be reported for more model sizes, especially the H-sized model in 512 * 512 resolution .

References

[1]Ma Z, Wei L, Wang S, et al. Deco: Frequency-decoupled pixel diffusion for end-to-end image generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 43600-43610.

[2]He Y, Ma L, Guo Z, et al. Hyperdit: Hyper-connected transformers for high-fidelity pixel-space diffusion[J]. arXiv preprint arXiv:2605.15741, 2026.

[3]Yu Y, Xiong W, Nie W, et al. Pixeldit: Pixel diffusion transformers for image generation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 14273-14282.

Questions for authors

In addition to the Weaknesses raised above:

  1. Does the semantic stream make a substantial contribution? Using a larger patch size but a smaller hidden dimension may lead to substantial information loss. Table 5 also suggests that including or excluding the semantic stream has only a small effect on the results, potentially smaller than the variation across random seeds.

  2. The functional division of "semantics/structure/details" is mainly focused on design interpretation, lacking direct verification, such as visualization or clustering of its features.

  3. Both the asymmetrical flow models[1] and EG-FM[2] mentioned that the FID evaluated using JiT would be lower. The other methods in Tables 1 and 2 were evaluated using ADM. What were the results of PatchBridge evaluated using ADM?

  4. Is there a qualitative result for T2I?

References

[1]Chen H, Ackermann J, Kim M, et al. Asymmetric Flow Models[J]. arXiv preprint arXiv:2605.12964, 2026.

[2]Tong H, He Y, Li F, et al. Energy-Guided Flow Matching[J]. arXiv preprint arXiv:2608.05811, 2026.

Reviewer_ifc4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces PatchBridge, a pixel-space diffusion transformer with three streams intended for global context modeling, full-depth denoising, and local detail refinement. The streams use different patch sizes and processing depths, exchanging information through top-down spatial modulation and bottom-up pooled feedback. An overlapping readout combines neighboring detail-token predictions. Experiments on class-conditional ImageNet and text-to-image generation report improvements over the corresponding JiT and MiniT2I baselines, supported by component ablations and a capacity-matched comparison.

Critical strengths
  1. The central architectural idea is reasonable: allocating different spatial resolutions and processing depths can provide fine-grained refinement without processing the finest token grid throughout the entire backbone.

  2. The reported results are competitive across model sizes and generation settings, including FIDs of 1.44 and 1.54 on ImageNet at 256×256 and 512×512, respectively, and improvements over MiniT2I on both GenEval and DPG-Bench.

Critical weaknesses
  1. The presentation is unnecessarily abstract in places. Terms such as “patch-role coupling,” “role-specific patch sizes and lifetimes,” and “views of one evolving denoising state” obscure otherwise concrete architectural choices. The dense notation in the method section and small labels in Figure 2 further hinder readability.

  2. The related-work discussion does not sufficiently establish the architectural distinction from the closest methods. PixelDiT, DeCo, and PixelU are cited, but their global/local or multiscale designs receive only brief descriptions, leaving the specific novelty of the proposed role allocation insufficiently contextualized.

  3. The claimed functional specialization is not directly validated. All three embedders process the same noisy image, and different patch sizes and computation schedules provide inductive biases rather than evidence of distinct semantic, structural, and detail roles. Table 5 demonstrates component utility, but overall FID improvements do not establish the claimed division of functions. The contribution of finer spatial granularity remains insufficiently separated from that of the proposed role allocation. At 256×256, the streams contain 64, 256, and 1024 image tokens, compared with 256 for JiT/16. At 512×512, the structural stream alone has 1024 tokens versus 256 for JiT-H/32, without a corresponding high-resolution FLOPs comparison.

Questions for authors
  1. Could the authors explain the architecture using a concrete configuration before introducing the full notation, clarify the network-depth meaning of “lifetimes” earlier, and improve Figure 2’s legibility?

  2. Could the authors compare PatchBridge more explicitly with PixelDiT, DeCo, and PixelU in terms of persistent multiscale states, processing depths, and cross-scale communication?

  3. Are the semantic, structural, and detail labels intended as design motivations or empirically established roles? Could feature probing or selective interventions demonstrate their relative functional specialization beyond aggregate FID changes?

  4. Could the authors add a baseline that increases fine-grained tokens without the proposed role-specific allocation under comparable parameter and FLOPs budgets? Please also report the corresponding FLOPs for PatchBridge and JiT at 512×512 to clarify the high-resolution quality–compute trade-off.

Reviewer_z7r1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a three-stream architecture for pixel-space image generation. It uses different patch sizes for three specialized streams: a semantic stream for global context, a structural stream for the main denoising process, and a detail stream for local refinement. Extensive experiments and ablation studies demonstrate the effectiveness of the proposed approach.

Critical strengths

The paper’s main strength is its extensive experimental evaluation. It demonstrates clear overall improvements at two image resolutions, across model scales, and when extended to text-to-image generation. The ablation studies are also thorough. In addition, the paper is clearly written and easy to follow, with a well-articulated motivation.

Critical weaknesses
  1. the contribution is primarily architectural, organizing computation into semantic, structural, and detail streams. However, related methods such as PixelDiT, DeCo, and PixelU have already explored related forms of semantic–detail separation or hierarchical processing. The theoretical justification is relatively limited, making the paper predominantly engineering-oriented.

  2. Although the paper assigns semantic, structural, and detail roles to the three streams, it does not directly analyze what each stream actually learns. The rationale for processing details only in the final layers also requires further validation. Comparisons between early, uniformly distributed, and late-stage detail processing would help substantiate this design choice. The discussion of relevant work as

[1] DiP: Taming Diffusion Models in Pixel Space

  1. The paper does not provide a comprehensive comparison of training time or report a user study assessing the perceptual quality of generated images.

  2. In the “Patch-boundary analysis,” the qualitative examples suggest improved boundary continuity, but they do not establish that this improvement accounts for the FID gains. Can the authors provide quantitative evidence linking reduced patch-boundary artifacts to improved generation performance?

Questions for authors

Please address the questions corresponding to the weaknesses listed above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.