- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the accelerating training technique for pixel diffusion. It proposes SemPix, which generates a semantic state in the frozen DINOv3 feature space with a semantic flow and conditions a shallow pixel flow on it. The claimed contribution is coupled training, allowing image-loss gradients to flow back into the semantic flow through the predicted condition.
Critical strengths
- It is a meaningful topic to accelerate pixel diffusion training, and decoupling it into semantic and pixel components is reasonable.
Critical weaknesses
-
The convergence comparison is confounded by LPIPS loss and other loss components. Every SemPix run includes in Eq. 7, while the baselines plotted in Fig. 4 do not, so the paper cannot distinguish its method from a known perceptual-loss effect.
-
The paper's own ablations do not support the claimed contribution. The paper's core contribution is CT, which it presents as gradient feedback. However, as shown in Tab. 3b, it only brings a FID improvement, which falls within the noise interval. Detailed ablations of each key component, including , are necessary.
-
Missing baselines. In Tab. 2, RAE, PiD, RAEv2, PixelDiT, and CoReDi are missing. Moreover, why are JiT+REPA and DeCo compared in Fig. 4 but not reported in Tab. 2?
-
Some potential inconsistency in the results.
- L320-321 claims "Table 1...SemPix achieves...IS 195.0", but Tab.1 reports IS 195.2.
- Tab.1 reports PixNerd-L/16 has params of 459M, but in Tab.2 it becomes 458M.
Questions for authors
-
Can the authors provide Tab. 3 rows without and without , and an oracle FID that conditions the pixel flow on real encoder features?
-
Since Eq. 2 and 4 are loss-equivalent to the velocity objective, what justifies -prediction for the semantic flow over -prediction?
-
This paper claims faster convergence on unguided scenarios. How does it perform in guided scenarios, which are more practical?
8 comments
Official Comment by Authors
W2: Is the CT gain within noise? Ablations of each component
We thank the reviewer for raising the question of statistical significance. To test whether the 0.1 FID difference lies within the noise range, we repeated the comparison in Table 3(b) of the paper with two more training seeds (three in total) and evaluated each model with six sampling seeds, i.e., 18 evaluations per setting. Gradient feedback gives lower FID in all 18 paired comparisons (mean 4.26 vs. 4.37; paired difference 0.11 ± 0.03, mean ± s.d. over the three training seeds). The gain is consistent across seeds and is not explained by noise.
This comment also helps us clarify the role of CT. The primary contribution of SemPix is generating an explicit semantic state and synthesizing pixels from it with a dedicated pixel flow, trained jointly with the semantic flow. The two-flow design accounts for most of the gain: with the same , the single-flow model reaches 10.00, while two independently trained flows already reach 4.56. CT is the training scheme that couples the two flows, and the two analyses below show what it changes in each flow. They use the protocol of Table 3 in the paper (L-size, 40 epochs, unguided, no ).
(a) CT adapts the pixel flow to the semantic flow's predictions. To examine this, we compare the oracle FID, obtained by conditioning the pixel flow on real encoder features (the analogue of rFID in latent methods), with the standard FID of sampled images (the analogue of gFID).
| Oracle FID | FID | |
|---|---|---|
| Independent training | 2.72 | 4.56 |
| + pixel flow trained on predicted states (detached) | 3.12 | 4.39 |
| + gradient feedback (CT) | 3.21 | 4.29 |
Although SemPix generates in pixel space, the independently trained pixel flow shows a reconstruction–generation trade-off akin to that known from latent diffusion (e.g., VA-VAE; Yao et al., 2025): it reconstructs best (oracle FID 2.72) but generates worst (4.56), because it is trained on real encoder features yet receives generated states at inference. Noise augmentation, which our independent baseline already uses (Table 3(d) of the paper), only approximates this gap. CT addresses it more directly: the pixel flow is trained on the semantic flow's predictions (4.56 → 4.39), and the semantic flow adapts to the pixel flow through image-loss gradients (4.39 → 4.29). The slightly higher oracle FID does not affect generation: it measures the pixel flow on real encoder features, a setting that never occurs at inference, where the pixel flow is always conditioned on states produced by the semantic flow.
(b) The feedback improves the semantic flow itself. Table 3(b) of the paper compares two complete models, so it cannot tell which flow the 0.10 comes from. We cross-pair them at sampling time, taking the semantic flow from one and the pixel flow from the other. In the CT model, image-loss gradients reach the semantic flow; in the detached model, the condition is detached, so they do not. Both pixel flows are trained on predicted states in the same way.
| Semantic flow ↓ / Pixel flow → | CT model | Detached model |
|---|---|---|
| CT model | 4.29 | 4.27 |
| Detached model | 4.48 | 4.39 |
Within each column the pixel flow is fixed, and in both columns the semantic flow from the CT model is better (by 0.19 and 0.12). Trained with a feature-space loss alone, the semantic flow cannot tell which deviations matter for the image; the image-loss gradient supplies that signal. Notably, the semantic flow from the CT model helps even a pixel flow it was never trained with: paired with the detached model's pixel flow, it reaches 4.27, on par with its own jointly trained pixel flow (4.29). If the feedback merely co-adapted the two flows, this gain would disappear once the pixel flow is swapped. Instead, the feedback improves the sampled states themselves, and since both semantic flows target the same frozen DINOv3 space, the improvement transfers to another pixel flow. The semantic-side gain is no smaller than the overall gap in Table 3(b) (0.10), and the pattern holds across several sampling seeds.
Ablations of the individual loss terms, including , are in our response to W1 & Q1; Table 3(c, d) of the paper covers initialization and condition noise.
Official Comment by Authors
W3: Missing baselines
We thank the reviewer for these suggestions. The table below adds all the mentioned methods, as well as further pixel-space baselines: DeCo-XL/16, PixelREPA (representation alignment on JiT), PixelGen-XL/16 and V-Co (ImageNet 256×256; "–": not reported; the second term in Parameters is the separately trained decoder).
| Method | Parameters | Epochs | Unguided FID↓ / IS↑ | Guided FID↓ / IS↑ |
|---|---|---|---|---|
| Latent Space | ||||
| RAE, DiT$^\text{DH}$-XL | 839M + 415M | 800 | 1.51 / 242.9 | 1.13 / 262.6 |
| RAEv2, DiT$^\text{DH}$-XL | 839M + 415M | 80 | 1.65 / 228.0 | 1.06 / 255.3 |
| CoReDi, SiT-XL/2 | 675M + 50M | 400 | 3.3 / 176.8 | 1.58 / 297.2 |
| Pixel Space | ||||
| DeCo-XL/16 | 682M | 320 | 14.88 / 88.2 | 1.90 / 303 |
| PixelDiT-XL | 797M | 80 | – | 2.36 / 282.3 |
| PixelDiT-XL | 797M | 320 | – | 1.61 / 292.7 |
| PixelGen-XL/16 | 676M | 80 | 5.11 / 159.2 | – |
| PixelGen-XL/16 | 676M | 160 | – | 1.83 / 293.6 |
| PixelREPA-L/16 | 459M | 600 | – | 2.11 / 309.5 |
| PixelREPA-H/16 | 953M | 600 | – | 1.81 / 317.2 |
| V-Co-L/16 | 918M | 200 | – | 2.10 / 243.0 |
| V-Co-H/16 | 1.9B | 200 | – | 1.85 / 246.5 |
| SemPix-L/16 | 471M | 80 | 3.27 / 202.9 | 2.17 / 245.0 |
| SemPix-H/16 | 970M | 160 | 2.40 / 225.2 | 1.83 / 250.4 |
Among pixel-space methods, SemPix reaches lower FID with less training. Without guidance, SemPix-L/16 reaches 3.27 at 80 epochs, compared with 5.11 for PixelGen-XL/16 at the same budget and 14.88 for DeCo-XL/16 at 320 epochs. With guidance, it reaches 2.17 at 80 epochs, below PixelDiT-XL at the same budget (2.36) with fewer parameters. SemPix-H/16 reaches 1.83 at 160 epochs, comparable to models trained considerably longer (V-Co-H/16: 1.85 at 200 epochs; PixelREPA-H/16: 1.81 at 600 epochs).
RAE and RAEv2 generate in a representation latent space and reconstruct images with a separately trained ViT-XL decoder, whereas SemPix has no separate decoder stage. PiD, which we discuss in the related work of the paper, is not included in the table, as it evaluates latent decoding and upsampling for text-to-image models and reports no class-conditional ImageNet results. For CoReDi, the latent-space model is listed above; its pixel-space model reports only an unguided result at 200K steps (FID 21.5), which is already in Table 1 of the paper.
Official Comment by Authors
We sincerely thank the reviewer for the detailed and technical review and the constructive suggestions. Following them, we conducted new experiments, summarized below:
We are particularly grateful for the suggestion of oracle FID. Because SemPix splits generation into a semantic flow and a pixel flow, this diagnostic lets us localize where each loss acts, and it has substantially strengthened our analysis.
W1 & Q1: Perceptual loss, loss ablations, and oracle FID
We thank the reviewer for raising this point. Following this comment, we added loss-matched comparisons, the ablation rows requested in Q1, and the corresponding oracle FIDs. The results show that the gain from the semantic flow holds with or without Lperc, which acts on the rendering quality of the pixel flow.
(a) Loss-matched comparison with a single-flow model (40 epochs, unguided FID):
† Without Lperc, the pixel flow relies more on the low-noise end of the trajectory to recover fine detail, so we allocate more steps there: 25 pixel steps with a time shift of 0.75 (default: 10 steps, shift 10), and 25 semantic steps to keep the total at 50. The same applies to † in (b).
The single-flow numbers are from the PixelGen ablation, and within each column the pixel-space losses match. Without any perceptual loss, adding the semantic flow already lowers FID from 23.67 to 15.83, so the gain does not depend on Lperc. The higher FID in this column comes from the pixel side: without perceptual supervision, the pixel flow renders texture and fine detail less well even from real features (oracle FID ≈ 12.4). The semantic flow trained without Lperc remains strong: paired with a pixel flow trained with Lperc, it reaches 4.05.
(b) Where each loss acts: ablations with oracle FID (Q1). Oracle FID conditions the pixel flow on real DINOv3 features, bypassing the semantic flow, and thus measures the pixel flow alone.
(c) Convergence under matched losses. Without a perceptual loss on either side, SemPix at 40 epochs (15.83) is already below JiT + REPA at 80 epochs (18.60, Fig. 4 of the paper). With perceptual supervision on both sides, Table 1 of the paper already contains the comparison: 3.87 for SemPix-L/16 vs. 7.53 for PixelGen-L/16 at 40 epochs, while the larger PixelGen-XL/16 (676M) reaches 5.11 at 80 epochs. Against either class of baseline, SemPix reaches lower FID with less training.