- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper targets the inference cost of the VAE decoder in latent diffusion models and the oversmoothed textures of pixel diffusion models. It adds a linear pixel head in parallel with the denoising head of a pretrained DiT, finetunes the last few blocks with L1, LPIPS and adversarial losses plus a trajectory regularization toward the frozen model, and uses this head only at the final step to output pixels directly. The method is evaluated on ImageNet (SiT-REPA, REPA-E, JiT, AnisoFlow), text-to-image (FLUX.2 klein, AnisoFlow-T2I) and text-to-video (Wan2.1 14B), where it removes the decoder with comparable or better FID and lowers warping error on video.
Critical strengths
The motivation and intuition are sound.
The linear probe in Fig. 3 makes a convincing case that last-layer DiT tokens already carry rich pixel information. The idea of separating the final pixel output from the denoising output is simple and well motivated.
Critical weaknesses
Weakness 1. The method seems does not really decode, and its logic does not extend to recent few/one-step models.
At t=0, where the head acts as a pure decoder, the quality gain disappears. Sec. 3.2 states that quality “substantially worsens”, and Fig. 4(b) shows results as smooth as the baseline, although the Fig. 4 caption’s “matches or outperforms the baseline’s” conflicts with its own “recovers the gain” and should be clarified. The quality gain therefore depends on evaluating the head at a nonzero timestep, which for the latent models is t=0.08 on ImageNet and t=0.3 for FLUX.2 klein Base and Wan (Table 4). Wan’s pixel head is never even trained below t≈0.125 after shifting (p_pix = [0.02, 0.1] with shift 7). At this noise level fine structure such as small text or mesh is likely not yet determined, so the head generates the remaining trajectory in one shot under L1+LPIPS+GAN losses. This looks much closer to GAN-like generation than to decoding. The comparison thus mixes decoder removal, a sampling schedule that ends at t=0.3, and adversarial generation, and no control separates them on Wan, where the largest latent-model gain is reported (e.g., the same final timestep with a GAN-tuned latent final step and the original decoder). The t_final sweep (Fig. 4(a)) and the adversarial-loss ablation (Table 2) address this only partially, on ImageNet and text-to-image. Notably, without the adversarial loss the COCO FID gain on AnisoFlow-T2I disappears (28.93 vs. 28.86 for the baseline, Table 2). The design is also close to the denoising decoder of LTX-Video (HaCohen et al., 2024) and the early termination in PiD (Lu et al., 2026). LTX-Video is cited only as a generic latent model, and PiD’s early termination is not discussed. The argument also weakens as the step count drops. In the 4-step FLUX model the head already jumps from t=0.7 and only ties the baseline. In one-step generators (DMD, flow maps) there is no trajectory to preserve, and paired L1/LPIPS becomes ill-posed. One-step models and few-step video models, including the 4-step Wan 1.3B used as motivation in the introduction, are not evaluated, even though this is where decoder cost matters most. Since few- and single-step generation is now widely used and the paper targets generation efficiency, this limits its generality. If DWD does not apply to one-step models, simply using such a model with its decoder would likely be faster.
Weakness 2. Baselines are limited to the vanilla models, with no cost-matched methods that share the same goal.
The clearest gap is AsymFlow (Chen et al., 2026a), which converts the same FLUX.2 klein 9B into a decoder-free pixel model at roughly the same per-step cost and reports gains over its latent base, whereas DWD only ties it. The paper cites AsymFlow and borrows its Oklab output and captions but reports no numbers. L2P (Chen et al., 2026c) also removes the VAE from a pretrained latent model by training only shallow layers and reports much faster native 4K inference. FlashDecoder (arXiv:2607.14898) targets the same Wan decoder bottleneck without touching the generator. These are only examples; my main point is that, since DWD aims at efficiency, it should be compared against other methods with the same goal, not only against the vanilla base models. I appreciate that the main goal is cost reduction, so matching the baseline quality would be enough if the savings were meaningful. However, since the inference cost of DWD is reported only in FLOPs, without wall-clock time or memory, it is unclear whether it is more efficient than these methods. Finally, the existing comparisons are not controlled. For the latent models, DWD alone gets a different final timestep, retuned guidance for SiT-REPA (App. A.3), and curated ≥720p training data for Wan. Its frozen blocks are also quantized to FP8 during finetuning, which may explain its robustness under 8-bit inference (Table 3).
Also, the pixel-space results rely mainly on AnisoFlow, which is unpublished and cannot be verified. In text-to-image, the only pixel baseline is AnisoFlow-L/16-T2I, which is much weaker than current state-of-the-art generators (COCO FID 28.86 vs. 15.13 for FLUX.2 klein Base). The gains from DWD may therefore look significant mainly because of this weak baseline.
References
- Chen et al., Asymmetric Flow Models, arXiv:2605.12964, 2026.
- Lu et al., PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion, arXiv:2605.23902, 2026.
- HaCohen et al., LTX-Video: Realtime Video Latent Diffusion, arXiv:2501.00103, 2024.
- Chen et al., L2P: Unlocking Latent Potential for Pixel Generation, arXiv:2605.12013, 2026.
- FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers, arXiv:2607.14898, 2026.
Questions for authors
The main claim of the paper feels too broad. The title and introduction frame DWD as a way to remove the decoder for efficiency, but the strongest results come from pixel models that have no decoder at all (texture detail and ImageNet FID), and the video section focuses on warping error. It is unclear which of these is the main contribution. Could the authors clarify which one they consider central? Narrowing the scope, either to efficiency with wall-clock evidence in few-step settings or to the dual-head separation principle, might make the paper more convincing.
6 comments
Response to Reviewer P8dt (2/2)
W1: Applicability to few-step and one-step models, including the 4-step Wan 1.3B used as motivation
We added the rCM-distilled 4-step Wan2.1 1.3B model from our introduction. We tune its last 8 of 30 blocks and keep the baseline's exact timesteps:
| Method | Warping error (×10⁻³)↓ | HPSv3↑ | HPSv2.1↑ | Subject consist.↑ | Backgr. consist.↑ | Temporal flicker.↑ | Motion smooth.↑ | Dynamic degree↑ | Aesthetic quality↑ | Imaging quality↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| Wan2.1 1.3B (4-step) | 2.84 | 8.32 | 0.265 | 0.938 | 0.962 | 0.976 | 0.979 | 0.825 | 0.660 | 0.669 |
| DWD | 2.07 | 8.46 | 0.258 | 0.946 | 0.959 | 0.984 | 0.983 | 0.811 | 0.657 | 0.648 |
| DWD (self-distillation) | 2.45 | 8.45 | 0.261 | 0.942 | 0.956 | 0.982 | 0.981 | 0.822 | 0.656 | 0.631 |
DWD reduces warping error by 27% and improves temporal flickering. Note that the 1.3B model is an unfavorable setting for DWD. It is much smaller, and tuning only its last 8 of 30 blocks leaves limited learnable capacity. The 14B model would not have this limitation, but we could not run it within the rebuttal period.
DWD does not target one-step models, since they have no trajectory to preserve. For two-step models, we acknowledge that the paired L1/perceptual loss may become ill-posed, but its target can be easily swapped. There are two options:
- Self-distillation. The regular DWD loss is still applied at small timesteps, and the final step's L1/perceptual target becomes a chain of the reference model's native final step, renoising to the DWD timestep, and the DWD pixel output. This is the self-distillation row above, which also reduces warping error and improves temporal consistency.
- Distill the native decoder. The final step's L1/perceptual target becomes the reference model's native final step followed by the native decoder. This may not improve quality, but it still cuts inference cost substantially. In a preliminary experiment on a 2-step MiniMax-H3 model, it reduces overall latency from 9.9 s to 6.2 s (1.6× faster).
W1: A one-step model with its decoder would likely be faster
On Wan 1.3B and MiniMax-H3, the decoder is slower than one denoiser step. Also, one-step T2I/T2V generation typically has much lower quality than two-step generation, and a decoder cannot recover the corrupted content.
W2: Comparison with methods that share the same goal (AsymFlow, L2P, FlashDecoder)
We evaluated AsymFlow finetuned from the same FLUX.2 klein 9B Base model on the same data:
| Method | FID↓ | pFID↓ | OneIG↑ | LongText↑ | Training cost (H100) |
|---|---|---|---|---|---|
| FLUX.2 klein Base 9B | 15.13 | 18.69 | 0.556 | 0.909 | -- |
| AsymFLUX.2 klein | 18.10 | 22.62 | 0.506 | 0.777 | 1100 GPU hours (15K iter) |
| FLUX.2 klein DWD | 15.05 | 18.38 | 0.556 | 0.908 | 350 GPU hours (26K iter) |
AsymFlow is substantially weaker, partly due to its sensitivity to training data and partly due to the difficulty of adapting a latent trajectory to a pixel trajectory. DWD matches the base model at about a third of the training cost. L2P is also substantially weaker than its base model (Z-Image Turbo): OneIG 0.499 vs. 0.528, LongText 0.787 vs. 0.917. FlashDecoder is not open source, so we could not compare with it.
W2/Q1: Main goal of the paper
Practical cost reduction is not the main goal. The main goal is to find a minimal working recipe for latent models, and to test whether the same principle transfers to pixel models. We show that a second decoder network is redundant, and that two separate heads on the same backbone make a compact, high-quality design. To our knowledge, no decoding acceleration work makes this point. This understanding is our primary contribution, and cost reduction is a secondary benefit.
W2: Pixel-space results rely mainly on AnisoFlow, which is unpublished, and its text-to-image baseline is much weaker than FLUX.2
AnisoFlow is included in the supplementary material. On ImageNet it gets a much better FID than JiT without REPA (1.55 vs. 1.90). On text-to-image it is stronger than the larger latent i1-1.3B model, and much stronger than MiniT2I of the same size (0.9B):
| Model | OneIG↑ | LongText↑ | COCO FID↓ |
|---|---|---|---|
| i1-1.3B (latent) | 0.511 | 0.853 | 62.3 |
| MiniT2I | 0.356 | 0.106 | 32.6 |
| AnisoFlow-L/16-T2I | 0.516 | 0.864 | 20.1 |
It is expected to be weaker than FLUX.2 klein 9B, given its smaller size and different training data. DWD also improves JiT on ImageNet (1.90 to 1.85), so the pixel-model gains do not depend on AnisoFlow alone.
We hope our replies have addressed the reviewer's concerns. In particular, DWD keeps the sampler's own final timestep for the pixel and 4-step models, and our new 4-step Wan results use the baseline's exact timesteps. We kindly ask the reviewer to consider raising the score, and we would be glad to know if any concerns remain.
Thank you for the detailed response. The new wall-clock and memory results, the 4-step Wan 1.3B experiment and the finetuned-decoder baseline address several of my concerns, and stating that one-step models are out of scope is helpful. A few points remain.
1. AsymFlow (W2). Your retrained AsymFlow is much worse than the FLUX.2 base (FID 18.10 vs. 15.13, LongText 0.777 vs. 0.909), while the AsymFlow paper reports gains over its latent base. Could you evaluate the official AsymFlow checkpoint if available, or report the metrics used in that paper, so that the comparison does not depend on your retraining?
2. AnisoFlow numbers (W2). The new table lists COCO FID 20.1, OneIG 0.516 and LongText 0.864 for AnisoFlow-L/16-T2I, while Table 2 reports 28.86, 0.528 and 0.850. Could you clarify which evaluation protocol each uses?
3. Main goal (Question). The response says cost reduction is a secondary benefit, but the title and abstract present DWD mainly as removing the decoder to reduce inference cost. I would recommend at least revising the abstract, both in the PDF and on OpenReview, to match this clarified goal.
Response to Reviewer P8dt (1/2)
We thank the reviewer for finding the motivation and intuition sound, the linear probe in Fig. 3 convincing, and the separate pixel output simple and well motivated.
W1: Whether DWD "decodes", given that the quality gain disappears at t=0
DWD is designed not to decode the clean latents. Its main point is that the final pixels should come from a separate output on the DiT, evaluated before the trajectory is fully denoised. The t=0 experiment is meant to support this, so we do not consider it a weakness. In the Fig. 4 caption, we meant that at tfinal=0 the FID goes back to roughly the baseline level, and a small positive timestep recovers DWD's gain. We will rephrase the caption.
W1: The nonzero final timestep means the head generates the remaining trajectory in one shot, closer to GAN-like generation than to decoding
t=0.3 for video is a very small timestep under a shift of 7, which corresponds to t≈0.058 before shifting. This needs to compensate for the resolution and frame count of video. The same principle applies to decoder training. Recent decoders such as RAE add noise with a standard deviation up to 0.8 to unit-variance latents during training. The adversarial loss is not essential to DWD (Table 2). The improved moving details on Wan and the fully preserved text rendering on FLUX.2 (LongText-Bench 0.909 vs. 0.908) also show that the head does not damage content.
We skip tiny timesteps for Wan (t<0.02 before shifting) because Wan's own outputs there have strong color drift and detail degradation (presumably due to logit-normal pretraining and shared modulation). Regular sampling grids do not use these timesteps anyway. We will clarify this in the revised paper.
W1/W2: Controlled comparisons for the final timestep and the adversarial loss
On ImageNet and text-to-image, Fig. 4 controls the final timestep and Table 2 the adversarial loss. DWD keeps the sampler's own final timestep for the pixel models and the 4-step models (listed as "sampler" in Table 4). The pixel models still improve substantially under the same final timestep (Table 1), so the gains come from the dual-head separation. Fig. 4(a) also sweeps the final timestep of a latent baseline (REPA-E) with its native decoder, and this does not improve over its default setting. Setting the final timestep explicitly is also common in prior work, e.g., SiT (also used by REPA and REPA-E) sets the final sigma to 0.04, much earlier than a 250-step uniform grid.
Repeating these controls on Wan 14B is very expensive (VBench inference alone costs 260 GPU hours), so we could not run them within the rebuttal period. Instead, our new 4-step Wan 1.3B experiment keeps the baseline's exact timesteps, and DWD still reduces warping error (part 2).
W1: Without the adversarial loss, the COCO FID gain on AnisoFlow-T2I disappears
DWD's largest gain on pixel models is in patch FID, which reflects fine-grained detail. Without the adversarial loss, patch FID is still clearly better than the baseline (34.52 vs. 36.59, Table 2).
W1: Similarity to the denoising decoder of LTX-Video and the early termination in PiD
LTX-Video and PiD both use a separate decoder network (PiD uses a second pixel diffusion model), while DWD removes the decoder and reads the pixels out from the DiT itself. Early termination is not the core idea of DWD. We will add a discussion of these works and make the architectural distinction clearer.
W2: Retuned guidance for REPA
Tuning the guidance is common practice on ImageNet, and the baselines themselves use tuned guidance (e.g., AsymFlow states this explicitly). We verified that retuning the baseline guidance did not give a clearly better FID. For a fair comparison, we also retune it for each DWD model. DWD does not change the optimal guidance for most models, and REPA is the only exception.
W2: Frozen blocks quantized to FP8 during finetuning, possibly explaining the robustness under 8-bit inference
The FP8 quantization of the frozen blocks is only there to speed up training. The reference model is also quantized to FP8, while the learnable layers are trained in FP32 with BF16 mixed precision. The BF16 tail therefore learns from a slightly degraded FP8 reference, which works against DWD. Our main results and all qualitative results are under BF16. The 8-bit results are not central to the paper, and we can remove them if the reviewer prefers.
W2: Curated ≥720p training data for Wan
We added a baseline that finetunes the native Wan decoder on the same data with the same pixel losses (see our response to Reviewer Yxpz W2).
W2: Inference cost reported only in FLOPs, without wall-clock time or memory
See our response to Reviewer 5V5K (W1/Q1) for the full latency and peak VRAM table. DWD makes end-to-end generation 1.12× faster on 4-step FLUX.2 klein 9B and 1.69× faster on 4-step Wan2.1 1.3B, with 5% and 16% lower peak VRAM.