acceptodds
Withdrawn from ICLR 2027

Normalized Perceptual Pullbacks for One-Step Generative Modeling

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

One step image generation places most distribution correction into training, making the construction of regression targets central to optimization. Feature guided drifting provides a natural way to build such targets, but the effect of transferring a feature space correction back to a pixel generator remains unclear. We show that, under a matched frozen field and scaling, feature target regression and an unnormalized pixel pullback induce the same local first order generator update. The pulled back correction is unevenly scaled across samples: with DINOv3-L, the largest 10% of samples account for 41.8% of the total correction magnitude. We therefore normalize each pulled back direction before forming stop gradient pixel targets. Normalizing in feature space leaves substantial variation after the encoder Jacobian, while normalization after the pullback directly controls pixel correction scale. Experiments show the same pattern across representation families, encoder capacities, image resolutions, and generator scales. On ImageNet, the method reaches FID 1.47, 2.04, and 3.18 at 256, 512, and 1024 resolution. On BLIP3o 60k, it reaches cFreD 3.03 and CLIPScore 23.23 under one step inference.

Primary area:
generative models
Keywords:
One Step Generative Modeling · Optimization Geometry · Perceptual Representations · Distribution Matching

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NYT6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The author try to reduce the out of distribution sample of one step generative modeling. They do so by training the generator to map the sample closed to the "correct sample." The define the correct sample as the original sample x + d, where d is defined as the Jacobian of a pretrained vision encoder like DINO, and some sort of latent signal. The main contribution of this paper is that the author propose this "correction signal" should be normalized, and doing so slightly improve the model's generation quality, measured in FID.

Critical strengths

The author show some ablation study that normalizing the "correction signal" during this Feature guided drifting post-training is necessary, in a sense it improve FID a little bit.

Critical weaknesses

The main weakness is the writing. I think this paper is a AI generated report. I spend lots of time try to understand the paper, but still got confused with the notation and still could not quite understand the main method figure (Figure 1) and the method section. I think the author need to improve the writing. For example, the author use the word "Feature guided drifting" in their abstract to describe this post-training correction method they use. But I think this is a new term, and the author never describe it or explain the connection to author literature. In the related literature section, the author simply cite a bunch of literature but did not explain the connection to their works.

Overall, it's extremely tiring for reader to understand this paper, and I recommend the rejection of this paper solely for this reason. I strongly recommend the author to rewrite the paper without AI. I don't think rebuttal will change my assessment.

Questions for authors

N/A

Reviewer_aP9fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes normalizing perceptual corrections after their Jacobian pullback to pixel space. It analyzes the equivalence of feature-space and pixel-space regression gradients and evaluates the method across ImageNet resolutions, representations, and text-to-image generation.

Critical strengths

The paper isolates the effect of normalization location through carefully designed ablations. Post-pullback normalization improves ImageNet-256 FID from 1.55 to 1.49 over batch-RMS normalization, consistently across five seeds. The improvements are further supported by gradient-variability and training-stability measurements.

The full method achieves competitive one-step generation quality (FID 1.47 at ImageNet-256), with improvements extending across multiple representations, resolutions, and generator sizes. Independent evaluation checks strengthen the empirical evidence.

Critical weaknesses

The theoretical equivalence follows directly from the chain rule, while the core method is a relatively simple per-sample normalization operation. Its placement after the encoder Jacobian is meaningful, but comparisons against conventional per-sample gradient clipping and magnitude-matched alternatives are missing.

Normalizing small corrections could amplify stochastic noise near distributional equilibrium, an issue not thoroughly analyzed. The method also increases training computation by 1.55–2.04×. Since the improvement over an already strong feature-based baseline is relatively modest, quality versus total training cost deserves more attention.

Questions for authors

How does the method compare with per-sample norm clipping under matched correction magnitudes?

Does normalization remain beneficial and stable as the distributional gradient approaches zero?

What is the improvement in quality at matched total training time rather than matched optimizer steps?

Reviewer_k2hUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates feature-guided target construction for one-step pixel-space generative models trained via drifting objectives. The authors show theoretically and numerically that, under matched step sizes and a fixed feature discrepancy field (computed via minibatch MMD in a frozen encoder's representation space), regressing toward a stop-gradient feature target induces the exact same local first-order generator parameter update as regressing toward an unnormalized pixel-space target pulled back via the encoder's Jacobian transpose. Practically, the paper proposes per-sample Euclidean normalization of the pulled-back pixel directions to form stop-gradient pixel targets, combined with an auxiliary LPIPS loss to obtain their numerical results. Experiments on ImageNet and text-to-image generation demonstrate consistent FID and cFreD improvements over unnormalized and feature-target baselines.

Critical strengths

Proposition 1 ensures that feature-space regression to an explicit pixel-space VJP pullback does not alter the local first-order optimization direction.

The empirical characterization of pullback norm imbalance is thorough and convincingly demonstrates that stronger semantic encoders induce larger Jacobian scale variance.

All overhead is strictly confined to training, preserving single-step (1-NFE) pixel generation at inference time.

Critical weaknesses

Despite the Reproducibility Statement claiming that Appendix L provides "complete training and evaluation protocol... including generator architectures," the paper never actually specifies what the generator architecture is. Calling it an "approximately 120M-parameter one-step generator" and "320M-parameter generator" is insufficient for reproducibility. Similarly, details on patchification, positional encodings, noise, and class/text conditioning injected into the blocks are omitted.

The paper relies heavily on quantitative metrics and a manual 100-sample failure count audit, relegating qualitative outputs to a single 6-image figure in the appendix (Figure A3) that only shows isolated "success" and "failure" crops for the proposed method at 1024. There are no side-by-side qualitative comparisons across methods.

There is no visual demonstration of the core problem of one-step generators the paper claims to be solving. The paper needs to show concrete visual examples where unnormalized/few-step corrections fail to update artifacts and demonstrate that post-pullback normalization resolves them.

Furthermore, no qualitative examples are provided for the text-to-image tasks or for 256/512 ImageNet. In Table 2 and Table A2, LPIPS is only ever added on top of the final "Pixel target + Per-sample norm" configuration. The experimental suite does not properly isolate the benefit of the proposed normalized pullback from the benefit of applying LPIPS alongside a strong feature representation.

In Equation 16, the normalization means that every single sample is stepped by a fixed Euclidean distance in pixel space, regardless of whether the sample is already very close to the target distribution or far away. Table A14 reveals that the method is quite sensitive to . Small changes (+/- 0.25) eliminate all of the proposed benefits of the method over baselines.

In Section 6.1, the main text discusses training time overhead (1.55x to 2.04x) but completely omits the substantial GPU memory overhead, relegating it to Table A17 in the appendix. At the 256 resolution, the proposed method increases peak GPU memory from 18.8 GB to 31.1 GB, nearly doubling the induced memory footprint compared to pixel drifting. For the higher resolutions, the peak memory increase is not as severe, but this is because the authors apply gradient accumulation steps, which raises an important question (see below).

Limited Baselines and Datasets. Baselines: In Table 1, DMD and Consistency Models are only evaluated at 256256 and are missing at 512 and 1024. Moreover, stronger recent one-step and pixel-space baselines cited in the related work (e.g., MeanFlow, rCM, or PixelGen) are not empirically compared. Datasets: Evaluating on additional standard datasets with different distributional characteristics (e.g., FFHQ, LSUN, or CIFAR-10) would strengthen claims of generality.

Minor: The x-axis tick labels of the center bar plot in Figure 2 are overlapping and illegible.

Questions for authors

What is the exact architecture of the 120M and 320M parameter generators?

What is the ImageNet-256 performance of Feature target (Batch RMS) + LPIPS and Pixel target (Batch RMS) + LPIPS?

At 512x512 and 1024x1024 (where gradient accumulation of 4 and 16 is used), is the MMD discrepancy and same-class LPIPS pairing computed over the 256 global batch or only over the per-step micro-batch of 16 / 64 images?

Can you provide side-by-side qualitative samples comparing Pixel Drifting, Feature Target, Unnormalized Pullback, and Normalized Pullback to visually demonstrate the correction failures that the method proposes to resolve?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.