acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Pixel Mean Flows via Interval Denoiser

11 views

Abstract

Modern diffusion and flow-based models are increasingly moving toward few-step, latent-free generation to bypass the computational overhead of multi-step sampling and the reconstruction bottlenecks of external autoencoders. We propose the Interval Denoiser, a principled framework for latent-free generation. Derived directly from the flow matching ODE, it establishes an exact analytical mapping for intermediate trajectory states and identifies the image-space prediction target as an interval-weighted average of instantaneous denoisers, which supports the generalized manifold hypothesis under which regression is tractable for a network operating directly on pixels. The derivation recovers the substitution of Pixel MeanFlow (pMF) and the decoder of CTM as special cases. Furthermore, by placing the stop-gradient only on the time derivative, our update is the exact first-order gradient of the Interval Denoiser identity, whereas pMF's stop-gradient placement damps the bootstrapped derivative term by a factor . By analyzing this objective, we show that the exact weighting is ill-conditioned for wide intervals and equip our framework with residual clipping and a time-sampling curriculum, enabling effective long-interval training and improving few-step performance. Trained from scratch on ImageNet , our model achieves an FID of 4.55 in one step (1-NFE) and 3.98 in two steps (2-NFE) without perceptual losses.

Primary area:
generative models
Keywords:
diffusion models, flow matching, few-step generation, pixel-space generation, MeanFlow, consistency models, image generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GidhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper derives an Interval Denoiser from the flow-matching ODE: a weighted average of denoising predictions that enables one- or two-step generation directly in pixel space. It explains how this target relates to Pixel MeanFlow and CTM, analyzes a difference in their training updates, and uses residual clipping and a time-sampling curriculum to stabilize large steps.

On class-conditional ImageNet 256×256, it reports 4.55 FID with one network evaluation and 3.98 with two, without an autoencoder or perceptual loss. Ablations show that the curriculum provides most of the measured gain; with clipping alone, the new method and Pixel MeanFlow perform similarly.

Critical strengths

A clear theoretical account of the prediction target. Deriving the Interval Denoiser from the flow-matching ODE gives a precise interpretation of the quantity predicted by Pixel MeanFlow and connects it to CTM. This makes the paper useful beyond its specific implementation.

Critical weaknesses
  1. The Pixel MeanFlow comparison is not matched to the paper’s strongest training recipe. The main baseline is not retrained with the same residual clipping and time-sampling curriculum, even though the curriculum accounts for most of pID’s measured improvement and could be applied to pMF. The reported FID gap therefore does not establish that pID is the better method.

  2. The practical benefit of the new objective remains unclear. With clipping and a static sampler, pID and pMF achieve nearly identical FIDs (9.25 vs. 9.34). The paper explains their theoretical update difference, but does not show that this difference yields a meaningful, reproducible gain under matched conditions.

Questions for authors
  1. Decision-critical comparison: Can pID match or surpass pMF’s best reported result under a comparable setup, including perceptual loss if that result uses it? If the claim is specifically about generation without perceptual loss, please instead compare against pMF trained with the same full recipe, including clipping and curriculum.

  2. Clarification of the contribution: Beyond a more principled derivation, what measurable benefit does the pID objective provide over pMF? With clipping alone, their reported results are nearly identical; please isolate any gain in generation quality, training stability, or efficiency attributable to the new objective.

2 comments

We greatly appreciate the reviewer’s feedback on our paper. We respond to each point below.

(1) Can pID match or outperform pMF when both methods use the same full training recipe? In our matched 160-epoch comparison on ImageNet-256, we used one training seed per method and no perceptual losses. pMF-B/16 achieved a lower 1-NFE FID than pID-B/16 (7.06 vs. 7.55). With the pMF objective fixed, the curriculum lowered its FID by 2.28 points relative to the static LN(0.8, 0.8) baseline of 9.34.

(2) What does the pID formulation offer beyond pMF? In the matched B/16 experiments, we did not establish a quality, stability, or efficiency advantage for the pID objective over pMF. Our formulation addresses gaps in the interpretation and evaluation of pMF [3] and provides guidance for training interval predictions:

  • What the network should learn. We derive an image-space counterpart of MeanFlow [2] directly from the ODE, recovering the pMF conversion [3]. This connects image prediction and average velocity, similarly to the image/velocity relation in flow matching. Proposition 1 identifies the exact target for every interior interval as a weighted average of denoisers. Therefore, the image-like interpretation that pMF supports through visualization receives a mathematical explanation, supporting the motivation for pixel-space regression in Li and He [1]. This ODE-based approach suggests a recipe for other geometries, including high-dimensional hyperspheres [6]: derive a geometry-specific denoiser identity rather than reuse the Euclidean conversion.
  • How to use the predictions beyond one step. A limitation of the pMF study is that sampling beyond 1-NFE is not explored in its reported experiments. We explicitly derive the update between arbitrary times for the image-space parameterization. We demonstrate its practical use for pID: two-step sampling improves pID-L/16 from FID 4.55 to 3.98.
  • Which intervals to learn at each training stage. Resolving pID's gradient suppression with residual clipping (Fig. 2) motivated us to schedule (r,t)(r,t) sampling from short to longer intervals, keeping the objective fixed and improving FID from 9.25 to 7.55. The difference from related work is what we schedule. Kim et al. [4] progressively change interval loss weights, while α-Flow [5] changes the training objective.

[1] Li and He. Back to Basics: Let Denoising Generative Models Denoise. CVPR, 2026.

[2] Geng et al. Mean Flows for One-step Generative Modeling. NeurIPS, 2025.

[3] Lu et al. One-step Latent-free Image Generation with Pixel Mean Flows. ICML, 2026.

[4] Kim et al. Understanding, Accelerating, and Improving MeanFlow Training. CVPR, 2026.

[5] Zhang et al. AlphaFlow: Understanding and Improving MeanFlow Models. ICLR, 2026.

[6] Chen and Lipman. Flow Matching on General Geometries. ICLR, 2024.

Reviewer_GidhOpenReview ↗

Matched comparison addresses concern; overall assessment unchanged

Thank you for adding the matched comparison and clarifying the contribution. This addresses my concern about the missing control experiment. The results demonstrate the curriculum’s usefulness for both methods, but do not establish a practical advantage of the pID objective over pMF.

I appreciate the theoretical interpretation and explicit sampling rule. However, I remain unconvinced that the additional contribution beyond pMF and CTM is sufficiently substantial to change my overall assessment. I will therefore maintain my weak-reject recommendation.

Reviewer_c9xMOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces the Interval Denoiser (pID), a reformulation of pixel-space MeanFlow derived directly from the flow-matching ODE. The authors show that the intermediate image-space prediction target can be expressed as an interval-weighted average of instantaneous denoisers along the probability-flow trajectory. This provides a theoretical interpretation of Pixel MeanFlow (pMF) and establishes connections to Consistency Trajectory Models (CTM).

The paper further analyzes the effect of stop-gradient placement. Under the idealized objective, pID follows the first-order gradient of the Interval Denoiser identity, whereas pMF attenuates the bootstrapped derivative term. Since the resulting pID target becomes ill-conditioned for wide intervals, the authors introduce residual clipping and a two-phase time-sampling curriculum to stabilize training.

Experiments on class-conditional ImageNet-256 report FID 4.55 at 1-NFE and 3.98 at 2-NFE without pretrained autoencoders or perceptual losses. Ablations examine residual clipping, sampling curricula, and intermediate sampling times.

Critical strengths
  1. The theoretical clarification is useful. The Interval Denoiser provides an explicit interpretation of the image-space target underlying pixel MeanFlow. The connections to CTM and the analysis of stop-gradient placement help explain the relationship between existing few-step generation objectives. The identification of wide-interval ill-conditioning is particularly informative.

  2. The few-step generation results are strong. The reported FID of 4.55 at 1-NFE and 3.98 at 2-NFE on ImageNet-256, without perceptual losses or pretrained autoencoders, is impressive. The ablations also provide useful evidence about the importance of residual clipping and progressive exposure to wider sampling intervals.

Critical weaknesses
  1. The main performance improvement is not clearly attributable to the Interval Denoiser objective. With residual clipping, pID and pMF achieve nearly identical FIDs of 9.25 and 9.34, respectively, under the reported single-seed comparison. The larger improvement, from 9.25 to 7.55, comes from introducing the two-phase time-sampling curriculum. However, the authors do not evaluate pMF under the same curriculum. Consequently, the main comparison at 320 epochs (6.28 versus 8.71 for B/16) cannot establish that the proposed formulation itself improves generation quality. This is my primary reservation and prevents a clear-accept recommendation.

  2. The theoretical novelty and practical advantage should be distinguished more carefully. The Interval Denoiser is derived through an exact reformulation of the existing flow-matching trajectory, and its parameterization is closely related to pMF and CTM. While the interpretation is useful, the theoretical results do not establish a new, more expressive class of generative models. Moreover, the claimed first-order exactness does not fully characterize the clipped and reweighted objective used in practice. Since both pID and pMF share the same fixed point and perform similarly after clipping, the practical importance of their optimization difference remains uncertain.

A secondary concern is experimental uncertainty. The ablation studies use a single training seed, and the final large-model results are not accompanied by multi-seed variability. This makes it difficult to assess the reliability of smaller improvements. The paper also lacks a fully matched pMF comparison using the complete proposed training recipe.

Questions for authors
  1. Matched pMF comparison: Can the authors evaluate pMF with the same residual clipping, two-phase curriculum, optimizer, guidance settings, and training budget as pID? Given that the two methods perform similarly under static sampling, this is the most important missing experiment. A consistent advantage for pID under matched conditions would materially strengthen my assessment.

  2. Practical significance of first-order exactness: Proposition 4 establishes a difference between the idealized pID and pMF updates. However, the implemented method uses clipped residuals and logarithmic weighting. How frequently does clipping activate, particularly for wide intervals, and to what extent does the effective update retain the theoretically derived advantage?

  3. Manifold interpretation: Proposition 1 establishes convex-hull membership but does not bound the dimensionality of the intermediate prediction set. Is there additional theoretical or empirical evidence that these predictions remain concentrated near a low-dimensional image-like manifold? How does this interpretation extend to the guidance-conditioned setting used in the experiments?

  4. Statistical reliability: Can the authors provide additional training seeds for the B/16 comparison, particularly the clipped pID versus clipped pMF experiment? This would help determine whether the small reported difference reflects a consistent optimization advantage or ordinary training variability.

1 comment

We are grateful for the reviewer’s positive assessment and thoughtful feedback.

(1) How does pID compare with pMF under a fully matched training recipe?

Section 5.2 now reports this comparison. We trained both B/16 models on ImageNet-256 for 160 epochs without perceptual losses, using the same optimizer, guidance settings, and residual clipping.

Time sampling during trainingpMF 1-NFE FIDpID 1-NFE FID
Static LN(0.8, 0.8)9.349.25
LN(0.8, 0.8) → uniform at epoch 1407.067.55

Under the matched full recipe, pMF achieves the lower FID. The curriculum reduces FID by 2.28 for pMF and 1.70 for pID.

(2) What does the pID formulation contribute beyond pMF, independently of the training recipe?

Beyond recovering pMF's conversion, our derivation explains the denoising structure of the finite-interval image-space target and how this target determines trajectory transitions.

  • What the network should learn. Image and velocity prediction can describe the same flow while giving the network different prediction tasks [1]. We derive the corresponding denoiser view of MeanFlow's average velocity [2]. Proposition 1 identifies the ideal interior target as a weighted average of instantaneous denoisers along one ODE trajectory. This explains the structure behind pMF's image-like target [3].

  • How to sample beyond one step. We derive a sampling rule for both pID and pMF (Eq. 12). To reach an intermediate time, we combine the current noisy image with the model’s prediction for that interval. At the final step, we use the prediction directly. For pID-L/16, FID improves from 4.55 with one step to 3.98 with two.

  • What changes across intervals. The identity connects local denoising at r=tr=t with trajectory consistency at r=0r=0. The curriculum is one practical application of this understanding, and its gain comes from the training recipe.

The ODE-based strategy also suggests a recipe for exploring other geometries, such as hyperspheres [4], whose denoiser identities would require a separate derivation.

(3) How often does clipping activate, especially over wide intervals?

We evaluated a 160-epoch pID checkpoint trained with clipping and the curriculum over one full evaluation epoch. For each example, we measured the fraction of residual coordinates with absolute value above 1 before clipping, then averaged this fraction over all examples or those with t−r>0.5t-r > 0.5. The overall average includes the 50% of examples assigned r=tr=t.

Time samplingAll examplest−r>0.5t-r > 0.5
LN(0.8, 0.8)1.5%27%
Uniform8%52%

(4) How do clipping and logarithmic weighting affect the idealized update?

We retain the analyzed stop-gradient placement, detaching the time derivative. Logarithmic weighting rescales gradients per example, coordinatewise clipping can change directions, and denominator clamping changes residuals. Where clipping and clamping are inactive, the per-example pID-loss gradient follows the ideal first-order direction up to a positive scalar, although reweighting changes examples’ relative contributions. The clipping rates in (3) count affected residual coordinates. We did not measure the resulting change in gradient direction.

Exactness does not guarantee better FID than pMF. Its practical value is that it exposes the wide-interval trade-off: the exact identity weight can produce large residuals but small gradients under the logarithmic loss. This predicts the suppression observed in Fig. 2 and motivates the stabilization verified in Figs. 2 and A1.

(5) What supports the manifold interpretation, including with guidance?

Proposition 1 establishes that the ideal unguided target is a weighted average of denoisers along one trajectory, with more weight near the lower-noise end (Eq. 7). This supports its image-like interpretation, but does not bound its dimensionality. Figure A2(a) provides qualitative examples, rather than a dimension estimate. Appendix C incorporates guidance through modified denoising targets, but this does not establish low-dimensional concentration under guidance.

(6) What conclusions can be drawn from the single-seed comparisons?

We used one training seed for each B/16 result in Q1 and could not run additional seeds within our compute budget. We cannot tell whether either difference would persist across seeds. We also do not report variability across training seeds for the L/16 results in Table 1.

[1] Li and He. Back to Basics: Let Denoising Generative Models Denoise. CVPR, 2026.

[2] Geng et al. Mean Flows for One-step Generative Modeling. NeurIPS, 2025.

[3] Lu et al. One-step Latent-free Image Generation with Pixel Mean Flows. ICML, 2026.

[4] Chen and Lipman. Flow Matching on General Geometries. ICLR, 2024.

Reviewer_EPXMOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper is about few-step generation in pixel space. It proposes interval denoiser based on pixel meanFlow and connects interval denoiser to MeanFlow, Improved MeanFlow, and CTM.

Critical strengths
  1. This paper propose interval denoiser and connects it with meanFlow, improved meanFlow, and CTM.
Critical weaknesses
  1. this paper provides an interpretation of pMF. However, it remains unclear what additional insights follow from this interpretation. The paper would benefit from explaining how it reveals limitations of existing methods or leads to new design principles beyond reformulating the prediction target.
  2. the larger coefficients in the unclipped pID objective introduce additional instability over long intervals. Current experiments are only conducted over small models and the potential benefits of pID need to verified over larger scale data and models.
  3. I think the motivation of method over pixel space needs more explanation, due to current extreme high dimensions of more and more complex data, like long video data.
Questions for authors

see weaknesses.

2 comments

We thank the reviewer for the helpful comments. We clarify the additional insights and the scope of our experiments below.

(1) What additional insights follow from the pID interpretation?

In flow matching, velocity, image, and noise parameterizations can describe the same flow while giving the network different prediction tasks and loss weightings [1]. For example, an x-prediction network can be trained with a velocity-space loss. We derive the corresponding image-space interpretation of MeanFlow's average velocity [2], recovering the pMF conversion [3]. Proposition 1 identifies the ideal interior target as a weighted average of instantaneous denoisers along one ODE trajectory, with more weight near the lower-noise end (Eq. 7). Together with the Interval Denoiser identity, this gives a deeper understanding of what the image network learns across intervals, from local denoising to endpoint consistency.

(2) What can we learn about the limitations of existing methods?

The pMF study leaves two gaps that our work addresses. First, the denoising structure of its interior target is motivated through visualization. Proposition 1 supplies the mathematical explanation. Second, its reported experiments focus on 1-NFE generation. We make the image-space update between arbitrary times explicit for both pID and pMF's parameterization, and demonstrate its use for pID-L/16 at 1-NFE and 2-NFE (FID 4.55 and 3.98).

(3) What new design principles follow from this interpretation?

  • Analyze updates after changing parameterization. Equivalent identities need not give equivalent updates once stop-gradient is applied. Proposition 4 reveals a trade-off between following the image-space identity's idealized first-order update and keeping wide-interval weights bounded. Therefore, stop-gradient placement should be examined together with the parameterization.

  • Check the effective learning signal across intervals. In pID, large wide-interval residuals produce small gradients under the logarithmic loss. Figure 2 confirms this suppression and shows how residual clipping restores the learning signal.

  • Adapt which intervals are learned at each stage. Once wide-interval learning is stabilized, its coverage still matters. With LN(0.8, 0.8), training initially favors short transitions at high noise. Switching both draws to uniform later increases exposure to wide intervals while keeping the objective fixed. This differs from progressive interval loss weighting [4] and objective annealing [5]. In our single-seed pID-B/16 ablation, the curriculum improves FID from 9.25 to 7.55, compared with 9.26 for static uniform sampling. This supports the importance of the training sequence. In a 160-epoch pMF-B/16 ablation, changing only the sampling schedule from static LN(0.8, 0.8) to the same curriculum lowers 1-NFE FID from 9.34 to 7.06.

(4) How does pID handle wide intervals, and at what scales has it been tested?

Residual clipping restores the wide-interval gradient signal in our measurements (Fig. 2a–b). Our matched comparison with pMF uses B/16 (118M parameters) on ImageNet-256. We also evaluated pID-L/16 (411M parameters) on the same dataset with one and two sampling steps (Table 1). Pu et al. [6] study MeanFlow-based distillation with an 8B latent-space text-to-image model. Their practical guidelines offer a starting point for future work on larger models and datasets.

(5) Why work directly in pixel space despite its high dimensionality?

Pixel-space generation removes the separate autoencoder's decoding stage and reconstruction bottleneck. Under the manifold assumption, natural images have much lower intrinsic dimensionality than their pixel representation. This motivates predicting structured image targets directly. Back to Basics [1] demonstrates the feasibility of this direction with multi-step JiT-G, achieving FID 1.82 on ImageNet-256 and 1.78 on ImageNet-512 without an autoencoder. Our work continues this direction toward one- and two-step generation. Few-NFE sampling reduces the number of network evaluations, but does not remove the cost of processing large inputs. We do not establish applicability to long videos, which requires separate investigation.

[1] Li and He. Back to Basics: Let Denoising Generative Models Denoise. CVPR, 2026.

[2] Geng et al. Mean Flows for One-step Generative Modeling. NeurIPS, 2025.

[3] Lu et al. One-step Latent-free Image Generation with Pixel Mean Flows. ICML, 2026.

[4] Kim et al. Understanding, Accelerating, and Improving MeanFlow Training. CVPR, 2026.

[5] Zhang et al. AlphaFlow: Understanding and Improving MeanFlow Models. ICLR, 2026.

[6] Pu et al. Few-Step Distillation for Text-to-Image Generation: A Practical Guide. arXiv, 2025.

Thank you for the detailed response. The clarifications regarding the additional insights provided by the interval-denoiser formulation, the handling of wide intervals, and the motivation for pixel-space generation have addressed my main concerns. I am therefore raising my rating from 2 to 3.

Given my limited familiarity with this specific area, my confidence remains low. I hope the AC will take this into account when weighing my assessment alongside those of the other reviewers.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank the reviewers for their feedback. We revised the abstract and the contributions paragraph in §1 to clarify our aim. The original Pixel MeanFlow (pMF) paper illustrated its image predictions on interior intervals and focused its experiments on one-step generation. For few-step sampling, we need to know how to use each image prediction to construct the next noisy state. We show that the ideal prediction is a weighted average of denoising estimates along one trajectory and derive the corresponding sampling rule for pMF’s image parameterization. The same recipe could extend x-prediction beyond Euclidean space, for example by deriving a denoiser identity directly from a flow on a hypersphere. Since our few-step sampler defines intermediate states directly in pixel space, we can also explore scale-wise generation at progressively finer resolutions without an autoencoder.

We also clarified in §6 that our experiments do not establish a quality advantage for the pID objective. Our analysis explains why pMF’s stop-gradient keeps the weight on errors in the denoiser identity bounded even on wide intervals. In the experiment added to §5.2, we held pMF’s objective fixed. Sampling wide intervals more often late in training lowered its one-step FID relative to static sampling.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.