acceptodds
Under review as a conference paper at ICLR 2027

Anisotropic Stochasticity for Inference-Time Scaling of Flow Models

Abstract

Inference-time reward alignment explores multiple candidates during generation and favors those with higher intermediate rewards. Since flow models generate samples deterministically, such exploration relies on marginal-preserving stochastic differential equations (SDEs), which let duplicated candidates branch from a shared intermediate state into distinct trajectories. Prior works add Gaussian noise with equal variance across all dimensions, but much of this isotropic noise falls on directions that barely change the final generated sample. We analyze how such perturbations translate into variation among the predicted endpoints, explaining why isotropic noise can lead to inefficient exploration. Building on this analysis, we propose an anisotropic stochastic transition framework formulated as an SDE that preserves the original flow marginals. Our framework shapes the noise with the posterior covariance of the current state, allocating more noise to directions where perturbations change the final sample more. We further show that the drift adjustment required to preserve the original marginals reduces to an acceleration of the flow. Experiments on text-to-image reward alignment and offline-RL demonstrate improved reward–compute trade-offs, even after accounting for the additional computational overhead.

Primary area:
generative models
Keywords:
Generative models, Flow matching, Reward alignment

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_8eUtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work studies anisotropic stochastic transitions for more efficient exploration in inference-time reward alignment. The authors analyze the Jacobian of the predicted endpoint with respect to the intermediate state and observe that it is effectively low-rank. This motivates their argument that isotropic noise uses exploration budget inefficiently, leading them to propose covariance-shaped noise transitions. To sample from these transitions efficiently, they derive a marginal-preserving SDE whose drift correction reduces to a first-order flow-acceleration term. Experiments on text-to-image generation and offline RL show improved reward-alignment performance under matched compute budgets.

Critical strengths
  1. The method is well motivated. The empirical observation that the endpoint-prediction Jacobian is effectively low-rank, together with the endpoint-variance analysis, provides a clear rationale for allocating noise anisotropically.

  2. Although covariance-shaped transitions incur additional computational overhead, the authors show improved reward-alignment performance relative to isotropic noise under matched compute budgets.

Critical weaknesses
  1. The empirical results, particularly for text-to-image generation, do not yet demonstrate a practically meaningful advantage over isotropic noise relative to the proposed method’s additional computational overhead. In the adaptive-resampling experiment in Table 5, covariance-shaped noise improves the aesthetic score by only 0.0480.048 (a 0.73% improvement) while increasing runtime by 2.53 seconds (a 6.1% deterioration). Moreover, the text-to-image results are reported without error bars, standard deviations, or confidence intervals. Given the small difference in mean reward, these missing statistics make it difficult to determine whether the observed improvement is robust and practically significant rather than attributable to evaluation variability.
Questions for authors
  1. For the text-to-image results in Figure 4, can the authors report the mean and standard deviation over prompts at each NFE budget? A table would be sufficient and would clarify whether the reported reward improvements are robust.

  2. The spectral and endpoint-variance analyses in Figure 3 and Table 1 appear to use just a single prompt (“a photo of a bench”). Can the authors extend this analysis to a more diverse set of prompts, such as a subset of the GenEval prompts used in the text-to-image experiments? This would clarify whether the proposed endpoint-diversification mechanism generalizes beyond the illustrated example.

4 comments
AuthorsOpenReview ↗

Rebuttal Response

We sincerely thank the reviewer for carefully reading our paper and providing constructive feedback.

[W1, Q1] Statistical Reliability of Performance Gains

Thank you for highlighting the importance of quantifying variability. We have updated Figures 4 and 5 to report mean performance with error bars, displayed as shaded bands representing the standard deviation across seeds and examples.

The updated results show a consistent overall improvement trend over isotropic sampling. Although the absolute scale of aesthetic score gains seem modest, our method generally achieves higher mean rewards with non-overlapping standard deviation bands, with a clearer advantage at higher NFE budgets. Improvements are also observed for ImageReward and both offline-RL environments. These results provide further evidence for the benefit of covariance-shaped exploration, with the magnitude of improvement varying across objectives and tasks.

To directly assess uncertainty in the text-to-image reward improvements, Tables A and B report mean paired reward differences and their 95% confidence intervals. For both Aesthetic alignment and ImageReward, the confidence intervals for improvements over isotropic sampling lie entirely above zero in most evaluated settings, providing statistical support for the observed gains.

Table A. Aesthetic differences between our method and each baseline: mean difference [95% paired confidence interval].

ComparisonN = 2N = 3N = 4N = 5N = 6
Ours − BoN0.037 [−0.024, 0.098]0.124 [0.052, 0.195]0.126 [0.064, 0.187]0.108 [0.061, 0.155]0.116 [0.039, 0.193]
Ours − Isotropic−0.016 [−0.069, 0.038]0.047 [0.024, 0.070]0.047 [0.010, 0.083]0.052 [0.006, 0.098]0.062 [0.014, 0.110]

Table B. ImageReward differences between our method and each baseline: mean difference [95% paired confidence interval].

ComparisonN = 2N = 3N = 4N = 5N = 6
Ours − BoN0.038 [0.008, 0.068]0.052 [0.030, 0.074]0.049 [0.027, 0.072]0.053 [0.021, 0.085]0.057 [0.028, 0.086]
Ours − Isotropic0.032 [0.014, 0.051]0.016 [−0.002, 0.034]0.034 [0.004, 0.063]0.033 [0.013, 0.053]0.042 [0.021, 0.063]

[Q2] Endpoint variance analyses on various prompts

We thank the reviewer for suggesting this additional analysis. We first clarify that Figure 3 averages the spectra over 64 independent initial noise seeds at each timestep, while Table 1 averages the endpoint-variance estimates over these seeds and the three timesteps.

We agree that evaluating additional prompts is important for assessing the generality of the observed behavior. We have therefore extended the analysis to additional prompts randomly sampled from GenEval and provided the results. For each prompt, we evaluate 10 independent initial noise seeds at the same three timesteps, t∈{0.2,0.5,0.8}t\in\{0.2,0.5,0.8\}, using the measurement protocol described in Appendix E.2.

We updated Figure 3 to include spectra from multiple prompts with standard-deviation bands. The averaged spectra remain strongly concentrated, consistent with the behavior observed in the original example.

Table C reports the endpoint variance V~h\widetilde{\mathcal{V}}_h under matched perturbation norms, following the same measurement procedure as Table 1. For each prompt, the estimates are averaged over seeds and three timesteps. The ratio is the covariance-shaped mean divided by the isotropic mean, and the pooled row averages the variance estimates across all eight prompts. These results provide further evidence that the proposed mechanism extends beyond the original bench example.

Table C. Endpoint variance across prompts

PromptIsotropicCov.-shapedRatio
a photo of a bench1.418133.66× 94.3
a photo of a backpack below a cake1.17583.11× 70.7
a photo of a cat below a backpack1.755132.31× 75.4
a photo of a person and a bear1.470102.13× 69.5
a photo of a purple parking meter and a red laptop1.355177.63× 131.1
a photo of a vase0.92352.81× 57.2
a photo of an orange tv1.159218.15× 188.2
a photo of two teddy bears1.31498.24× 74.8
Pooled (8 prompts)1.321124.76× 94.4

Dear reviewer, thank you again for your time and for the detailed review.

As the discussion period is closing in a day, we would greatly appreciate it if we could know whether our rebuttal has properly addressed your concerns. If any concern remains, we would be happy to provide further clarification.

Dear Reviewer 8eUt,

As the rebuttal period closes in just a few hours, we would be truly grateful if you could take a moment to review our response. We sincerely hope it addresses your concerns, and we would greatly appreciate any further feedback.

I thank the authors for their work during this review period and will address their points now.

Q2 [Endpoint Variance across prompts] Thank you for pursuing this result. This extension to multiple prompts indicates that the observed low rank behavior is not isolated to the original prompt and fully addresses my concern.

[W1, Q1] Statistical Reliability of Performance Gains I appreciate the authors' presentation of the error bars in Figures 4 and 5, and the anlaysis presented in Table A. I still find the numerical improvements somewhat modest but the authors have convinced me that they are nevertheless robust.

I will therefore raise my score to weak accept.

Reviewer_CTYTOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript is about inference-time reward alignment, where existing works add Gaussian noise with equal variance across all dimensions and have an exploration limitation, i.e., much of this isotropic noise falls on directions that barely change the final generated sample. To address this, this work proposes a covariance-shaped marginal-preserving SDE as the noise.

This manuscript gives a sound analysis of why the problem happens, which is valuable and gave me a lot of insight, and the method is driven by this analysis. Of course, the limitation is that making the theory practical is a bit hard: it only holds in the ideal case, and the actual implementation stacks four approximations (the learned uθu_\theta, Σ\Sigma estimated from the Jacobian, VJP in place of JVP under a symmetry assumption, and Lanczos with only 2 iterations).

Critical strengths
  1. The manuscript is very well organised and easy to read, and points out an interesting view for understanding diversity and exploration of Sequential Monte Carlo in inference-time scaling.
  2. This work is well motivated, and sound in both theory and practice.
  3. The gain on OGBench offline-RL is large (about 65% vs 48% on pointmaze-giant at the highest compute budget).
Critical weaknesses

The weakness is in making the theory practical, i.e., it only holds in the ideal case, while the actual implementation stacks four approximations (the learned uθu_\theta, Σ\Sigma estimated from the Jacobian, VJP in place of JVP under a symmetry assumption, and Lanczos with only 2 iterations). In addition, the gain on FLUX.1-dev text-to-image generation is small (about +0.03 to +0.05 in aesthetic score).

Questions for authors
  1. Could the authors give an analysis of why the gain on FLUX.1-dev text-to-image generation is small? I think this would help readers better understand what the method can bring us.
  2. How many prompts and seeds are used for the text-to-image results? The difference between m=1m=1 in Table 6 and matched-iso in Table 2 is already about 0.03, which is the same order as the reported gain. Is the gain within the noise level?
  3. The two methods use different noise schedules (c(1−t)2c(1-t)^2 vs ctct). Chould the authors explain this a little bit?
4 comments
AuthorsOpenReview ↗

Rebuttal Response [1/2]

We thank the reviewer for the positive assessment of our work and the constructive feedback.

[W1] On the Validity of the Practical Approximations

We agree that translating our theoretical framework into a practical sampler introduces approximations. Our implementation aims to make the proposed construction computationally feasible while keeping the additional overhead minimal. We appreciate the reviewer’s feedback and the opportunity to examine the approximations involved more carefully.

First, the Jacobian–covariance relation is an exact identity for the true marginal velocity (Eq. (6)). Since this velocity is unavailable in practice, we substitute the learned velocity (u_\theta) into the identity. Thus, the covariance formula itself is exact; the approximation arises from applying it to a learned velocity field. While its accuracy depends on both the learned velocity and its derivatives, this approximation has shown to be effective in practice [1,2].

To assess the practical impact of replacing JVPs with VJPs under the symmetry assumption, we compare our default VJP-based Lanczos implementation with a JVP-based variant in Table A. The JVP variant improves the mean aesthetic score from 6.591 to 6.613, while increasing runtime from 44.25 to 48.16 seconds (8.8% increase). These results quantify the quality–runtime tradeoff associated with this implementation choice. We use VJPs by default to reduce computational cost, while acknowledging the higher mean reward obtained by the JVP variant.

Table A. Lanczos JVP results with overhead

ImplementationAestheticRuntime (s)
VJP6.591 ± 0.02844.25
JVP6.613 ± 0.01748.16

We also examine the number of Lanczos iterations in Appendix D, Table 6. Increasing the iteration count from two to five produces similar aesthetic scores while increasing runtime, suggesting limited additional reward gains beyond (m=2) in this setting. We therefore use two iterations as our default to obtain a favorable reward–runtime tradeoff.

We expanded the discussion in Appendix G to clearly distinguish the exact theoretical guarantees from the practical approximations and their empirical support. Reducing approximation errors while keeping the additional computational cost low remains a direction for future work.

[1] On the Posterior Distribution in Denoising: Application to Uncertainty Quantification (Manor & Michaeli, ICLR 2024)
[2] Covariance-aware sampling for Diffusion Models (Schioppa & Salimans, arXiv 2026)


[W1, Q1] Statistical Reliability of Performance Gains on Text-to-Image Generation

Thank you for highlighting the importance of quantifying variability. We agree that the absolute aesthetic score gains seem modest which may reflect the numerical scale of the reward. To address this issue, we updated Figures 4 and 5 to report mean performance with error bars, displayed as shaded bands representing the standard deviation across seeds and examples.

Although the absolute Aesthetic score gains are modest, our method shows a clear and consistent advantage over isotropic sampling on both Aesthetic and ImageReward when viewed alongside the reported variability. In particular, our method with four particles surpasses isotropic sampling with five particles on both metrics. These results demonstrate that our covariance-shaped exploration can achieve higher rewards with fewer particles, providing clear empirical evidence of our framework’s effectiveness.

To directly assess uncertainty in the text-to-image reward improvements, Tables B and C report mean paired reward differences and their 95% confidence intervals. For both Aesthetic alignment and ImageReward, the confidence intervals for improvements over isotropic sampling lie entirely above zero in most evaluated settings, providing statistical support for the observed gains.

Table B. Aesthetic differences between our method and each baseline: mean difference [95% paired confidence interval].

ComparisonN = 2N = 3N = 4N = 5N = 6
Ours − BoN0.037 [−0.024, 0.098]0.124 [0.052, 0.195]0.126 [0.064, 0.187]0.108 [0.061, 0.155]0.116 [0.039, 0.193]
Ours − Isotropic−0.016 [−0.069, 0.038]0.047 [0.024, 0.070]0.047 [0.010, 0.083]0.052 [0.006, 0.098]0.062 [0.014, 0.110]

Table C. ImageReward differences between our method and each baseline: mean difference [95% paired confidence interval].

ComparisonN = 2N = 3N = 4N = 5N = 6
Ours − BoN0.038 [0.008, 0.068]0.052 [0.030, 0.074]0.049 [0.027, 0.072]0.053 [0.021, 0.085]0.057 [0.028, 0.086]
Ours − Isotropic0.032 [0.014, 0.051]0.016 [−0.002, 0.034]0.034 [0.004, 0.063]0.033 [0.013, 0.053]0.042 [0.021, 0.063]
AuthorsOpenReview ↗

Rebuttal Response [2/2]

[Q2-1] Evaluation Protocol on Text-to-Image Generation

The original text-to-image experiments used 45 Animal prompts for Aesthetic alignment and 553 GenEval prompts for ImageReward alignment, with one seed per prompt. We agree that a single generation per prompt provides limited evidence about variability due to generation randomness. We have therefore repeated the evaluation using five generation seeds per prompt and updated the main experiment results in Figure 4. We also revised the manuscript to explicitly report the prompt and seed counts in Section 6.1.

[Q2-2] Differences Between m=1m=1 Lanczos and Matched-Isotropic Sampling

We thank the reviewer for this careful observation. A key distinction between the two configurations is their drift correction. In Table 6, m=1m=1 approximates covariance-shaped noise using only one Lanczos iteration while retaining the acceleration-based drift correction derived for the covariance-shaped SDE (Eq. (12)). Matched-isotropic sampling in Table 2 uses isotropic noise together with the drift correction for isotropic diffusion (Eq. (8)).

At m=1m=1, the coarse covariance-shaped noise approximation is paired with a drift correction derived to preserve marginals for the full covariance-shaped perturbation. This mismatch between the approximated noise and the retained drift correction may contribute to the lower performance. Since the two configurations use different stochastic transitions, their observed performance gap is not a direct estimate of experimental variability.


[Q3] Clarification on different noise schedules

We use different noise schedules to match the overall perturbation scale between the two methods. Our aim is to keep the stochasticity scales comparable to isolate the effect of directional noise allocation.

We use c(1−t)2c(1-t)^2 for the isotropic baseline, following the default setup in [1]. For our method, the perturbation also includes the covariance factor Σ1∣t\Sigma_{1|t}, whose scale varies with time (Eq. (6)). Therefore, we choose ctct as a practical designed schedule to account for this additional time-dependent scaling. Figure 9 shows that this choice yields broadly comparable perturbation norms, although they are not identical at every timestep. We clarified this motivation in Section 4.2 of the revised manuscript.

[1] Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing (Kim et al., NeurIPS 2025)

Dear reviewer, thank you again for your time and for the detailed review.

As the discussion period is closing in a day, we would greatly appreciate it if we could know whether our rebuttal has properly addressed your concerns. If any concern remains, we would be happy to provide further clarification.

Reviewer_CTYTOpenReview ↗

Response to authors

Thank you for the authors' effort and time in this work, they strengthen the empirical evidence. Although the practical sampler’s theoretical guarantees remain limited, I continue to find the analysis and contribution valuable and would like to maintain my positive rating.

Reviewer_Di5COpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors target inference-time reward alignment problem. In particular, the authors dive into the common assumption of isotropic Gaussian noise that turns the flow ODE into SDE in order to make SMC-based reward alignment possible and identify significant limitations of this assumption in terms of inefficient exploration. They then propose to replace the isotropic noise with anisotropic perturbation that also preserves the marginal. Their proposed method has clear advantages when tested on text-to-image generation and offline RL applications.

Critical strengths

I really like the perspective and the motivation of this paper. I personally have always found the isotropic assumption really unsatisfying as it does not necessarily have much practical benefit other than its simplicity. I am very glad to see the authors take it apart and actually show that it can hurt exploration and thus performance!

Critical weaknesses

My main concern regarding this paper lies in its computational cost: In page 6 “Computational overhead”, the authors claim that each covariance-shaped transition costs four NFE-equivalent. However, I don’t think that is necessarily true, considering 3 out of 4 network evaluations are actually JVP/VJP. These operations can be significantly more expensive than regular forward passes depending on the deployment setting and kernel implementation. Because of this peculiar point, the NFE matching results that the authors provide are not necessarily valid. It would be great if the authors can provide another metric that reflect the actual computational cost (e.g. wall clock time on matching machines).

Another concern is that even with this significant compute overhead, it seems like the performance gain from the method in comparison to isotropic noise is relatively limited and unclear if it holds any statistical significance. Maybe it would be a good idea to include confidence intervals or error bars in the plots and tables.

Questions for authors

It would be great if the authors can answer my concerns above. I am more than happy to accept this paper if the authors can address my concerns.

4 comments
AuthorsOpenReview ↗

Rebuttal Response

We thank the reviewer for the constructive feedback which has helped us improve the clarity and quality of the paper.

[W1] Computational Overhead.

We appreciate the reviewer for carefully examining the computational cost. We acknowledge that JVP and VJP computations can differ in runtime from a standard velocity forward evaluation. We first clarify that, as described in Appendix B, our implementation incorporates computation reuse to reduce overhead. It therefore does not treat the forward evaluation, JVP, and two VJPs as independent computations performed from scratch. We also kindly note that we validate this NFE accounting in actual runtime, in both an SMC transition (Table 4) and an entire sampling trajectory (Table 5). We clarify the accounting and these measurements below.

Under our convention, the velocity forward pass and additional derivative pass required for a JVP or VJP are each counted as one NFE. Our implementation evaluates the velocity once while retaining the computational graph and caching the intermediate values needed for differentiation. The JVP for computing acceleration reuses these cached values without repeating the forward velocity evaluation. Each of the two Lanczos iterations similarly computes a VJP using the retained graph, requiring one additional derivative pass each and no additional velocity forward evaluation, as noted in Appendix B. Overall, our accounting includes one velocity forward pass (including cache preparation) and three derivative passes.

Measured transition cost. To assess the reliability of this approximation, Table 4 compares the complete covariance-shaped transition—including the acceleration computation and both Lanczos iterations—with an isotropic transition under matched hardware and evaluation settings. The measured runtime ratios are 3.41× for FLUX and 2.05× for QGF, both below the factor of four charged in Figures 4–5. Thus, the four NFE-equivalent charge conservatively accounts for the measured aggregate transition cost in both settings.

Measured trajectory cost. Covariance-shaped transitions are applied only at SMC branching/resampling steps; at all other steps, our method uses the same sampling procedure as the isotropic baseline, with the same computational cost. As shown in Table 5, resampling is sparse, averaging only 2.4 events over 100 sampling steps in the reported text-to-image configuration. Table 5 also presents the resulting end-to-end runtime with four particles, including search-time reward evaluations and selection. For text-to-image generation, runtime increases from 41.72 to 44.25 seconds, corresponding to an overall overhead of 6.1%.

We revised the computational overhead discussion in Section 4.2 to clarify this accounting and highlight the measured runtimes. Thank you for drawing our attention to the ambiguity in our original explanation.


[W2] Statistical Reliability of Performance Gains

We appreciate the reviewer for the suggestion. We updated Figures 4 and 5 to report mean performance with error bars, displayed as shaded bands representing the standard deviation across seeds and examples.

The updated results preserve the overall trend of improvement over isotropic sampling in both text-to-image generation and offline-RL, supporting the effectiveness of our framework. Although gains in aesthetic score seem modest in absolute scale, the updated plot shows that our method outperforms isotropic sampling by a clear margin with non-overlapping standard deviation bands, especially at higher compute budgets. In particular, our method with four particles surpasses isotropic sampling with five particles on both metrics. These results demonstrate that our covariance-shaped exploration can achieve higher rewards with fewer particles even after accounting for its computational overhead, providing clear empirical evidence of our framework’s effectiveness. We appreciate the comment and revised the discussion in the paper accordingly.

Dear reviewer, thank you again for your time and for the detailed review.

As the discussion period is closing in a day, we would greatly appreciate it if we could know whether our rebuttal has properly addressed your concerns. If any concern remains, we would be happy to provide further clarification.

Dear Reviewer Di5C,

As the rebuttal period closes in just a few hours, we would be truly grateful if you could take a moment to review our response. We sincerely hope it addresses your concerns, and we would greatly appreciate any further feedback.

Thank you for your rebuttal! It has answered my concerns so I am updating my score from 2 to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.