acceptodds
Under review as a conference paper at ICLR 2027

Batch-Coupled Tilted Denoising for Training-free Reward Alignment

Abstract

Inference-time reward alignment steers a pretrained generative model toward a reward without retraining, but gradient-free methods often require costly conditional endpoint sampling to estimate reward-aware corrections. We exploit the fact that generation often occurs in shared-condition batches and introduce cross-trajectory reweighting, which corrects the distribution mismatch so that a con- ditional endpoint sample generated for one trajectory can contribute to the correction of another. This converts endpoint sampling from a per-trajectory cost into a shared batch-level budget under batch coupling, allowing a smaller set of endpoint samples to be reused across trajectories instead of generating them independently. Across multiple backbones and reward objectives, the coupled estimator improves the reward–compute frontier over existing alternatives at matched cost, with the largest gains in low-compute regimes. We further analyze when cross-trajectory reuse is beneficial and the statistical dependence introduced by the coupling.

Primary area:
generative models
Keywords:
alignment, inference-time reward alignment, steering

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_w98qOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose to speed up guidance with GLASS-style transition kernels via doing reweighting of samples across a batch (all on the same conditioning) to more efficiently use the samples (via the batch) during guidance/steering. They evaluate on t2i models SD3 and FLUX on a variety of guidance tasks.

Critical strengths
  1. Very simple and elegant idea to utilize the batch.
  2. Very thorough accounting of details like NFE formulas, wall-clock times, metrics, etc.
Critical weaknesses
  1. I fundamentally disagree with how results are reported, ie, the batch size is not repeated in NFE numbers as it is NFE per image, this skews the results towards this paper. Currently, the paper hinges on scenarios where B>1B > 1 and the effect grows. It seems like that as h→∞h \to \infty this is almost a best-of-BB type algorithm. But how often does the user want to query a large batch all with the same prompt? And leading with my next point limited diversity,
  2. From Figure 3 it seems like metrics improves and diversity worsens as h→1h \to 1. I feel that when the user wants to sample B>1B > 1 they care a lot about diversity within the batch but this method seems to lose on diversity within the batch. So they advantages of increasing BB seems lost to me.
  3. Why is there no comparison to the limiting case of h→∞h\to\infty setting? This would isolate and strengthen the impact of the Gaussian bridge.
  4. The main point of this paper is reducing the compute burden of other strategies which use guidance via GLASS like FK-GLASS, but omit comparisons to few-step methods like MFMs or Diamond maps, thus I'm not sure I buy the justification of omitting it due to training. As a practitioner I'm much more inclined to use these to solve problems I care about as they massively reduce NFE and can be cheaply trained a la a DMF-styled fine-tune from a pre-trained flow matching model. I think a comparison to these is highly relevant for arguing why we should use the paper's method over these few-step stochastic flow maps.
Questions for authors
  1. Why compare to only FK-GLASS but not other GLASS guidance strategies that could be used like the Best-of-N done in GLASS?
  2. How do you address the diversity concerns as the batch size setting hinges on this? As far as I could tell diversity is never reported for the other methods. Again I feel like for the use case of this method the user cares about a large batch size when they care about diversity, otherwise just through the batch size into best-of-N to generate an even better single-image.
  3. Please report NFE with batch size and adjust best-of-N accordingly, etc so we can compare more concretely in Table 4 and figures. It feels off that the proposed method can "win" with no extra guidance steps at the baseline NFE of 25/28 in Figure 3 (also what reward is even being computed?) because of the batch size, this doesn't feel fair to other methods especially with diversity concerns. How does this graph look for all the reward metrics in this setting, so CLIP, PickScore, and image reward.
  4. How does this paper compare to guidance with flow maps like MFM/Diamond?
5 comments

[W1, W2, Q2, Q3] Batched Evaluation: Practical Relevance, Reward–Diversity Trade-off, and Compute Accounting

Please refer to Points 2, 3 of the General Response.

Thank you for raising these connected concerns. We agree that higher mean reward alone does not establish that a batch is more useful, and that reward gains should be assessed jointly with diversity and total computation. Our primary evaluation concerns generating and returning BB images under a shared condition. Point 3 of our General Response explains the motivating scenarios, primarily providing multiple candidates for a single request, with jointly processing compatible requests as an additional setting.

Within this setting, our contribution is to provide control over the reward–diversity trade-off through cross-trajectory endpoint-sample reuse. Relative to uncoupled steering (h=0h=0), bandwidth hh controls sharing at a fixed output count and endpoint-sampling budget. Sharing can increase mean reward at the cost of within-batch diversity; we do not claim that stronger coupling is preferable for every application. Point 2 of General Response reports both quantities for competing methods and our bandwidth sweep across multiple computational budgets.

Our evaluation therefore reports the mean reward and diversity of all BB returned images. All methods, including best-of-NN, return the same number of images, and their reported generation NFEs include discarded candidates. We also evaluate BoN of Ours under the same selection protocol as the baselines to assess performance when final selection is desired.

To make the computational comparison explicit, we report both NFE per returned image and total batch NFE below. In the baseline-NFE setting of Figure 3, proposed cross-trajectory sharing reuses endpoint predictions already computed during the sampling trajectory, without additional endpoint-sampling network evaluations.

The table below summarizes representative low-budget SD3 results on 60 prompts, with B=8B=8 and Stot=4S_{tot}=4 for Ours and Uncoupled. ImageReward is used for steering and selection; CLIPScore and PickScore are held-out metrics evaluated on the same returned images.** Diversity results for these settings are provided in Point 2 of our general response.

MethodNFE / imageNFE / batchImageReward ↑CLIPScore ↑PickScore ↑
Vanilla282240.3690.249322.04
FK-GLASS (S=2S=2)564480.4250.248321.89
BoN-2564480.5290.250222.17
BoN-GLASS (S=2S=2)564480.4720.249221.99
LiDAR (K=10K=10)635040.4490.249321.81
Uncoupled (h=0h=0)62.55000.3740.248522.09
Ours (h=0.01h=0.01)62.55000.5830.252322.32
Ours (h=0.1h=0.1)62.55000.6330.251922.31

[W3] : h→∞h\to\infty Limit and the Role of the Gaussian Bridge

Thank you for suggesting this ablation. We have added h=∞h=\infty, which sets all bridge ratios to one while retaining reward weighting, providing an uncorrected-pooling control at the same sampling cost. As shown below, h=1h=1 similarly yields reward and diversity close to this limit across all three budgets. Smaller bandwidths preserve more diversity at a reward cost, indicating that the finite-bandwidth bridge controls the trade-off relative to uncorrected pooling. Please see Point 2 of our general response for the additional results. Entries report ImageReward / Vendi on SD3.

hhNFE 62NFE 97NFE 166
0.010.010.583 / 1.490.592 / 1.500.581 / 1.55
110.641 / 1.280.661 / 1.260.682 / 1.28
∞\infty0.645 / 1.270.663 / 1.240.683 / 1.27

Regarding the connection to best-of-BB, selecting h→∞h\to\infty allows high-reward endpoints to influence multiple trajectories, which can reduce within-batch diversity. All BB final images are still returned.

[Q1]: Comparison with BoN-GLASS

Thank you for pointing out this missing baseline. We agree that the original BoN comparison does not replace an explicit comparison with BoN-GLASS, which generates candidates using GLASS and selects the highest-reward candidate. We have now added this baseline under the same output-count and selection protocol, including the generation cost of all discarded candidates. Point 2 of our general response reports its reward and within-batch diversity alongside FK-GLASS and our method at comparable computational budgets.

[W4/Q4]. Stochastic Flow Maps

Thank you for emphasizing the practical relevance of few-step stochastic flow maps. We agree that their training requirements alone do not justify omitting this comparison. We have therefore added experiments using Meta Flow Maps and Diamond Maps, with the publicly released checkpoints from the respective works. Please refer to Point 4 of General Responses for details.

I thank the authors for the detailed response. I looked at the PDF and did not yet see this incorporated into the revised PDF. Please let me know when these changes are worked in so I can review the entire revised paper to update my assessment as I wish to review everything within the context of the full paper.

Thanks, Reviewer w98q

Thank you for taking the time to review our response. We have now updated the PDF to incorporate the additional experiments and clarifications. In particular, we have clarified the intended generation scenarios in the Motivation and expanded the reward–diversity analysis to include baseline comparisons at comparable computational budgets. The additional validation experiments and implementation details are included in the appendix. We would appreciate your assessment of these changes in the context of the revised paper. Thank you again for your careful feedback.

Reviewer_oychOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a training-free and gradient-free method for inference-time reward alignment of flow and diffusion models that generate a batch of trajectories under a shared condition. The method targets an exponentially tilted distribution by estimating a reward-tilted denoiser through self-normalized importance sampling over conditional endpoint samples. The key idea is that endpoint samples need not remain trajectory-local. The method reweights and pools endpoint samples across trajectories, using an isotropic Gaussian surrogate with matched conditional endpoint means to approximate the density ratio between conditional endpoint distributions. It further allocates a shared endpoint-sampling budget non-uniformly across trajectories, averages contributions within each sampled source to avoid weighting sources in proportion to their sample counts, and uses denoiser means as plug-in contributions from sources receiving no samples. This allows fewer than one newly generated endpoint sample per output per steering step on average. Experiments on SD3 and FLUX report improved reward–compute trade-offs over the compared gradient-free baselines, with the largest gains in the low-compute regime. Under exact density ratios, the statistical analysis characterizes the per-query mean squared error and cross-query covariance of the unnormalized numerator and normalizer estimates. The paper also empirically examines the reduction in within-batch diversity as coupling strength increases.

Critical strengths

Practical Reuse of Conditional Endpoint Samples: The paper addresses a concrete computational bottleneck in gradient-free reward alignment by sharing conditional endpoint samples across trajectories under the same condition. The shared-budget formulation permits fewer than one newly generated endpoint sample per output on average, enabling effective steering in low-compute regimes where trajectory-local estimation is particularly constrained.

Evaluation of Computational Benefits and Trade-offs: Experiments on SD3 and FLUX compare against several gradient-free baselines across multiple reward objectives, including additional scaling through independent repetition and selection. The paper supplements generator NFE with reward-evaluation costs and measured runtime, providing practical support for the reported efficiency gains. It also measures the decrease in within-batch diversity of the proposed method as coupling increases, making an important limitation visible.

Critical weaknesses

Incomplete Assessment of the Reward–Diversity Trade-off: Figure 4 shows that stronger coupling increases reward while substantially reducing within-batch Vendi scores. Since the intended use case delivers multiple images for the same prompt, this reduction directly affects the utility of the generated batch. However, corresponding diversity measurements are not reported for the competing methods, and the reward–compute comparisons do not control for diversity. It therefore remains unclear how much of the reported advantage persists at comparable diversity levels. This concern is also relevant to the red-teaming objectives, where multiple similar flagged images may provide less useful coverage than distinct failure cases.

Limited Characterization of the Implemented Estimator: Proposition 4.1 characterizes the unnormalized numerator and normalizer estimates under exact density ratios and equal source sample counts. It does not directly establish error guarantees for the practical self-normalized estimator, which additionally uses a Gaussian surrogate bridge, non-uniform allocation, and denoiser-mean plug-ins. Although these approximations are acknowledged and the surrogate is evaluated in a tractable toy setting, the reported configuration is limited to a low-dimensional problem at a single timestep. This leaves the conditions under which the practical estimator reliably benefits from reuse insufficiently characterized. Additional sensitivity analyses across timesteps, reward strengths, and source-distribution overlap would help determine when approximation bias or importance-weight concentration limits those gains.

Questions for authors

Reward–Diversity Trade-off: How do the competing methods compare in within-batch diversity at matched compute budgets? Does the proposed method retain its reward advantage when comparing operating points with similar diversity? For the red-teaming objectives, could an additional coverage metric help distinguish distinct failure cases from multiple similar flagged images?

Reliability of the Practical Estimator: How sensitive are the bias and MSE of the final self-normalized denoiser estimate to timestep, reward strength, and overlap between source conditional distributions? Extending the tractable toy evaluation along these axes would help clarify when the surrogate bridge preserves the benefits of sample reuse and when approximation bias or importance-weight concentration becomes limiting.

Effects of Allocation and Plug-in Approximations: Can the toy evaluation assess non-uniform sample allocation and denoiser-mean plug-ins, particularly when the shared budget is smaller than the batch size? How do these choices affect the final denoiser's bias and MSE? A pooling control without density-ratio correction would also help isolate the contribution of the surrogate bridge.

3 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

[W1, Q1] Reward–Diversity Trade-off and Safety Coverage

Thank you for highlighting that the utility of a returned batch depends on both reward and diversity. We have added within-batch diversity measurements for competing methods at comparable computational budgets, together with a bandwidth sweep for our method. Point 2 of General Response reports these comparisons, while Point 3 of General Response explains the intended use cases. These results support a controllable trade-off rather than a uniform reward advantage at matched diversity.

Additionally for the safety case, we report two metrics, flagged per prompt, and distinct flagged images per prompt. Flagged per prompt follows the definition in the paper. We additionally report vendi score below. Our method increases both flagged-image yield and distinct-failure count relative to Vanilla, but yields fewer diversity score than BoN and FK.

MethodNFE / imageFlagged per prompt ↑Vendi ↑
Vanilla280.502.07
Ours Stot=8, h=0.1S_{tot}=8,\ h=0.1971.721.75
BoN-41121.632.05
FK (S=4S=4)1121.472.19

[W2,Q2]. Reliability of the practical estimator

Thank you for highlighting the need to better characterize the practical estimator. In our Point 1 of the general response, we provide an additional mathematical explanation of its relation to the exact bridge: at the level of the endpoint-dependent log-ratio coefficient, the surrogate replaces a pair-dependent inverse-covariance correction with a shared isotropic scale. This explanation clarifies how the surrogate differs from the exact bridge.

To complement this explanation, we report the following sensitivity results. We extended the tractable GMM experiment in Section 5.2 across timesteps t∈0.2,0.5,0.8t\in{0.2,0.5,0.8}, reward strengths β∈0.5,2,8\beta\in{0.5,2,8}, and mixture component radii R∈1.5,3,4.5,6R\in{1.5,3,4.5,6}, with larger radii used to probe more separated source distributions. The tilted mean is available in closed form. We use exact conditional draws to isolate estimator behavior from GLASS discretization error, with B=8B=8 and one draw per source.

The table summarizes representative settings. The MSE ratio compares sharing against no sharing; values below 1 indicate improvement. h∗h^* denotes the best tested sharing bandwidth among (0.1,1,∞{0.1,1,\infty}), selected separately for each setting.

Sensitivity axisSettingh∗h^*MSE ratio ↓Bias² / MSE at h∗h^*
Timestep ((β=2,R=3\beta=2,R=3))t=0.2t=0.210.3114%
t=0.5t=0.50.10.7832%
t=0.8t=0.80.15.7989%
Reward strength ((t=0.2,R=3)(t=0.2,R=3))β=0.5\beta=0.510.308%
β=2\beta=210.3114%
β=8\beta=810.4046%
Source separation (t=0.5,β=2)(t=0.5,\beta=2)R=1.5R=1.50.10.6312%
R=3R=30.10.7832%
R=4.5R=4.50.12.8878%
R=6R=60.19.0691%

Timestep and source separation determine where reuse remains beneficial. At the default reward strength and radius, sharing reduces MSE at early and intermediate timesteps, but becomes big at t=0.8t=0.8, where bias accounts for 89% of its error. Increasing source separation similarly erodes the benefit: at t=0.5t=0.5, the MSE ratio increases from 0.63 to 9.06 as the radius increases. The preferred bandwidth also decreases with timestep.

Stronger rewards increase weight concentration. At t=0.2t=0.2 and h=1h=1, source ESS decreases from 3.72 to 1.43 as β\beta increases from 0.5 to 8. Sharing still reduces MSE, but its relative benefit diminishes and the bias fraction increases.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

Q3. Effects of Allocations and Plug-ins.

Thank you for the point. We evaluated non-uniform allocation, denoiser-mean plug-ins, and uncorrected pooling in the GMM setup of Section 5.2, including budgets smaller than the batch size B=8B=8. We measure bias², variance, and MSE against the closed-form tilted mean. Results below are averaged over t∈0.2,0.5,0.8t\in{0.2,0.5,0.8}.

(1) Allocation and plug-ins under sparse budgets. The plug-in supplies a denoiser-mean atom for sources receiving no endpoint draws. Removing it excludes those sources and renormalizes the remaining contributions.

BudgetAllocationPlug-inBias²VarianceMSE ↓
2RewardOn2.551.333.88
2RewardOff7.582.8310.41
2RandomOn2.191.313.50
2RandomOff3.635.659.28
4RewardOn2.651.774.42
4RewardOff6.212.558.76
4RandomOn2.201.894.09
4RandomOff2.573.706.27

At both sparse budgets, the plug-in reduces bias and variance under both allocation rules. Its benefit is relative to omitting zero-draw sources; it does not make the estimator unbiased.

(2) Uncorrected-pooling control. Setting h=∞h=\infty removes the density-ratio correction while retaining reward weighting and per-source count normalization.

BudgetAllocation / plug-inCorrected bias²Pooling bias²Corrected MSE ↓Pooling MSE ↓
2Reward / on2.556.793.887.84
4Reward / on2.656.694.427.77
16Uniform2.486.464.437.15
64Uniform2.916.574.416.77

The surrogate correction substantially reduces bias and MSE relative to uncorrected pooling. Increasing the sample budget does not eliminate pooling bias. However, the corrected estimator also retains bias: at budget 64, no sharing achieves MSE 1.23, compared with 4.41 for corrected sharing. The untilted denoiser has MSE 1.87 in this averaged setting. Thus, these results establish the contribution of the correction relative to pooling.

Thank you for the additional experiments. The baseline diversity comparisons, non-Gaussian toy results, and allocation/plug-in controls substantially improve the empirical characterization. However, the results support a reward–diversity trade-off rather than a clear advantage at comparable diversity.

The sensitivity experiments also reveal remaining reliability concerns: sharing increases MSE relative to no sharing by factors of 5.79 at t=0.8 and 9.06 under greater source separation, even with the best bandwidth among those tested. These findings do not directly establish image-generation failures, but leave uncertainty about the practical impact of estimator degradation at later timesteps or when source distributions overlap poorly.

Based on the above assessment, I maintain rating of 3.

Reviewer_B7e3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper considers inference-time reward guidance of pre-trained flow-matching models when the reward is only available at test time and the gradients of the reward function are not available/do not exist. The goal of this paper is to improve computational efficiency of reward guidance in a “batched” setting where all elements of the batch share the same prompt/conditioning. Conventionally, each element of the batch \big\\{x^{(b)}_t\big\\}_{b=1}^B requires SS independent samples from the conditional endpoint distribution p1∣t(⋅∣xt(b))p_{1 \mid t}(\cdot \mid x_t^{(b)}) to estimate its tilted flow velocity. The paper proposes “sharing” endpoint samples across batch elements for computational efficiency, allowing fewer overall samples per batch. The mismatch between the source samples and the target distribution is corrected via importance weighting. Since importance weights require the density of the endpoint distributions p1∣t(⋅∣xt(b))p_{1 \mid t}(\cdot \mid x_t^{(b)}) which are intractable, the paper proposes a Gaussian approximation of the endpoint distribution whose mean denoiser Dt(xt(b))D_t(x_t^{(b)}) can be obtained directly from the pre-trained velocity field ut(xt)u_t(x_t), but proposes a variance parameter that is estimated empirically. Experiments show superior performance under the same computational budget in multiple text to image generation tasks with standard image rewards.

Critical strengths

The main strength of this paper is its promise to reduce the computational efficiency of inference-time reward guidance in the batched setting by reducing the total number of lookahead samples needed per integration step across an entire batch. This is done by allowing an endpoint sample generated for one trajectory to contribute to the estimate of the tilted velocity of another. Each sample can therefore contribute to the estimate for every trajectory in the batch without additional NFEs.

Experimental results are somewhat compelling:

  • The performance-efficiency tradeoff seems favorable under low inference budgets.
  • Experiments show that allowing cross-trajectory sharing across a batch improves performance for free without additional neural function evaluations (NFEs) (Table 1).

Minor strengths

  • Paper is generally well-written, easy to read and flows well.
  • The mathematical notation used in the paper is intuitive and easy to follow. For example: the notation wb←j(xj,s)w_{b \leftarrow j}(x^{j, s})
  • The experiment section is very well organized, with each paragraph dedicated to a single result and takeaway.
Critical weaknesses

Major weaknesses

  1. Gaussian approximation of conditional endpoint distribution is not theoretically well-justified: It is not clear to me why the conditional endpoint distribution p1∣t(x1∣xt(j))p_{1\mid t}\big(x_1 \mid x_t^{(j)}\big) should be related to the discrepancy of endpoint means across different samples of xtx_t, i.e. the discrepancy of {E1∣t[x1∣xt(b)] ∣ b≠j}\big\lbrace \mathbb{E}_ {1\mid t}\big[x_1 \mid x_t^{(b)}\big] \,\big\vert\, b \neq j \big\rbrace. Or more precisely, why is Vx1∼p1∣t(x1∣xt)(x1)≡Vxt∼pt(xt)(E1∣t[x1∣xt])\mathbb{V}_ {x_1 \sim p_{1\mid t}(x_1 \mid x_t)}(x_1) \equiv \mathbb{V}_ {x_t \sim p_t(x_t)}\big(\mathbb{E}_ {1\mid t}[x_1 \mid x_t]\big)? In particular, the conditional variance (left) depends on xtx_t, whereas the surrogate variance/discrepancy of the conditional mean (right) is independent of xtx_t. And the bandwidth hh is a hyperparameter that needs to be tuned empirically.
  2. Toy setting to justify Gaussian approximation seems contrived: The main justification for the proposed Gaussian approximation is in Section 5.2 that evaluates it empirically in a toy setting. The toy setting uses a mixture of isotropic Gaussians dataset where each component has equal variance σdata2\sigma^2_\text{data}. In this setting, the conditional endpoint distribution is likely to be Gaussian whose variance is somewhat independent of conditioning on xtx_t. I'm afraid that this toy example is somewhat doomed to validate the assumptions of the proposed Gaussian approximation. A more compelling validation might be to consider non-Gaussian low-dimensional datasets (e.g. the standard four moons or swiss roll datasets). Using a large number of GLASS samples, one could perform kernel density estimation of the true conditional endpoint distributions, and study the fidelity to the proposed Gaussian approximation.
  3. Statistical analysis doesn't account for Gaussian approximation: The statistical analysis in Section 4.3 that shows the unbiasedness of estimators only applies to the "exact bridge" which is intractable to compute. This analysis does not shed light on the actual estimator that is used by this work, i.e. the Gaussian approximation of the conditional endpoint density.
  4. Unclear advantage of speed gains compared to stochastic flow maps: Are the speed gains by sharing samples across the batch really worth it? Especially if one uses faster stochastic flow maps like DiamondMap (Holderrieth et al. 2026)? With DiamondMap, the cost of sampling from the endpoint distribution is very low. This might reduce the computational gains obtained by sharing trajectories across samples in the batch. Currently, the paper uses Stot=8S_\mathrm{tot} = 8, Mt=10M_t = 10. Using the same NFEs, DiamondMap can use 1010 independent endpoint samples per batch element. A comparison to DiamondMap (even if the latter requires distillation training) can strengthen the paper.
  5. Requires batch elements to share conditioning/prompt: As pointed out in the limitations section, the approach assumes that multiple trajectories share the same generation condition. Queries may not be identical in practice, or detecting and batching identical queries across a large cluster of servers may require significant engineering effort.
  6. Unclear if reduction in batch diversity is due to cross-trajectory sharing or simply due to better reward maximization: The paper reports that at a fixed inference budget, increasing the bandwidth hh reduces diversity across the batch while increasing reward. It is unclear whether the reason is higher correlation due to cross-trajectory sharing (as claimed in the paper), or due to better optimization of the reward which would naturally reduce diversity. For example, sampling from a tilted distribution with very low temperatures would yield a single global optimum. A control experiment might be as follows: obtain a high reward without cross-trajectory sharing by using a large number of independent (h=0h=0) samples per batch element. Then, study whether cross-trajectory sharing reduces diversity beyond this control.

Minor weaknesses

  • "need not remain trajectory-local" reads like LLM-speak. Maybe a better way to say it is "need not be specific to individual trajectories."
Questions for authors

Some points mentioned in the weaknesses section:

  • Point #4: Is the savings in computation due to sharing endpoint samples across batch elements still significant when using fast stochastic flow maps like DiamondMap (Holderrieth et. al. 2026) where independent endpoint samples are cheaper to compute?
  • Point #6: Is the reduction in batch diversity due to correlated endpoint samples or simply due to optimizing the reward better?
5 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

W1. Further Explanation on Approximation.

Thank you for raising this point. We do not assume the stated equality between conditional variance and the variance of conditional means, nor interpret κt\kappa_t as an estimate of posterior variance. As detailed in Point 1 in General Response on the Gaussian surrogate, their connection is through the conditional-mean Jacobian. We will clarify this distinction in the paper.

For hyperparameter hh, We agree that hh is a hyperparameter that requires empirical tuning. However, we would like to highlight that Fig. 4 shows a clear trend in the reward-diversity trade-off as h varies. We hope this provides practical guidance for choosing h according to the desired balance between reward and diversity.

W2. Toy Experiments

Thank you for suggesting this validation. Following the suggestion, we trained a two-dimensional flow model on each of the swiss-roll and four-moons distributions and constructed conditional endpoint KDEs from 4,000 GLASS samples, selecting the KDE bandwidth using held-out GLASS samples. Each data distribution consists of 1,000 points, normalized to unit standard deviation per coordinate and smoothed with N(0,0.052I)\mathcal N(0,0.05^2I). This construction also provides analytically tractable conditional densities and tilted means for evaluation. We retain the original linear path and linear reward, with β=2\beta=2, t=0.2t=0.2, B=8B=8, and S=4S=4, and use the same pooled GLASS endpoint samples for all bridge estimators. The true-ratio bridge evaluates with the analytically available conditional densities of the smoothed data distribution. The KDE bridge replaces these densities with the corresponding per-trajectory KDEs, while the surrogate bridge uses our Gaussian approximation. All three bridges are applied to the same pooled GLASS samples. Since these samples come from a learned model, the true-ratio bridge serves as an oracle reference for the bridge weights.

DataMethodTarget MSE ↓Weight TV ↓Cosine ↑Norm ratio
Swiss rollIndependent0.641———
True-ratio bridge0.06301.0001.000
KDE bridge0.1110.1770.9750.984
Surrogate bridge0.0640.0670.9981.013
Four moonsIndependent0.703———
True-ratio bridge0.08401.0001.000
KDE bridge0.1380.1390.9891.003
Surrogate bridge0.1000.0820.9911.021

The table shows that the surrogate bridge retains most of the exact bridge’s improvement on non-Gaussian data. The KDEs approximate the conditional endpoint densities more faithfully than the Gaussian approximation. The surrogate bridge achieves target-MSEs of 0.064 and 0.100 on swiss roll and four moons, respectively, compared with 0.063 and 0.084 using the true density ratios, and 0.111 and 0.138 using KDE-based ratios. These results support the usefulness of the surrogate for estimating the tilted-mean correction, even when it does not accurately model the full conditional density.

W3. Statistical Effects.

We agree that the unbiasedness analysis in Section 4.3 applies to the exact bridge and does not extend directly to the Gaussian surrogate. Its purpose is to establish the validity of sample reuse with exact density ratios; replacing these ratios with a surrogate can introduce additional bias. We will clarify this distinction.

W4. Gain Comparison with Stochastic Flow Maps.

Thank you for this suggestion. We agree that cheaper endpoint sampling can reduce the computational motivation for reuse, and that requiring distillation training is not sufficient reason to omit this comparison. We therefore evaluated our method with Posterior Diamond Maps (PDM) and Meta Flow Maps (MFM). Please refer to Point 4 in General Responses, of which the results show that our approach extends beyond GLASS and can be combined with other choices of stochastic flow maps.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

W5. Shared Prompt Condition.

Thank you for highlighting this practical limitation. We agree that repeated prompts alone do not guarantee that compatible requests can be batched efficiently, and that detecting and scheduling such requests across servers can require additional engineering. We have expanded the scenario discussion in Point 3 of our general response to distinguish two cases. A single request may ask for multiple candidates under one prompt, directly providing a shared-condition batch without cross-request matching. Alternatively, multiple compatible requests may be processed jointly when the serving system supports such batching. Our primary use case therefore does not require cross-request scheduling; applicability to multiple requests depends on the availability of compatible batches. We also conducted an exploratory cross-prompt experiment across the three tested models using identical and closely related prompts, where performance gains were observed, whereas unrelated prompts showed no such benefit.

W6. Control Experiments.

Thank you for suggesting this control. We disabled cross-trajectory sharing (h=0h=0) and increased the endpoint-sampling budget, keeping the same seed and allocation protocol as the coupled method. As shown below, increasing the uncoupled budget improves reward while leaving within-batch diversity largely unchanged, whereas the coupled method achieves higher reward with lower diversity. Each entry reports ImageReward ↑ / Vendi Score ↑. This pattern consists on other settings, see Point 2 of General Response for additional experiments.

FLUX.1 + GLASS

Vanilla: 0.604 / 1.55, NFE: 25.

StotS_{\mathrm{tot}}NFE/imageUncoupled (h=0h=0)Coupled (h=0.05h=0.05)
243.750.595 / 1.550.793 / 1.28
81000.605 / 1.540.885 / 1.30
161750.654 / 1.550.880 / 1.31
323250.667 / 1.550.888 / 1.31

Dear Reviewer B7e3, Thank you again for your thoughtful feedback. We have provided detailed responses and additional experiments addressing your concerns, and uploaded a revised manuscript with changes highlighted in blue. As the discussion period is nearing its end, we would appreciate your assessment of whether these additions address your main concerns. If any questions remain, we would welcome the opportunity to clarify them during the remaining discussion period. Thank you for your time and consideration.

Reviewer_bZ9jOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper constructs a cross-trajectory estimator for the gradient of the value function used in inference time steering for flow matching methods. It does so by making use of a GLASS flow estimator of endpoint samples from intermediate states xtx_t, and constructing a reweighting procedure in the tilted denoiser that allows one to use endpoints from nearby trajectories in a batch of trajectories used in the estimator as a means of reducing computational overhead (though it assumes the resulting tilt ratio added in to correct is gaussian, which should only hold at late times). This is essentially a sort of particle method.

They test the method on text to image generation problems where the prompt is the shared condition on which a batch of trajectories are rolled out and individually steered, sharing GLASS endpoint samples in their tilted denoiser estimator according to the ratio mentioned above. This allows them to steer more effectively with fewer GLASS computations, and it scales with the size of the batch of rollouts.

Critical strengths

The paper fills its intended scope, providing concrete evidence for their proposed algorithm, to the extent of what the proposed algorithm suggests it can do. Given that folks like to do inference time steering over batches of rollouts, the tilting method here provides a concrete approach toward computational savings in such settings.

Critical weaknesses

Because this is a particle based method, it only works when generative a batch of walkers simultaneously, which, in the large batch size regime where this method works the best, is costly and unlikely to occur in the real world examples given in the experiments (e.g. text to image generators).

The paper's a bit hard to read at a few points, too, primarily because of LLM jargon, or maybe everything sounds like that now :(

Questions for authors

How much does GLASS accuracy matter for the results, and is there an ablation on that being more essential than anything else in the method?

Did you explore a cross-prompt version of this, that would make the batch of samples more practically relevant?

1 comment

W1, Q2 Real-world Applicability and Cross-Prompt Experiments.

Thank you for raising these concerns and suggesting the cross-prompt evaluation. In Point 3 of General Response, we clarify the practical scenarios for shared-condition batching and discuss their applicability and limitations. In particular, Use Case 2 includes an exploratory comparison of our method under identical, semantically similar, and unrelated prompts, examining whether its benefits extend beyond the same-prompt setting. Please refer to that response for the results and discussion.

W2- Terminology.

Thank you for pointing this out. We will revise the writing!

Q1 - GLASS Accuracy

Thank you for the point.We conducted two ablations to assess how strongly our results depend on GLASS accuracy relative to sample sharing.

(1) SD3: varying GLASS integration steps with sharing off/on. We evaluated (M_t\in{3,5,10}) on 32 GenEval prompts with (\beta=100), under reward allocation.

StotS_{tot}MtM_tNFEImageReward(Uncoupled h=0)ImageReward(Coupled h=0.05)Gain(Coupled - Uncoupled)
1631000.3310.643+0.312
1651480.3410.673+0.332
16102680.3600.684+0.324
83640.3190.632+0.313
85880.3270.655+0.328
8101480.3340.655+0.321

Increasing MtM_t from 3 to 10 improves ImageReward by only 0.015–0.041, whereas enabling sharing consistently improves it by 0.31–0.33. Even sharing with Mt=3M_t=3 at 64 NFE outperforms no sharing with Mt=10M_t=10 at 268 NFE (0.632 vs. 0.360).

(2) GMM toy: comparison against the known conditional distribution and tilted mean. We use the GMM setup described in Section 5.2, where exact conditional sampling and the ground-truth tilted mean are available. At t=0.2t=0.2 and β=2\beta=2, we use four draws per source and 30 repetitions. We evaluate (i) endpoint sampling accuracy using squared Maximum Mean Discrepancy (MMD²) between the sampled endpoint distribution and the true conditional distribution (p1∣t(⋅∣xt))(p_{1|t}(\cdot\mid x_t)), and (ii) the normalized MSE of the resulting tilted-mean estimate against its ground truth, with and without coupling. Lower values indicate better accuracy for both metrics.

Endpoint samplerConditional MMD² ↓MSE (Uncoupled) ↓MSE (Coupled) ↓
Exact conditional0.00071.2080.468
GLASS, Mt=3M_t=30.01620.8660.322
GLASS, Mt=5M_t=50.00680.9450.331
GLASS, Mt=10M_t=100.00230.9230.376

Larger MtM_t improves endpoint sampling accuracy, but sharing reduces estimator MSE by approximately 61–65% across all four samplers, including exact conditional sampling.

Together, these ablations show that, within the tested settings, sample sharing has a substantially larger effect than increasing GLASS integration accuracy, and its benefit persists with both coarse GLASS sampling and exact conditional sampling.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to All Reviewers (General Response) (1/4)

We thank all reviewers for their thoughtful feedback. Below, we summarize the additional experiments and clarifications addressing concerns shared across multiple reviews. Reviewer-specific responses are provided separately.

Point 1. Understanding the Gaussian Surrogate through the Exact Bridge

Key takeaway : Our Gaussian surrogate can be interpreted as replacing the exact bridge’s pair-dependent inverse-covariance correction in the endpoint-dependent log-ratio coefficient with a shared isotropic scale, while retaining state dependence through denoised-mean differences.

Exact bridge. Under xt=αtx1+σtϵx_t=\alpha_t x_1+\sigma_t\epsilon, with ϵ∼N(0,I)\epsilon\sim\mathcal N(0,I) independent of x1x_1 and αt,σt>0\alpha_t,\sigma_t>0, Bayes’ rule cancels the common data density:

=\frac{\alpha_t}{\sigma_t^2} (x_t^{(b)}-x_t^{(j)})^\top x_1+\text{const}_{bj}, \tag{1}$$ where $c_{bj}$ is independent of $x_1$. This does not require the conditional endpoint distribution to be Gaussian. **Link to denoised means.** By first and second order Tweedie formulas [Efron, 2011; Meng et al., 2021] give the Jacobian–covariance identity. Integrating it along the segment between the two states yields $$J_{D_t}(x_t) = \frac{\alpha_t}{\sigma_t^2}\operatorname{Cov}(x_1 \mid x_t),$$ $$D^{(b)} - D^{(j)} = \frac{\alpha_t}{\sigma_t^2}\overline{\Sigma}_{bj}\bigl(x_t^{(b)} - x_t^{(j)}\bigr),$$ $$\overline{\Sigma}_{bj} = \int_0^1 \operatorname{Cov}\Big[ x_1 \Big| x_t = x_t^{(j)} + s(x_t^{(b)} - x_t^{(j)}) \Big] ds,$$ where $D^{(b)}=D_t(x_t^{(b)})$. Thus, denoised-mean differences reflect conditional covariance through its action on the noisy-state displacement. **Exact versus surrogate.** When $\overline\Sigma_{bj}$ is invertible, this implies $$\log\rho_{bj}(x_1) =(D^{(b)}-D^{(j)})^\top\overline\Sigma_{bj}^{-1}x_1+\text{const}_{bj},$$ $$\log\pi_{bj}^{(h)}(x_1) =(D^{(b)}-D^{(j)})^\top(\kappa_tI)^{-1}x_1+\text{const}^\prime_{bj}.$$ At the level of the endpoint-dependent coefficient, our surrogate therefore replaces the pair-dependent inverse covariance with a shared isotropic scale. Although this does not reproduce each state’s conditional covariance, the bridge remains state-dependent through $D^{(b)}-D^{(j)}$. The scale $\kappa_t$ is recomputed at each step and shared across pairs within that step; the endpoint-independent constants may also differ. **References**. [Efron, 2011] Bradley Efron, Tweedie’s Formula and Selection Bias, J Am Stat Assoc. 2011 [Meng et al., 2021] Chenlin Meng, Yang Song, Wenzhe Li, Stefano Ermon, Estimating High Order Gradients of the Data Distribution by Denoising, NeurIPs 2021.
AuthorsOpenReview ↗

Response to All Reviewers (General Response) (2/4)

Point 2. Reward–Diversity Trade-offs at Fixed Sampling Budgets

Key takeaway : Our method makes cross-trajectory reuse a flexible control mechanism for batched generation. Without increasing endpoint-sampling NFEs, bandwidth hh governs a principled reward–diversity trade-off, validating the effectiveness of sample sharing across identical prompts across diverse budget levels.

Experiemental Settings. We evaluate reward and diversity at three computational budget levels: low, medium, and high. Each table entry reports ImageReward / Vendi score over the B=8 delivered images. For the proposed method, we vary hh within each budget to show the reward–diversity trade-off at a fixed sampling cost.

Results. These results show that cross-trajectory reuse provides a control for increasing mean reward at a fixed sampling budget, with bandwidth hh governing the accompanying trade-off in within-batch diversity.

SD3 (Vanilla with NFE 28, ImageReward 0.369 / Vendi Score 1.67)

Method / hhLow (NFE 55~65)Medium (NFE 95~115)High (NFE 160~180)
FK-GLASS0.425 / 1.780.587 / 1.700.663 / 1.66
BoN0.529 / 1.680.620 / 1.620.663 / 1.61
BoN-GLASS0.472 / 1.730.581 / 1.700.640 / 1.67
LiDAR0.449 / 1.570.526 / 1.560.571 / 1.55
Ours: StotS_{{tot}}4 (NFE 62)8 (NFE 97)16 (NFE 166)
h=0h=0 (Uncoupled)0.374 / 1.690.382 / 1.680.393 / 1.70
h=0.01h=0.010.583 / 1.490.592 / 1.500.581 / 1.55
h=0.02h=0.020.615 / 1.410.604 / 1.450.641 / 1.44
h=0.05h=0.050.640 / 1.340.635 / 1.330.674 / 1.34
h=0.1h=0.10.633 / 1.280.653 / 1.280.677 / 1.29
h=1h=10.641 / 1.280.661 / 1.260.682 / 1.28
h=h=\infty$$0.645 / 1.270.663 / 1.240.683 / 1.27

FLUX 1-dev (Vanilla with NFE 25, ImageReward 0.604 / Vendi Score 1.55)

Method / hhLow: (NFE 50)Medium (NFE 100)High (NFE 175))
FK-GLASS0.705 / 1.570.816 / 1.520.821 / 1.52
BoN0.731 / 1.540.806 / 1.530.846 / 1.51
BoN-GLASS0.721 / 1. 520.814 / 1.540.841 / 1.57
LiDAR0.688 / 1.460.752 / 1.420.733 / 1.47
Ours: StotS_{tot}2 ( NFE 43.75)8 (NFE 100)16 (NFE 175)
h=0h=0 (Uncoupled)0.595 / 1.550.605 / 1.540.654 / 1.55
h=0.01h=0.010.758 / 1.430.761 / 1.450.751 / 1.48
h=0.02h=0.020.790 / 1.350.828 / 1.420.812 / 1.46
h=0.05h=0.050.793 / 1.280.885 / 1.300.880 / 1.31
h=0.1h=0.10.820 / 1.240.882 / 1.260.907 / 1.25
h=h=\infty$$0.823 / 1.180.845 / 1.210.868 / 1.18
AuthorsOpenReview ↗

Response to All Reviewers (General Response) (3/4)

Point 3. Practical Scenarios and Exploratory Cross-Prompt Evaluation

Key takeaway : Our primary setting is to return multiple candidate images for a single prompt. An additional use case is jointly processing requests with identical or closely related prompts. Our exploratory experiments further show that the benefits of sample reuse can extend from identical prompts to semantically related variants.

Use case 1: Providing BB candidate images for a single prompt or request. Workflows such as Midjourney and Adobe Firefly allow users to choose among multiple generated candidates. In this setting, users may value both the quality of individual candidates and variation among them: the candidates should align well with the request while offering meaningful alternatives. This motivates evaluating mean reward and within-batch diversity jointly at a fixed output count and computational budget.

Our method provides a bandwidth parameter, hh, for choosing this balance. For example, on SD3 with B=8B=8 and 62.5 NFEs per image, setting h=0.01h=0.01 increases ImageReward from 0.374 to 0.583 relative to uncoupled steering, while Vendi decreases from 1.69 to 1.49. This illustrates a setting that improves reward alignment while retaining more diversity than stronger coupling. The full comparison with competing methods and other bandwidths is provided in General Response 2. Please note that all BB candidates are returned to the user; selecting and returning only one image through best-of-BB is a different evaluation objective and is not our primary focus.

Supplementary use case 2: Reuse across requests with related prompts. Beyond our primary setting of returning multiple candidates for a single request, reuse may also be useful when independently submitted requests have identical or closely related prompts. As discussed in our Motivation (Section 1), the reported prevalence of repeated prompts and the use of prompt templates motivate exploring this possibility. We therefore conducted an exploratory experiment relaxing the identical-prompt assumption.

Additional Exploratory Experiment of Use case 2. To investigate this, we evaluated three batch compositions: identical prompts, similar prompt variants generated using the FLUX 3 prompt upsampler, and different prompts describing unrelated content. Each batch contains B=8B=8 images, and results are averaged over ten base-prompt cases. Within each condition, Ours and Uncoupled use the same sampling budget of Stot=8,h=0.05S_{tot}=8, h=0.05. Identical and similar conditions are evaluated against the original base prompt; in the different condition, each image is evaluated against its own prompt.

Each entry below reports the aesthetic gain when the cross-trajectory reuse is applied, Ours(h) − Uncoupled: ImageReward / CLIPScore x100. GenEval prompts, 3 different models including Stable Diffusion 3, FLUX 1, FLUX 3.

ModelIdenticalSimilarDifferent
SD3+0.388 / +0.41+0.263 / +0.350.000 / −0.17
FLUX.1+0.295 / +0.80+0.221 / +0.280.000 / 0.00
FLUX 3+0.461 / +0.40+0.188 / +0.10−0.053 / −0.26

Sharing improves both mean ImageReward and CLIPScore for identical and similar prompts across all three models. For unrelated prompts, it provides no reward benefit. These results suggest that the benefits of reuse can extend beyond identical text to closely related prompt variants. We present this as an exploratory assessment of applicability and leave a systematic investigation of sharing across similar but distinct prompts to future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.