- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work studies anisotropic stochastic transitions for more efficient exploration in inference-time reward alignment. The authors analyze the Jacobian of the predicted endpoint with respect to the intermediate state and observe that it is effectively low-rank. This motivates their argument that isotropic noise uses exploration budget inefficiently, leading them to propose covariance-shaped noise transitions. To sample from these transitions efficiently, they derive a marginal-preserving SDE whose drift correction reduces to a first-order flow-acceleration term. Experiments on text-to-image generation and offline RL show improved reward-alignment performance under matched compute budgets.
Critical strengths
-
The method is well motivated. The empirical observation that the endpoint-prediction Jacobian is effectively low-rank, together with the endpoint-variance analysis, provides a clear rationale for allocating noise anisotropically.
-
Although covariance-shaped transitions incur additional computational overhead, the authors show improved reward-alignment performance relative to isotropic noise under matched compute budgets.
Critical weaknesses
- The empirical results, particularly for text-to-image generation, do not yet demonstrate a practically meaningful advantage over isotropic noise relative to the proposed method’s additional computational overhead. In the adaptive-resampling experiment in Table 5, covariance-shaped noise improves the aesthetic score by only (a 0.73% improvement) while increasing runtime by 2.53 seconds (a 6.1% deterioration). Moreover, the text-to-image results are reported without error bars, standard deviations, or confidence intervals. Given the small difference in mean reward, these missing statistics make it difficult to determine whether the observed improvement is robust and practically significant rather than attributable to evaluation variability.
Questions for authors
-
For the text-to-image results in Figure 4, can the authors report the mean and standard deviation over prompts at each NFE budget? A table would be sufficient and would clarify whether the reported reward improvements are robust.
-
The spectral and endpoint-variance analyses in Figure 3 and Table 1 appear to use just a single prompt (“a photo of a bench”). Can the authors extend this analysis to a more diverse set of prompts, such as a subset of the GenEval prompts used in the text-to-image experiments? This would clarify whether the proposed endpoint-diversification mechanism generalizes beyond the illustrated example.
4 comments
Dear reviewer, thank you again for your time and for the detailed review.
As the discussion period is closing in a day, we would greatly appreciate it if we could know whether our rebuttal has properly addressed your concerns. If any concern remains, we would be happy to provide further clarification.
Dear Reviewer 8eUt,
As the rebuttal period closes in just a few hours, we would be truly grateful if you could take a moment to review our response. We sincerely hope it addresses your concerns, and we would greatly appreciate any further feedback.
I thank the authors for their work during this review period and will address their points now.
Q2 [Endpoint Variance across prompts] Thank you for pursuing this result. This extension to multiple prompts indicates that the observed low rank behavior is not isolated to the original prompt and fully addresses my concern.
[W1, Q1] Statistical Reliability of Performance Gains I appreciate the authors' presentation of the error bars in Figures 4 and 5, and the anlaysis presented in Table A. I still find the numerical improvements somewhat modest but the authors have convinced me that they are nevertheless robust.
I will therefore raise my score to weak accept.
Rebuttal Response
We sincerely thank the reviewer for carefully reading our paper and providing constructive feedback.
[W1, Q1] Statistical Reliability of Performance Gains
Thank you for highlighting the importance of quantifying variability. We have updated Figures 4 and 5 to report mean performance with error bars, displayed as shaded bands representing the standard deviation across seeds and examples.
The updated results show a consistent overall improvement trend over isotropic sampling. Although the absolute scale of aesthetic score gains seem modest, our method generally achieves higher mean rewards with non-overlapping standard deviation bands, with a clearer advantage at higher NFE budgets. Improvements are also observed for ImageReward and both offline-RL environments. These results provide further evidence for the benefit of covariance-shaped exploration, with the magnitude of improvement varying across objectives and tasks.
To directly assess uncertainty in the text-to-image reward improvements, Tables A and B report mean paired reward differences and their 95% confidence intervals. For both Aesthetic alignment and ImageReward, the confidence intervals for improvements over isotropic sampling lie entirely above zero in most evaluated settings, providing statistical support for the observed gains.
Table A. Aesthetic differences between our method and each baseline: mean difference [95% paired confidence interval].
Table B. ImageReward differences between our method and each baseline: mean difference [95% paired confidence interval].
[Q2] Endpoint variance analyses on various prompts
We thank the reviewer for suggesting this additional analysis. We first clarify that Figure 3 averages the spectra over 64 independent initial noise seeds at each timestep, while Table 1 averages the endpoint-variance estimates over these seeds and the three timesteps.
We agree that evaluating additional prompts is important for assessing the generality of the observed behavior. We have therefore extended the analysis to additional prompts randomly sampled from GenEval and provided the results. For each prompt, we evaluate 10 independent initial noise seeds at the same three timesteps, t∈{0.2,0.5,0.8}, using the measurement protocol described in Appendix E.2.
We updated Figure 3 to include spectra from multiple prompts with standard-deviation bands. The averaged spectra remain strongly concentrated, consistent with the behavior observed in the original example.
Table C reports the endpoint variance Vh under matched perturbation norms, following the same measurement procedure as Table 1. For each prompt, the estimates are averaged over seeds and three timesteps. The ratio is the covariance-shaped mean divided by the isotropic mean, and the pooled row averages the variance estimates across all eight prompts. These results provide further evidence that the proposed mechanism extends beyond the original bench example.
Table C. Endpoint variance across prompts