- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents a training-time approach to initial noise optimization for deterministic generative models. The authors identify that existing test-time approaches require iterative refinement, requiring many generator and reward evaluations. To avoid this cost, they train a residual transport map using a local GRPO objective, resulting in reward-aligned samples with a single inference step. To mitigate reward over-optimization, they introduce a KL-based regularizer that constrains the corrected-noise distribution to remain close to the original prior. The method is evaluated on a toy problem, text-to-image generation, and protein structure generation. Results suggest that LeNO improves the target rewards while requiring substantially fewer inference-time function evaluations than iterative noise-optimization baselines.
Critical strengths
-
The evaluation suite is strong. The authors compare against both amortized training-time and iterative inference-time baselines across diverse settings, including text-to-image generation with differentiable preference rewards and protein structure generation with an expensive non-differentiable reward. This breadth helps contextualize the method’s advantages.
-
The empirical results are strong. LeNO consistently improves target rewards over the frozen text-to-image models. Although iterative inference-time baselines achieve higher absolute reward when given substantially larger search budgets, combining LeNO with Best-of-N yields competitive or stronger results using far fewer NFEs. This demonstrates a meaningful inference-efficiency tradeoff rather than simply reporting reward metrics in isolation.
Critical weaknesses
- LeNO trades expensive inference-time refinement for expensive pre-inference training. Although the paper specifies training hyperparameters, the authors do not directly report the computational overhead for training this transport. This omission make it difficult to assess the method's practical advantage over inference-time approaches.
Questions for authors
- Can you report the computational overhead for the text-to-image and protein experiments, including wall-clock training time, number and type of GPUs, and total generator/reward evaluations? This would help clarify the practical cost of amortizing optimization at train-time instead of test-time.
4 comments
A Kind Reminder for Comments
We sincerely thank Reviewer cVQv for the constructive feedback regarding training cost and the practical benefits of amortized noise optimization. Our response and revised manuscript address these concerns through the following analyses and clarifications:
- Training cost: Added computational cost estimates for SDXL-Turbo and protein generation, including wall-clock training time, GPU-hours, and reward-evaluation budgets (Appendix D.3).
- Amortization efficiency: Clarified the trade-off between training cost and inference efficiency. LeNO achieves comparable reward performance to inference-time ZeNO with substantially faster generation, making amortization beneficial for repeated generation under a fixed reward objective, particularly when reward evaluations are expensive.
We appreciate the reviewer's time and would welcome any remaining questions or concerns regarding these revisions. Changes in the revised manuscript are highlighted in blue.
I thank the authors for their thoughtful feedback to my concerns and their edits to the manuscript. Overall, I feel that my concerns have been addressed and I maintain my positive score.
We thank the reviewer for their careful consideration of our rebuttal and are glad that their concerns have been addressed.
Response to Reviewer cVQv
We thank the reviewer for the valuable feedback on training cost and computational efficiency. We provide additional runtime measurements and an amortized cost analysis to clarify when LeNO offers practical advantages over inference-time optimization. Detailed results and explanations are provided below.
W1 & Q1 : Training Cost
We profiled LeNO on eight RTX 4090 GPUs (24 GB each), measuring six image-training iterations and five protein-training iterations after warm-up. Initialization, periodic validation, and checkpoint saving are excluded.
Each protein reward evaluation uses four designed sequences, totaling 144,000 ProteinMPNN sequence designs and 144,000 ESMFold refoldings per model. Counts refer to generated samples, rather than batched calls; timing includes transport forward/backward computation. RMF-L's estimate is more variable: the eight-GPU-equivalent iteration times range from 12.93 to 45.82 seconds, with the delayed iteration retained in the mean.
On the same 100 prompt–seed pairs, 1,000-step LeNO achieves similar ImageReward to five-step ZeNO (0.970 vs. 0.961), with much faster generation (0.304 vs. 7.845 s/image on one RTX 4090). Including LeNO's estimated 71.8 GPU-hours of training, cumulative compute breaks even at approximately 34,300 images. LeNO therefore benefits repeated generation under a fixed reward objective; ten-step ZeNO can achieve higher reward (1.056) at greater inference cost. We will add these costs and amortization curves to the revision.
The amortization benefit is particularly relevant for protein generation, where reward evaluation requires costly ProteinMPNN sequence design and ESMFold refolding. LeNO incurs these evaluations during training but requires no reward calls at inference. Although inference-time methods can parallelize samples and candidate evaluations, successive refinement iterations remain sequential within each sample. LeNO replaces this iterative search with a learned transport that supports batched generation, making amortization especially attractive when reward evaluation dominates computational cost.