- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents RACE (Reward-Guided Adaptive Candidate Exploration), aiming to address candidate construction in reward-weighted diffusion fine-tuning, building on DiffusionNFT’s reward-weighted forward-process objective. In the algorithm, a frozen collection copy (v_old) generates B independent base samples per prompt; the top-P are selected as parents. A lightweight state-conditioned predictor Γ_φ(x_t, t) replaces DiffusionNFT’s single global scalar perturbation coefficient γ with an entry-wise map, used to construct additional “child” candidates around selected parents by mixing their implied noise direction with fresh Gaussian noise at varying, learned, per-coordinate strength.
RACE claims a learnable, state-conditioned, and entry-wise perturbation predictor replacing DiffusionNFT’s scalar perturbation for constructing candidates, jointly trained this light weight predictor with the generator. Under the same shared objective, RACE+’s reference-guided extension using inverted paired images as added exploration centers. These are validated via matched-budget comparisons against DiffusionNFT across natural-image, medical-imaging, and remote-sensing benchmarks, showing higher mean scores.
Critical strengths
1. The idea is interesting. Learning a small predictor to adaptively shape candidate exploration (rather than using a fixed scalar perturbation) is an interesting direction — letting the model decide the perturbation based on the current state is slightly more flexible than a hand-tuned global constant.
2. Clean integration with existing objective. Both the learned perturbation and the reference-guided candidates are folded into DiffusionNFT’s existing reward-weighted loss without introducing a separate objective, keeping the training pipeline relatively simple.
3. Broad empirical coverage. Experiments span natural images, medical imaging, and remote sensing, showing the method isn’t narrowly tuned to one domain.
Critical weaknesses
1. Marginal technical contribution compared with DiffusionNFT:
RACE’s core change relative to DiffusionNFT is replacing a single fixed scalar perturbation coefficient γ with a small online-learned, state-conditioned function Γ_φ(x_t, t) that predicts a per-coordinate version of the same scalar — the construction mechanics, the shared reward-weighted regression loss (Eq. 15), and the overall collection procedure are otherwise unchanged from DiffusionNFT. RACE+’s further addition is reusing inverted reference images as alternative exploration centers for the same predictor, which is a data-source substitution, not a new mechanism. Neither change touches the reward weighting scheme or the generator. This reads as an incremental refinement on top of an existing method, and the paper does not establish that the entry-wise structure itself is doing meaningful work beyond what a well-tuned scalar could achieve.
2. Added training complexity is not justified by the empirical gains, and RACE+ shows worse stability:
Jointly optimizing (θ, φ) introduces complexity the paper doesn’t adequately analyze: the construction relies on an engineered stop-gradient boundary, and the authors’ own derivation shows φ's gradient loses reward-dependence entirely whenever v_θ and v_old coincide. Despite this added machinery, the measured benefit is marginal — in Tables 2 and 4, even RACE+, which carries the most additional construction machinery, shows only modest mean improvement over DiffusionNFT, and on top of that its variance is larger. The added complexity of jointly learning φ alongside θ is not shown to buy a reliable improvement over the simpler baseline. (Even though it's slightly more flexible)
3. (Minor) The presentation of the paper is not clear:
Though the problem itself and the methods are interesting, the presentation of the paper is not very clear, and much of the setup is left under-specified, apparently assuming the reader already has DiffusionNFT’s formulation in hand — e.g. the problem formulation is not clearly stated.
Questions for authors
1. On the degenerate joint-training regime (relates to Weakness 2): Appendix A.4 (Eq. 35) shows that φ's gradient loses reward-dependence entirely whenever v_θ and v_old coincide. Given that the collection copy is refreshed via EMA (Table 9), how often does this degenerate regime actually occur during training?
2. On computational cost: The paper acknowledges that “matched scored-candidate budgets” do not imply “matched compute” (inversion and continuation add extra overhead). Could the authors provide actual computation cost comparisons between RACE/RACE+ and DiffusionNFT?
3. On statistical reliability of the reported gains: Across most tables, RACE’s improvement over DiffusionNFT appears quite small. Given that only 3 training seeds are used throughout, is it possible that these gains are not statistically distinguishable from noise?
5 comments
Response to Reviewer 2S4z (Part 2/2)
Thank you again for the detailed questions; we continue below with Q1-Q3 on gradient behavior, computation, and statistical reliability.
Q1. The four additional probes show a nonzero reward-dependent predictor-gradient component at the sampled updates. These probes use the detached-old-output graph at updates 1, 10, 100, and 600. The equal-field cancellation argument in Sec. 3.4 and Appendix A.4 assumes that both fields coincide as functions with identical input-gradient paths. Detaching the old output changes that path, so equal forward values alone do not establish reward independence. Moreover, reward independence would not imply a zero total gradient, since a residual-loss gradient may remain. Appendix C.2 specifies the implementation scope. These four probes do not measure the frequency of cancellation over the full training run.
Q2. The recorded MRI jobs show lower mean whole-job GPUh with RACE and RACE+, and the additional validation comparison shows higher scores at common GPUh budgets. The resource measurements cover three jobs per method, each with 600 updates and 4,800 scored training candidates:
Table S2. Recorded MRI resources.
| Method | A100 GPUh | Peak GB |
|---|---|---|
| DiffusionNFT | 5.899 ± 0.381 | 44.2 |
| RACE | 5.102 ± 0.165 | 44.5 |
| RACE+ | 4.871 ± 0.526 | 44.6 |
GPUh are measured whole-job allocated A100 time, including optimization (mean ± sample SD); memory is the jobstats-sampled peak. Appendix C.1 reports these measurements, and Appendix C.2 specifies their implementation scope.
Table S3. Additional MRI validation joint score versus reported cumulative A100 GPUh.
| Method | 1 h | 2 h | 3 h | 4 h | 5 h |
|---|---|---|---|---|---|
| DiffusionNFT | 0.745 | 0.762 | 0.775 | 0.784 | 0.790 |
| RACE | 0.758 | 0.785 | 0.805 | 0.819 | 0.827 |
| RACE+ | 0.762 | 0.792 | 0.813 | 0.827 | 0.835 |
Both variants attain higher validation scores at the displayed compute points. Table S3 is an additional validation diagnostic; Table S2 reports the recorded resource cohort. The validation scores are separate from the final held-out test scores.
Q3. The additional AID comparison shows gains over DiffusionNFT in all three seeds, with a reported paired-test result of for RACE versus DiffusionNFT. All methods below use the same 120-image test bank:
Table S4. Additional AID comparison; mean ± sample SD across three seeds.
| Method | AID score | Mean paired gain over NFT | Seeds above NFT |
|---|---|---|---|
| DiffusionNFT | 0.268 ± 0.005 | — | — |
| RACE | 0.279 ± 0.003 | +0.0110 | 3/3 |
| Reference + entry-wise | 0.302 ± 0.004 | +0.0337 | 3/3 |
The RACE-minus-DiffusionNFT differences are +0.011, +0.009, and +0.013; the reported test is a two-sided paired -test over the three seeds. Reference + entry-wise uses the matched-initialization configuration in Table S1 and also improves over DiffusionNFT in each seed. These additional AID results address seed variability in this setting. They do not establish statistical significance for every original benchmark, whose means and SDs remain descriptive. Appendix D.4 defines the aggregation units.
Dear Reviewer 2S4z, could you check if the responses from the authors have addressed your concerns?
Response to Reviewer 2S4z (Part 1/2)
Thank you for the careful assessment of our contribution and for raising questions about entry-wise construction, joint optimization, statistical reliability, and presentation. We have clarified the problem formulation and the relation to DiffusionNFT, and added control experiments and resource measurements to support the responses below; manuscript revisions are marked in blue. W1-W3 are addressed here; Q1-Q3 follow in Part 2.
W1. RACE changes candidate construction while retaining DiffusionNFT's reward-weighted loss form; the new controls test whether entry-wise structure adds value. DiffusionNFT collects and scores clean images before forward-process training. RACE adds children around selected intermediate states through a state-conditioned, entry-wise map; RACE+ additionally uses inverted training references within the same scoring budget. The shared-coefficient construction is a restricted ablation of our child map, not the entire DiffusionNFT baseline.
Table S1. Matched-initialization reference-aware AID controls; mean ± sample SD.
All three constructions initialize at γ0=1 and use the same 120-image test bank. The mean entry-wise-minus-fixed difference is approximately +0.017; entry-wise-minus-learned-shared is +0.0083, positive in every seed (for entry-wise versus learned shared: reported two-sided paired t-test, p=0.011). Sec. 4.6 presents this controlled comparison, which supports an additional benefit of entry-wise structure in the tested setting.
W2. Measured cost and matched-initialization controls make the benefit–complexity tradeoff explicit. The predictor has 19,024 parameters (approximately 0.10% of trainable LoRA parameters) and is unused at inference. Appendix C.1 reports GPUh and memory. Table S1 shows similar seed SDs in the three reference-aware arms. The fixed-context diagnostic additionally reports mean child-reward gains of +0.0051 over frozen initialization and +0.0063 over the tuned-fixed control (Sec. 4.6; Appendix C.4). These observations concern the measured configurations; they do not establish lower variance across every original benchmark. Part 2, Q1 clarifies the gradient-path condition; Q2 gives measured costs, and Q3 adds direct comparisons against DiffusionNFT.
W3. Sec. 3 now starts with a self-contained problem statement. The blue paragraph defines the pretrained generator, conditions, reward evaluator, scored-candidate budget, optional training references, and held-out reward objective. It states that test generation starts from fresh noise and uses the condition alone. Sec. 4.6 summarizes the new coefficient and candidate-reward controls; Appendices C.1-C.4 give the resources, gradient scope, inversion procedure, and checkpoint measurements.