- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper frames multi-reward DiffusionNFT as the problem of selecting a single scalar optimality target. The authors use the reward covariance to predict the reward gains of a candidate target, constrain the velocity-field change the target requires with a residual-weighted Gram matrix, and solve a per-group Nash bargaining problem to choose the combination coefficients. On two four-reward suites with SD3.5-Medium, the method achieves the best worst-channel normalized gain.
Critical strengths
S1: The covariance gain map directly explains why fixed weights fail: nonnegative weights can still lower a reward through negative correlations between rewards. The raw-sum result in Table 1 (worst gain −0.34) is consistent with this analysis.
S2: On the compressibility suite, raw sum, group-z sum, reward-only bargaining, and MARBLE all collapse to black images, while FCB-NFT improves all four rewards. Table 8 further shows that even with the controller off, the residual-weighted matrix avoids collapse while the scale-matched covariance does not. This is an interesting finding.
S3: The scale-matched covariance and shuffled-q controls are well designed, separating the effect of the matrix shape from its scale and from finite-sample noise.
Critical weaknesses
W1: The motivation for the field Gram matrix M does not match the estimator M ̂ used in practice. The paper motivates M by its ability to capture whether reward-induced field responses reinforce, oppose, or are orthogonal to each other. However, M ̂ (Eq. 19) only weights reward outer products by a scalar residual energy and contains no directional information; it is essentially a residual-weighted covariance. In Table 4, the deviation of M ̂ from the scale-matched covariance is only 1.5 to 1.8 times the noise floor. Moreover, the field budget \Delta is calibrated with M̂ itself (Eq. 56), so the "conservative estimate" carries limited meaning.
W2: The feedback controller in Appendix C has a large effect on the results: in Table 8, turning it off drops the worst gain from 0.43 to 0.29. Yet the main text mentions it in only one sentence. Apart from reward-only bargaining, no baseline has a comparable mechanism, and all ablations on the OCR suite keep the controller on. This makes it difficult to attribute the improvement to the field constraint.
W3: In Table 3, M ̂ differs from the scale-matched covariance by only about 20%, yet the worst gain rises from 0.60 to 0.89, and the paper does not explain this. The mechanism proposed in Appendix D for the compressibility suite (that residuals from low-frequency changes correlate with the compressibility reward) is also not verified with data.
W4: Balance-oriented baselines are missing, such as Tchebycheff / max-min aggregation, geometric-mean scalarization, and a weighted sum with a variance trust region. In addition, the evaluation uses only the four reward models used in training and covers a single base model and prompt distribution, with no held-out metrics or human evaluation.
Questions for authors
Q1: Since M ̂(Eq. 19) is a scalar residual-weighted covariance with no directional information, can you state precisely which quantity in your method actually captures the "reinforce / oppose / orthogonal" structure that motivates the field Gram matrix? If none does, what is the remaining justification for calling M ̂a field Gram matrix?
Q2: Given that Δ is calibrated with (M ) ̂itself (Eq. 56), what independent evidence shows the resulting budget is conservative rather than self-consistent by construction? Please give a calibration check that does not use M ̂.
Q3: Table 4 reports M ̂'s deviation from scale-matched covariance at only 1.5–1.8× the noise floor. Does this imply M ̂is statistically distinguishable from a plain covariance? If so, what test establishes that, and if not, how should the reader interpret the estimator?
Q4: Table 8 shows worst gain drops from 0.43 to 0.29 when the controller is off. Can you report the field-constraint ablation with the controller off, so the improvement can be attributed to the field constraint rather than the controller?
Q5: In Table 3, a ~20% difference between M ̂and scale-matched covariance coincides with worst gain rising from 0.60 to 0.89. What mechanism explains this magnitude of gain from such a small estimator difference? Can you provide an ablation or diagnostic that isolates the cause?
Q6: For the compressibility suite (Appendix D), can you verify with data that residuals from low-frequency changes correlate with the compressibility reward, as the proposed mechanism claims? Without this, what is the basis for the mechanism?
Q7: Can you add the balance-oriented baselines you omitted—Tchebycheff / max-min aggregation, geometric-mean scalarization, and weighted sum with a variance trust region—so the field constraint can be compared against established balancing methods?
Q8: The evaluation uses only the four training reward models, one base model, and one prompt distribution, with no held-out metrics or human evaluation. Can you report held-out reward models and at least one additional base model / prompt distribution to establish generalization?
2 comments
I thank the authors for the constructive rebuttal. The added evidences to Q3–Q8 are helpful. However, the core soundness concern—the mismatch between the field Gram matrix motivation and the trained residual-weighted covariance M̂, which lacks directional information and is clarified rather than resolved. The controller attribution (Q4) also remains incomplete. I keep my scores unchanged.
Response to Reviewer gLVe
We thank the reviewer for the detailed questions.
Q1–Q2. Directional structure and conservativeness.
The directional interpretation belongs to the exact field Gram matrix $ M $ in Section 4.2. Section 4.3 calls $ \hat M $ an empirical field metric: it estimates the residual-weighted population upper bound and captures reward–residual-energy associations. We will make this distinction explicit in the abstract and conclusion: the estimator targets the population upper bound, and the training ablations use M^. General Response 3, G8 shows a controlled case where the bound preserves an interaction sign missed by covariance and selects a target closer to the exact-M solution. The inequality $ M\preceq\bar M $ follows from conditional Jensen under the stated assumption, before budget calibration. The toy checks population costs independently of the chosen budget. The upper-bound guarantee applies at the population level. The training metric estimates this bound from samples.
Q3–Q5. Noise, attribution, and the size of the reward gap.
Table 4 describes the matrix deviations relative to the shuffled-q noise floor. Training evidence for the metric’s contribution comes from General Response 1, G1: full metric versus covariance gives OCR Worst 0.85 versus 0.58 without feedback, and 0.89 versus 0.60 with feedback. The submitted shuffled-q control gives 0.61. The field budget binds in 100% of groups (Table 3), and Table 4 locates the additional structure primarily in off-diagonal interactions. Changing these terms can alter the preferred coefficient direction. G8 illustrates target-direction sensitivity, while General Response 1, G1 isolates the training benefit.
Q6. Image-structure diagnostics.
On compressibility rollouts at iterations 100/300/1,000, mean within-group correlations between residual energy q and image high-frequency energy are 0.02/0.17/0.28. At iteration 1,000, high-frequency energy correlates with compressibility at −0.55, and q with compressibility at −0.31. Pooled q–high-frequency correlation is 0.33. These measurements show an association between residual energy and image structure relevant to compressibility that strengthens later in training. We will revise Appendix D to describe this measured association and the metric-dependent controller effects in G6.
Q7. Balancing alternatives.
General Response 2, G3 reports percentile-min, percentile geometric mean, and weighted-sum/variance-budget baselines, compared with FCB-off. FCB-off has higher Worst than all three added alternatives on OCR and both completed alternatives on compressibility. G3 specifies the sample-level reward transformations and reports the percentile-min run's early degeneration and numerical failure.
Q8. Held-out evaluation.
General Response 1, G2 adds two evaluators not used for training and a second evaluation prompt distribution. FCB-NFT improves both external scores over GroupZ on both distributions.