- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to realize structured requests through frozen diffusion generators, changing selected output properties while preserving others. The authors analyze the Jacobian of the requested properties through the complete remaining sampling process and show that first-order guidance can distort the relative components of a request. The method uses damped least-squares correction and repeatedly recomputes the Jacobian and residual at a fixed diffusion time to handle finite edits. The paper also introduces a request-specific score that predicts improvement in cosine alignment over first-order guidance. Experiments cover images, audio, robotics, character motion, and planning.
Critical strengths
-
The problem is meaningful, and the mechanistic analysis is supported by carefully designed controls. Reducing the overall objective does not necessarily mean accurately realizing multiple requested changes. The paper usefully distinguishes these outcomes. The scalar equivalence test and the experiments matching latent displacement or predicted output movement help separate the benefits of update direction from those of intervention magnitude.
-
Some experiments demonstrate substantial downstream benefits, and the limitations are discussed openly. In the paired inpainting experiment, four corrections improve missing-pixel PSNR over Adam by 3.25 dB at equal total derivative traversals. The paper also distinguishes request alignment, task outcomes, and executed behavior, and reports failures of local approximation, cases where generic optimizers perform better, and attenuation of improvements during execution. These analyses help clarify the scope of the method.
Critical weaknesses
-
The significance of the contribution beyond prior work remains unclear. The core correction builds on classical damped least squares and is closely related to existing inverse-correction and constraint-projection methods [1–3]. While the request-specific analysis is useful, the paper needs to explain more clearly what substantial new insight or capability it adds.
-
The practical value of the prediction score is not sufficiently established. The score predicts directional improvement within a local regime, but it remains unclear whether it can reliably guide steering decisions or anticipate better task outcomes. This limits the significance of one of the paper's central contributions.
-
The practical advantage over strong optimizers needs clearer characterization. The experiments show useful gains on some tasks, while generic optimizers remain competitive on others. Given the additional computation required for geometric correction, the evidence does not yet clearly establish when this approach is preferable in practice.
References
[1] Weiguo Gao, Ming Li, and Qianxiao Li. Terminally constrained flow-based generative models from an optimal control perspective. arXiv:2601.09474, 2026.
[2] Utkarsh et al. Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints. NeurIPS, 2025. arXiv:2506.04171.
[3] Alexandros Graikos, Nebojsa Jojic, and Dimitris Samaras. Fast constrained sampling in pre-trained diffusion models. NeurIPS, 2025. arXiv:2410.18804.
Questions for authors
-
How does the core update differ mathematically, and in its assumptions, from GNSolver in the TOCFlow paper [1]? Which differences concern damping, update timing, or implementation, and which constitute an independent methodological or analytical contribution? After accounting for PCFM and Fast Constrained Sampling, how would the authors characterize the main contribution of this work?
-
Can the reliability of the prediction score be assessed without using post-edit outcomes? How does it perform across the complete diagnostic collection? Is there evidence that it helps select methods or correction depths and thereby improves final target error or task success?
-
On a few representative tasks, could the authors compare spectral correction with well-tuned Adam and L-BFGS using consistent tuning and stopping rules, reporting end-to-end runtime and memory consumption alongside directional alignment, target error, and task success? This would help clarify the method's practical scope.
Response to Reviewer uk6b
Thank you for recognizing the value of the work, the questions, and for the references. We clarify the solver relationship and test the predictor on fresh seeds.
Q1. Relation to GNSolver and constrained sampling. With M=JhDΦ, TOCFlow's GNSolver returns M⊤(I+sMM⊤)−1h. Up to sign convention and an outer scalar factor, this has the same damped Gauss–Newton form as our J⊤(JJ⊤+λI)−1d. We acknowledge this algebraic overlap. Relative damping, fixed-time relinearization and differentiation through the implemented remaining sampler are design choices, not a new Gauss–Newton update. TOCFlow, Eq. 148 and Algorithm 3.
PCFM projects using the constraint Jacobian at a predicted endpoint and maps back by interpolation. Fast constrained sampling develops an approximate Newton construction through the one-step denoiser. PCFM, Fast constrained sampling.
The independent contribution is request-specific prediction and controlled evaluation of directional benefit. Request coefficients in the Jacobian's singular basis determine distortion; conditioning alone is insufficient. This yields a scalar null, movement-matched tests and predictions evaluated against nonlinear edits. Holding solver and actual residual fixed, full-sampler differentiation improves K1 cosine on 75–95% of paired vision instances across tested windows. At eight remaining steps, predictor–gain correlation is 0.94 versus 0.68 for the one-step map on inpainting, and 0.80 versus 0.62 on SR. The one-step surrogate nevertheless reconstructs inpainting better and differentiates fewer sampler transitions. This isolates the map's effect without claiming superiority over the complete PCFM or fast-constrained-sampling algorithms.
Q2. Reliability without observing the evaluated outcome. We froze two robotics tasks, calibration seeds 20000–20009, evaluation seeds 20100–20129 and threshold G=0.05 before evaluation. All scales of a seed remain in one split. With no regime filter, the fresh-cohort result is:
Intervals bootstrap seeds. Five Panda requests were rejected during task construction. The outcome is K1-minus-FO cosine gain; FO here uses its one-iteration damped least-squares step, not movement matching. Both tasks meet the preregistered correlation ≥0.5 prediction.
The original unfiltered collection gives rho=0.626 over 10,362 instances versus −0.061 for log condition number; after removing population means, 0.599 versus −0.017. The earlier pilot split requires pilot-edit outcomes.
Decisions and overhead. We executed the router, charging its score computation on both branches and reusing the Krylov prefix. It saves only 0–6% of correction-to-trajectory CPU time, below the preregistered 15%; FO routing also fails the request-error noninferiority criterion. With only five or six request coordinates, four predictor iterations cost most of the full solve. Thus directional ranking transfers to these fresh instances, but we claim no routing speed-up. The predictor also does not establish a final-success or optimizer-selection rule. A separate residual-based discrepancy stop gives mean spectral PSNR within 0.02 dB of cross-fitted selection on all four vision operators.
Q3. Tuned optimizers and computational cost. Archived runs use common cross-fitted budgets and disjoint learning-rate tuning; grids and compute differ.
Deblurring reports means only, without paired inference; sample count and paired outputs are unavailable.
The paired inpainting gain is +1.77 dB [0.95, 2.67]; cross-fitted bandwidth LSD is 12.874 versus Adam’s 12.525 (paired Wilcoxon p=.15). New profiles cover all ten configurations on a RTX 5090. Medians use three images/task, one run each, including initialization and final sampling. K4 takes 85.3 s versus Adam@64's 49.3 s and DPS's approximately 62 s. With m=8, K4 takes 43.4 s; the separate 30-image quality ablation changes its PSNR by −0.03/+0.08 dB on inpainting/SR. The paired m=16 margin was not re-measured at m=8. Peak spectral allocation is 2.54–2.59 GiB versus Adam's 2.51, read before metrics. Small timing samples do not establish quality-matched frontiers. Full GPU/CPU results are in the appendix.
We have uploaded a revised manuscript incorporating these clarifications and additional experiments. We welcome further questions and would be happy to discuss any remaining concerns.