- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper addresses follow-up rectification of an existing generation: given a reference image and VLM feedback on its residual errors, how to fix those errors within a small extra budget while keeping what is already correct. RGDR asks a VLM (Doubao Seed-2.1 Pro) to write two short texts, a Target describing the desired state and a Reference describing the current erroneous state, and at every sampling step adds the difference of the frozen model's conditional velocities, V(c_target) − V(c_reference), rescaled to the norm of the base velocity and multiplied by a strength η and a fixed schedule s(t). Starting from the reference's initial noise, one complete image is generated for each η in a grid (ten for text-to-image models, five for the editing model), and the best of these candidates and the reference is selected by the VLM. Two theorems state that endpoints vary Lipschitz-continuously with η and that, if a first-order progress-minus-interference quantity is positive, net progress is positive for small η. Experiments on tasks drawn from PRISM-Bench and SciIR-Bench with SD3.5 Medium, SANA-1.5, FLUX.1-dev and Qwen-Image-Edit-2511 report Gains of +3.74 to +4.37 checklist points over Best@20 / Best@5 references, versus +1.24 to +3.41 for BoN, TIR and a constant-CFG sweep, plus ablations of the direction construction and the temporal schedule.
Critical strengths
-
A simple, training-free mechanism that applies uniformly to three text-to-image models and one editing model. RGDR needs only the frozen model's conditional velocity predictions and a fixed strength grid, keeps the reference's initial noise, and in Table 1 yields the highest Final, Gain, Improved and Repair on all four models with Preserve of 94–97%. The qualitative examples (Fig. 1) show localized changes, such as adding a bicycle while keeping the subject and scene, which is the behaviour one wants for follow-up correction.
-
The direction ablation targets the right question. Table 2 compares the Target–Reference difference with an operation-style instruction, a mismatched Target–Reference pair and a random Gaussian direction under the same relative reparameterization. The fact that mismatched feedback (overall Gain 1.15) and a random direction (1.31) do much worse than the correct Target–Reference direction (4.34) is the most direct evidence in the paper that the semantic content of the direction matters, although (see Weakness 1) these numbers also reveal how much gain comes from selection alone.
Critical weaknesses
1. The evaluation cannot separate genuine rectification from best-of-N selection under the evaluation judge, and its scale and uncertainty are unreported.
- Selector equals evaluator. Appendix C.5 (l. 839–840) states that Doubao Seed-2.1 Pro provides the feedback, scores the candidates, and that final outputs are selected under the same evaluation protocol; Algorithm 1 adds the reference to the candidate set before taking the argmax (l. 353, 360). Every reported Final/Gain is therefore the maximum of eleven candidates under the test-time judge, which is non-negative by construction and inflated by judge noise.
- The paper's own numbers show a large selection floor. A random Gaussian direction obtains an overall Gain of 1.31 and mismatched feedback 1.15 (Table 2), and BoN averages 1.56 across the four models (Table 1). RGDR's margin over the strongest baseline is only 0.82–1.59 points per model (SD3.5 +1.59 over Dynamic CFG; SANA +0.96, FLUX +1.37, Qwen-Image-Edit +0.82 over TIR).
- Unknown n, no uncertainty. The number of tasks per model is never stated. The Improved percentages for Qwen-Image-Edit (24.55, 30.91, 29.09, 33.64) are all multiples of 1/110, suggesting about 110 tasks for that branch. No seeds, standard deviations, confidence intervals or significance tests are reported, although the reproducibility statement (l. 499–501) says random seeds are disclosed.
- Test-set choice of the schedule. Appendix D.2 (l. 889–891) selects s(t) = j/(N−1) for SD3.5 and s(t) = 1 for SANA and FLUX by comparing Gains on the same evaluation set. With a single schedule for all models the picture changes: with j/(N−1), RGDR falls below TIR on SANA (+3.01 vs +3.41) and FLUX (+2.64 vs +2.99) (Table S2). Baselines received no comparable tuning.
- Compute is not matched. Budgets are matched in number of images only. Eq. 13 requires two additional conditional evaluations per step, roughly doubling the per-step cost for CFG models and tripling it for guidance-distilled FLUX.1-dev; no NFE or wall-clock numbers are given.
- Internal inconsistency. The direction ablation reports an overall Gain of 4.34 and Repair of 15.28% for TR Correct (Table 2), whereas the four-model averages of RGDR in Table 1 are 4.18 and 17.19%. Appendix C.4 (l. 834–835) indicates that the direction study comes from an "archived" experiment and that the Operation direction used a much smaller strength grid (0.0075–0.075) than the others (0.02–0.20), so the comparison between directions is not matched.
2. Novelty relative to existing semantic guidance and editing is not established, and the theory does not support the empirical claims.
- Closest prior methods are not compared. The correction field V(c_target) − V(c_reference) added to the base velocity is essentially SEGA-style semantic guidance (discussed in l. 162–164) or negative prompting with c_reference, and norm-relative rescaling of an added guidance term is close to existing guidance-rescaling methods. None of these is run with the same VLM-derived texts. For the text-to-image models, the most natural baseline for correcting an existing image — applying an editing model (e.g., Qwen-Image-Edit, which the paper already uses, or FLUX.1 Kontext) or an inversion-based flow editor to the reference with the same feedback — is also missing. The "Dynamic CFG" baseline is in fact a sweep over ten constant guidance scales with fixed noise (Appendix C.3, l. 800–805), not the online-feedback method of Papalampidi et al.
- The theorems are standard and disconnected from the experiments. Theorem 3.1 is the textbook Grönwall continuous-dependence bound; its constant depends on a Lipschitz constant containing 2C_V·L_D/ε (Appendix B.1, l. 689–694) that enters exponentially, so the bound is not informative for small ε. Theorem 3.2 is a second-order Taylor statement. The quantities Φ_q, e_fix and P_keep are never instantiated and κ_q(D_F) > 0 is never estimated. The paper itself notes that positive Δq does not imply a higher checklist score (l. 302–303) and that the Target–Reference construction does not guarantee the sufficient condition (l. 331–333).
- Presentation and reproducibility. The abstract and introduction are hard to parse in several places (e.g., l. 040–043, 080–081, 088, 097, 130). The introduction reports three-model averages (2.77/1.61/2.09, l. 135–136) while the abstract reports four-model ratios (1.51×/1.81×) without saying so. The reproducibility statement refers to data splits, training-time normalization, checkpoint selection and inversion settings, none of which apply to this training-free, inversion-free method; the VLM prompts promised by the title of Appendix C.4 are not given, and no code is provided.
Questions for authors
Clarifications that could change my assessment
- How many tasks are in the evaluation set for each model and each source (PRISM / SciIR), and how were they selected from the two benchmarks? Please report per-model mean ± std over several seeds and paired confidence intervals for RGDR versus the strongest baseline.
- Please re-score all final outputs (all methods) with an evaluator that was not used for feedback or selection (a different VLM and/or a human study), and report Final/Gain under that evaluator.
- Is the reference score b_i the score recorded when the Best@20 / Best@5 reference was selected, or a fresh re-evaluation? If the former, how large is the re-evaluation variance of the judge on the same image?
- Please add, with the same VLM-derived Target/Reference texts and the same image budget: (a) SEGA, (b) negative prompting with c_reference plus c_target appended to the prompt from the same noise, and (c) an editing model applied to the reference (for the text-to-image models).
- Please report NFEs and wall-clock time per additional image for each method.
- Why do the TR Correct numbers in Table 2 (Gain 4.34, Repair 15.28%) differ from the RGDR averages in Table 1 (4.18, 17.19%)? Can the direction ablation be re-run with the same strength grid for all directions?
Suggestions (not required for my assessment)
- Please clarify what "figure generation" refers to in the AI-use statement (l. 488–489); I assume it refers to plotting or layout rather than to the generated images shown as qualitative results.
- Report results with a single, pre-declared schedule for all models, and correct the reproducibility statement so that it describes what is actually released.
5 comments
Thank you for the detailed response and for releasing the evaluation data. I have read the rebuttal PDF (cited as "rb l.") and checked the released data/ folder.
Resolved or clarified. R3.1: releasing the 400 tasks with prompts, checklists and starting-image configurations clearly improves reproducibility. R3.3: the reference score is a fresh re-evaluation. R3.6: the Table 1/Table 2 difference comes from the 200-task ablation subset, which I confirmed is a subset of the main set with identical starting points. R3.5: Table B answers my cost question.
Remaining concerns
-
Independent evaluation (R3.2). The same Doubao model (Seed2.1 Pro) gives the feedback, selects the final output among the reference and the candidates, and computes the reported scores. The response changes only the feedback VLM; the evaluator is unchanged. With GPT-5.6 Sol feedback, the Target–Reference gain drops from 4.34 to 2.86 and its lead over the best alternative from 1.00 to 0.45 (Table C), close to the random-direction (1.31) and mismatched-feedback (1.15) gains in Table 2 of the paper. Moreover, since the reference is re-evaluated and the maximum over once-scored candidates is taken, judge noise alone yields a positive expected Gain. The judge's re-evaluation variance, which you can compute from the selection-time and re-evaluated reference scores, is still not reported.
-
Uncertainty (R3.1). I did not ask for resampled starting points. With them fixed, the methods remain stochastic (feedback, judging, BoN seeds), and paired bootstrap CIs over the 400 tasks can be computed from the per-task gains you already have. Given RGDR's margins of 0.82–1.59 points over the strongest baseline, these are necessary. Please also describe how the 400 tasks were selected (pool size, criterion, whether the judge was involved).
-
Editing branch. The data contains 400 Qwen-Image-Edit starting points, but the Improved percentages for this model in Table 1 (24.55, 30.91, 29.09, 33.64%) are multiples of 1/110 and are not attainable with 400 tasks, unlike those of the three text-to-image models. Which tasks underlie these results?
-
Ablation subset (R3.6). The 200 ablation tasks were prioritised for substantial RGDR gains in earlier main experiments (rb l. 154–156). Selecting tasks on the outcome of one compared direction biases the comparison in its favour, so Table 2 and rebuttal Tables A/C cannot by themselves support the Target–Reference principle. Please repeat the ablation on a random subset or on all 400 tasks. The strength grids also still differ across directions.
-
Baselines (R3.4). There is still no comparison with SEGA, negative prompting, or an editing model using the same VLM-derived texts and budget; R1 and R2 raised the same point. Since the original Dynamic CFG could not be reproduced (rb l. 058–066), please rename this baseline as a constant-CFG sweep.
-
Cost (R3.5). RGDR takes 49.92 s per image versus about 23.2 s for the other methods (about 2.15×), and its gain per minute (4.90) is below that of TIR (6.46) and Dynamic CFG (5.12). The comparison is matched in images, not in compute; an efficiency claim would need a wall-clock-matched comparison (e.g., BoN and TIR with about twice as many images). Do the times include the VLM calls? The 0.5–3% loss with the 75% window (Fig. C) is up to about 2 points, comparable to RGDR's margin.
-
Not addressed: the per-model temporal schedule chosen on the evaluation set (Appendix D.2); κ_q is never estimated, and a positive Δq does not imply a higher checklist score; the reproducibility statement; "figure generation" in the AI-use statement.
If the final outputs of all four methods could be re-scored by an independent judge (e.g., GPT-5.6 Sol or a small human study), together with paired CIs and a direction ablation on an unselected subset, I would be glad to reconsider. For now, I maintain my score.
Response to Reviewer DeAB’s Follow-up Comments — Part I
Thank you for carefully reading our response and the released materials, and for identifying the remaining points that need clarification. We have updated the manuscript; please refer to the revised paper PDF. Below, we respond to your points in order.
1–2. Independent evaluation and statistical uncertainty
The starting images were selected through manual review. We began with 1,500 tasks from PRISM-Bench and SciIR-Bench, used different VLMs to assist with initial screening, and excluded cases where the first output already met all requirements. We then used a limited number of generation or editing attempts to examine the remaining errors. Using the initial scores as a guide, we manually selected images that met most requirements but still contained clear errors. Tasks where an occasional candidate received a perfect score could also be retained after manual review confirmed remaining errors. We selected 400 tasks and used Seed-2.1 Pro to score their starting images under a common protocol, obtaining the Base scores in Table 1. Regarding the coverage of improvement, Improved in Table 1 already reports the percentage of tasks on which each method increased the score. Given the time available for this discussion and the budget required for additional evaluation, we will not add independent rescoring in this round.
3. Image-editing branch
The Qwen branch was constructed after the 400 main evaluation tasks had been selected. It starts from the corresponding 400 SD3.5 images, generates up to five Qwen editing candidates per task, and selects one. It therefore does not involve independently selecting another subset from the original pool of 1,500 tasks. At the first formal evaluation, 70 selected images already met all checklist requirements, so the system skipped further rectification for those tasks. The remaining 330 tasks entered the subsequent rectification experiments. The released data contains the complete set of 400 starting-point configurations, whereas the Qwen results in Table 1 are calculated over the 330 tasks that entered further rectification. This explains the difference between the number of released configurations and the number of tasks included in the table.
4. Ablation subset
The direction ablation examines the semantic-description criterion in Section 3.2 and compares several concrete constructions. We selected 200 informative tasks from the main experiments, prioritizing cases where RGDR had corrected some errors but clear residual errors remained. We did not select tasks according to how much Target–Reference outperformed the other constructions. If all errors have already been corrected, the score has reached its ceiling; if there is no correction response at all, it is also difficult to see what changing the construction does. Cases with partial correction allow us to start from the same image and observe whether another construction corrects more, a similar amount, or less. If another construction performs better, this will also appear in the results. The strength ranges follow the explanation in R3.6: in the flamingo example, the same strength is suitable for Target–Reference but already too strong for Global Operation. Different constructions tolerate different intervention strengths, so we search within an appropriate range for each construction while keeping the candidate count the same. Given the time available for this discussion, we will not rerun the ablation on a random subset or on all 400 tasks in this round.
Our responses to points 5–7 continue in the next comment, titled “Response to Reviewer DeAB’s Follow-up Comments — Part II”.
Dear Reviewer DeAB,
Thank you for your time and constructive feedback. We have included our first responses, together with supporting tables and figures, in
ICLR_Rebuttal.pdfincluded in the supplementary material.We group related comments by topic and label each response with the corresponding reviewer and comment IDs. For consistency, R3 would be a reference to DeAB. Besides,
Rk.jdenotes the reviewer and the j-th comment, whileRk.Wjcorrespondingly denotes the j-th weakness raised by reviewers. Please refer to the sections marked with the comment IDs.We welcome further questions and discussions regarding our responses. We will upload a revised manuscript incorporating the updates before the rebuttal deadline. Thank you again for helping us improve our work.