- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper shows that image-to-text feedback in mmdits can reinforce generation errors, and proposes a cfg-like guidance method to suppress this feedback and strengthen text conditioning. The method is training-free, improves alignment across several t2i and t2v backbones.
Critical strengths
- The paper provides a clear mechanism of how I2T feedback in mmdit can preserve generation errors. The method is well motivated.
- SFG requires no model training, which makes the method easy to integrate into existing mmdit pipelines.
- The gains across multiple settings are relatively strong, and the generalization experiment on t2b is great
Critical weaknesses
- Limitations of older text encoders. The evaluation is limited to earlier mmdits such as sd3/3.5 and flux.1. Testing newer backbones such as flux.2 klein 9b would strengthen the generality claim, especially since modern llm-based text encoders like qwen3 in flux.2 klein may induce different text–image interaction patterns than older T5.
- The paper mainly evaluates text-to-image generation, while newer models such as flux.1 kontext, flux.2 klein unify generation and editing. It would be valuable to study whether the identified image-text interactions play a similar role in image-conditioned generation.
- Since SFG directly intervenes in internal attention pathways at inference, it would be useful to evaluate whether it introduces visual artifacts. Therefore, a dedicated artifact evaluation using a strong multimodal judge (e.g., Gemini 3.1 Pro) with artifact-focused prompts/criteria, and ideally a human preference study is welcome.
Questions for authors
- It seems different models favor different guidance strengths. Have you tried making the guidance strength learnable, or just distilling the effect of SFG into the model, for example via LoRA? This could substantially improve the practical usability of the method.
- How localized is the harmful i2t feedback? SFG currently scales the i2t message in all joint-attention blocks. Is the effect concentrated in particular attention heads / frequency areas?
4 comments
W3. Does SFG introduce visible artifacts?
Thank you for suggesting that we score artifacts directly to evaluate our method more fully. We used 500 prompts and generated a pair of images per prompt on FLUX.1. Within each pair, we used the same random seed to generate one image with CFG and the other with CFG + SFG. We shuffled the 1,000 images, hid the method labels, and used Claude Sonnet 5.5 to rate structural and rendering artifacts. We counted artifacts rated moderate or worse as visible.
| Blinded evaluation of 500 image pairs | CFG | CFG + SFG |
|---|---|---|
| Images with visible artifacts | 74 / 500 | 37 / 500 |
| Visible artifact rate | 14.8% | 7.4% |
Adding SFG halved the visible artifact rate in this evaluation. In 64 pairs, only the CFG image had visible artifacts. In 27 pairs, only the CFG + SFG image had visible artifacts. In another 10 pairs, both images had visible artifacts. These results show that SFG clearly reduced visible artifacts compared with the CFG baseline.
Q1. Can a LoRA adapter learn the SFG correction?
We agree with your suggestion. We ran the distillation experiment you suggested. On SD3-Medium, we trained a rank-16 LoRA adapter to match the SFG teacher’s guided velocity prediction in a single model evaluation. The adapter covers the attention projections in all joint-attention blocks. We trained it for 6,000 steps on COCO image–text pairs with a batch size of 16. The teacher controlled only I2T feedback (SFG-X), with , , and a guidance scale of .
| SD3-Medium, 40 sampling steps, no CFG | Model evaluations per step | GenEval |
|---|---|---|
| Base model | 1 | 32.95 |
| SFG at inference (teacher) | 2 | 56.23 |
| SFG distilled into LoRA | 1 | 57.41 |
The LoRA adapter achieved a better result than training-free guidance. It requires only one model evaluation per sampling step, with no auxiliary branch or guidance scale to tune at inference.
Q2. How localized is the harmful I2T feedback?
Thank you for this insightful suggestion. It helped us identify which part of the I2T feedback SFG acts on.
| Source of the I2T perturbation | GenEval2 AM | GenEval2 GM |
|---|---|---|
| Full SFG | 83.67 | 45.24 |
| Low-frequency component of the values only | 80.10 | 41.54 |
These results show that attenuating just the low-frequency component of image feedback can already give good results. This may suggest that the errors already present in the image are global and semantic, with high-frequency details playing little role. Suppressing this feedback may therefore prevent much of the harm these errors cause to the semantic information in the text stream.
Thank you again for the constructive feedback. We will incorporate these revisions into the final manuscript. We are glad to discuss and address any remaining question, and if you find your concerns resolved, we would appreciate if you could reflect this in your evaluation.
Thank you for your active response. I would like to raise my rating to 3.
Dear Reviewer bc8T,
Thank you for your follow-up and for your willingness to raise your rating. We will incorporate your suggestions into the final manuscript. Please let us know if you have any further suggestions before the discussion closes.
Thank you again for your time and effort.
Best regards,
The Authors of Submission 2377
Response to Reviewer bc8T
Thank you for reading our paper carefully. We appreciate your comments on the experimental evidence for image-to-text (I2T) feedback and on how easily SFG can be integrated into existing MMDiT pipelines. Your questions prompted us to test SFG on a newer backbone, apply it to image editing, and directly evaluate visible artifacts. Following your suggestion, we also tried distilling SFG into a LoRA adapter.
W1. Does SFG transfer to a backbone with an LLM-based text encoder?
We agree that changing the text encoder may change how the image and text streams interact. Thank you for suggesting that we test FLUX.2 [klein] Base 9B, which uses a Qwen3 text encoder. We ran two 25-step text-to-image comparisons on this model, each with its corresponding classifier-free guidance (CFG) baseline. AM and GM denote the arithmetic and geometric means on GenEval2, respectively.
On FLUX.2, adding SFG to CFG improved all four scores. GenEval2 AM rose by 7.25 points and GM by 12.25 points. For comparison, with SFG, the 9B model even came close to or exceeded the scores of the larger FLUX.2-dev (32B) from the same family.
In the paper, we found that the SFG correction direction was nearly orthogonal to the CFG correction direction. The FLUX.2 results show that SFG can further improve scores when combined with CFG and can also replace CFG when the inference budget is fixed. With two model evaluations per sampling step, SFG alone scored higher than CFG on all four metrics. Thank you for this insightful suggestion, which helped us see more of what SFG can do.
W2. Does SFG help with image editing?
Thank you for suggesting image editing as a test of whether this feedback mechanism generalizes to image-conditioned generation. We evaluated FLUX.2 [klein] Base 9B on all 606 editing instructions in GEdit-Bench-EN, using Qwen2.5-VL-72B as the judge under the VIEScore protocol.
With SFG, the model followed the editing instructions more closely. The overall score rose by 0.293 points.