- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The work focuses on conditional image generation and text-to-image generation. It proposes Semantic Self-Flow (SSF), which investigates whether deliberately introducing information asymmetry, rather than relying on random masking, can make selected semantic information a necessary condition for the recovery process. This is a well-motivated angle and offers useful insights for existing self-supervised representation learning.
Critical strengths
The explicit design of alternative-relation decoys, which forces the model to rely on relation words and reference locations to infer the subject position, is a concrete and feasible solution. The leakage analysis of spatial uncertainty also provides solid support for the experimental motivation.
Critical weaknesses
However, the claim that “the distribution is approximately independent of r” requires more rigorous measurement. In addition, only about 8.2% of ImageNet samples can be automatically annotated with spatial relations, which may significantly limit scalability to larger datasets.
Questions for authors
1 In Table 1, the sensitivity of the proposed method under SiT-XL/2 is only 10.0%. Such a low sensitivity suggests substantial limitations of the method, and the paper should discuss this more thoroughly.
2 The method requires relation annotations. How are these annotations obtained? What is their scale? Can this approach scale to large-scale databases?
3 As shown in Figure 4, Self-Flow does not exhibit flipping behavior. Why does Self-Flow ignore the relation condition, whereas the proposed method appears to utilize it? If Dq consists of object-region tokens cropped from other training images, how are these tokens sampled? Does the sampling depend on the relation word? How sensitive are the results to this design?
4 Figure 1 describes a mixed sampling strategy. How are the 8.2% relation-aware images selected? The paper should provide quantitative results on how this proportion affects image generation and parameter sensitivity.
5 The statement that “masked regions follow approximately the same distribution regardless of the true relation” needs clarification. How should “approximately” be understood or guaranteed? Could the proportion of relation-aware images or the sampling strategy affect this approximation?
6 The “look better” evaluation in Figure 3 appears subjective and lacks an objective metric. Moreover, why does Self-Flow achieve a higher “look better” preference rate?
2 comments
Thank you
Thank you for the positive and constructive review. We will add the analyses from our response to the final version.
We thank the reviewer for finding our angle well motivated and the decoys a concrete and feasible solution.
Approximate independence of the mask from the relation (W1, Q5)
We measure the approximation with a classifier that predicts the relation from the reference box and a mask, without the relation word, on 1,830 held-out pairs of COCO images.
The subject mask reveals the relation (93.2%), whereas the mask of SSF adds nothing over the box alone (60.2% vs. 62.6%). The proportion of relation-aware images changes how often this mask is used, not what it reveals, whereas random decoy locations break the construction and lower Both accuracy (33.7% vs. 42.1%).
Relation annotations and scalability (W2, Q2)
The annotations are automatic (Section 3.3.1 and Appendix A.2), and annotating all 1.28M ImageNet images takes about 33 hours on four B200 GPUs. For text-to-image, we use the SCOP pairs of COCO (Appendix A.5). The approach scales, since SSF needs annotations for only part of the images, the pipeline is linear in the number of images, and newer models or existing object annotations can replace its stages, e.g., Gemma 4 and Qwen3.8 confirm 88% and 86% of 1,000 sampled labels.
Why the sensitivity is only 10% (Q1)
The sensitivity (Appendix B.1) measures how much the model's prediction for the same noisy image changes when we flip only the relation in the prompt, e.g., from left of to right of. A value near 100% is not expected, since only the location of the subject should change, whereas its appearance, the reference, and the background should remain. Further, we find that even large changes yield small values, e.g., swapping the subject category, from bird above branch to cup above branch, changes the prediction by only 14.9% (Self-Flow 11.2%). Flipping the relation changes it by 10.0% for SSF vs. 1.1% for Self-Flow.
Why Self-Flow ignores the relation, and how the decoys are sampled (Q3)
With a random mask, most of the subject stays at the lower noise level and reveals its location, and hence the relation, so the denoiser reaches a low loss without the relation word. SSF hides the subject and the three alternative locations while keeping the reference clean, so the relation word is needed to locate the subject. The decoys come from a bank of subject regions of other training images, indexed by their relation and the cell of their reference. For each alternative relation, we draw 16 regions from the cell of the current reference and use the first one that does not overlap the reference, at its original position. Hence, the sampling depends on the relation word only through the choice of the alternatives. The design matters, since random decoys reach 33.7% Both accuracy and no decoys 27.6%, compared to 42.1% for SSF, and with a second training seed, SSF again outperforms both (43.6% vs. at most 40.9%).
Selection of the 8.2% and the effect of the proportion (Q4)
The 8.2% are all images retained by the annotation pipeline, without further selection. The table reports other proportions with SiT-B/2 at 200k steps.
Up to 50%, a larger proportion raises both the generation accuracy and the sensitivity, but oversampling to 50% increases the FID-50k, whereas at 8.2%, SSF keeps the FID of Self-Flow (11.84). With the same oversampling, Self-Flow reaches 48.2%, and SSF remains 6.3 points ahead, significant with 95% confidence. Training on the labelled images only (100%) reduces the data to 105k images, which lowers the image quality of both methods (FID-50k of 70.73 vs. 86.68 for Self-Flow) and their generation accuracy compared to 50%. SSF remains ahead (31.4% vs. 25.6%), significant with 95% confidence.
The look evaluation and why raters prefer Self-Flow (Q6)
In a new single-image study, 34 raters rate the look of each image from 1 to 5 and, as an objective criterion, state whether it shows its caption. They find a small look gap (2.53 vs. 2.66), whereas the images of SSF show the caption more often (61% vs. 46%). A head-to-head comparison can show a large preference gap even for a small difference in quality, and the preference for Self-Flow often arises when SSF draws an unusual layout that the prompt requests, e.g., keyboard above monitor.
We are happy to discuss any further questions or suggestions.