- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces RINFD, a non-adversarial framework for adding real-data supervision to few-step text-to-image distillation. First, Teacher-to-Data correction (T2D) constructs displacement targets from teacher reconstructions of real images, trains a correction adapter on noisy points sampled along bridges between the reconstruction and the real latent, and injects the predicted correction into student updates. Second, FD-SWD matches multi-scale spatial and frequency-sensitive patch statistics between student clean predictions and real latents using sliced Wasserstein distance. Combined with DMD and ISG, RINFD obtains the best reported average rank on SDXL and FLUX.1-dev and the second-best on SD3.5 Large among the evaluated four-step systems.
Critical strengths
- The paper asks a relevant question: how to use real images in DMD-style distillation without relying on a discriminator. The separation between prediction-level correction and distribution-level statistical matching is intuitive.
- The SDXL ablations are more informative than a simple full-versus-base comparison. In particular, the paper examines plain versus frequency-sensitive SWD, the two components individually, several loss weights, and bridge sampling versus reconstruction-only queries.
Critical weaknesses
- Although RINFD introduces no inference-time overhead, training requires an online generator-field estimator, a T2D correction adapter, and FD-SWD, with alternating generator, field-regression, and real-pair correction updates. T2D further requires teacher reconstructions, bridge queries, and additional frozen-network evaluations. The resulting wall-clock time, memory footprint, and number of teacher calls may therefore be substantially higher than those of the DMD/ISG baseline. The paper reports the hardware and batch size but does not provide training time, peak memory, or a compute-matched comparison with DMD, SenseFlow, or SwD, making the practical cost difficult to assess.
- The main component ablations are conducted only on SDXL and are reported as single-run point estimates. Relative to FD-SWD alone, the full method improves PickScore, ImageReward, and CFM, but decreases HPSv2 and GenEval. Thus, the current results may reflect a shift to a different metric operating point rather than a consistent synergistic improvement.
- In the first example of Figure 7, the prompt asks for “a dog with a disc on the beach.” Compared with SwD, the RINFD outputs on both SD3.5 Large and FLUX.1-dev place the disc in visually implausible positions. Its relation to the dog’s mouth or body is unclear, and the FLUX result resembles a ring attached around the dog’s neck rather than a disc being carried or placed naturally in the scene. This example suggests that RINFD may improve aggregate preference metrics while still producing locally plausible but spatially inconsistent objects.
Questions for authors
see weaknesses
4 comments
Further Discussion with the Reviewer
Dear Reviewer WJ1g,
Thank you for your time and constructive feedback. We have carefully considered your comments and provided detailed responses in our previous official comment. We summarize the main points below:
-
We quantified the training cost, reporting approximately 30% more GPU-hours than Base and 32% fewer than SenseFlow. We also clarified that the additional computation is confined to training, with no additional inference overhead..
-
We provided a new component-ablation results on FLUX.1-dev, showing additional preference gains from combining T2D with FD-SWD, with GenEval essentially maintained. We also acknowledged the SDXL trade-offs and will narrow our claims to reflect these backbone-dependent results.
-
We discussed the spatial inconsistency in Figure 7, acknowledging that RINFD does not explicitly constrain object relationships. We will clarify this limitation and distinguish improvements in aggregate preference metrics from relational correctness in individual outputs.
We hope these clarifications and new results help address your concerns. We would appreciate hearing whether any points remain unresolved and would be happy to provide further clarification.
Thank you again for your thoughtful feedback and for helping us improve our work.
Best regards,
The Authors
Thank you for the authors’ response. Nevertheless, my concern has not been fully addressed. As also noted by other reviewers, T2D itself does not appear to provide a sufficiently substantial improvement. Although the newly added FLUX experiments show that T2D can improve HPSv2 and PickScore, the gains on these metrics appear smaller than those achieved by FD-SWD in Table 2. Moreover, given the motivation behind the Teacher-Constructed Correction loss, I would expect its introduction to improve the perceptual realism of the generated images. However, based on the qualitative examples presented in the paper, I do not observe a clear improvement in realism compared with SwD-FLUX. Therefore, I am inclined to maintain my current score.
Dear Reviewer WJ1g,
Thank you for your follow-up and for clarifying your remaining concerns. Your comments helped us recognize that our previous response focused too narrowly on metric improvements and did not sufficiently clarify the central contribution of our work.
1. Our main contribution: non-adversarial real-data supervision for few-step distillation
Our work investigates how real-data supervision can be incorporated into few-step distillation without introducing an adversarial discriminator. We implement this through two complementary forms of supervision: FD-SWD matches local statistics, while T2D learns a teacher-to-data correction. Within this framework, T2D provides a learned correction toward real-data targets beyond statistical matching. Our contribution is this approach to incorporating real-data information, rather than a claim that T2D produces a large improvement on all metric.
2. What the additional ablation establishes
We agree that FD-SWD contributes more to the FLUX PickScore and ImageReward gains. Nevertheless, T2D provides further improvements when added to FD-SWD: HPSv2 increases from 0.2907 to 0.2939, PickScore from 21.87 to 21.93, and ImageReward from 0.9562 to 0.9935, while GenEval is essentially maintained and CFM remains unchanged. For HPSv2, the additional gain from T2D (+0.0032) is larger than the gain from adding FD-SWD to Base (+0.0017). These results support a complementary contribution, although its magnitude varies across metrics and backbones. We will make this scope explicit in the revised manuscript.
3. The expectation of improved perceptual realism
We understand your expectation that real-data correction should improve perceptual realism. However, learning a conditional-mean correction does not directly optimize human judgments of realism, and preference-score improvements alone cannot establish such an improvement. We acknowledge that the presented qualitative examples do not clearly demonstrate a realism advantage over SwD-FLUX. Accordingly, we will distinguish our methodological contribution—incorporating real-data supervision without adversarial training—from the claim of visibly superior realism. We will clarify the observed benefits, trade-offs, and limitations in the revised manuscript.
Thank you again for helping us clarify both the contribution and the evidence supporting it.
Best regards,
The Authors
We thank the reviewer for recognizing the relevance of non-adversarial real-data supervision and the informative SDXL ablations. We address the concerns about computational cost, metric trade-offs, and spatial consistency below.
1、Training cost and comparison fairness
Having no additional inference-time modules does not imply negligible training overhead. The generator-field estimator is already part of the DMD/ISG baseline, while T2D and FD-SWD introduce additional training computation. As shown in the Table above, our training-cost comparison shows that Base (DMD + ISG) requires approximately 45–55 GPU-hours, SenseFlow requires 90–100 GPU-hours, and full RINFD requires 60–70 GPU-hours. Using the midpoints of these ranges, RINFD requires approximately 30% more training GPU-hours than Base and 32% fewer than SenseFlow. We consider the additional training cost over Base worthwhile for performance development. As shown in Table 2 in our manuscript, RINFD improves ImageReward from 0.4273 to 0.6517 (+52.5%), PickScore from 21.27 to 21.77, and HPSv2 from 0.2800 to 0.2916. CLIP and CFM also increase from 0.3297 to 0.3336 and from 13.01 to 13.15, respectively. The additional training computation yields gains across five of the six reported metrics, with the largest relative improvement in ImageReward. This cost is incurred during training, while the deployed student retains the same four-NFE inference procedure without additional inference overhead.
2、Additional component ablations on FLUX.1 dev
We agree that the SDXL results do not establish a consistent improvement. Relative to FD-SWD alone, the full method improves PickScore, ImageReward, and CFM but reduces HPSv2 and GenEval. Our intended claim is that T2D can provide additional preference gains when combined with FD-SWD, with trade-offs that depend on the backbone. To examine whether this behavior extends beyond SDXL, we have now conducted a component ablation on FLUX.1-dev (see our response to reviewer cAUz). Adding T2D improves HPSv2 from 0.2907 to 0.2939, PickScore from 21.87 to 21.93, and ImageReward from 0.9562 to 0.9935. GenEval increases from 0.7010 to 0.7024, while CFM remains unchanged. Thus, the FLUX estimates show additional preference gains without the HPSv2 or GenEval decreases observed on SDXL. This additional ablation provides evidence of complementary benefits on another backbone. We will narrow our claim in the revised manuscript accordingly and explicitly acknowledge both the SDXL trade-offs.
3、Spatial inconsistency in Figure 7
The first example in Figure 7 reveals a failure in object relationships. The dog-and-disc example highlights an ambiguity in the spatial relationship between the objects. Similar ambiguity is also visible in the SwD and SenseFlow outputs on SD3.5 Large, where the disc appears to overlap the dog’s mouth–neck region. This issue is not unique to RINFD. RINFD does not include an explicit objective for aligning or constraining spatial relationships between objects and therefore cannot guarantee relational correctness in individual outputs. Our reported improvements in aggregate preference metrics should be interpreted within this scope: they do not imply uniformly improved spatial consistency or object interactions. We will explicitly discuss this limitation in the revised manuscript, using this example to distinguish visual appearance quality from relational correctness.