- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Many existing image-conditioned image generation methods treat conditioning modality as isolated components, overlooking the relationships among different conditions and consequently causing training cost and memory consumption to scale linearly with the number of conditions. To address this problem, this paper proposes UNITY, which divides the training process into two stages: a Universal Stage to capture cross-modal joint representations and a Specialization Stage to refine modality-specific features. The paper also designs a Morphable Attention Flow Network for condition injection. Experiments involving four conditioning modalities are conducted on SD1.5 and SDXL, where UNITY achieves the best FID and CLIP Score results.
Critical strengths
- The motivation of this work is well justified. Joint learning across multiple conditions can avoid the repeated computational costs incurred by training a separate model for each condition.
- The method is evaluated on both SD1.5 and SDXL across multiple conditioning modalities. The consistent improvements demonstrate its generalizability across different diffusion backbones.
Critical weaknesses
- The presentation of the proposed method is somewhat confusing and difficult to follow. In particular, the terminology related to “MorphWrapper,” “morphological wrapping,” and “warping” is not used consistently, which further affects the clarity of the method description.
- The memory column in Table 1 appears to report the number of 24GB GPUs rather than the measured memory consumption. Aggregating device capacities is not a reliable measure of training memory. The paper should report the peak allocated memory for each method instead.
- FID and text-image CLIPScore do not directly measure whether the generated image adheres to the given spatial condition. Image-image CLIP similarity and SSIM may be reported as complementary metrics where appropriate.
- Tables 3 and 4 report ablation results only for Sketch and Segmentation. Since the main experiments consider four conditioning modalities, the ablation results for Canny and Depth should also be provided to demonstrate that the conclusions generalize across all evaluated conditions.
- and first appear in Tables 1 and 2, whereas their precise definitions are provided only in the appendix. These two settings should be introduced in the main text when they are first used.
- The equal division between the Universal Stage and the Specialization Stage appears to be heuristic. The paper would benefit from an ablation study using different stage ratios to justify the current design.
- The current ablation study does not clearly isolate the contributions of the Universal Stage, the Specialization Stage, and the MAF Network. A more direct ablation would be: (1) Universal Stage only with ControlNet, (2) Universal Stage and Specialization Stage with ControlNet, and (3) Universal Stage and Specialization Stage with the proposed MAF Network. Such a comparison, under the same total training budget, would more clearly demonstrate the individual contributions of the two-stage training strategy and the proposed MAF architecture.
Questions for authors
The training-cost analysis assumes a cost of for baselines supporting conditions. However, this assumption may not apply to unified methods such as CtrLoRA and Uni-ControlNet. A unified baseline can be trained jointly with a cost of approximately , whereas UNITY requires training steps. When , UNITY seems to incur a higher computational cost. Could the authors clarify the training costs of CtrLoRA, Uni-ControlNet, and the other baselines?