- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work introduce a multi-modal conditional control for layout-to-image generation. The conditions include a global prompt, per-instance images, local descriptions, and bounding boxes. The multimodal conditions are tokenized by MM-DiT backbone, avoiding the modality misalignment in the feature space. It also introduces a network for spatial control, which incorporates spatial embeddings into low-rank adaptation. To train the model, the authors generate 400K synthetic training images. And the method is evaluated on COCO-MIG and MS-Bench.
Critical strengths
The proposed method is a great unification of textual, spatial, and visual control. Integrating these conditions within an existing multimodal backbone is effective and practical.
Critical weaknesses
-
Missing failure cases and failure-pattern analysis. The paper presents successful examples without systematically examining limitations. Please include representative failures and quantify performance across challenging conditions, such as overlapping boxes, small objects, and conflicts between reference appearance and local descriptions.
-
Missing comparisons with current SOTA VLM baselines. Current Claude and GPT models can also take the specialized layouts for image generation. Please also compare with the these VLMs with same conditions and prompts. if a VLM such as Claude needs external adapter to achieve the same generation quality as your method, it deserves to be called out.
-
Unclear ability to follow long, detailed instance descriptions. The evaluation does not establish whether MoLaGen preserves fine-grained local instructions as description length and complexity increase, particularly when reference images already convey much of the requested appearance. Previous work Rethinking the Training and Evaluation of Rich-Context Layout-to-Image Generation (Cheng et al., NeurIPS 2024) already addressed the long description challenge and offered evaluation method. Please have a discussion or evaluation on this.
Questions for authors
-
How much does each conditional modality contribute to the final performance? The manuscript lacks the ablation study on the component effectiveness including spatial CondLoRA, modality-specific branches and instance isolation.
-
Can the model follow local text when it requests an attribute or pose change relative to the reference image?