- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Transformation, Plan then Place (TPP), a framework for sequential image generation that aims to preserve subject identity while allowing state changes and natural interactions with surrounding scenes. Its main contribution, CoPlace, jointly predicts subject placement and generates an empty background through iterative communication between a layout branch and an image generation branch. It also introduces a space-gating mechanism to prevent unwanted content in the reserved subject region. The framework uses Concept Sliders for subject transformation and AnyDoor for final placement. Experiments on three datasets report improvements in image quality, subject consistency, and subject-scene compatibility, alongside qualitative examples of multi-step procedural generation.
Critical strengths
-
Coupling the layout branch (LB) with the image generator is an interesting idea. Where the subject should go plausibly depends on the scene, and the scene on where the subject goes, so predicting the box and rendering the background jointly seems better motivated than predicting layout first and rendering afterwards.
-
Reciprocal Evolution (SE) is a good design choice. After the first 20% of the denoising steps, the layout branch and the empty-scene branch start exchanging information: the layout branch predicts the bbox (with a confidence score) from the current preview scene, the scene branch is conditioned on that bbox, and every 10% of the denoising process the bbox and the preview scene are updated and exchanged again. In this way the placement and the background progressively adapt to each other instead of one being fixed before the other. While iterative refinement between two components is not new, it is a sensible way to couple the two branches here.
Critical weaknesses
-
The paper claims improvements in sequential visual generation (subject consistency, state transition coherence; contribution 4), but there is no evaluation of it. Tables 1 and 2 evaluate single images with a single object, and sequences appear only in two qualitative examples (Fig. 4, against TIP and MMP) with no metric.
-
Important baselines are missing. I would prioritize two: (a) For final image generation: PlanGen, then inpainting to remove the object, then AnyDoor to add it back. The removal mask can be obtained automatically with SAM3, so the baseline needs no manual annotation. It differs from the included PlanGen+AnyDoor because it involves no PlanGen fine-tuning. The authors themselves build the empty-background ground truth by cropping subjects and inpainting with Big-LaMa (l.322-323), so generate-then-remove is essentially their own data pipeline. (b) A multi-reference image model such as FLUX.2 or Qwen-Image-Edit, given the reference subject (or its edited version) and the step description, and asked each time to generate the full image. Why can such a model not do the same editing, i.e., apply the state change to the subject and place it in a fitting background, without Concept Sliders and the bbox/empty-scene planning? In CoPlace the bbox comes from a VLM rather than the user, but my intuition is that these models already place the object sensibly while generating the background. If CoPlace's advantage appears mainly in long sequences, this should be shown with more visual examples. Optionally, commercial models (GPT Image, Nano Banana) given the reference or edited reference would be a useful reference point on the single-object experiments.
-
The qualitative evidence is too weak. The number of qualitative examples is low, the image quality is noticeably below current models (likely in part due to the SD 1.5 backbone of the empty-scene branch, l.366), and Fig. 5a is very small. Some baseline inputs are unfair: the reference labelled "fishing boat" actually shows a person, so it is not a fair input to IP-Adapter. In Fig. 5b the reference labelled "older couple walking a dog through the woods" shows a single person. The SG and SE ablations are only quantitative (Table 2); there are no visual with/without comparisons (Fig. 5b only visualizes the SE evolution).
-
Several claims are unexplained or not supported.
- In CFG, the unconditional branch is not a way of removing objects: it still generates objects, just not ones guided by the text. So using it inside the box (Eq. 7) does not make the region "empty", and the paper needs either an intuition for why it should be empty or a visual ablation showing it (e.g., side-by-side examples of standard CFG vs. space-gated CFG).
- Missing ablations: the spatially weighted loss (Eqs. 13-15) and the confidence head (Eq. 12), and how much each helps.
- The user study has only 5 raters.
- The training time of the Concept Sliders (per concept) is never discussed, although the state transformation depends on them.
The story would be easier to follow if SCRM were simply described as Concept Sliders for preserving identity while applying the edit, and SPM as AnyDoor for the final generation (as in the CoPlace+AnyDoor evaluation), which would also reduce the number of components a new reader has to track.
Questions for authors
- Why are the TPP and MMP step-1 images in the Fig. 4 pan-frying example identical?
- Why does the space-gated CFG still produce objects inside the reserved region in Fig. 5b, given that the SG is meant to remove objects?
- In Fig. 5b, the reference "older couple walking a dog through the woods" shows a single person. Is this intended?
- Can you provide an evaluation of the sequential setting (multi-step subject consistency and state-transition coherence), given that it is the main claim?
- How does CoPlace compare with (i) PlanGen followed by SAM3-based object removal and AnyDoor, and (ii) a multi-reference model such as FLUX.2 or Qwen-Image-Edit given the reference or edited reference? Commercial models (GPT Image, Nano Banana) would be a useful optional reference.
- What is the training time of the Concept Sliders per concept?
- What is the contribution of the confidence head (Eq. 12) and of the spatially weighted loss (Eqs. 13-15)? Please add ablations, and visual examples for the SG and SE ablations.
Overall, I lean towards rejection. I will read the other reviews and the rebuttal and take them into account, but I expect the main concerns would need more than a rebuttal to address, so a change in my assessment is unlikely.
reviewer 3: Thank you for your constructive comments. We sincerely apologize for overstating Contribution 4 without sufficient experimental validation. We will revise the contribution statement accordingly and provide additional experimental results in the multimodal procedural learning scenario in our future work. We will also report a full combination of Spacial Gating Mechanism and Spiral Evolution ablation test and discutions in future submission. Reply to Q1: We apply TPP only when the step text is detected to mention the reference subject. Otherwise, we follow MMPlanner and generate the image using a prompt refined by an LLM through chain-of-thought reasoning. Reply to Q2: Our main concentration rest on placing the main subject complex background with various objects, which is a common scene in multimodal procedure setting, thus we aim to soly prevent the main subject appearance inside the reserved region. We will consider replace the term "empty" with other statement to prevent misleading. Reply to Q3: Due to automatic postpocessing, the dataset might be quite noisy and not strictly semantically aligned. Considering that it is labor intensive, we will try to carlibrate the dataset manually if possible in future work. Reply to Q6: Phragmatically, train one slider from 3-5 pairs of contrasting images cost approximately 3-5 minites. Reply to Q5 and Q7: We acknowledge these questions and will provide a more detailed technical discussion in our future work.