- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposed a simple method for controllable image generation targeting on joint control of external object compositions and structure/pose of individual objects. External composition is represented via 3D bounding boxes color coded with coordinate map and internal object poses are represented as 2D skeletons. Both of these representations are rendered as 2D images and are used to condition the text-to-image generative model. The conditioning is performed by concatenating the tokens of these 2D conditioning image tokens along with the noise image tokens of the flow model. Additionally, to disambiguate between multiple boxes and their text, an attention masking operation is for the object specific text and reference tokens attend only to the object region during generation. A combination of synthetic and real proprietary dataset is used to train this model.
Critical strengths
[Simple approach]:
The proposed method is simple and generalizes for additional downstream tasks such as image editing and reference driven generation without training. The method leverages established in-context conditioning mechanism of DiT where the conditioning can be directly concatenated to condition the image generation on different control signals. Adding pose control along with the scene composition control is new seems to be working well with in-context conditioning.
[Results]:
The qualitative and quantitative results present in the paper are promising. The generated scenes look more realistic as compared to the baseline approaches and follows the conditioning layouts well.
Critical weaknesses
[Novelty concern]:
- Presented approach is incremental on top of the existing works SceneDesigner, SeeThrough3D, BoxCtrl as they have already proposed 3D bounding box based control for image generation and editing. The major distinction is adding the skeleton rendering as an additional conditioning. The conditioning mechanism is straightforward where the tokens of new skeleton map is additionally appended to the bounding box conditioning map. Additionally the attention mask operation to disambiguate objects is already present in the SeeThrough3D work. Though adding skeleton conditioning improves the results qualitatively and qualitatively, I believe the contribution is marginal. Forming a new representation that compactly represent both the internal and external structure in a common space can be considered as a good contribution over existing methods but this paper lacks in that respect.
[Representation]:
-
The proposed representation of bounding boxes and 3D skeletons share some common information. For example, skeletons already capture extent of the object and 3D orientation as well. And for rigid objects 3D bounding boxes with orientation based color coding is sufficient to capture the objects internal structure as it is fixed. Why do we have two separate conditioning mechanism? an ideal would be creating a new representation that encodes both the information efficiently.
-
During inference, a user has to draw both the bounding boxes and skeleton to perform conditioning image generation. Drawing bounding boxes is relatively easier but drawing accurate skeleton is challenging to draw manually for most of the users. How does the proposed method performs if only bounding box conditioning is passed during inference?
[Experiments]:
-
The paper has not compared with state-of-the-art methods for bounding box based image generation SeeThrough3D[1] and editing method BoxCtrl[2]. A direction comparison for these methods is important to understand the contribution of the proposed method.
-
Why not use the attention routing during training also? As the dataset consists of object masks and 3D bounding boxes, it makes intuitive sense to use it during the training stage also. An ablation experiment should be provided to understand this design choice.
-
The method a combination of real and synthetic datasets an ablation on real vs synthetic dataset should be provided to understand the contribution of each of the components.
[Presentation]:
The paper is easy to read but there are some concerns about presentation particularly about missing important details, as discussed below:
-
Dataset details: The dataset generation process is not clearly presented, the paper claims use of 'internal image/asset collection', and just report the total number of images in the appendix. As dataset is one of the core component of the proposed method, detailed analysis about the dataset should be provided for understanding the data distribution - types of images, distribution of categories, and a sufficient number of representative images from the dataset.
-
Evaluation benchmarks: All the three evaluation benchmarks are not detailed with the number of samples and distribution of images. Disclosing evaluation set or evaluating on public benchmarks is important to validate the correctness of the comparisons presented.
-
Section 3.1 - The geometric conditioning for the external map and the internal maps should we explained in detail about their construction as it the important part of the proposed model pipeline.
-
L218: The training objective is not desire in details about the loss function and how the time steps were sampling for training. Though these details are provided in appendix, presenting them in the main paper is crutial.
[1] SeeThrough3D: Occlusion aware 3d control in text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026
[2] BoxCtrl: 3D-Aware Visual Prompting for Geometric Image Editing, In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH), 2026
Questions for authors
[Clarification]:
-
L169: Disjoint masks {M_i} . What happens if there is some occlusion in the box as it is common for real world scenes where objects don’t fill the complete box tightly such as bicycle, hence the objects may not occlude in the imager but their boxes can occlude upto some extent.
-
L219: Image editing without the mask can be ambiguous: Imaging two humans in the input scene and the user defined different conditioning for each of the humans, how will the model be able to accurately assign the human to its correct conditioning form.
-
L278: What does geometry mean extracted from SAM 3 means? Is this segmentation masks?
-
L279: What is reconstruction rate? Aren’t you doing generation, how to measure the reconstruction?
6 comments
Response to Reviewer iqmA (2/3)
W4: Comparison with SeeThrough3D and BoxCtrl
We added additional comparisons with relevant baselines. For SeeThrough3D, we compare on the shared generation task; for BoxCtrl, we add a comparison on its related rigid editing task.
Table R4: Additional comparison with SeeThrough3D (reference-free generation)
| Method | Rec. (up) | Ang.Err (down) | [email protected] (up) | Trans.Err (down) | Scale.Err (down) | Chamfer (down) | [email protected] (up) | PickScore (up) |
|---|---|---|---|---|---|---|---|---|
| SeeThrough3D | 98.9 | 30.43 | 67.1 | 0.1319 | 0.2631 | 0.0446 | 0.664 | 20.532 |
| Ours | 100.0 | 24.77 | 74.6 | 0.1025 | 0.1941 | 0.0272 | 0.873 | 20.314 |
Table R5: Additional comparison with BoxCtrl (rigid editing)
| Method | Rec. (up) | Ang.Err (down) | [email protected] (up) | Trans.Err (down) | Scale.Err (down) | PSNR (up) | SSIM (up) | DINO (up) |
|---|---|---|---|---|---|---|---|---|
| BoxCtrl | 100.0 | 24.30 | 78.7 | 0.1366 | 0.1980 | 24.58 | 0.834 | 0.798 |
| Ours | 100.0 | 18.15 | 80.6 | 0.1464 | 0.1946 | 21.87 | 0.760 | 0.826 |
W5: Ablation of Training-Time Routing
We did not introduce routing during training mainly because we want a single model to support multi-object generation, single-object generation, and image editing within one unified framework, while these tasks do not require the same degree of instance-level isolation. We therefore designed routing as a training-free inference-time mechanism that can be enabled or disabled flexibly depending on the task. As shown in Section 5.2 (Table 6), this design already improves instance binding in multi-object scenes without requiring additional training.
W6: Contribution of Real and Synthetic Data
To disentangle the effect of data source from that of data size and training budget, Table R2 compares real-only, synthetic-only, and mixed training under matched data size and matched budget. The results indicate that the real-only model produces more natural image appearance but has weaker geometric precision, while the synthetic-only model achieves higher geometric accuracy but shows a stronger synthetic style. Figure 7 in the revised supplementary material (Appendix A.7) also illustrates this difference more directly. By contrast, the mixed model achieves a better balance between image quality and geometric accuracy.
W7: Dataset Details
The initial submission was limited by internal disclosure review, so the dataset statistics were not sufficiently detailed. We have now added detailed category distributions for the training data in Section 1 of the General Response.
W8: Benchmark Scale and Verifiability
In Section 1 of the General Response, we added detailed benchmark sizes and asset provenance information. To improve verifiability, we plan to release the test prompts, control maps, and representative evaluation examples after the necessary anonymization and compliance review.
W9: Additional Details on Geometric-Map Construction
Thank you for the suggestion. In the revised main paper, we have clarified the coordinate-color encoding, the construction of the 3D geometric graph, and the projection-based rendering process.
W10: Additional Details on Loss and Sampling
We have also clarified key training and sampling settings in the revised main paper, including the weighted flow-matching MSE loss and the discrete time indices of the uniform-sampling scheduler.
Response to Reviewer iqmA (3/3)
Q1: Details of Disjoint Masks
The disjoint masks here are not pixel-accurate segmentations. What we mean is the following: after downsampling the projected regions to the latent grid, if different objects overlap, we assign the overlapping locations to one instance according to user input order, producing mutually exclusive token-assignment masks. The purpose is to provide an instance-level partition for instance routing and reduce cross-instance leakage from text or reference-image conditions as much as possible. We agree with the concern that this partition is not perfect in heavily occluded scenes. Still, it is sufficient to support the routing mechanism in practice and improves instance binding in multi-object generation.
Q2: Identity Correspondence Without Masks
In the current editing setup, identity correspondence mainly relies on the global appearance context from the source image, the spatial position and pose in the target control map, and explicit object descriptions in the editing instruction. As a result, when different objects can be clearly distinguished by semantic descriptions, the model can usually perform the correct edit. We also agree with the reviewer's concern that, when a scene contains multiple visually similar people and each needs a different instance-level condition, the correspondence becomes more challenging without explicit masks or routing. This more difficult multi-object editing setting is a meaningful direction for future work and calls for finer-grained instance-level control mechanisms.
Q3: Clarification of the Division of Labor in the SAM 3 Geometry Module
SAM 3 performs 2D instance segmentation, and SAM 3D then uses the segmentation results for 3D reconstruction.
Q4: Clarification of Rec
Rec (Reconstruction Rate) refers to the success rate of reconstruction, i.e., the fraction of cases in which the evaluation pipeline successfully recovers a 3D bounding box for the target instance. If the generated image misses the target instance, or if the instance is too blurry for SAM to identify, reconstruction fails.
Response to Reviewer iqmA (1/3)
Thank you for the thoughtful questions on novelty, representation design, training strategy, and evaluation. In response to these points, we added new comparisons and further clarified the method design and evaluation protocol.
W1a: Novelty at the Capability Level
Thank you for the in-depth discussion of novelty and representation. To clarify our view, the contribution of this work at the capability level lies not only in applying box control or attention isolation, but also in enabling joint control of external pose and internal structure for open categories and multiple instances, and in showing that the same model design can be reused across reference-free generation, reference-guided generation, and editing.
W1b: Novelty at the Representation Level
At the representation level, the internal map in the paper is not a standard 2D skeleton. Instead, it is a control map obtained by projecting a 3D geometric graph and encoding colors with canonical coordinates. This 3D-aware representation provides richer spatial information than a 2D skeleton.
Regarding your questions about why we do not use a more compact unified representation and about sequence concatenation, Table R1 provides ablations that test the practical value of separating the two conditions before fusion:
W2: Complementarity of the Two Condition Representations
In response to your question about using two separate condition representations, we agree that the internal structure map and the external box do share some spatial information (for example, the color encoding in the internal structure map also implicitly contains pose information). However, the two are functionally complementary: the external box provides an explicit global anchor for absolute position and scale, whereas the internal structure map provides a more detailed description of internal shape. As shown in the geometric-condition ablation in Table 5 of the original submission, using both conditions together yields better geometry metrics than using either condition alone (external only or internal only).
W3a: Cost of Using Skeletons
Regarding the cost of using skeletons, the framework is flexible in practice: users can decide how many joints to create depending on the desired level of control precision. In addition, to further lower the interaction burden, we provide preset templates for common poses, which users can edit directly.
W3b: Support for Box-Only Inference
To further clarify the question of box-only inference, the original model in the paper always receives both the external map and the internal map during training and therefore is not directly exposed to inputs with only the external map. It thus cannot directly support box-only inference. To answer this question, we additionally trained a model with dropout on the internal condition at probability 0.5 and evaluated it under both external-only input and full dual-condition input (Table R8). The results indicate that, under external-only input, this dropout model achieves higher geometric accuracy than the external-only model in Table 5 of the original paper, which was both trained and tested with only the external condition. Under full dual-condition input, its performance is again close to that of the full model. This suggests that simple condition-dropout training can make the model reasonably compatible with box-only inference while preserving most of the benefit of dual-condition input.
Table R8: Support for box-only inference