acceptodds
Withdrawn from ICLR 2027

COMPASS: COnditioned Manipulation via Precise Aligned Spatial Specification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image editing models are essential for content creation, and instruction-based editors make semantic modifications accessible through natural language. Precise spatial control (i.e., relocating, rotating, or rescaling an object) remains highly unreliable and challenging. We present COMPASS, which conditions a pretrained image editor on rendered 3D bounding-box proxies: we rasterize a proxy box at the source pose and another at the target pose, and paint each face with a fixed color shared across boxes. This allows the spatial control to be represented as local, spatially aligned differences in the pixel domain, which a diffusion transformer can follow without learning cross-position geometry from scratch. COMPASS also supports multi-object edits in a single pass. To demonstrate our approach, we use Qwen-Image-Edit as the base backbone and fine-tune it on a new dataset of 37,000 source/target image pairs, of which 30,000 are used for training and 7,000 are held out as a benchmark with single-object, multi-object, and overlapping-box splits; we further transfer the model zero-shot to the WildDet3D real-image dataset. We compare COMPASS with state-of-the-art editors on this benchmark. COMPASS outperforms existing techniques: (1) on the new dataset, COMPASS achieves a translation error of 0.27 m and a target-mask IoU of 0.765, where competing methods range from 0.31 to 1.19 m and from 0.262 to 0.633; and (2) on the real-image dataset, COMPASS reduces translation error to 4.15 m, nearly half of the best competing method at 7.93 m and under 40% of the worst at 11.83 m.

Primary area:
generative models
Keywords:
Image Editing, Computer Vision

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RUWBOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents COMPASS, a 3D-aware image editing method. In particular, it represents each object to be edited, together with its desired edited state, using a pair of oriented 3D bounding boxes to specify the pose transformation. To distinguish multiple objects, the method introduces an object-ID embedding for each object and adds it to the image tokens associated with the corresponding object. To train the editing model, the authors also construct a synthetic dataset of paired 3D edits, allowing the model to learn diverse 7-DoF pose transformations.

Critical strengths

The paper introduces oriented 3D bounding boxes as a control representation for image editing. This representation provides the model with explicit 3D-aware editing control.

Critical weaknesses
  1. Limited methodological novelty. Several of the paper’s claimed contributions—including oriented 3D bounding boxes, object-ID embeddings, and the synthetic dataset pipeline—closely resemble mechanisms that have already been implemented in prior works.
  • To the best of my knowledge, the oriented 3D bounding box was previously introduced in SceneDesigner[1] (NeurIPS 2025). In particular, SceneDesigner proposed a more expressive representation called a CNOCS map, which can also be viewed as a color-coded 3D bounding-box representation. Unlike COMPASS, which assigns a single color to each box face, a CNOCS map assigns a distinct color to every pixel according to its normalized 3D coordinate. From this perspective, the 3D representation used by COMPASS appears to be a simplified or degenerate version of the CNOCS map. Besides, the CNOCS map is neither properly discussed nor experimentally compared in the paper, which may prevent readers from accurately assessing the originality of the proposed representation.
  • Object-ID embeddings have also been widely used in subject-driven image generation methods to distinguish tokens associated with different objects.
  • Similarly, synthetic dataset pipelines for 3D-aware image editing have already been widely adopted in prior works, including OBJect-3DIT[2] and Free-Form Scene Editor[3]. The paper does not clearly explain how the COMPASS data generation pipeline differs from or improves upon these existing pipelines.
  1. The experiments do not sufficiently demonstrate the effectiveness and generalizability of the method.
  • Most of the qualitative examples use synthetic images as source inputs, which is insufficient to establish generalization to real-world scenes. Although Figure 4 includes three real-world examples, their editing quality appears relatively poor, raising concerns about the practical performance of COMPASS on real images.
  • The qualitative experiments also cover only a limited range of pose transformations. It is therefore unclear whether the method can handle more challenging inputs, larger rotations, or substantially more diverse 3D transformations.

[1] Qin, Zhenyuan, Xincheng Shuai, and Henghui Ding. "Scenedesigner: Controllable multi-object image generation with 9-dof pose manipulation."

[2] Michel, Oscar, et al. "Object 3dit: Language-guided 3d-aware image editing."

[3] Shuai, Xincheng, et al. "Free-form scene editor: Enabling multi-round object manipulation like in a 3d engine."

Questions for authors
  1. Novelty relative to prior 3D control representations and data pipelines.
  • Please clarify the main differences between the proposed oriented 3D bounding-box representation and previous representations, particularly the CNOCS map introduced in SceneDesigner[1].
  • Please also clarify how the proposed dataset pipeline differs from existing synthetic 3D-editing pipelines, such as those used in OBJect-3DIT[2] and Free-Form Scene Editor[3]. A clear comparison is necessary to establish the originality and novelty of the paper.
  1. Role of the role embedding. What is the specific purpose of the role embedding? The source- and target- oriented 3D bounding boxes are already injected into the network at different positions, which appears to distinguish their roles. Moreover, the current ablation studies do not provide evidence that the role embedding independently improves performance. Please provide a separate ablation of the role embedding.

  2. More experiments.

  • Please compare CNOCS maps and the proposed oriented 3D bounding-box representation under the same backbone, training data, and evaluation protocol. Such an experiment would help establish whether the proposed representation offers an empirical advantage over the more fine-grained CNOCS representation.
  • Please include substantially more real-world examples for both single-object and multi-object editing. These experiments should demonstrate the specific advantages of COMPASS over prior methods in realistic settings.
  • Please evaluate the method on more challenging and diverse transformations, including full 360-degree rotations around different axes. This would clarify whether COMPASS supports broad pose diversity or is primarily effective within the relatively restricted transformation distribution used during training.

[1] Qin, Zhenyuan, Xincheng Shuai, and Henghui Ding. "Scenedesigner: Controllable multi-object image generation with 9-dof pose manipulation."

[2] Michel, Oscar, et al. "Object 3dit: Language-guided 3d-aware image editing."

[3] Shuai, Xincheng, et al. "Free-form scene editor: Enabling multi-round object manipulation like in a 3d engine."

Reviewer_sSQ9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents COMPASS, a framework for precise 3D-aware image editing using rendered source and target 3D bounding boxes as visual conditions. Each box face is assigned a fixed color to encode position, scale, and orientation. The method further introduces object-ID embeddings to support multi-object editing and reduce correspondence errors. The model is trained on a newly generated synthetic dataset and evaluated on both synthetic and real images.

Critical strengths
  1. 3D-aware image editing is an important and practically relevant problem that remains challenging for existing instruction-based editing models.
  2. Representing 3D transformations through visual conditions is intuitive and well suited to instruction-based editing models.
Critical weaknesses
  1. The task formulation, visual prompting representation, and central motivation of using colored 3D boxes instead of pose vectors are highly similar to BoxCtrl [1]. Consequently, the methodological novelty appears limited.
  2. The experimental evaluation is not sufficiently comprehensive. Most qualitative examples involve small objects with relatively simple geometry. Results on larger and more complex objects, such as humans or animals, would make the performance easier to assess. More importantly, it would be better to compare with BoxCtrl, given that BoxCtrl was published more than two months before the ICLR submission deadline.

[1] BoxCtrl: 3D-Aware Visual Prompting for Geometric Image Editing, SIGGRAPH 2026.

Questions for authors
  1. What are the main differences between COMPASS and BoxCtrl beyond multi-object editing?
  2. How sensitive is COMPASS to errors in the source 3D box, camera parameters, depth, and orientation?
  3. Pose errors are computed only when the evaluation detector successfully identifies the edited object. What would the results look like under a unified success metric that counts detection misses, swaps, merges, and other unmatched cases as failures?
Reviewer_oDzAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes COMPASS, a spatially controlled image editing framework that uses colored 3D bounding boxes to specify object transformations. It introduces role and object-ID embeddings for multi-object editing and evaluates the approach on synthetic and real-world datasets.

Critical strengths
  1. The method is simple and effective, achieving promising results on spatially controlled image editing.
  2. The synthetic dataset also provides useful benchmarks for multi-object manipulation.
Critical weaknesses
  1. Limited novelty. The colored 3D box representation has already been explored in concurrent works, while the proposed role and object-ID embeddings are relatively straightforward.
  2. Insufficient validation. The ablation study does not isolate the contributions of individual embeddings and depth ordering. Moreover, the high failure rate on real-world images raises concerns about practical robustness.
Questions for authors
  1. How does the method handle cases where object depth ordering changes after manipulation?
  2. Can the authors provide separate ablations for role embeddings, object-ID embeddings, and depth-based ordering?

After reading the analysis of Reviewer RUWB, I think it makes sense and i've changed my score into 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.