acceptodds
Withdrawn from ICLR 2027

PhysLayer: Learning Object–Effect Separation through Physics-Grounded Layer Decomposition

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent diffusion-based approaches have advanced image layer decomposition, enabling flexible extraction and editing of individual image components. However, complex natural scenes remain challenging when a target is partially occluded, covers previously invisible content, or induces effects such as shadows and reflections. Existing representations often entangle these effects with the foreground or background, while natural 2D imagery rarely provides aligned supervision for the complete target, the target-free scene, and their interactions. We propose PhysLayer, a target-conditioned framework that decomposes a composite image, given a target bounding box, into three complementary components: an amodal foreground RGBA layer, a clean target-removed background, and a signed residual capturing target-induced scene interactions. To supervise this formulation, we construct PhysLayer-30K through controllable 3D scene manipulation and counterfactual rendering, providing matched foreground, background, visibility, and interaction annotations from the same scene. A joint diffusion transformer predicts the three layers within a shared denoising process using layer-aware spatial encoding, targeted foreground and background supervision, and residual attention alignment. We evaluate PhysLayer on foreground matting, object removal, and layer decomposition. Compared with the box-conditioned RevealLayer baseline, PhysLayer improves object-removal PSNR by 2.51 dB on OBER-Test and reduces background LPIPS by 47.4% on PhysLayer-Test. It also achieves alpha SoftIoU scores of 0.9333 on RefMatte-RW100 and 0.945 on PhysLayer-Test, demonstrating improved foreground recovery and target-free scene reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Layer Decomposition, Physics-Grounded Supervision, Object–Effect Separation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ztUpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes PhysLayer, a box-conditioned image decomposition framework. It separates a composite image into three layers: an amodal foreground RGBA layer, a clean background with the target removed, and a signed additive interaction residual for capturing shadows and reflections. To train the model, the authors build PhysLayer-30K, a synthetic dataset rendered with Blender Cycles using counterfactual object removal, and mix it with 10K samples from RevealLayer. The proposed Phys-MMDiT is built upon Qwen-Image-Edit-2511 and the OmniAlpha RGBA-VAE and is fine-tuned with LoRA using flow matching. The training also includes several regional, perceptual, reconstruction, and cross-attention alignment losses. The authors evaluate the method on foreground matting (RefMatte-RW100), object removal (OBER-Test), and three-layer decomposition. For the last task, the evaluation relies mainly on their self-rendered PhysLayer-Test split, which contains 150 images.

Critical strengths
  1. The proposed PhysLayer-30K dataset is valuable. Using Blender Cycles to render paired scene states—with and without the target object under fixed lighting and camera parameters—is a clean way to obtain aligned ground truth for amodal completion and effect removal, bypassing the hallucination artifacts common in 2D pseudo-labeling pipelines.
Critical weaknesses

1. Quality of the proposed PhysLayer-30K and PhysLayer-Test datasets

1.1. Reproducibility concerns

It is unclear whether PhysLayer-30K and PhysLayer-Test will be made publicly available. If not, this would raise reproducibility concerns.

1.2. Low diversity, asset reuse, and lack of transparency about the datasets

a. Limited diversity. Based on the visualizations provided in the paper and the statistics in Figure 2, PhysLayer-30K is centered on household scenes. There is a severe lack of human figures, animals, vegetation, soft, deformable objects and materials (such as clothing, hair, and fluids), and scenes involving complex interactions in natural outdoor environments, which significantly limits its practical value. This is echoed by Table 7: the final training dataset includes 10K external RevealLayer samples for foreground generation, in addition to the proposed PhysLayer-30K. These external samples are likely to contribute substantially to tasks related to foreground generation, such as foreground matting (Table 1) and layer decomposition (Table 3), which weakens the case for the usefulness of PhysLayer-30K.

b. Missing asset statistics. The paper does not provide necessary details about the diversity of PhysLayer-30K and PhysLayer-Test. It reports only the number of images, without specifying the numbers of unique foreground and background assets. Therefore, we do not know how much redundancy exists in PhysLayer-30K and PhysLayer-Test. For example, as shown in Figure 15, the last two test cases use the same background image.

2. Unconvincing generalization and selective evaluation

2.1. In-domain testing

Based on the visualizations and descriptions in the paper, PhysLayer-Test is generated using exactly the same Blender pipeline as PhysLayer-30K. Therefore, PhysLayer’s better performance than the comparison models and baselines on this test set is expected. Moreover, PhysLayer-Test is a small test set of only 150 images. Details such as the proportions of scenes involving occlusions, shadows, or reflections, as well as the proportion of normal scenes, are not provided. It is also unclear whether the dataset’s quality has been verified by humans. Without such details, the evaluation on PhysLayer-Test may be less informative and unrepresentative.

On the external RevealLayer benchmark (Table 10), PhysLayer underperforms the RevealLayer baseline across almost all metrics. Furthermore, the real-world comparison in Table 9 relies on a manually screened 150-image subset of PrismLayer Real, without specifying the selection criteria, raising concerns about cherry-picking.

2.2. Insufficient testing and evaluation of the core claim: effect separation

Residuals are evaluated only within the ground-truth interaction support, without comparison to a simple residual baseline. The evaluations do not establish whether the model invents effects elsewhere.

2.3. Ill-defined downstream utility of the proposed setting: amodal foreground FF and scene-locked residual RR

Although line 046 explicitly claims that “the residual represents target-induced effects rather than serving only as an auxiliary training signal,” the paper never demonstrates a single downstream application that actually uses the predicted residual R^\hat{R}. In fact, R^\hat{R} is completely discarded in matting (Table 1), object removal (Table 2), and even the iterative decomposition examples (Appendix, Figures 8–11). Worse, combining amodal foreground completion with a signed additive RGB residual creates a fundamental contradiction for downstream editing:

a. No relocatability. The main practical reason to hallucinate occluded foreground regions through amodal completion is to move, scale, or reorder the object. Unlike LayerDecomp, which represents shadows and reflections as transparent RGBA layers that can move with the object, PhysLayer predicts a position-locked RGB difference baked into the original background texture. As acknowledged in the paper, moving the foreground invalidates R^\hat{R} and requires synthesizing new effects from scratch.

b. Broken recomposition under occlusion. Because F^\hat{F} completes occluded regions while the occluder remains baked into B^\hat{B}, directly compositing (F^,B^,R^)(\hat{F}, \hat{B}, \hat{R}) causes the occluded target to appear in front of its occluder. Preventing this requires the visibility mask VV (Eq. 2), which the model does not predict at inference time. Table 11 relies on privileged ground-truth VgtV^{\mathrm{gt}}.

2.4. Missing user study

For image generation tasks, especially in this decomposition setting, human evaluation is necessary to assess the details of the decomposed layers. Given the authors’ claim in Figure 6 that existing real-world ground-truth backgrounds retain shadow artifacts, a blind human preference study is particularly essential to validate perceptual quality, effect cleanliness, and amodal plausibility on real images, yet none is provided.

2.5. Missing key ablations

The paper uses a mixture of training datasets: PhysLayer-30K and 10K external RevealLayer samples. Therefore, we do not know which part of the training data accounts for the improvements on the real-world test datasets, RefMatte-RW100 and OBER-Test. How does a baseline trained only on the 10K RevealLayer samples, with exactly the same architecture and losses where applicable, perform on RefMatte-RW100, OBER-Test, and the RevealLayer benchmark? Similarly, why does adding PhysLayer-30K lead to a performance drop across most metrics on the RevealLayer benchmark in Table 10?

Moreover, the w/o R_layer ablation also turns off LrecL_{\mathrm{rec}}, so the residual’s contribution is confounded.

2.6. Limited novelty

Considering the concerns discussed above, the relatively novel component is the residual layer, which is ill-defined and insufficiently evaluated.

Questions for authors

Please take a look at the weaknesses. That’s all I wanted to ask.

Reviewer_i5ULOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

PhysLayer tackles target-level layer decomposition: given an image and a bounding box, it outputs three aligned layers — an amodal foreground RGBA layer, a target-removed background, and a signed interaction residual capturing target-induced shadows and reflections — related to the input by a visibility-aware compositing operator. To supervise this, the authors build PhysLayer-30K by rendering each Blender scene twice under identical camera and lighting, with and without the target, yielding matched foreground, background, and a separated amodal/visible alpha pair (29,896 records from ~65k VQA-screened candidates). The model, Phys-MMDiT, is a LoRA fine-tune of Qwen-Image-Edit-2511 that jointly denoises four slots {C,B,F,R}\{C,B,F,R\} in one diffusion process, with layer-aware RoPE, layer-specific auxiliary losses, attention alignment on the residual, and a compositing consistency term. Evaluation covers matting, object removal, and layer decomposition, reporting 32.93 dB on OBER-Test (+2.51 dB over RevealLayer) and background LPIPS 0.0505 on PhysLayer-Test.

Critical strengths

S1. The dataset is a useful contribution to the field. Paired data for object removal is scarce, and a corpus of this scale with this breadth of annotation would be a helpful resource for the community.

S2. Strong results on the main benchmarks, including OBER-Test, which is not the authors' own.

S3. Reproducibility is above the norm: all 13 loss coefficients, the exact support construction, supervised attention block indices, and a per-source supervision-availability table.

S4. Proper asset/scene splitting. Training and test foreground assets and background scenes are assigned to disjoint pools before scene composition, which removes the usual concern about a self-built test set.

Critical weaknesses

W1. The reported gains are likely driven by the base model rather than the method. PhysLayer is built on Qwen-Image-Edit-2511, a substantially stronger and more heavily post-trained editing model than the baselines' backbones, RevealLayer and OmniPaint use FLUX.1[dev], ObjectClear uses SDXL-Inpainting.

W2. ObjectClear is weakened in the comparison. Table 2 strips its Attention-Guided Fusion, which is ObjectClear's own contribution and only possible for a model predicting an object-effect map.

W3. Limited novelty. The residual RR plays much the same role as the effect part of ObjectClear's object-effect mask: both carry the object's shadow and effect information, and once that region is available, extracting the effect itself is straightforward. The cross-attention loss LattnL_{attn} is used in the same way as ObjectClear's LmaskL_{mask}, supervising attention with that effect region. And LrecL_{rec} serves the same role as LayerDecomp's consistency loss.

W4. Reflections appear absent from the data and results. The signed residual's advantage over a shadow-only layer is representing brightening, but every description of an actual residual output concerns shadows, and I saw no reflection cases in the qualitative figures. The paper does not report how many reflection cases the dataset contains.

Questions for authors

See weaknesses.

Reviewer_gvgdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PhysLayer, which decomposes an image into an amodal foreground, a target-free background, and a target-induced interaction residual. It introduces PhysLayer-30K, generated through counterfactual 3D rendering, and trains a joint diffusion model for this decomposition. The method improves the reported matting, object removal, and synthetic layer decomposition results, although its real-world generalization and comparison protocol require further clarification.

Critical strengths
  1. The paper explicitly separates the target object, target-free background, and target-induced effects, addressing an important limitation of conventional two-layer decomposition.
  2. The counterfactual 3D rendering pipeline provides aligned supervision for the amodal foreground, target-removed background, visibility, and interaction effects, enabling systematic study of the proposed decomposition.
Critical weaknesses
  1. Equation (2) requires a target visibility map VV, but the paper states that inference uses only the input image and the target bounding box, without predicting or providing VV. It is therefore unclear how the amodal foreground is composited with the recovered background at test time.

  2. Both PhysLayer-30K and PhysLayer-Test are generated with the same 3D rendering pipeline, while the complete three-layer decomposition is not evaluated on real images. Therefore, the reported gains mainly demonstrate performance within the synthetic domain and do not establish generalization to natural scenes.

  3. PhysLayer and the baselines use different guidance types, textual conditions, training data, and post-processing procedures. In particular, OmniAlpha combines outputs from two separate tasks, while the RevealLayer reproduction details are not provided.

  4. The w/o Rlayerw/o\,R_{\text{layer}} variant removes both the residual layer and the reconstruction-consistency loss. Therefore, the reported performance degradation cannot be attributed specifically to residual modeling.

Questions for authors

Equation (2) requires a visibility map VV. At test time, is VV predicted, provided, or derived from the input? Please clarify the exact inference and compositing procedure.

Since both PhysLayer-30K and PhysLayer-Test are generated with the same rendering pipeline, are the test scenes and assets strictly held out from training? Is the complete three-layer decomposition evaluated on any real images?

For a fair comparison, what guidance, prompts, training data, and post-processing are used for each baseline? Does the RevealLayer training data overlap with any evaluation data?

In Table 4, can the authors provide an ablation that removes only the residual branch while retaining the reconstruction-consistency loss, so that the contribution of residual modeling can be isolated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.