acceptodds
Withdrawn from ICLR 2027

GeoCo-SAVi: Geometry-Consistent Slot Attention for Explicitly Editable Object Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Object-centric video models represent scenes with slots, yet exposed geometry can vary in meaning with appearance. In Invariant Slot Attention (ISA), explicit position and scale can disagree with the decoded center and extent; edits can yield unexpected motion or resizing, and replacing appearance can shift geometry. GeoCo-SAVi promotes geometric authority and semantic alignment. Its spatially equivariant, object-wise decoder makes position and scale effective commands: changing them moves or resizes the rendered support. Factual position alignment ties position to the decoded center, and normalized attention overlap discourages duplicate allocation. Appearance transplantation aligns geometry semantics across objects, so recipient geometry governs layout while donor appearance supplies shape. On Obj3D, GeoCo-SAVi matches ISA reconstruction, reduces latent-position-to-decoded-centroid error by nearly 90%, and reduces appearance-induced size variation while producing the expected translation and scale responses. On 250 MOVi-C videos, it improves reconstruction and instance grouping over both same-protocol references, and video editing over STAITUS. GeoCo-SAVi transforms explicit geometry into compositional control, making both position and scale more readable and editable. Project page: https://GeoCo-SAVi.github.io/.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Object-Centric Learning, Slot Attention, Video Representation Learning, Unsupervised Scene Decomposition, Geometric Control

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_3ikfOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes GeoCo-SAVi to address the inconsistency between explicit position and scale in Invariant Slot Attention (ISA) and the decoded center and extent. The method combines a spatially equivariant, object-wise decoder with factual position alignment and counterfactual appearance transplantation. The authors evaluate the method on the synthetic Obj3D and MOVi-C datasets.

Critical strengths

The paper presents a range of quantitative experiments, along with mathematical derivations and analysis.

Critical weaknesses
  1. Concerns about the practical value of the task. All evaluations are conducted on synthetic datasets, and the qualitative demonstrations, such as Fig. 3, focus on simple geometric objects. Without results on real-world scenes, the practical relevance of the task remains unclear. For these simple geometric objects, the authors should explain what practical benefit the learned representation offers beyond parameter-based manipulation of available scene assets in software such as Blender, and whether this benefit justifies the model’s complexity. The paper also lacks clear qualitative demonstration of position editing, and its current formulation is limited to 2D image-plane position. Moreover, recent instruction-guided image and video editing models support increasingly complex object manipulation like pose editing beyond the simple editing the paper show. The paper does not adequately discuss how its task and capabilities differ from these approaches or what distinct practical need its formulation addresses.

  2. Poor writing and presentation. For example, the Introduction section largely describes the proposed components without sufficiently explaining the motivation behind the individual design choice. The method appears to be an incremental improvement of prior ISA work, yet the paper does not clearly states why each proposed component is needed to address what specific limitations of prior work. Besideds, in Fig. 2(B2), the images before and after appearance transplantation look almost identical, making it difficult to understand what this comparison is intended to demonstrate.

  3. Insufficient qualitative evaluation. The paper presents very few qualitative example. Although it focuses on object-centric representation learning in videos, the paper do not provide video demonstrations. This makes it difficult to assess the temporal behavior and consistency of the video edits. Qualitative ablation comparisons are also insufficient to clearly illustrate the contribution of individual components.

Questions for authors

Please address the critical weaknesses above.

Reviewer_A8KROpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a slot-based representation for object-centric image reconstruction and editing. Specifically, the authors leverage Invariant Slot-Attention (ISA), that exposes disentangled representations for appearance, position and scale, and further improve it with several contributions. First, they introduce a novel spatially equivariant decoding scheme. Then, they introduce a two stage optimization for factual and counterfactual training respectively. The authors evaluate their proposal by running reconstruction and editing experiments on the Obj3D and MOVi-C datasets.

Critical strengths

N/A

Critical weaknesses
  • The manuscript is a very challenging read. Even at the second or third pass, most parts miss clarity and do not flow naturally.

    • For instance, the abstract and introduction already assume a lot of context about the task and do not welcome the reader. The abstract mentions concepts such as "duplicate allocation", "appearance transplantation", "donor appearance" without explaining what they mean. Where the model is introduced in lines 54 to 58, it is very hard to understand what the exact contributions are, as well as their assumptions and tradeoffs. These are just two examples of an issue that affects the paper as a whole.
    • The mathematical formulation is not rigurous either. For example, line 104 introduces a fnf_n vector that is unspecified, or Eq 3 introduces a ξi\xi_i that is not used anywhere else. The authors are encouraged to revise their paper and polish / clarify the formalism.
    • In the experimental section, several metrics are not introduced rigorously or not presented without definition altogether. For example, it's hard to understand how FposF_pos, FsclF_scl, FappF_app scores are computed, whether others like EΔpE_{\Delta p} or βr→1\beta_r \rightarrow 1 are not defined at all.

    In light of the above, it's hard to appreciate any value out of the proposed method.

  • The proposed method seems to showcase reasonable editing results on toy datasets (Obj3D). However, no qualitative results are presented on the more challenging MOVi-C dataset. In general, it is hard to assess the generalization of slot-based representation for real-world use-cases, especially in light of the state-of-the-art of 3D or VLM foundation models, that might perform position, scale and appearance editing just as well or better.

Questions for authors

N/A

Reviewer_AiwTOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets object-centric video models whose slots expose explicit geometry: an appearance vector, a position p and a scale s, as in Invariant Slot Attention (ISA). The authors observe that in ISA these variables are not reliable commands. Editing p or s does not move or resize the rendered object predictably, and swapping appearance at fixed geometry can shift it. GeoCo-SAVi addresses this in three parts:

A spatially equivariant per-slot decoder whose convolution sampling radius scales with s. The paper proves translation and scale covariance of the raw alpha logits in the continuous idealization. Factual calibration losses: decoded centroid alignment with p, and a normalized attention-overlap penalty. A counterfactual “appearance transplantation” loss. It decodes donor appearance at recipient geometry and asks that center and extent follow the recipient while compactness follows the donor.

On single-frame Obj3D the method matches ISA reconstruction and greatly improves alignment between latent and decoded position, with near-ideal scale-response slopes. On MOVi-C it reports better reconstruction, segmentation and editing metrics than reimplemented STATM-SAVi and STAITUS baselines.

Critical strengths

The question is well posed. Asking whether exposed latents behave as commands under intervention is more concrete and useful than generic disentanglement claims.

The scale-response slope diagnostic (target 1 for radius, 2 for area) is clean and interpretable. The Obj3D results here are convincing: ISA is nearly unresponsive to scale, while GeoCo-SAVi is close to ideal.

The evaluation of editing is thorough: no-op floors, independent RGB oracle targets, preservation of non-target pixels, counts of disappearing objects, and finite-grid covariance measurements. Code is released.

Critical weaknesses

The writing is very dense and hard to follow. The main text leans on undefined or idiosyncratic terminology (“geometric authority”, “OneFG support”, “measurement support”, “reference-scale continuation”, “detached factual confidence”). Much of what is needed to understand the method is deferred to a 20+ page appendix: pair construction, the center penalty, the selector, the terminal readout and the curricula. Sec. 3 reads as a compressed list of design decisions rather than an explanation of why each is needed. The paper needs a substantial rewrite: a clear problem statement, a minimal core method in the main text, and the optional refinements moved out or clearly separated.

Missing baselines. No strong published video object-centric baseline is included at native settings, such as a DINOSAUR- or VideoSAUR-style model with feature reconstruction. ISA, the method being improved, is missing on MOVi-C entirely.

Significance is limited, and the claimed novelty of the decoder is undercut by the paper’s own results. The appendix (C.6, Table 5) shows that a standard AIR/SPACE-style decoder performs on par with the proposed scale-steered decoder. That decoder renders on a canonical canvas and then bilinearly warps by (p, s). It is better on F_scl, transplant PSNR, translation error and transplant drift, and slightly worse on PSNR and F_pos.

Separately, the gains are shown on 64×64 synthetic data with isotropic scale only, and no rotation, depth or occlusion handling. Segmentation quality on MOVi-C is low in absolute terms (FG-ARI 0.19, Table 2). It is therefore unclear what the method buys beyond older what/where models, and whether the editing claims hold for objects the model actually segments well.

No qualitative results on MOVi-C. All qualitative figures (Figs. 1, 3, 4) show Obj3D. The paper offers no visual evidence at all for its headline video setting: no decompositions, no slot masks over time, and no position, scale or transplant edits. That matters here because MOVi-C segmentation is weak in absolute terms (FG-ARI 0.19). Without seeing the masks, a reader can’t tell whether slots correspond to real objects, whether an “edit” moves an object or only a fragment, or what the edited frames look like (e.g., artifacts or holes left behind). The MOVi-C editing F1 scores are measured against targets derived from the model’s own decoded masks, so they can’t answer these questions alone. For a paper whose main claim is predictable editing, qualitative video edits are necessary to support the claim.

Questions for authors

Can you provide qualitative MOVi-C results: slot decompositions over time next to the GT segmentation, and position, scale and appearance-transplant edits across full videos? Please include failure cases and the baselines’ outputs for the same edits.

What do you get from ISA with a canonical-canvas warp decoder and no additional losses, and from the same decoder with your losses? This would separate the decoder’s contribution from the calibration’s. Given Table 5, what do you see as the essential contribution of the scale-steered decoder?

Which components are necessary for the core claim? Could you present a minimal version (decoder + L_pos + L_ov + L_geo) and report how much the selector, terminal readout and tail penalty add on each dataset?

Reviewer_A4nTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes GeoCo-SAVi, an extension of Invariant Slot Attention for editable object representations. It combines a spatially equivariant decoder with factual and counterfactual geometry calibration. Experiments on Obj3D and MOVi-C evaluate reconstruction, object grouping, and position, scale, and appearance editing.

Critical strengths

The proposed method achieves the best performance on most reported metrics compared with the evaluated baselines on Obj3D and MOVi-C, particularly for position, scale, and appearance editing.

Critical weaknesses
  1. The contributions are limited. In particular, the first stated contribution merely summarizes an empirical finding rather than introducing a theoretical or methodological contribution.

  2. The architecture diagram is confusing, and the method is difficult to reproduce. The ISA backbone, the actual decoder architecture, and how the model outputs used in the loss functions are obtained are all unclear. The main methodology section also fails to explain the components and data flow depicted in the architecture diagram.

Questions for authors
  1. What distinct theoretical or methodological advance does the first contribution represent beyond an empirical finding?

  2. Can you clarify the architecture and data flow in Figure 2, including the ISA backbone, decoder, and outputs used by each loss?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.