- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether image manipulations can be approximated by spatially shared linear mappings in the feature spaces of pretrained vision models. It evaluates geometric, photometric, masking, and semantic transformations across ConvNeXt, SwinV2, and DINOv3, comparing linear mappings with MLPs, CNNs, and Transformers. The results suggest that simple linear mappings can often approximate complex semantic edits, particularly in deeper feature representations. An additional SVD analysis suggests that leading singular components capture much of the visible editing effect.
Critical strengths
An interesting research question. The paper explores whether complex semantic transformations can be approximated by simple linear operators in pretrained visual representations. This question is relevant to representation learning, interpretability, and feature-space editing.
Critical weaknesses
- Potentially unfair comparison between linear and nonlinear mappings. The MLP and Transformer use a fixed hidden dimension of 512, while deeper features have 1024 channels. Without appropriate residual connections, this introduces dimensional bottlenecks that may limit their expressiveness relative to the linear baseline. Consequently, the small performance gap may reflect architectural constraints rather than the sufficiency of linear mappings, potentially also confounding the claimed improvement with feature depth.
- Insufficient distinction between feature preservation and actual editing. High similarity between predicted and target features does not necessarily indicate successful editing, particularly when most features remain unchanged. Appendix A.10 reports substantially weaker displacement alignment for localized edits and a higher displacement prediction error for Linear than Transformer (RMSE: 5.99 vs. 4.42). These findings raise questions about whether linear mappings accurately capture editing-induced changes rather than primarily preserving the original representations.
- Questionable interpretation of SVD components. The SVD analysis decomposes the complete mapping , whereas the actual editing displacement is determined by . Therefore, the apparent concentration of semantic edits in leading singular components may simply result from the loss of original image details. Analyzing would provide more convincing evidence.
- Potential overestimation of semantic editing success. The VLM evaluation adopts permissive criteria, such as accepting faded mirrors as successful removal or indistinct objects as teddy bears. It also allows successful judgments when the desired property already exists in the original image. Without no-edit controls, these criteria may conflate genuine semantic manipulation with reconstruction artifacts or unchanged attributes.
Questions for authors
- Fairness of model comparisons. Can the paper clarify whether the nonlinear probes include residual connections and evaluate wider nonlinear architectures to ensure that their performance is not limited by dimensional bottlenecks?
- Actual editing effectiveness. How much do the learned mappings improve over an identity mapping? Can the paper report editing-displacement errors and explain the relatively poor directional alignment observed for localized manipulations?
- Semantic evaluation reliability. What is the false-positive rate of the VLM evaluator on original images and FeatInv reconstructions without editing? Can the paper provide stricter semantic success rates, ideally validated through human evaluation?
Rebuttal to Reviewer Afwe
We thank the reviewer for the careful reading. We have uploaded an intermediate revision of the paper in which all changes are marked in blue. We will upload the final revised version, incorporating all reviewer feedback, before the end of the discussion period.
The reviewer identified important limitations in the original comparison. We now distinguish edit displacement from preservation and report results relative to an identity/no-edit mapping. In feature space, RΔ2=1−∥Fmap−Ftgt∥2/∥Ftgt−Forig∥2; RΔ2=0 is the no-edit reference, 1 is exact displacement recovery, and a negative value is worse than no-edit. Masked RΔ2 restricts both errors to the edited region. Feature-space no-edit masked cosine is a separate quantity, not zero by definition.
W1/Q1 (probe capacity and residuals)
The reviewer is right that a 512-wide MLP or Transformer without a skip connection creates an identity-path bottleneck when the feature dimension is 1024. We ran an ablation on five semantic edits at ConvNeXt and SwinV2 feat3, comparing the paper-width probes with 512-wide residual probes and full-width probes (MLP width 4d, Transformer dmodel=d), with and without a zero-initialized residual connection. Values below are means of per-edit medians, reported as RΔ2 / masked RΔ2:
W2/Q2 (edit capture vs. preservation)
Across the 297 dataset/backbone/layer/edit combinations, means of per-combination medians show positive masked displacement capture for semantic edits: masked RΔ2 is 0.13 for Linear and 0.21 for the paper Transformer, with Linear positive in 80% of combinations. For masking, the corresponding values are 0.21 and 0.46; for geometric edits, 0.43 and 0.45 (with the spatial-only transform as zero reference). Globally, semantic RΔ2 is −0.17 for Linear and −0.02 for the paper Transformer: small errors on many unchanged tokens can outweigh gains in the edited region. This explains weak global alignment for localized edits and motivates reporting masked alongside global results. In the Cars ConvNeXt feat3 residual-probe results above, global semantic RΔ2 is positive for every tested edit (0.28–0.58), compared with −0.22–0.46 for the paper MLP.
W3 (W versus W−I)
For the reported 297 linear probes, the effective rank of W−I for semantic edits is 0.75d, compared with 0.53d for W; it is higher for every semantic probe. The Frobenius norm ratio ∥W−I∥F/∥I∥F is 0.95 for semantic edits (masking 0.52, photometric 0.68). For masking edits, W−I is more compact (0.54d versus 0.82d for W). In the ConvNeXt feat3 truncation analysis, top-32 W falls below the no-edit reference for three of four edits (RΔ2=−0.26 to −0.84), whereas top-32 W−I retains 46–75% of the full probe’s RΔ2; reaching 90% requires k=256–1024. We will show both decompositions and describe the semantic displacement as distributed across many directions, with a substantial portion in the leading ones.
W4/Q3 (VLM reliability and human validation)
We agree that a permissive judge can overestimate success, particularly when the requested property is already present or an object is indistinct. We will report per-edit rates under the original rubric and include no-edit controls to investigate the False Positive Rate. We will also report blinded human ratings on a subset rather than imply that the VLM alone establishes semantic success.