acceptodds
Under review as a conference paper at ICLR 2027

Learning The Changes: Transferable Visual Transformations From Image-Pair Compression

Abstract

A pair of images demonstrates a visual change precisely but binds it to the identity, layout, and texture of one scene. Existing systems typically learn to reuse the change from matched demonstration–query examples or represent it in a pretrained semantic space. We ask whether a reusable representation can instead emerge from reconstruction alone. A pair encoder compresses a source–target pair into a short edit code of a few continuous tokens, and a jointly trained diffusion editor reconstructs the target from it, with no text and no example of the edit acting on a second image. The learned codes apply the demonstrated change to new pairings of source images and transformations. Longer codes reconstruct the demonstrated pair more faithfully; shorter codes transfer more accurately and organize more clearly by transformation. The editor also applies two compatible changes at once from the concatenated codes of separate demonstrations, a format it never receives in training. This compositionality emerges from reconstruction as well, once the training targets themselves contain more than one change, as natural-image edits often do. The compressed representation thus retains the transformation itself, enough to carry it to new images and to combine it with another.

Primary area:
generative models
Keywords:
visual transformation learning, exemplar-based image editing, task representations, representation compression, compositional generation, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_p9vPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper tackles the task of image analogies (calls it only in-context learning though for some reason), and proposes to train a Flux-based encoder and another decoder to "compress" the image analogy exemplar-pair into a "code" vector that represents the analogy, and then to condition the decoder on that vector to apply the analogy to a new query input image. This approach also allows for averaging the code vectors of multiple exemplars to refine the needed analogy.

Critical strengths

I like the idea of compressing the analogy to a vector which can be reapplied later and refined this way via averaging the code vectors. Usually in analogy editing it's just one exemplar and this approach allows for an intuitive solution to that.

I appreciate the author's efforts in analyzing the transformation space and its 'semantics', although I have some worries about it (in the weaknesses section)

Critical weaknesses
  • The paper writing is problematic in my opinion. The phrasing is a bit vague, cumbersome, and imprecise, making the reading experience very tiering. Also, many instances of not well-defined words suddenly being used to represent something that the reader should guess what the authors meant. For example, L314 – "Simliar transfer" to what? L367 – withhold what "experience"? I feel the paper needs a massive editing effort to make it presentable. Minor but related, why are there both experiments, discussion, and conclusion? The discussion is mostly repetitive of ideas present in the experiments section.

    • One important example is the VAP-data. The text in Sec 4 mentioned that only VAP-data frames are used, making the reader think the targets are also from VAP-Data. But the appendix reveals that actually QWEN was used to edit the images, and it seems only the source images are from VAP. Will the source frames and the generated dataset be made public? Otherwise it's quite hard to reproduce the results later, since nothing about code release, checkpoint release, or data release was mentioned in the reproducibility statement.
  • The paper doesn't present the task clearly. It's an image analogy task (Hertzman et al. 2001), not just in-context learning (which is a new, very LLM-centric viewpoint). The paper doesn't acknowledge seminal works in this field. Related, in the related work on visual learning from examples – Image analogies are presented as in-context learning, which it is not (always), for example – Hertzman et al. 2001 which is patch-based. Also Diffusion Image Analogies from the paragraph is not an in-context learning approach, as it uses inversion. Image analogies should be discussed explicitly, and make the distinction to in-context learning based approaches. Nguyen et al (2023) is only mentioned as an afterthought even though it's a conceptual predecessor.

  • The experiments are lacking.

    • Most importantly, the authors don't test generalization to held-out transformations (L744) at all. In the appendix when they train and test on Relation252K, the original authors provided specifically 10 unsees transformations just to check this. However, the authors ignore this and only test on analogy types seen during training. While it is true the comparison to RelationAdapter is harder since they don't give the train-test-split, it was nonetheless possible to compare all approaches on the unseen analogies. It is possible that the models just learned to act as a classifier to distinguish between the different types of analogies, instead of actually compression the analogy data (Since K=2 query tokens, each 3072 in dimensions, is quite a large space to overfit for analogy-type classification).
    • Also very important - the main paper has no comparisons to baselines at all. The comparison in App F, as I mentioned above, is built on purpose for some reason to be not comparable with prev works. This is not a new task, and even if it was it is expected to provide some baseline comparisons.
    • Next, the dataset is confusing to me, specifically the edit-groups. Why is the Naruto in fig 2 considered a styling edit but the sculpture is a subject edit? Why is the man in the city a scene edit but when the man is at sea on a dolphin it's not a scene edit? Fig.8 is also weird on this regard, why is eg giant figurine viewpoint and geometry? This this is relevant to the clustering experiment, which uses colors to distrinugish the different classes, but seems like the grouping is a bit random.
    • The paper reiles heavily on paired metrics (MSE, SSIM, etc). "Qwen3.5-9B judge (QwenTeam,2026) sees the support pair, query, and prediction, but not the paired query target." (the wording isn't clear, because the word "target" wasn't used as the GT image until now, but rather interchangeability with prediction (eg L162)). I don't agree it makes sense to compare to the GT in pixel-wise metrics, since analogies can be applied in many ways, for multiple realizations.
    • The user study is problematic.
      • Related to the unclear writing, I could not understand what really the human study results are from Sec 6.4. What is "per-edit application"? The study should be expanded upon. I couldn't even understand from the appendix what's the meaning there.
      • Next, the user study from my understanding is on a different encoder-decoder models, which was continue to be trained for 480x480 images. No automated metrics computed on those images.
      • The appendix (only) mentions that the images in the user study included quite excessive filtering (L1050 incompatible edits, inapplicable or unclear demonstrations, duplicates, and invalid assets). Also some ratings include additional exclusions (L1058). This doesn't allow seeing if the method actually edits successfully in general, and instead introduces survivorship bias: WHEN the method happens to edit in a correct way, then it will also produce a good-looking result.
      • L1061 seems to say the results were filtered after receiving the answers from 2-7 humans (why not a balanced questioning?). So not only this is an extremely small set, but it was filtered even further? If this is the case the user study is barely representative.
  • Table 13 shows the variance of the VLMs scores is 1.8. It's not reported in other VLM metrics, but if it's similar for all the other VLM results, then it seems the VLM-based metrics do no mean much, sadly, as all fall within the same STDs, making the claims on different Ks statistically indistinguishable.

Questions for authors

My most important questions are basically the weaknesses. Most importantly:

  • the transfer of the results to held-out novel transformations, unseen during training. How would the 1-NN accuracy in code space behave in this space of transformations?
  • Why are there no comparison to baselines?

Minor questions:

  • Which GPU was used for training? How long did training 45K steps take
  • Just out of interest - can the longer codes maybe be used for personalization? Since you say they represent the exemplar pair "too much" for analogies.
Reviewer_VPBgOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
3
Read review
Paper summary

I am going to be honest, this was a really hard paper to follow, it assumed the reader knew exactly what they are talking about from the start, so if this summary does not resonate, it is from lack of comprehension and clarity. The paper is an analysis of the properties of a particular conditioning technique for image editing. Their technique (unclear from the paper if the training procedure and architecture are part of their contributions or only the indepth analysis of properties) uses minimal edit pairs of images and a query to train an encoder that produces a "code", here a matrix encoding of dimensions K x d, that represents the transformation which is then given to a decoder model with the original unedited image from the pair to reconstruct the edited images. The paper then goes on to analyse the properties of these "codes", specifically, (1) their reusibility on new images, (2) whether they can be combined and how for more complex edits, (3) the effects of the size of K on properties (1) and (2). Their main conclusions are: they do seem reusable, especially if you average "code" extracted by encoder from multiple similar edit pairs; they are combinable and the best way to do so is through concatenation; smaller K is better for transferability of "codes".

Critical strengths

I like the use of the controlled color edit experiment to better understand the impact of different combination techniques. Figure 1 really helped with understanding. I appreciate the rigor used in experimental protocols. There seems to be a thorough set of additional experimentation in the appendix, though it is over 20 pages long so I did not read it as carefully as the main paper.

Critical weaknesses

My by far biggest concern in the complete lack of clarity. The paper is written like a technical report for those who were on the project, not for an outside reader and assumes way too much insider knowledge, leaving the reader to have to infer a lot of information to try to fill in the blanks. For starters, the abstract in no way gives a sense of what the paper is about. "A pair of images demonstrates a visual change precisely but binds it to the identity, layout, and texture of one scene", what does this mean? The reader does not yet know we are taking about pairs of matched images with a minimal edit or transformation distinguishing them. Or the use of "edit codes" again the reader does not know what these are. Goes for the introduction, line 031, what kind of image pairs?, line 036-043, what are the formulations in question and "whether either is necessary" for what? "is an open question?" but is this the question you answer in the paper? I still don't really know after reading it. It is also unclear if the encoder-decoder architecture describred and training procedure is one of your contributions or if only the in depth analysis of parameterizations and operations over "codes" is. line 174 what does it mean for a source and target image pair to be "accepted"? lines 204-213 What is a "support" I still do not know after reading the whole paper and some of your results are based on this concept and I could not interpret them. Section 6.3 on the controlled experiment introduces two different training regimes "atomic" and "combined", this is the first we hear of this. Did all your other experiments try both or only one? If based on figure 1 your main encoder-decoder model was trained only with the atomic approach, yet in the controlled experiment the "combined" approach played a big roll in the performance of the concatenation method for combining codes, what does this mean for the concatenation results in the main model? lines 450-451 "Routing the code through different conditioning streams shows the same dependence, since routes with similar transfer differ in composition quality", this is the first mention of "routes" in the main text and its in the discussion, what are routes? Lines 452-455 I do not know what this paragraph is trying to say. In the literature review and conclusion, you mention that others have typically used "matched demonstration and query examples or on a semantic space fixed in advance" for conditioning image edits/transformation; what is a fixed semantic space here concretely?

I am left after reading the paper with more questions than answers. I think there could be something interesting here so I don't want to just reject it, but the real lack of clarity made this paper incredibly difficult to read and review and would require a very significant rewrite. For this reason my recommendation is weak reject.

Other minor suggestions: 3.3 "Reusing the code" call it something that matches with your figure 1, like code reuse at inference.

UPDATE POST REBUTTALS AND NEW REVISION: I find clarity much improved, I am moving my score from 2 -> 3.

Questions for authors

What is dimension d? I couldn't find it anywhere.

Also see questions in critical weakness section.

1 comment

Thanks for the review. Please make an effort to comment on the authors' rebuttal.

Reviewer_ySYsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The method proposes a method to propagate visual transformations from an image pair to new images. It is done through an information bottleneck. First, an encoder takes in the image pair and outputs a small number of tokens as "editing code", and then a decoder takes in the first image and the editing code to generate the second image. From the paper, as the number of tokens is small enough, the tokens carry more information about the visual transformation instead of leaking the image content, which helps better propagation of the transformation. The paper shows some qualitative results that look promising and reports evaluations to ensure the transformation is propagated faithfully.

Critical strengths
  1. I found some experiments in the paper interesting and informative. Since the information bottleneck is the key ingredient for transferring the edit properly, the analysis in Figure 4 really helps us understand how small this bottleneck should be. Figure 6 also provides logical edits that are visually verifiable, which makes the method more convincing.
  2. The qualitative results are pretty strong and convincing.
Critical weaknesses
  1. Unfortunately, the biggest weakness of this paper is the lack of baseline comparisons, and this is the biggest factor in my current rating. While I understand visual analogy is not a common application for modern image editing models, the problem itself is not new. It would be great if there were some comparisons to Deep Image Analogy [1] and other relevant work. In fact, it would be informative to compare with proprietary models like nanobanana, even if the method doesn't outperform them. Another potential baseline would be to convert the image pair into text descriptions and apply text-based steering to text-to-image models [2]. An even simpler version would be to use a proprietary VLM (e.g., GPT-6) to describe the visual transformation of the image pair as an edit instruction, then pass the instruction into an image editing model. Please discuss these options and include at least 2-3 strong baselines to compare with the proposed method

  2. It would be great to discuss failures of the method. For example, are there particular types of visual transformations that the proposed method fails at?

[1] Liao et al. Visual Attribute Transfer through Deep Image Analogy.

[2] Ekin et al. The Unreasonable Effectiveness of Text Embedding Interpolation for Continuous Image Steering.

Questions for authors
  1. For Figure 6, is the same type of visual transformation included in the dataset, or is this a generalized behavior that emerged from training on other types of visual transformations?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.