acceptodds
Withdrawn from ICLR 2027

VIRAL: Visual In-Context Reasoning via Analogy in Diffusion Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual in-context learning (V-ICL) uses a source–target image pair to specify a transformation for a new query. A central challenge is to support structured prediction, restoration, and open-domain editing through this interface, despite their different output structures and appearances. We present VIRAL, a V-ICL framework for pre-trained Diffusion Transformers (DiTs), instantiated with Qwen-Image-Edit. VIRAL encodes the exemplar and query as separate image sequences, distinguishes their roles through 3D-MSRoPE, and uses MoE-LoRA to accommodate heterogeneous transformations while keeping pre-trained weights frozen. We construct a dataset of transformation-consistent exemplar–query quadruplets through standard-task pairing and analogous edit mining and synthesis. The resulting model follows visual demonstrations without task-specific text instructions or test-time parameter updates. Across eight task groups, VIRAL outperforms four V-ICL baselines on all 14 reported metrics. It also performs open-domain editing and generalizes to held-out styles and lineart generation, which are excluded from VIRAL training. These results show that one pre-trained DiT can support a shared visual-demonstration interface across diverse tasks and edits.

Primary area:
generative models
Keywords:
Visual In-Context Learning, Image Generation, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fPdFOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a method for in-contextl learning by training LoRAs and MoE-LoRAs on a text-to-image editing foundation model (Qwen-Image-Edit). The methods accomplished image analogy editing on some varied tasks (from depth estimation to image editing), and show the method generalizes to unseen image-editing tasks. This is done without any guidance from text.

Critical strengths
  • The paper was written well and easy to follow. The method itself is quite simple.
  • The method does not require text instructions (Although I have reservations about this, in the weaknesses section)
  • The evaluation includes held-out styles, which is very important to see generalization, and is something sometimes missed by works. (I still have reservations in the weakness section)
  • the performances compare to the baselines seem as well, qualitatively as well.
Critical weaknesses
  • My main concern with the paper is its novelty. The task as described in sec 3.1 is essentially Image analogies (Hertzman et al. SigGraph 2001), which was studied extensively, including by using pretrained DiTs and adapting them with various LoRA based architectures (including MoE-LoRAs), but this and other seminal works in the field remain uncited. The paper doesn’t acknowledge any Image analogies works, and simply acknowledges in context learning. In-context learning can be considered a subset of image anaolgies, and even is a new, LLM-centric viewpoint. For example, the same task can be solved by patch-based methods (such as the original Hertzman et al. 2001). Image analogies should be discussed explicitly, and make the distinction to in-context learning based approaches.
    • Importantly, this I think is the cause of missing so many highly related work, maybe the most is VisualCloze (Li et al. ICCV 2025), which solve many of the same tasks by adapting a DiT (Flux.1) with LoRAs. They also leverage 3D-RoPE for multi-image conditioning.
    • The paper doesn’t acknowledge other DiT based editing works such as RelationAdapter (Gong et al. NeurIPS 2025) which investigate positional encoding for DiT-based editing or LoRWeB (Manor et al. ECCV 2026) which uses a similar mixture of LoRAs on DiTs.
    • The paper compares to ICEdit (Zhang et al. NeurIPS 2025), but fails to mention it in the related work section, where it would highly be beneficial to distinguish it from ICEdit’s LoRA-MoE routing in DiTs.
    • The authors claim in the last line of the abstract that their results show that one pre-trained DiT can support lots of visual transofmations. As expresses here, this is not the first time this was shown and cannot be claimed as a contribution.
    • Of course, this translates to the much needed comparisons to the recent DiT-based analogy methods, rather than relying on the older baselines that currently the paper compares to.
  • The proposed method doesn’t use text, and it’s true that recent analogy methods opt for text-based guidance as well, however, I’m not sure how much of a strength it really is. For starters, the used model is a text-to-image model, so there’s no need for additional training for text. This is related to the notion of how much info is present in xsx_s. Specifically, while it might be difficult to describe all transformations exactly with text, text is usually very useful when given xtx_t, without xsx_s. This is especially important since it’s usually very hard to get paired source exemplars in the wild. For example, the third line in Fig1 shows masks and bounding boxes on an object, which can easily be described by text (mask the chicken, get the bounding box of the car). This might even be easier then extracting some bounding box or mask of the original image. Similarly with style transfer, usually the original xsx_s image doesn’t hold any important information which isn’t shown already in xtx_t. For example, the anime style applied in the bottom row can be inferred from xtx_t alone. The added information from xsx_s is that a hat was added and the mouth was opened for a smile. This also can be easily described in text. Practically speaking, usually in the real world you might even only have access to xtx_t, and you’d want to “copy” the same edit to your own image, without any access to xsx_s. Indeed, Tab 6 compares to baselines with text, but it doesn’t really reveal if xsx_s is in itself necessary or the model just learned to imitatie xtx_t directly. An experiment with mismatched exemplar pairs can help here, and even ablating what happens when a guiding text is given to Qwen (since it’s already built for it).
  • Figure 4 seems weird to me. I went to the relevant tab 12 that shows the automatic text instructions given to the baslines, however they depict a description of the exemplar pair sometimes - instead of the transformation. For example, ‘replace the cupcake with a galaxy theme’ - when the query image is a cake. Applying these instructions might bias against methods that use these automatic prompt. Actually, this figure is very confusing to me, as it seems for example for this cake example only ICEdit actually used xtx_t for editing. Specifically, it doesn’t look at all like galaxy cupcakes - but just blue cupcakes, yet somehow your and Qwen-Edit applied an unrelated galaxy edit. Did figure 4 actually use the prompts for your method? In any case, this casts doubts on the comparisons.
  • While lineart is considered an unseen task, one of the trained tasks is edge detection. This means, in my opinion, it’s not really an unseen tasks as the 2 tasks are too similar. Additionally, since the training data also includes some stylization tasks, without the dataset being public here in this submission it’s hard to see if the evaluated data truly captures enough variety and if the unseen tasks are really unseen.
  • I’d like to make the wording in the abstact a bit more nuanced. In the abstract L36 the authors frame in a sentence the framework as using just pretrained models. Only next it says it’s LoRAs instantiated from them. This can be misleading, and make it seem like it’s a zero shot method.
  • A lot of the evaluation is heavily dependant on synthetic images, generated with Qwen. Then, the used model is Qwen as well. This introduces a bias risk where maybe the model is collapsed towards these results and no learning was actually really done.
    • Similarly, this approach was trained and then tested on labels generated via ControlNetAux, but is compared to completely different methods trained on other datasets. This also introduces an evaluation bias towards the distribution of ControlNetAux.
  • The B.2 appendix says tab 8 will include task sampling probabilities, noise-level distribution, etc. but it doesn’t?
Questions for authors
  • You have tasks in the dataset such as pose estimation or segmentation but you do not report any metrics on them. Why?
  • Table 10 says ‘category and fold selection’ yet the word fold appears once in the entire manuscript - what fold do you mean here? I'd also appreciate answers regarding my concerns in the weakness section.
Reviewer_hxMpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VIRAL, which is an visual in-context learning framework built on pretrained DiTs. For the context of in context learning, a source-target example pair specifies the transformation that should be applied for a given input image. Given the pair of images, VIRAL encodes them as distinct image sequences and attempts to separate the roles of these images with 3D-MSRoPE indices dedicated to each image, which is an extension to the RoPE embeddings in Qwen-Image-Edit (MSRoPE), that is the model selected for the task. Following, the authors adapt the model for the task given the pair with a MoE-LoRA. Additionally, the authors construct a dataset of quadruplets that are consistent in the subjected transformation. The paper shows the effectiveness of the proposed method with qualitative and quantitative experiments, along with generalization experiments with unseen styles and lineart generation.

Critical strengths
  • The paper provides a simple and effective way of representing the example pair and the query as separate sequences with role-specific positional indices, enabling in-context signal for diffusion models which also avoids dividing the input resolution across a grid.
  • The proposed framework covers common control tasks with the combination of input adaptation and MoE-LoRA adapter, where the qualitative effectiveness is evident in Fig. 3. The provided examples also show strong evidence across the generalization of the method across different control types, which makes the method useful for different data curation pipelines for control oriented tasks.
  • The proposed dataset construction pipeline for editing quadruplets is preactical and well documented.
Critical weaknesses
  • One major concern is the positioning of the paper with respect to recent DiT-based approaches that address the same problem, such as [1] and [2]. Both methods investigate the problem of translating similar controls wiht an in-context learning motivation but neither work is discussed or compared, which makes it difficult to understand the main contribution of the paper that created the differentiation in terms of success of the method.
  • The compared V-ICL baselines are earlier models that operate in lower resolutions and are not trained on the curated dataset, whereas VIRAL uses a considerably larger pre-trained backbone. Therefore, the improvements in Table 1 may be model dependent and authors are encouraged to provide comparisons with the baseline model, to isolate the contribution of their method.
  • The ground-truth source for depth, normal, and edge evaluation is not clearly stated. If the test samples are labeled with the same predictors used in dataset construction, the comparison with Depth Anything and StableNormal in Table 3 can be misleading, and results on standard benchmarks would be helpful (depth estimation and pose estimation could be good targets for these tasks). Furthermore, the selected task-specific baselines (an unsupervised deraining method and a watermark removal method from 2021) are not sufficient to support the claim of being comparable to state-of-the-art specialists.

[1] Gong, Yan, et al. “RelationAdapter: Learning and transferring visual relation with diffusion transformers.” Advances in Neural Information Processing Systems. Vol. 38. 2025. [2] Li, Zhong-Yu, et al. “VisualCloze: A universal image generation framework via visual in-context learning.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025.

Questions for authors
  • How does VIRAL compare with [1] and [2] on the tasks they share? As mentioned in the weaknesses section, this comparison is important for understanding the contribution of the proposed design.
  • Can the authors clarify the ground-truth source for the depth, normal, and edge evaluations, and provide results on a standard benchmark (e.g. depth annotations)?
  • Did the authors observe any task specialization among the MoE-LoRA experts? An analysis of the routing behavior would support the motivation for using MoE-LoRA.
Reviewer_STyTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes VIRAL, a visual in-context learning framework built upon a pretrained diffusion transformer. Specifically, VIRAL introduces 3D-MSRope together with MoE-LoRA for fine-tuning QwenImage-Edit on a newly constructed dataset. VIRAL aims to apply the transformation demonstrated by an exemplar image pair to a query image, particularly in a one-shot setting. They evaluate their model across eight task groups, where it outperforms the baselines. In addition, they conduct experiments to assess its generalization ability in open-domain settings. The results show that a single pretrained DiT can support a shared visual-demonstration interface across diverse tasks and edits.

Critical strengths
  1. The paper is well written, with detailed descriptions of the method, dataset construction, and experimental analyses.

  2. Extensive experiments evaluate both performance on tasks covered during training and generalization to open-domain settings.

  3. The qualitative examples clearly illustrate VIRAL’s capabilities across diverse tasks.

Critical weaknesses
  1. The qualitative examples for segmentation and object detection show only single-object cases. How does VIRAL perform when multiple objects are present in the query image? In addition, this paper does not discuss failure cases, which would help clarify the method’s limitations.

  2. How does VIRAL compare with directly using models such as GPT-Image-2, Nano Banana, Qwen-Image-2.1, and FLUX.2-dev, etc.?

  3. The method is evaluated only with Qwen-Image-Edit-2511 as the backbone. Evaluating backbones of different sizes and architectures would help distinguish the contribution of VIRAL from the capabilities of the underlying backbone.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.