acceptodds
Under review as a conference paper at ICLR 2027

Source Retrieval for Generative Stylized Images via Destylized Feature Learning

Abstract

Generative image editing enables users to easily create highly customized and stylized derivatives from reference images, posing new challenges to copyright protection and visual source tracing, particularly in the portrait domain. To address this problem, we formalize the task of Source Retrieval for Stylized Images, which aims to trace a stylized query image back to its exact source image. The main challenge is that existing pretrained visual representations can be sensitive to style variations and generation-induced appearance changes, weakening source discrimination under large style shifts. To enable systematic evaluation of this problem, we build MSIR-Bench, comprising 56K images derived from 5,500 source images and covering 25 artistic styles and five generative editing models. Building on this benchmark, we further present Destylized Feature Learning (DFL), a joint learning framework applicable to various visual backbones that encourages source-discriminative representations with reduced reliance on style-predictive cues by jointly employing a Source Discriminative Loss, a Destylized Triplet Loss, and Style Confusion Regularization. Across seven visual backbones, DFL consistently improves source retrieval over zero-shot and task-adapted baselines in the primary evaluation, while maintaining gains under unseen-source and unseen-style settings and achieving positive average improvements across datasets and generators. Our benchmark and associated resources will be made publicly available at https://anonymous.4open.science/r/Source-Image-Retrieval-8172/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Source Retrieval for Stylized Images, Visual Source Tracing, Generative Image Editing, Style-Invariant Representation Learning, Image Retrieval

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_nbQCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to retrieve the exact source image used to generate a stylized image. MSIR-Bench contains 56K images, including 5,500 source images, and evaluates generalization to new sources, styles, generators, and datasets. DFL jointly trains the final backbone blocks and the embedding head using the Source Discriminative Loss, Destylized Triplet Loss, and Style Confusion Regularization. It brings features from different styles of the same source closer together and separates features from different sources. It also trains the encoder to make the predictions of a frozen style classifier close to a uniform distribution, aiming to reduce reliance on style cues. Across seven backbones, DFL generally achieves higher retrieval accuracy than zero-shot models.

Critical strengths
  1. The task is clearly defined, and MSIR-Bench separately evaluates generalization to new sources, styles, generators, and datasets. It provides a useful benchmark for comparing source retrieval methods for stylized images.

  2. DFL achieves higher Top-1 accuracy than a baseline trained only with the Source Discriminative Loss for all seven backbones. Results with DINOv2 are also stable across different random seeds.

Critical weaknesses
  1. Recent reference-conditioned generation models can change a person's pose or action, alter the composition, or combine multiple reference images. The experiments in this paper focus on stylization from a single source image, so it is unclear whether the method can identify the source images in these broader settings.

  2. The paper distinguishes image-level source retrieval from identity-level matching, but the experiments do not test this distinction. Since the portrait data use one source image per person, it is difficult to tell whether the model identifies the exact image or simply matches the person. An evaluation with other photos of the same person in the gallery is needed.

Questions for authors
  1. Please clarify whether the proposed method can be applied to generated images with changes in composition, pose, or action, to images generated using multiple references, and to domains beyond portraits and landscapes, such as character or product images. Related experimental results, if available, would be helpful.

  2. Please report retrieval accuracy when other photos of the same person are added to the gallery, and how often these photos are retrieved instead of the exact source image.

  3. Please clarify whether face detection, alignment, and cropping were applied to both source and query images when evaluating ArcFace and FaceNet512, and specify their input resolutions.

  4. Please explain how the styles were selected and divided into seen and unseen sets. It would also be helpful to explain how you checked whether generated images matched the target style and preserved source content, including any criteria for excluding or regenerating failed outputs.

3 comments
AuthorsOpenReview ↗

Response to Reviewer nbQC—Part 1 of 3

Response to W1 and Q1: Evaluation on Broader Reference-Conditioned Generation Settings

Thank you for your valuable suggestions. To evaluate DFL beyond single-source image stylization, we conducted six additional controlled experiments covering Pose, Action, Composition, Multi-reference, Character, and Product. The six settings include a total of 1,800 query images, which were further processed using unseen styles. All methods were evaluated using the same queries and galleries. The results are presented below.

Table R9. DINOv2 Source Image Retrieval Performance across Reference-Conditioned Generation Settings.

SettingSource Disc.DFLGain
Pose79.0081.002.00
Action78.0081.503.50
Composition76.6788.3311.66
Multi-reference51.3381.0029.67
Character77.7582.254.50
Product90.0091.501.50

Specifically, since the primary experiments use FFHQ face images as source images, the Pose setting mainly involves head pose changes, such as looking up, looking down, and turning the head to the side. The Action setting mainly involves facial expressions and local movements, such as smiling and blinking. The Composition setting examines retrieval under changes in image layout, while Character and Product assess retrieval performance in other image domains. For Multi-reference, we generate each query using two reference images, one of a person and one of a scene. We use Any-reference Top-1 as the evaluation metric, where a retrieval is considered correct if it matches either of the two input reference images. The other five settings use Exact-source Top-1.

As shown in Table R9, DFL with DINOv2 outperforms Source Disc. across all six settings. In particular, Composition and Multi-reference achieve improvements of 11.66 and 29.67 percentage points, respectively. These results indicate that DFL maintains its retrieval advantage over the baseline under composition changes, multi-reference image generation, and across different image domains.

It is worth noting that this work primarily focuses on Source Retrieval for Stylized Images, where a clear correspondence remains between the generated image and its original source image. Generated images involving substantial content changes that make the original source difficult to identify fall outside the main scope of this work. Nevertheless, the additional experiments show that DFL exhibits a certain degree of robustness to controlled pose, expression, and local action changes, while maintaining its retrieval advantage over the baseline under composition adjustments, multi-reference generation, and in broader image domains such as Character and Product. We will include these additional experiments and their evaluation protocols in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer nbQC—Part 2 of 3

Response to W2 and Q2: Exact-Source Retrieval and Same-Identity Confusion

Thank you for your valuable comments. Our retrieval objective is always to recover the exact source image corresponding to a generated query. The CelebA-HQ test gallery used in Table 3 of the previous manuscript contains different photos of the same person, with each photo treated as an independent source image during retrieval. Following your suggestion, we further analyzed how the presence of other photos of the same person affects retrieval performance.

We used DINOv2-B/14 as the visual backbone and compared Source Disc., trained using only the Source Discriminative Loss, with DFL, trained using the full DFL framework. The test set contains 1,000 source images and 2,000 stylized queries covering 761 identities. We grouped the queries according to the number of additional photos of the same person in the gallery and calculated two metrics:

  • Exact-source Top-1 (Exact): A retrieval is counted as correct only when the Top-1 result is the exact source image corresponding to the query. Accuracy is calculated as the number of correct retrievals divided by the total number of queries in each group.
  • Same-Identity Confusion Rate (Confusion): A retrieval is counted as a same-identity confusion when the Top-1 result is another photo of the same person rather than the exact source image. The confusion rate is calculated as the number of such incorrect retrievals divided by the total number of queries in each group.

Table R10. Exact-Source Retrieval and Same-Identity Confusion on CelebA-HQ.

Extra Same-ID PhotosQueriesSource Disc. ExactDFL ExactSource Disc. ConfusionDFL Confusion
01,15680.1984.000.000.00
156076.4380.890.710.36
218681.7283.871.611.08
≥39876.5379.591.020.00
Overall2,00079.1082.90.400.20

As shown in Table R10, Full DFL achieves an overall Exact-source Top-1 accuracy of 82.90%, improving upon Source Disc. by 3.80 percentage points, while reducing the Same-Identity Confusion Rate from 0.40% to 0.20%. Even when other photos of the same person are present in the gallery, DFL achieves higher exact-source retrieval accuracy across all groups.

These results further show that DFL not only matches the person's identity but can also distinguish between different photos of the same person and retrieve the exact source image corresponding to the generated query. We will include this statistical analysis in the revised manuscript to more clearly demonstrate how the existing evaluation protocol assesses image-level source retrieval.

Response to Q3: Face Preprocessing Protocol

Thank you for your suggestion. In the evaluation reported in Table 1 of the previous manuscript, ArcFace and FaceNet512 use the same preprocessing pipeline for both gallery source images and query images. Specifically, faces are detected using OpenCV and cropped according to the detected bounding boxes. If face detection fails, the full image is used as a fallback. The input resolutions are 112 × 112 for ArcFace and 160 × 160 for FaceNet512, respectively. For each backbone, Source Disc. and Full DFL use exactly the same preprocessing pipeline and input resolution to ensure a consistent comparison. We will clarify these details in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer nbQC—Part 3 of 3

Response to Q4: Style Selection and Generation Quality Assessment

Thank you for your valuable suggestion. We further clarify the style selection, Seen/Unseen split, and generation quality assessment procedures used in MSIR-Bench.

(1) Style Selection and Split

When selecting styles, we considered both visual diversity and source retrieval difficulty. MSIR-Bench contains 25 styles covering different types of visual transformations, including traditional painting, line art, cartoonization, material changes, and digital rendering, with varying levels of retrieval difficulty. For example, zero-shot DINOv2 achieves a Top-1 accuracy of only 5.00% on StreetArt, compared with 70.25% on ClassicOil. These styles cover a range of retrieval difficulties, from easy to challenging cases.

Based on these considerations, we selected 19 styles for training and Seen-style testing, while the remaining six styles were reserved for Unseen-style testing. The Unseen styles cover different visual types and retrieval difficulties, allowing us to evaluate the model's generalization ability to style variations not encountered during training.

(2) Generation Quality Assessment

During the construction of the final test set, we conducted a Mean Absolute Error (MAE) analysis on the generated images, covering three dataset domains, five generators, and 15 test styles. We analyzed the MAE distributions across different styles and generators to identify the typical range of image changes within each group, and applied different thresholds for different generators and styles. Stylized images with MAE values below the corresponding thresholds were considered too similar to their source images, potentially indicating that the stylization instructions had not been properly followed, and were therefore excluded and regenerated. Images with MAE values above the corresponding thresholds were subjected to manual review, and those that failed the quality assessment were also regenerated.

The manual review mainly focused on whether the generated images matched the target styles and preserved the source content. We examined cases of insufficient stylization where the target style was not adequately reflected, as well as cases of excessive stylization that caused subject deformation or the loss of important structures or content. In addition, we conducted a final random inspection of the generated results to further assess overall generation quality. This process combines automated anomaly detection with manual quality assessment.

Reviewer_idj8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a method to retrieve the original image from a stylized image . To achieve the goal, it introduces MSIR-Bench, containing 56,000 images from 5,500 sources, then a Destylized Feature Learning (DFL) method has been proposed. DFL combines normalized source classification, a triplet objective using source–style relationships, and a regularizer that encourages uniform predictions from a frozen style classifier. Experiments evaluate seven backbones, unseen sources and styles, transfer across generators and datasets, and robustness to post-processing.

Critical strengths

The main contribution comes from the benchmark which provides a useful starting point for protecting the copyright and privacy in generative editing. The benchmark contains 56,000 images from 5,500 sources, which is a decent number for the community.

The experiments and ablation studies is comprehensive. For example, improvements over source-classification fine-tuning are consistent across all seven backbones in Table 1. Unseen-style accuracy improves from 80.23% to 84.10% and in Table 3 it improves from 73.94% to 78.62%.

Critical weaknesses

The proposed method is a similar approach for identity ( face) recognition, in a destylization manner. The existing face-recognition baselines is not a fair comparison. [1] already proposed a general approach for destylize any stylization image. Although its task and generative formulation differ from the proposed retrieval framework, the conceptual relationship should be same and apply to the baseline for face recongintion.

Baseline suggestion: Stylized query → OmniStyle2’s destylization component → ArcFace/FaceNet embedding → search the original-image gallery.

Unlike this existing work, it is unclear that the proposed classifier has removed the style. Moreover, given existing style transfer methods, do we guarantee all kinds of style can be removed clearly?

[1] OmniStyle2: Learning to Stylize by Learning to Destylize

Questions for authors

How does the proposed method with strong face recognition+ image destylization baseline? How dose the methods handle extreme difficult styles? For Impressionism, the paper does not provide enough evidence to judge extreme cases. When we refer to the table, the methods is not robust to different styles such as classic oil and street art.

2 comments
AuthorsOpenReview ↗

Response to Reviewer idj8—Part 1 of 2

Response to W1 and Q1: Destylization Baselines, Style Suppression, and Generalization

Thank you for your valuable suggestion. We conducted additional experiments on destylization-based face recognition baselines and further examined the cross-generator and unseen-style evaluation results reported in the previous manuscript.

(1) Destylization-Based Retrieval

Following your suggestion, we evaluated a destylization-based face recognition baseline using the following pipeline: Stylized query → OmniStyle2's destylization component → ArcFace/FaceNet embedding → Search the original-image gallery.

We also conducted an additional destylization experiment using Qwen-Image-Edit and included direct retrieval using the original stylized queries as a baseline to evaluate the effect of destylization on retrieval performance.

Specifically, we used 2,000 stylized queries from FFHQ to evaluate both pretrained ArcFace and FaceNet512 and their variants fine-tuned with the Source Discriminative Loss. For each model, we compared three settings: Direct Retrieval, OmniStyle2 Destylize-then-Retrieve, and Qwen-Image-Edit Destylize-then-Retrieve. The results are reported in Table R7.

Table R7. Source Image Retrieval Performance of ArcFace and FaceNet512 with Query Destylization.

Embedding ModelDirect RetrievalOmniStyle2 Destylize-then-RetrieveQwen Destylize-then-Retrieve
ArcFace (Pretrained)45.0547.0543.55
ArcFace (Fine-tuned)53.5553.1548.3
FaceNet512 (Pretrained)47.64943.7
FaceNet512 (Fine-tuned)57.9557.1550.85

As shown in Table R7, OmniStyle2 destylization slightly improves retrieval accuracy for both pretrained face recognition models. However, for models fine-tuned with the Source Discriminative Loss, retrieval performance remains nearly the same as direct retrieval. After destylization with Qwen-Image-Edit, retrieval accuracy decreases for all four model variants. Overall, explicit destylization provides only limited improvements in the retrieval performance of face recognition baselines. One possible reason is that stylization substantially changes image textures, colors, and local structures, making it difficult for destylization models to accurately recover details from the source images, which may affect subsequent retrieval performance. We will include these baseline results and the corresponding discussion in the revised manuscript.

(2) Effect of DFL on Style Information

Regarding your concern about style removal, DFL aims to reduce the interference of style information in retrieval by learning style-robust source image features. To examine this effect, we trained an independent style classifier on the final learned features, following the style classifier training settings in the previous manuscript.

Specifically, we trained a fresh style classifier on the final DFL features and evaluated its style classification accuracy, as reported in the "DFL" column of Table R8. To better demonstrate the effect of the destylization components in DFL, we trained another style classifier for comparison under the same experimental settings. This classifier was trained on retrieval features obtained using only the Source Discriminative Loss, without the destylization components, as reported in the "Source Disc." column of Table R8. If the style classification accuracy in the "DFL" column is lower than the corresponding accuracy in the "Source Disc." column, it indicates that the destylization components in DFL reduce the influence of style information to some extent.

Table R8. Style Classification Top-1 Accuracy on Final Retrieval Features.

BackboneSource Disc.DFLReduction
DINOv2-B44.98%34.30%10.68%
MAE62.26%36.22%26.04%
ResNet5075.67%28.87%46.80%
CLIP ViT-B/3276.62%61.49%15.13%
Avg.64.88%40.22%24.66%

As shown in Table R8, DFL achieves lower style classification accuracy than Source Disc. across all four backbones, with reductions ranging from 10.68% to 46.80% and an average reduction of 24.66%. These results indicate that the destylization components in DFL reduce the influence of style information in the final retrieval features to some extent.

AuthorsOpenReview ↗

Response to Reviewer idj8—Part 2 of 2

Response to W1 and Q1: Destylization Baselines, Style Suppression, and Generalization

(3) Generalization Across Different Generators and Styles

To further evaluate DFL's ability to handle different style transfer methods and style variations, we analyzed the cross-generator and per-style evaluation results reported in the previously submitted manuscript.

Table 3 of the previous manuscript evaluates DFL across five generators: Qwen, BAGEL, FLUX.2, OmniGen2, and Step1X. DFL outperforms Source Disc. across all five generators, with an average improvement of 4.68%. In particular, the improvements on BAGEL, FLUX.2, and Step1X are 5.38%, 5.32%, and 7.64%, respectively. In addition, the per-style evaluation in Table 11 of the previous manuscript shows that DFL also achieves the highest retrieval accuracy across all six unseen styles.

These results indicate that DFL demonstrates a certain degree of generalization ability across different generative models and unseen styles. As generative models continue to evolve, more unseen styles may emerge. Since our method has not yet been evaluated on these future unknown styles, its generalization ability to such styles remains to be further verified.

Response to Q2: Performance on Challenging Styles

Thank you for the valuable comments. DFL addresses challenging styles through training on diverse styles and learning destylized features. Table 11 of the previous manuscript evaluates DFL on six unseen styles. For ClassicOil and StreetArt, which you specifically mentioned, DFL improves Top-1 retrieval accuracy by 7.54% and 16.02%, respectively, over the strongest baseline. In addition, on Bauhaus, where zero-shot DINOv2 achieves a Top-1 accuracy of only 11.76%, DFL reaches 84.45%, outperforming the strongest baseline by 54.54%. These results show that DFL achieves substantial improvements over baseline methods on challenging unseen styles.

In fact, the primary motivation of this work is to provide an initial step toward addressing this challenging task. First, we introduce the task of Source Retrieval for Stylized Images and construct MSIR-Bench to systematically assess the out-of-the-box retrieval capabilities of existing pretrained visual backbones on this task. Since its anonymous release, MSIR-Bench has been downloaded 17,859 times, suggesting notable interest from the research community. Second, we introduce DFL to further boost retrieval accuracy on top of representative visual backbones. We expect this work to serve as a useful basis for the community and to spur additional research on this task.

Reviewer_LMjFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Current methods can detect copyright-generated content and identify the generator used to create the output image. However, no model indicates which image in a gallery is most important in the output guidance process. To address this, the authors propose a pipeline that learns features that discriminate the source image without being biased toward image style. The authors propose a two-stage framework: first, they train a style classifier, and then they fine-tune the last layers of a visual encoder and an embedding head using a loss composed of three terms: a cross-entropy term that enforces source discrimination, a regularization term that prevents the features from being biased toward style, and a contrastive loss term that encourages feature similarity based on the image source rather than style. The approach shows good results on the source retrieval task, generalizes across different visual encoders, and is robust to image degradation in the source gallery images.

Critical strengths
  • The paper is easy to read and clearly presents the problem it addresses. In addition, the proposed method is easy to follow and understand.
  • The authors show results with different visual encoders as backbones and on different datasets. This shows that the model can generalize to different feature environments.
  • The model shows robust source image retrieval even in difficult conditions, when the reference gallery contains degraded images.
Critical weaknesses
  • The authors do not show clear visual examples for the source image retrieval task; a section with qualitative results is necessary.
  • The visual encoders used for the comparison are quite old. The authors should provide results with more recent visual encoders, such as DINOv3.
  • In the supplementary material, the authors provide results across different seeds, but similar results should be included in the main table, reporting the standard deviation.
Questions for authors
  • The approach requires access to the training data for the image retrieval task. To clearly show its performance, the authors should train a small generative model from scratch on a controlled dataset and evaluate the framework in a controlled setting.
2 comments
AuthorsOpenReview ↗

Response to Reviewer LMjF—Part 1 of 2

Response to W2: Evaluation of DFL on DINOv3

Thank you for the valuable suggestion. We conducted additional experiments using DINOv3 ViT-B/16, comparing three settings: Zero-shot, Source Disc., and Full DFL. "Zero-shot" refers to the retrieval accuracy obtained by directly using pretrained DINOv3 features without task-specific training. "Source Disc." refers to the retrieval accuracy after fine-tuning DINOv3 with the Source Discriminative Loss. "DFL" refers to the retrieval accuracy after training DINOv3 with the full DFL framework, which jointly employs Source Discriminative Loss, Destylized Triplet Loss, and Style Confusion Regularization. We used the same training images and evaluation protocol as the DINOv2 experiments in the original manuscript to examine the performance of DFL on a more recent visual encoder. The results are presented in Table R6.

Table R6. Source Image Retrieval Performance with DINOv3 ViT-B/16.

MethodSeenUnseenAvg.
Zero-shot66.9466.466.67
Source Disc.95.0882.9389.01
Full DFL96.3184.4390.37

Full DFL achieves higher retrieval performance than Source Disc. This result is consistent with the trend observed on DINOv2, indicating that DFL can also be applied to more recent visual encoders and further improve source image retrieval performance.

The magnitude of improvement, however, varies across different visual backbones. As shown in Table 1 of the original manuscript, DFL improves the overall retrieval accuracy by 6.98% for the widely used CLIP model and by 8.69% for SigLIP. Therefore, the overall performance gains of DFL vary depending on the backbone used.

Response to W1 and W3: Visualization and Multi-Seed Results

Thank you for your valuable suggestions. We have added some visualization examples of source image retrieval to the anonymous link provided in the abstract. We will also include clearer and more intuitive visualization examples of source image retrieval in the appendix of the revised manuscript.

Regarding your suggestion to include multi-seed results and standard deviations in the main table, we have already reported some multi-seed results in the appendix and would also like to extend this evaluation to all seven backbones and two training configurations in Table 1. However, this would require 7 × 2 × n training runs for n random seeds, making it challenging to complete all the experiments within the limited rebuttal period. We will follow your suggestion to include multi-seed evaluation results and standard deviations for all seven visual backbones in Table 1 of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer LMjF—Part 2 of 2

Response to Q1: Controlled Experiment with a Generator Trained from Scratch

Thank you for the valuable suggestion. We conducted an additional experiment using a generative model trained from scratch. Together with the cross-generator evaluation in Table 3 of the previous manuscript, this experiment allows us to evaluate DFL's retrieval performance on images generated by different generators.

(a) Controlled Experiment with a Generator Trained from Scratch

We constructed paired images using FFHQ source images and their corresponding stylized images, and trained a conditional Pix2Pix model supporting six styles unseen during retrieval model training from scratch with random initialization, without using any pretrained generator weights. The training, validation, and test sets contained 1,800, 200, and 1,000 source identities, respectively, with no overlap between the three sets.

At test time, Pix2Pix generated 2,000 queries from 1,000 test source images, with the corresponding 1,000 source images serving as the retrieval gallery. Neither DINOv2 with Source Disc. nor Full DFL was retrained or fine-tuned for this experiment. Full DFL achieved a Top-1 retrieval accuracy of 76.65%, outperforming Source Disc. (67.55%) by 9.10%. This indicates that DFL maintains its retrieval advantage on queries generated by a model trained from scratch.

(b) Cross-Generator Generalization Evaluation in Table 3

Table 3 of the previous manuscript evaluates source retrieval performance on stylized images generated by five generators: Qwen, FLUX.2, OmniGen2, BAGEL, and Step1X. The evaluation includes a total of 30,000 query images from three datasets: FFHQ, CelebA-HQ, and LHQ. To evaluate cross-generator generalization, we trained the model using only images generated by Qwen-Image-Edit and tested it on the other four unseen generators.

As shown in Table 3, DFL outperforms Source Disc. across all five generators, with an average improvement of 4.68%. In particular, the improvements on BAGEL, FLUX.2, and Step1X are 5.38%, 5.32%, and 7.64%, respectively. These results demonstrate the cross-generator generalization ability of DFL.

Together with the additional controlled Pix2Pix experiment, these results show that DFL's performance advantage also holds for generators trained from scratch and other generators unseen during retrieval model training.

Reviewer_BZQaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The task is to find the exact original of a photo that has been restyled by an instruction-based editor. The authors build MSIR-Bench for it: 5,500 sources, 25 styles, five editors, 56K images. It covers seen and unseen styles as well as cross-generator, cross-dataset, robustness and open-set settings. Their method, DFL, fine-tunes a pretrained backbone with a source classification loss, a cross-style triplet loss and a style-confusion term. This adds 2 to 9 Top-1 points over classification-only fine-tuning (DINOv2: 95.7 seen / 84.1 unseen). Zero-shot SSCD, DreamSim and LPIPS are all around 62%.

Critical strengths
  1. The benchmark. Exact-source retrieval under generative restyling is a real provenance problem and there was no testbed for it. MSIR-Bench is well designed: source, style, generator and domain shifts are evaluated separately, and the robustness, scaling and open-set protocols are a nice addition. I think this is the real contribution of the paper.

  2. The results look trustworthy. DFL beats the classification-only baseline on all seven backbones, the five-seed std is 0.12/0.23, gains carry over to five generators and three datasets (+4.7 on average), and the failure analysis (338 queries fixed, 106 regressed) is reported honestly.

Critical weaknesses
  1. The method is a standard metric-learning recipe: NormFace-style classification, a triplet loss, a confusion regularizer and a hard-negative curriculum. The parts that are actually new buy less than a point each. The destylized triplet beats a plain triplet by 0.66 (Table 7). Style confusion adds ~0.8 on top of the rest (Table 2), the curriculum 0.58 (Table 9). The full method is 3.2 points over classification-only fine-tuning on DINOv2 (Table 1). That's a systems contribution, not a method.

  2. The headline comparison doesn't show what it claims. (a) Zero-shot accuracy is higher on unseen than seen styles (DINOv2 49.8 vs. 37.3). The unseen styles are simply easier, so 95.7 vs. 84.1 is not a generalization gap. (b) SSCD is only compared zero-shot. It could be fine-tuned with the same loss. (c) The obvious baseline is missing: destylize the query with the same editor, then retrieve zero-shot. (d) All primary queries come from one generator. The closed-model test has 80 queries.

These two points are why I'm at weak reject. It's a solid benchmark report, but the modeling part and the central comparison don't meet the bar yet.

Questions for authors
  1. (Clarification) Why is zero-shot accuracy higher on unseen than seen styles? Please add a difficulty-matched control, for example a subset of seen styles whose zero-shot difficulty matches the six unseen ones.

  2. (Clarification) Please report (a) a destylize-then-retrieve baseline (Qwen-Image-Edit with "convert back to a realistic photo", then zero-shot SSCD/DINOv2) and (b) SSCD fine-tuned on the same training set with the Source-Disc. loss.

  3. (Clarification) The style-confusion classifier is frozen after Phase I. If you train a fresh style classifier on the final features, what accuracy does it get? That would tell us whether style was actually removed or the model just learned to fool a stale classifier.

  4. (Clarification) Several differences are under one point (Table 2: Des.-Triplet-only vs. Source-Disc.-only; all rows of Table 9). Please give multi-seed std for these.

  5. (Suggestion) Results with a much larger gallery (50K+, e.g. all of FFHQ) and a larger backbone (ViT-L) would bring this closer to the provenance scenario the paper is motivated by.

4 comments
AuthorsOpenReview ↗

Response to Reviewer BZQa—Part 1 of 4

Response to W1: About Contribution

Thank you for the valuable comment. The performance gains of the proposed Destylized Feature Learning (DFL) framework vary by visual backbone. As shown in Table 1 of the manuscript, DFL improves overall retrieval accuracy by 3.19% with DINOv2, 6.98% with Meta’s CLIP, and 8.69% with Google DeepMind’s SigLIP. The impact of individual components also depends on the backbone: for CLIP (Table 2), Destylized Triplet Loss improves accuracy by 5.76%, while Style Confusion Regularization adds 2.10%. These results show that DFL’s overall and component-level gains depend on the backbone.

The DFL framework is proposed as an initial attempt to address Source Retrieval for Stylized Images, with the aim of drawing attention to this task. We have evaluated the performance of representative visual backbones on this task, and their low retrieval accuracy underscores its difficulty. We also propose a general method to improve these backbones, providing a stronger baseline for future work.

Another important contribution of this work is the MSIR-Bench. MSIR-Bench comprises 5,500 source images with unique identities, spans five advanced generative editing models, and covers 25 artistic styles. Since its anonymous release on Hugging Face in mid-May 2026, MSIR-Bench had received 17,859 downloads as of November 18, 2026. A screenshot of the download counts is accessible through the anonymous link in the abstract. The download statistics suggest that the task and dataset have attracted attention from the research community and have been adopted in related studies.

In summary, the main contributions of this work are:

  1. We present the challenging task of Source Retrieval for Stylized Images.
  2. We construct MSIR-Bench, a benchmark for this task, and conduct a systematic evaluation of widely used visual backbones.
  3. We propose the Destylized Feature Learning (DFL) framework, which improves the performance of general-purpose visual representations on this task to some extent.

We appreciate the reviewer’s helpful comments. In the revised manuscript, we will further elaborate on our research motivation, contributions, and limitations in the Introduction, Method, and Conclusion sections.

Response to Q1 and W2(a): Evaluation under Style Difficulty Matching

Thanks for the constructive suggestion. As suggested, we ran a difficulty-matched control experiment by pairing Seen and Unseen styles with similar zero-shot retrieval accuracy. Since no Seen style matched the retrieval difficulty of the Unseen StreetArt style, we ran the control experiment using five Seen–Unseen style pairs with similar difficulty. Table R1 reports the average retrieval accuracy across these five matched Seen styles and five matched Unseen styles. "Zero-shot" denotes retrieval using pretrained DINOv2 features without task-specific training, and "DFL" denotes retrieval after training DINOv2 with the full DFL framework.

Table R1. Difficulty-matched retrieval results for Seen and Unseen styles using DINOv2.

Test SetZero-shotDFL
Matched Seen Styles60.5597.62
Matched Unseen Styles61.5988.52

As shown in Table R1, after matching difficulty, zero-shot retrieval accuracy differs by 1.04% between the two groups, indicating similar initial retrieval difficulty. After DFL training, DINOv2 achieves 97.62% (Seen) and 88.52% (Unseen), a 9.10% gap, similar to the 11.61% gap (95.7% vs. 84.1%) reported in Table 1 of the original manuscript. These results indicate that, after matching style difficulty, the substantially lower retrieval accuracy on Unseen styles compared with Seen styles is mainly due to the generalization gap.

We thank you again for this suggestion and will add a discussion of style difficulty matching to the experimental section of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer BZQa—Part 2 of 4

Response to Q2 and W2(b)(c): Destylize-then-Retrieve and SSCD Fine-Tuning Experiments

Thank you for the helpful suggestion. As requested, we ran two additional experiments: a destylize-then-retrieve baseline and SSCD fine-tuning. We will report these results in the revised manuscript.

(1) Destylize-then-Retrieve Baseline

As suggested, we used Qwen-Image-Edit with the prompt "convert back to a realistic photo" to destylize 4,000 stylized FFHQ and CelebA-HQ images generated by the same editor. We then used zero-shot SSCD and DINOv2 to retrieve the source images from these destylized images. Retrieval accuracies are reported in Table R2.

Table R2. Comparison of Direct Retrieval and Same-Editor Destylization-Based Retrieval.

DatasetBackboneDirect RetrievalDestylize-then-Retrieve
FFHQDINOv260.760.9
SSCD64.664.25
CelebA-HQDINOv240.2536.25
SSCD66.7556.95
Avg.58.0854.59

As shown in Table R2, the Destylize-then-Retrieve baseline achieves nearly identical retrieval accuracy to the Direct Retrieval baseline on FFHQ. On CelebA-HQ, however, its retrieval accuracy is 4.0% to 9.8% lower than that of Direct Retrieval. These results indicate that destylization before retrieval brings little improvement or even negatively affects retrieval performance. One possible reason is that stylization substantially changes image textures, colors, and local structures, creating large differences between stylized images and their original sources. As a result, the destylization process may struggle to effectively recover information from the source image, thereby affecting subsequent retrieval accuracy.

We will also add these Destylize-then-Retrieve baseline results and the corresponding discussion in the revised manuscript.

(2) SSCD Fine-Tuning Experiment

As suggested, we fine-tuned SSCD on the same training set using the Source Discriminative Loss. The retrieval accuracy of fine-tuned SSCD increased from 62.82% to 69.55%. We will report these results in Table 8 of the revised manuscript.

Response to W2(d): Cross-Generator Generalization

Thank you for the comment. Table 3 of the original manuscript evaluates source retrieval performance on stylized images generated by five generators: Qwen, FLUX.2, OmniGen2, BAGEL, and Step1X. The evaluation includes a total of 30,000 query images from three datasets: FFHQ, CelebA-HQ, and LHQ. To evaluate cross-generator generalization, we trained the model using only images generated by Qwen-Image-Edit and tested it on the other four unseen generators.

As shown in Table 3, DFL outperforms Source Disc. across all generators, with an average improvement of 4.68%. In particular, the improvements on BAGEL, FLUX.2, and Step1X are 5.38%, 5.32%, and 7.64%, respectively. These results demonstrate the cross-generator generalization ability of DFL.

Our generalization experiments are primarily conducted using open-source models. The small-scale query experiment reported in Appendix C.8 is mainly intended to supplement the evaluation of DFL's retrieval performance on closed-source models and provide corresponding visualization examples. We will further clarify the purpose of this experiment and its relationship to Table 3 in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer BZQa—Part 3 of 4

Response to Q3: Style Classification Experiment

We thank the reviewer for the valuable suggestion. Following your suggestion, we trained a fresh style classifier on the final DFL features using the same training settings as the style classifier in the original manuscript, and evaluated its style classification accuracy, as reported in the "DFL" column of Table R3. To better demonstrate the effect of the destylization design in DFL, we trained another style classifier for comparison under the same experimental settings. This classifier was trained on the retrieval features obtained using only the Source Discriminative Loss, without the destylization components, and its results are reported in the "Source Disc." column of Table R3. If the style classification accuracy in the "DFL" column is lower than the corresponding accuracy in the "Source Disc." column, it indicates that the destylization components in DFL reduce the influence of style information to some extent.

Table R3. Style Classification Top-1 Accuracy on Final Retrieval Features.

BackboneSource Disc.DFLReduction
DINOv2-B44.98%34.30%10.68%
MAE62.26%36.22%26.04%
ResNet5075.67%28.87%46.80%
CLIP ViT-B/3276.62%61.49%15.13%
Avg.64.88%40.22%24.66%

As shown in Table R3, the style classification accuracy of DFL is lower than that of Source Disc. across all four backbones, with reductions ranging from 10.68% to 46.80% and an average reduction of 24.66%. These results indicate that the destylization components in DFL reduce the influence of style information in the final retrieval features to some extent.

Response to Q4: Multi-Seed Evaluation

Thank you for the valuable suggestion. We conducted additional experiments for Tables 2 and 9 of the original manuscript using five random seeds and report the mean and standard deviation (mean ± std) of Top-1 retrieval accuracy. The results are presented below.

(1) Table 2: Component Ablation Study

MethodSeenUnseenAvg.
Source Disc. only94.03 ± 0.4580.54 ± 0.3987.29 ± 0.39
Des. Triplet only92.24 ± 0.5479.69 ± 0.4385.97 ± 0.46
Source Disc. + Sty. Con.93.86 ± 0.4382.45 ± 0.6988.16 ± 0.22
Source Disc. + Des. Triplet95.63 ± 0.1982.92 ± 0.3689.28 ± 0.18
Full DFL95.92 ± 0.1283.69 ± 0.2389.81 ± 0.10

(2) Table 9: Hard Negative Curriculum Learning

Switching EpochSeenUnseenAvg.
094.96 ± 0.5383.00 ± 0.6788.98 ± 0.20
595.25 ± 0.1282.61 ± 0.4488.93 ± 0.17
1095.92 ± 0.1283.69 ± 0.2389.81 ± 0.10
1595.17 ± 0.2282.62 ± 0.2888.89 ± 0.17
2095.11 ± 0.1382.73 ± 0.4388.92 ± 0.24

The multi-seed evaluation results show that Full DFL maintains the highest mean Top-1 retrieval accuracy among all configurations in Table 2. For Table 9, the switching strategy at epoch 10 adopted in the original manuscript also achieves the highest mean accuracy, outperforming the other tested switching epochs. We will report these results in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer BZQa—Part 4 of 4

Response to Q5: Evaluation with a Larger Backbone and Gallery

Thank you for the valuable suggestion. We conducted two additional experiments. The first evaluates source retrieval for stylized images using a larger backbone, DINOv2-L/14 (ViT-L), while the second evaluates source retrieval with a larger gallery.

(1) Evaluation with a Larger Backbone

Using the same training data and retrieval evaluation protocol as in the original manuscript, we scaled up the visual backbone from DINOv2-B/14 (86M parameters) to DINOv2-L/14 (300M parameters) and compared the performance of Zero-shot, Source Disc., and Full DFL. The results are presented below.

Table R4. Top-1 Accuracy of Source Retrieval with DINOv2-L/14 (ViT-L).

DINOv2-L/14Seen Top-1Unseen Top-1Avg.
Zero-shot51.5853.9752.77
Source Disc.92.8881.9287.4
DFL96.3385.0390.68

As shown in Table R4, Full DFL achieves an average improvement of 3.28% over Source Disc. on DINOv2-L/14. These results indicate that DFL is also effective on larger visual backbones, demonstrating its applicability across different model scales.

(2) Evaluation on a Larger Gallery

Thank you for your suggestion. In this additional experiment, we first expanded the retrieval gallery from 3,000 to 10,000 images by adding 7,000 FFHQ images as distractors while retaining the original 3,000 ground-truth source images. We evaluated DFL on this expanded gallery using the same 6,000 Unseen queries as in the original manuscript. We also plan to further expand the gallery to 50K+ images in future work.

Table R5. DFL Retrieval Performance with Different Gallery Sizes.

GalleryTop-1Top-5Top-10
Original gallery (3,000)84.1089.2391.05
Larger gallery (10,000)81.2286.3088.08

As shown in Table R5, when the gallery size increases from 3,000 to 10,000 images (3.33 times the original size), DFL's Top-1 retrieval accuracy decreases from 84.10% to 81.22%, a reduction of 2.88%. This indicates that DFL's retrieval performance decreases to some extent as the number of candidate images increases, but it still maintains relatively high source retrieval accuracy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.