- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to retrieve the exact source image used to generate a stylized image. MSIR-Bench contains 56K images, including 5,500 source images, and evaluates generalization to new sources, styles, generators, and datasets. DFL jointly trains the final backbone blocks and the embedding head using the Source Discriminative Loss, Destylized Triplet Loss, and Style Confusion Regularization. It brings features from different styles of the same source closer together and separates features from different sources. It also trains the encoder to make the predictions of a frozen style classifier close to a uniform distribution, aiming to reduce reliance on style cues. Across seven backbones, DFL generally achieves higher retrieval accuracy than zero-shot models.
Critical strengths
-
The task is clearly defined, and MSIR-Bench separately evaluates generalization to new sources, styles, generators, and datasets. It provides a useful benchmark for comparing source retrieval methods for stylized images.
-
DFL achieves higher Top-1 accuracy than a baseline trained only with the Source Discriminative Loss for all seven backbones. Results with DINOv2 are also stable across different random seeds.
Critical weaknesses
-
Recent reference-conditioned generation models can change a person's pose or action, alter the composition, or combine multiple reference images. The experiments in this paper focus on stylization from a single source image, so it is unclear whether the method can identify the source images in these broader settings.
-
The paper distinguishes image-level source retrieval from identity-level matching, but the experiments do not test this distinction. Since the portrait data use one source image per person, it is difficult to tell whether the model identifies the exact image or simply matches the person. An evaluation with other photos of the same person in the gallery is needed.
Questions for authors
-
Please clarify whether the proposed method can be applied to generated images with changes in composition, pose, or action, to images generated using multiple references, and to domains beyond portraits and landscapes, such as character or product images. Related experimental results, if available, would be helpful.
-
Please report retrieval accuracy when other photos of the same person are added to the gallery, and how often these photos are retrieved instead of the exact source image.
-
Please clarify whether face detection, alignment, and cropping were applied to both source and query images when evaluating ArcFace and FaceNet512, and specify their input resolutions.
-
Please explain how the styles were selected and divided into seen and unseen sets. It would also be helpful to explain how you checked whether generated images matched the target style and preserved source content, including any criteria for excluding or regenerating failed outputs.
3 comments
Response to Reviewer nbQC—Part 2 of 3
Response to W2 and Q2: Exact-Source Retrieval and Same-Identity Confusion
Thank you for your valuable comments. Our retrieval objective is always to recover the exact source image corresponding to a generated query. The CelebA-HQ test gallery used in Table 3 of the previous manuscript contains different photos of the same person, with each photo treated as an independent source image during retrieval. Following your suggestion, we further analyzed how the presence of other photos of the same person affects retrieval performance.
We used DINOv2-B/14 as the visual backbone and compared Source Disc., trained using only the Source Discriminative Loss, with DFL, trained using the full DFL framework. The test set contains 1,000 source images and 2,000 stylized queries covering 761 identities. We grouped the queries according to the number of additional photos of the same person in the gallery and calculated two metrics:
- Exact-source Top-1 (Exact): A retrieval is counted as correct only when the Top-1 result is the exact source image corresponding to the query. Accuracy is calculated as the number of correct retrievals divided by the total number of queries in each group.
- Same-Identity Confusion Rate (Confusion): A retrieval is counted as a same-identity confusion when the Top-1 result is another photo of the same person rather than the exact source image. The confusion rate is calculated as the number of such incorrect retrievals divided by the total number of queries in each group.
Table R10. Exact-Source Retrieval and Same-Identity Confusion on CelebA-HQ.
| Extra Same-ID Photos | Queries | Source Disc. Exact | DFL Exact | Source Disc. Confusion | DFL Confusion |
|---|---|---|---|---|---|
| 0 | 1,156 | 80.19 | 84.00 | 0.00 | 0.00 |
| 1 | 560 | 76.43 | 80.89 | 0.71 | 0.36 |
| 2 | 186 | 81.72 | 83.87 | 1.61 | 1.08 |
| ≥3 | 98 | 76.53 | 79.59 | 1.02 | 0.00 |
| Overall | 2,000 | 79.10 | 82.9 | 0.40 | 0.20 |
As shown in Table R10, Full DFL achieves an overall Exact-source Top-1 accuracy of 82.90%, improving upon Source Disc. by 3.80 percentage points, while reducing the Same-Identity Confusion Rate from 0.40% to 0.20%. Even when other photos of the same person are present in the gallery, DFL achieves higher exact-source retrieval accuracy across all groups.
These results further show that DFL not only matches the person's identity but can also distinguish between different photos of the same person and retrieve the exact source image corresponding to the generated query. We will include this statistical analysis in the revised manuscript to more clearly demonstrate how the existing evaluation protocol assesses image-level source retrieval.
Response to Q3: Face Preprocessing Protocol
Thank you for your suggestion. In the evaluation reported in Table 1 of the previous manuscript, ArcFace and FaceNet512 use the same preprocessing pipeline for both gallery source images and query images. Specifically, faces are detected using OpenCV and cropped according to the detected bounding boxes. If face detection fails, the full image is used as a fallback. The input resolutions are 112 × 112 for ArcFace and 160 × 160 for FaceNet512, respectively. For each backbone, Source Disc. and Full DFL use exactly the same preprocessing pipeline and input resolution to ensure a consistent comparison. We will clarify these details in the revised manuscript.
Response to Reviewer nbQC—Part 3 of 3
Response to Q4: Style Selection and Generation Quality Assessment
Thank you for your valuable suggestion. We further clarify the style selection, Seen/Unseen split, and generation quality assessment procedures used in MSIR-Bench.
(1) Style Selection and Split
When selecting styles, we considered both visual diversity and source retrieval difficulty. MSIR-Bench contains 25 styles covering different types of visual transformations, including traditional painting, line art, cartoonization, material changes, and digital rendering, with varying levels of retrieval difficulty. For example, zero-shot DINOv2 achieves a Top-1 accuracy of only 5.00% on StreetArt, compared with 70.25% on ClassicOil. These styles cover a range of retrieval difficulties, from easy to challenging cases.
Based on these considerations, we selected 19 styles for training and Seen-style testing, while the remaining six styles were reserved for Unseen-style testing. The Unseen styles cover different visual types and retrieval difficulties, allowing us to evaluate the model's generalization ability to style variations not encountered during training.
(2) Generation Quality Assessment
During the construction of the final test set, we conducted a Mean Absolute Error (MAE) analysis on the generated images, covering three dataset domains, five generators, and 15 test styles. We analyzed the MAE distributions across different styles and generators to identify the typical range of image changes within each group, and applied different thresholds for different generators and styles. Stylized images with MAE values below the corresponding thresholds were considered too similar to their source images, potentially indicating that the stylization instructions had not been properly followed, and were therefore excluded and regenerated. Images with MAE values above the corresponding thresholds were subjected to manual review, and those that failed the quality assessment were also regenerated.
The manual review mainly focused on whether the generated images matched the target styles and preserved the source content. We examined cases of insufficient stylization where the target style was not adequately reflected, as well as cases of excessive stylization that caused subject deformation or the loss of important structures or content. In addition, we conducted a final random inspection of the generated results to further assess overall generation quality. This process combines automated anomaly detection with manual quality assessment.
Response to Reviewer nbQC—Part 1 of 3
Response to W1 and Q1: Evaluation on Broader Reference-Conditioned Generation Settings
Thank you for your valuable suggestions. To evaluate DFL beyond single-source image stylization, we conducted six additional controlled experiments covering Pose, Action, Composition, Multi-reference, Character, and Product. The six settings include a total of 1,800 query images, which were further processed using unseen styles. All methods were evaluated using the same queries and galleries. The results are presented below.
Table R9. DINOv2 Source Image Retrieval Performance across Reference-Conditioned Generation Settings.
Specifically, since the primary experiments use FFHQ face images as source images, the Pose setting mainly involves head pose changes, such as looking up, looking down, and turning the head to the side. The Action setting mainly involves facial expressions and local movements, such as smiling and blinking. The Composition setting examines retrieval under changes in image layout, while Character and Product assess retrieval performance in other image domains. For Multi-reference, we generate each query using two reference images, one of a person and one of a scene. We use Any-reference Top-1 as the evaluation metric, where a retrieval is considered correct if it matches either of the two input reference images. The other five settings use Exact-source Top-1.
As shown in Table R9, DFL with DINOv2 outperforms Source Disc. across all six settings. In particular, Composition and Multi-reference achieve improvements of 11.66 and 29.67 percentage points, respectively. These results indicate that DFL maintains its retrieval advantage over the baseline under composition changes, multi-reference image generation, and across different image domains.
It is worth noting that this work primarily focuses on Source Retrieval for Stylized Images, where a clear correspondence remains between the generated image and its original source image. Generated images involving substantial content changes that make the original source difficult to identify fall outside the main scope of this work. Nevertheless, the additional experiments show that DFL exhibits a certain degree of robustness to controlled pose, expression, and local action changes, while maintaining its retrieval advantage over the baseline under composition adjustments, multi-reference generation, and in broader image domains such as Character and Product. We will include these additional experiments and their evaluation protocols in the revised manuscript.