acceptodds
Under review as a conference paper at ICLR 2027

SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Auditing

Abstract

Auditing whether a text-to-image (T2I) model was fine-tuned on a particular image collection is difficult: a model can absorb collection-level visual characteristics without reproducing training images. Proactive dataset-use auditing plants a shared signal in a dataset before release, then later checks whether that signal appears in a suspect model's generated outputs. Existing approaches often trade off verifiability, fidelity, robustness, and deployability. We introduce SpectralCanary, which uses a smooth low-pass mask to blend a fixed canary image into every collection image while leaving captions unchanged and requiring neither auxiliary generation nor per-image optimization. A lightweight verifier detects the transferred signal in suspect-model outputs. Across eight tests on the Pokemon dataset and three realistic data transformations, spanning two architecture families, SpectralCanary achieves the highest average per-generation TPR@ FPR () and fixed-threshold model-level accuracy using 30 generations () among the evaluated methods. Across five datasets spanning artistic, medical, satellite, and fashion domains with SD3.5, SpectralCanary averages per-generation TPR@ FPR while preserving the released images (SSIM , LPIPS ) with a near-zero mean change FID (mean FID ) across datasets. Matched-energy ablations on Pokemon and ROCOv2 support our design choice, low-frequency placement transfers more reliably compared to mid/high-frequency placement. Under the evaluated adaptive attacks, the most effective removal requires aggressive modification and incurs more than FID points. These results identify low-frequency visual signals as effective carriers for collection-level T2I audit traces.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Dataset-Use Auditing, Text to Image, Data Provenance

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UC2cOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SpectralCanary protects an image collection by blending the low-frequency spectrum of one secret canary image into every image of the collection, then checks a suspect T2I model by scoring its generated images with a verifier trained on original-protected pairs. Across eight Pokemon settings, SpectralCanary achieves the highest average TPR at 1% FPR among the evaluated methods, and across five domains the protected images stay close to the originals while detection remains high.

Critical strengths
  • The motivation is clear, the threat model is well defined, and the experiments support the design choices.
  • The matched-energy band ablation is informative. Frequency-band robustness is studied in classical watermarking, but the transfer of a low-frequency trace through T2I fine-tuning is a new observation.
Critical weaknesses
  1. Overall novelty is low, and the method section and the contribution list do not separate the borrowed components from the new ones. The audit pipeline and the verifier design follow DIAGNOSIS and EnTruth, but Sec. 2 cites neither, and frequency-domain watermarking is not cited at all. These components need citations where the method uses them, so that the new contribution is isolated.
  2. Although the paper is clear, the writing is very dense. For example, instead of restating numbers every time, stating the interpretation and giving numbers only when it is necessary, would make the paper easier to read.
  3. The dependence of the framework on the canary image is not analyzed sufficiently. All main experiments use one fruit image, and only the band ablation uses three canaries. This may affect the validity of the framework vastly.
Questions for authors
  • How much do the main results vary across independently chosen canary images, and how should an owner choose the canary?
6 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for recognizing the clear motivation, well-defined threat model, experiments supporting our design choices, and informative matched-energy ablation. We especially appreciate the reviewer's recognition that low-frequency trace transfer through T2I fine-tuning is a new observation. In what follows, we address all reviewer's concerns.

W1 — Attribution of prior components and the new contribution

We thank the reviewer for highlighting the need to distinguish established components from our contributions. We agree that the general dataset-use auditing framework and classifier-based verification build on prior work, including DIAGNOSIS and EnTruth. In the revision, we explicitly cite these methods alongside the verifier formulation, cite Tree-Ring when discussing frequency-domain watermarking (Sec. 3), and revise the contribution list to distinguish borrowed components from our contributions

Our novelty lies in designing an effective and efficient frequency-selective canary that survives generative fine-tuning, rather than the general auditing pipeline. Specifically, we demonstrate that weak, shared low-frequency signals can remain detectable in newly generated outputs after T2I fine-tuning while preserving image fidelity. This behavior is non-obvious, as simply adding frequency-domain signals does not guarantee their transfer through model training.

We adapt Tree-Ring by applying inversion-based watermarking and reconstruction to existing training images, fine-tune SD1.5 on the resulting dataset, and evaluate generated outputs using the Tree-Ring detector. We additionally blend reconstructed watermarked images with the originals to reduce distortion.

MethodAUC ↑TPR@1% FPR ↑SSIM ↑LPIPS ↓
Tree-Ring adaptation0.794011.0%0.72010.4577
Tree-Ring adaptation, blended0.52660.8%0.99730.0023
SpectralCanary0.996096.6%0.99770.0071

These results illustrate that adapting generation-time frequency-domain watermarking does not automatically achieve effective, fidelity-preserving dataset-use auditing. Together with our matched-energy frequency-band ablations, they support the distinct contribution of SpectralCanary. To our knowledge, prior work has not established this frequency-dependent traceability mechanism for dataset-use auditing through generative fine-tuning.

W2 — Dense writing and repeated numerical descriptions

Thank you for your comment. In the revision, we reduce repetitive numerical descriptions, emphasize the interpretation of results, and improve the organization of the main text.

W3, Q1 — Dependence on canary choice and guidance for selection

Thank you for this valuable comment.

  • SpectralCanary maintains strong detection across five independently selected canaries, although performance varies with canary choice. We conduct additional experiments using five semantically diverse canaries under identical settings.

    CanaryAUC ↑TPR@5% FPR ↑TPR@1% FPR ↑SSIM ↑LPIPS ↓Regional color contrast
    Fruit0.999099.3%99.2%0.99710.0080326.23
    Vehicle0.998899.5%97.9%0.99790.0044326.10
    Animal0.998298.6%97.2%0.99830.0036526.72
    Indoor0.990493.9%91.2%0.99790.0041323.54
    Landscape0.990995.6%88.0%0.99850.0017614.45

    AUC remains consistently high across all five canaries (0.9904–0.9990), with TPR@1% FPR ranging from 88.0% to 99.2%, while all choices retain high image fidelity.

  • Regional color contrast offers a preliminary selection heuristic. Our exploratory analysis suggests that canaries with stronger regional color contrast (measured by average pairwise CIELAB distance across a 4×4 grid) tend to achieve higher traceability. Thus, dataset owners may consider selecting canaries with diverse regional color structure, although broader experiments are needed to validate this heuristic.

AuthorsOpenReview ↗

Follow-up discussion

We thank the reviewer again for the thoughtful review. Our rebuttal provides clarifications and additional experiments addressing the two main concerns: (1) the novelty of frequency-selective trace transfer through generative fine-tuning, supported by our comparison with the existing Tree-Ring frequency-domain watermarking method, and (2) Analysis on the dependence of detection performance on canary choice

As the discussion deadline approaches, we would be happy to clarify any further questions. We hope these clarifications and additional results help address your concerns. If your concerns have been resolved, we would appreciate it if you could reconsider the assessment with additional evidence.

Thank you for your efforts during the rebuttal period. I have read your responses and the revised manuscript. Several of my concerns remain:

  1. My original comment that "frequency-domain watermarking is not cited at all" was not a request to add a frequency-domain watermarking method as a baseline. My point was that the main observation, "low-frequency visual structure is an effective carrier," follows the same principle as conventional frequency-domain watermarking. As for the Tree-Ring comparison, Tree-Ring embeds its pattern in the Fourier space of the initial noise latent and detects it via DDIM inversion. It is therefore expected that this pattern does not transfer through fine-tuning on reconstructed training images, so this comparison does not provide strong evidence for the novelty of the proposed approach.
  2. I agree that showing low-frequency signals transfer well through T2I fine-tuning is a new empirical observation. However, the method itself (blending a low-pass-filtered canary into every image, followed by the classifier-based verification and KS-test procedure from prior work) offers limited methodological novelty.
  3. The new results of canary choice show that detection performance varies non-negligibly with canary choice (TPR@1% FPR from 88.0% to 99.2%). The trace strength likely depends on how much the canary's low-frequency content differs from the dataset's low-frequency content. If so, the canary should be chosen according to the target dataset. However, the revision only shows empirically that the choice matters. The proposed regional-color-contrast heuristic rests on five canaries and is not validated. In addition, Table 12 does not state which canary image, dataset, and architecture were used.
  4. For the dilution experiment requested by Reviewer 98oA (Appx. D.3), the source of the unprotected images does not seem to be described. Please let me know if I missed it. Appendix D.3 fixes the number of protected images (and their exposure), and Appendix D.4 shows the partial-use experiment. From my own perspective, the partial-use experiment seems more informative for practical scenarios. However, in Appendix D.4, performance is stable down to 50% but drops sharply as the number of protected images decreases further (TPR@1% FPR of 81.15% at 25% and 60.74% at 10%). This suggests that the scale of the protected collection is a key factor and an important limitation of the method.
Reviewer_ZfPWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SpectralCanary, a method aims to audit whether a text-to-image model is fine-tuned on a particular image dataset. The authors claim that existing methods in this field often trade off verifiability, fidelity, robustness, and deployability. SpectralCanary uses a low-frequency visual structure as an effective carrier for weak collection-level traces during T2I fine-tuning. Specifically, the dataset owner chooses one fixed canary image before release, and then SpectralCanary mixes a small amount of the canary's low-frequency Fourier content into the original while retaining most of its spectrum and leaving its caption unchanged. It protects each image with one inexpensive tranform, requiring neither an auxiliary generator nor per-image optimization. Experiments on distinct image domains show that SpectralCanary can be a practical method for T2I dataset-use auditing.

Critical strengths
  1. The datasets used in this paper cover distinct image domains, such as character art, sparse scene, medical, satellite, and fashion. They cover most kind of image, which can support a comprehensive evaluation of this paper. And the results on these image domains show that SpectralCanary works on them with generalizability.
  2. The paper discusses its limitations candidly, which benefits to subsequent research.
  3. The paper summarizes four requirements in the dataset-use auditing field and corresponding limitations for each item. These summaries are helpful to understand the development of this field.
Critical weaknesses
  1. SpectralCanary implement dataset-use auditing via mixing a small amount of the canary's low-frequency Fourier content into the original images. The potential impact of the added content is unclear. Indeed, the authors claimed the caption is unchanged, but what about other utilities. For example, will the added content affect detection performance on satellite images? Will the added content affect the retrieval ability on fashion images? The missing experiments leave the potential impact of SpectralCanary unclear.
  2. Why do the authors place the Section Related Work after Section Method? A deep learning paper ususlly follows the section order: Introduction, Related Work, Method, Experiment, and Conclusion. Of course, that order isn't set in stone, but makes more logical sense.
  3. The main paper has too many references to Appendix content. Almost every paragraph in the experiment section has sentences that refer to Appendix. Are thay necessary? The Appendix should be a supplementary for the main paper but the main paper should be self-completed. More clear appendix writing and less references from main paper can make the paper more readable.
  4. How about the performance of SpectralCanary on video datasets? In this paper, the evaluations are conducted on image data. The generalizability from images to videos is unclear.
  5. Some arrows in Figure 5 is confusing. For example, the "Better" refers to nothing. Please proofread experimental results and make the figures more easy to understand.
  6. The novelty is limited. The main processing of SpectralCanary is low-frequency visual content adding to protected images. What is the technical challenge of this method?
Questions for authors
  1. The potential impact to other utilities, such as detection and retrieval.
  2. The technical challenge of the paper.
  3. If someone uses a part of samples from protected dataset, not the full dataset, how does the proposed method perform?
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1/2)

We thank Reviewer for highlighting the strengths of our paper: the broad evaluation across distinct image domains, candid discussion of limitations, and helpful summary of four practical auditing requirements and the corresponding limitations of existing approaches. We also thank the reviewer for the constructive feedback and the opportunity to clarify our work. In what follows, we respond to all of the reviewer's concerns.

W1, Q1: Downstream utility for fashion retrieval and satellite detection

We thank the reviewer for this important point. SpectralCanary preserves downstream utility in the evaluated retrieval and detection tasks.

  • We conduct additional experiments using frozen CLIP ViT-B/32 for fashion retrieval on FACAD and frozen YOLOv8-S OBB for satellite object detection on DIOR-R. We compare predictions on original and protected images while keeping the task models fixed.

    Task / modelMetric (%)OriginalProtectedChange (pp)
    FACAD / CLIP ViT-B/32Recall@123.723.4−0.3
    FACAD / CLIP ViT-B/32Recall@551.251.3+0.1
    FACAD / CLIP ViT-B/32Recall@1067.367.0−0.3
    DIOR-R / YOLOv8-S OBBmAP5064.2364.07−0.16
    DIOR-R / YOLOv8-S OBBmAP50–9548.5448.40−0.14

    All measured changes are within 0.3 percentage points, providing additional evidence of utility preservation under these settings. These are frozen-model evaluations; they do not establish utility preservation for every task or for task models trained on protected images.

W2/W3/W4: Section order, appendix organization, and excessive appendix references

We appreciate these presentation suggestions and have made the following revisions:

  • Moved Related Work before Method.
  • Reorganized the appendix and improved its table of contents.
  • Removed repetitive appendix references and made the main experimental discussion more self-contained.

W5: Applicability to video generation

Thank you for raising this point. Our additional results provide preliminary evidence that SpectralCanary can extend to video generation. We evaluate Fashion-Text2Video using Wan2.1-T2V-1.3B, applying the same canary consistently across training-video frames with the default parameters α = 0.02, c = 0.20, and δ = 1.

AUC ↑TPR@1% FPR ↑TPR@5% FPR ↑SSIM ↑LPIPS ↓
0.939880.0%82.0%0.99110.0266

Each video is scored by averaging verifier probabilities over eight uniformly sampled frames. Detection uses 100 generated videos per clean/protected model; the 1% FPR operating point corresponds to one false positive among the 100 clean-model videos. SSIM and LPIPS measure protection distortion across 500 training videos. Broader evaluation across video datasets, architectures, and temporal attacks remains future work.

W6: Figure 5 arrows and “Better” labels

Thank you for pointing this out. Higher TPR and lower ΔFID, protection time, and LPIPS are preferred, so the desired region in each panel is the upper left. The caption states this direction. We will revise the arrows and “Better” labels to make their association with the axes unambiguous.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2/2)

W7, Q2: Technical challenge and novelty concern

We appreciate your comments and provide detailed clarification below.

  • The technical challenge is making a weak visual trace survive generative fine-tuning while preserving image fidelity. Simply adding low-frequency content does not guarantee that a T2I model will learn and reproduce a detectable signal. Existing frequency-domain watermarking methods, such as Tree-Ring, target watermarking during image generation rather than dataset-use auditing.

    We adapt Tree-Ring by applying inversion-based watermarking and reconstruction to existing training images, fine-tune SD1.5 on the resulting dataset, and evaluate generated outputs using the Tree-Ring detector. We additionally blend reconstructed watermarked images with the originals to reduce distortion.

    MethodAUC ↑TPR@1% FPR ↑SSIM ↑LPIPS ↓
    Tree-Ring adaptation0.794011.0%0.72010.4577
    Tree-Ring adaptation, blended0.52660.8%0.99730.0023
    SpectralCanary0.996096.6%0.99770.0071

    These results illustrate that adapting generation-time watermarking to dataset-use auditing does not automatically achieve both reliable detection and high image fidelity, whereas SpectralCanary achieves both under the evaluated setting.

  • We respectfully argue that simplicity does not imply a lack of novelty. The above Tree-Ring adaptation experiments provide further evidence that effective signal transfer through generative fine-tuning is non-obvious. Our contributions are threefold:

    • Effective low-frequency canary design: We identify a frequency-selective mechanism that enables weak visual signals to survive generative fine-tuning and remain detectable in newly generated images while preserving image fidelity.
    • Non-trivial empirical validation: Our controlled frequency-band experiments reveal the importance of frequency selection, while evaluations across architectures, datasets, fine-tuning strategies, and preprocessing attacks establish its effectiveness under diverse settings.
    • Practical simplicity: SpectralCanary requires no caption modification, auxiliary generators, or per-image optimization, with efficient CPU-based coating (0.39 s/image).

    To our knowledge, prior work has not demonstrated this combination of traceability, fidelity, robustness, and efficiency through a frequency-selective canary mechanism for dataset-use auditing.

Q3: Detection Performance using only part of the protected dataset

Thank you for the question. SpectralCanary remains effective when only part of the protected dataset is used, although detection generally weakens for smaller subsets. We fine-tune SD1.5 on subsets containing 10%, 25%, 50%, and 100% of the protected Pokémon collection. Each setting uses matched clean/protected models trained for 80 epochs and 833 generated images per condition. We use the same verifier without retraining.

Protected collection usedTraining imagesAUC ↑TPR@1% FPR ↑TPR@5% FPR ↑
10%830.932260.74%70.71%
25%2080.976881.15%86.79%
50%4160.992192.44%94.36%
100%8330.993491.96%96.64%

AUC remains above 0.93 even with 10% of the protected collection, although TPR@1% FPR decreases to 60.74%. Detection generally improves with larger subsets, highlighting the importance of protected dataset coverage.

Thanks for the rebuttal. The response has addressed most of my questions. My main concern still lies in the novelty of this paper. The authors claim that "The technical challenge is making a weak visual trace survive generative fine-tuning while preserving image fidelity.", but this paper does nothing during the model fine-tuning. The visual trace designs are heuristic and have no relationship with the downstream model fine-tuning.

AuthorsOpenReview ↗

Thank You and Further Clarification

We thank the reviewer for the follow-up and are glad that our rebuttal addressed most of your concerns. Regarding the remaining concern, we apologize if any part of our previous explanation was unclear and would like to make two further clarifications:

The authors claim that "The technical challenge is making a weak visual trace survive generative fine-tuning while preserving image fidelity.", but this paper does nothing during the model fine-tuning.

Clarification 1: Not intervening during fine-tuning is an intentional requirement of dataset-use auditing. Dataset owners cannot control how unauthorized parties train their models. Thus, the technical challenge is to design a canary signal that remains detectable after fine-tuning, without prior knowledge of or control over the downstream training algorithm. This setting is also shared by previous dataset-use auditing protection methods, such as EnTruth, SIREN and DIAGNOSIS

The visual trace designs are heuristic and have no relationship with the downstream model fine-tuning.

Clarification 2: Our design is directly informed by downstream fine-tuning behavior, rather than being an arbitrary heuristic. Our controlled frequency-band ablations (Fig. 6, Sec. 4.5) show that low-frequency signals survive generative fine-tuning more effectively than mid/high-frequency signals. This finding motivates our frequency-selective design. Furthermore, our Tree-Ring adaptation experiments show that simply applying an existing frequency-domain watermark does not guarantee reliable dataset-use auditing (11.0% TPR for Tree-Ring compared to 96.6% TPR for SpectralCanary at 1% FPR under the same setting)

We believe that identifying, designing and extensively validating an effective frequency-selective mechanism for dataset-use auditing is a meaningful contribution, particularly given its favorable practical trade-offs among detection performance, fidelity, robustness and efficiency compared with existing methods

We hope these clarifications help address the remaining novelty concern and would appreciate it if you could reconsider the assessment with additional evidence.

Reviewer_98oAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SpectralCanary, a proactive auditing method for detecting whether a text-to-image model was fine-tuned on a protected image collection. Before releasing a collection, the method blends a small amount of the low-frequency Fourier content of one fixed canary image into each protected image through a smooth low-pass mask. Captions remain unchanged. A verifier trained on original and protected image pairs is subsequently applied to generations from a suspect model. The evaluation covers SD1.5 and SD3.5, LoRA and full-parameter fine-tuning, five image domains, several training-data transformations, three direct auditing baselines, frequency-location ablations, and adaptive spectral-removal attacks. The reported results show strong average detection, low image distortion, and almost no average degradation in generated-image FID.

Critical strengths
  • Strong practical operating point across several competing requirements The method is notably simple: a single FFT-based deterministic transform, no caption modification, no auxiliary generator, and no per-image optimization. Despite this simplicity, it obtains an average per-generation TPR of 91.1% at 1% FPR across eight Pokémon settings and 94.8% fixed-threshold model-level accuracy using 30 generations.
Critical weaknesses
  • The threshold is calibrated separately for each method–architecture pair. However, five models are too few to support a 1% FPR. In a real audit, the suspect model's base checkpoint, architecture, fine-tuning setup and sampler are unknown, and clean models that differ in these ways could push false positives higher. Results also vary across settings: average model-level accuracy is 94.8%, but it is 58.5% for SD3.5 with augmentation. The authors may calibrate one global threshold on a diverse pool of clean models and test it on unseen architectures and pipelines.
Questions for authors
  • How does detection change when a fixed protected collection is mixed with progressively larger unprotected datasets, for example at protected sampling ratios of 50%, 10%, 1%, and 0.1%, while controlling the number of times each protected sample is seen?
  • The current attacks do not provide the attacker access to verifier feedback. Can the authors discuss whether a black-box attacker who can repeatedly query an audit endpoint could optimize sanitization against the verifier while preserving image quality?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for recognizing SpectralCanary's simple protection procedure and strong practical operating point across competing requirements, including detection performance and model-level verification. We appreciate the constructive feedback and address all concerns below

W1: Calibration across architectures and pipelines, scope of 1% FPR and augmentation

We thank the reviewer for highlighting the importance of calibration across heterogeneous models and pipelines. We provide experiments and clarifications below:

  • A single global threshold transfers to the tested unseen architecture and pipelines. Our original calibration enables controlled comparisons but does not fully reflect practical audits with unknown architectures and pipelines. Following the reviewer's suggestion, we calibrate one global threshold per method using only clean-trained SD1.5 and SD3.5 models without transformations, setting it just beyond the most protected-like clean score. We then freeze the threshold and evaluate 70 held-out models covering unseen preprocessing pipelines (augmentation, regeneration, recaptioning) and an unseen architecture (SDXL)

    SpectralCanary achieves the highest model-level TPR (90.70%) and balanced accuracy (92.24%) among the evaluated methods, with a model-level FPR of 6.22%. EnTruth achieves the second-highest balanced accuracy (89.71%) but requires approximately 17.7× longer CPU coating time. These results support transfer across the tested architectures and pipelines, although they do not establish a universal 1% model-level FPR guarantee

    MethodModel-level FPR↓ (%)Model-level TPR↑ (%)Balanced Acc↑ (%)Coating time/image (s) ↓
    EnTruth6.4885.8989.716.92 (CPU)
    SIREN8.3680.9286.2871.38 (GPU)
    DIAGNOSIS2.3559.6878.670.02 (CPU)
    SpectralCanary6.2290.7092.240.39 (CPU)
  • TPR@1% FPR is a per-generation operating point, not a model-level guarantee. We agree that five clean calibration models are insufficient to establish a reliable 1% model-level FPR. To clarify, the reported TPR@1% FPR measures detection of individual generated outputs at an empirical threshold selected from clean-model generation scores. We clarify this distinction and avoid implying a universal model-level FPR guarantee

  • Augmentation limitation. We acknowledge the weaker SD3.5/Pokemon result under training-time augmentation, despite the strong average performance across settings. We discuss this failure case in the limitations

Q1: Dilution while controlling protected-sample exposure

Thank you for this valuable comment. SpectralCanary's signal remains detectable under the tested dilution levels

  • We mix a fixed collection of 833 protected Pokemon images with progressively larger unprotected datasets. Each mixture is trained for 80 epochs, so each protected image is nominally presented 80 times regardless of the dilution ratio
    Protected fractionProtected/total imagesAUC↑TPR@1%FPR (%)↑
    50%833/1,6660.993897.72
    10%833/8,3300.988794.84
    5%833/16,6600.993592.80
  • TPR@1%FPR decreases from 97.72% to 92.80% as the protected fraction falls from 50% to 5%. Testing 1% and 0.1% would require 83,300 and 833,000 total images, and was infeasible within the rebuttal period. We acknowledge that detection performance may degrade further at these extreme dilution ratios and will discuss this limitation in the revision

Q2: Adaptive sanitization with black-box verifier feedback

We thank the reviewer for noting this threat that repeated verifier feedback could enable stronger adaptive sanitization attacks

  • Lower verifier scores on sanitized images do not automatically yield stronger evasion after fine-tuning. We conduct a preliminary black-box feedback-guided attack starting from collection-centering sanitization (λ = 0.50). We use GPT-6.1 Sol to propose modifications to the sanitization algorithm over 10 refinement rounds, evaluating three candidates per round on 100 protected images (3,000 verifier queries). The objective is to reduce verifier scores while preserving SSIM and LPIPS relative to the released protected images. We freeze the selected attack and evaluate it on 733 held-out images, followed by model fine-tuning and verification under the main Pokemon protocol
    Attack#QueriesSanitized-image verifier score↓TPR@1%FPR↓SSIM↑LPIPS↓ΔFID↓
    Starting attack-0.2860627.60.991780.0183217.17
    LLM-guided3,0000.2832543.30.991820.0183312.19
  • The LLM-guided attack slightly reduces verifier scores and improves ΔFID, but downstream TPR@1% FPR increases from 27.6% to 43.3%, indicating weaker evasion after fine-tuning. This suggests that reducing verifier scores on sanitized images does not necessarily suppress traces in generated outputs. Stronger feedback-guided attacks still remain possible, while restricting verifier access may help mitigate this threat

Thank you for the detailed rebuttal and additional experiments. I find the new results convincing and believe they adequately address my main concerns. In particular, the additional evaluations on global threshold calibration, dataset dilution, and adaptive sanitization provide useful evidence supporting the robustness and practical applicability of the proposed method. I therefore raise my rating to 3

AuthorsOpenReview ↗

Thank You for Your Positive Feedback

We thank the reviewer for the positive feedback and for increasing the score. We are glad that our rebuttal helps clarify and address your main concerns. We really appreciate your thoughtful review and constructive suggestions

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of our key contributions and rebuttal

We thank the Area Chair, Program Chairs, and all reviewers for their thoughtful feedback. As the discussion period concludes, we would like to summarize our key contributions and rebuttal.

1. Key contributions and recognized strengths

  • Novel observation: Reviewer UC2c explicitly recognized that low-frequency trace transfer through T2I fine-tuning is a new observation. Our matched-energy frequency-band experiments (Sec. 4.5) support this finding and our frequency-selective design.
  • Strong practical performance and simplicity: Reviewers 98oA and ZfPW recognized the method's effectiveness and practical value. Reviewer 98oA particularly highlighted its simple protection procedure and strong operating point across competing requirements.
  • Broad evaluation and well-motivated design: Reviewers ZfPW and UC2c recognized the diverse image domains, clear motivation, well-defined threat model, and experiments supporting the design choices.

2. Additional evidence during rebuttal

We conducted additional experiments and revised the manuscript to address the reviewers' concerns:

  • Global-threshold generalization: SpectralCanary achieves 92.24% model-level balanced accuracy (90.70% TPR, 6.22% FPR) across tested held-out architectures and preprocessing pipelines, outperforming the evaluated baselines.
  • Dataset dilution and partial collection use: Detection remains strong with 5% protected data (92.80% TPR@1% FPR) when protected-sample exposure is fixed. Under partial collection use, detection remains strong at 50% collection use (92.44% TPR@1% FPR), but weakens as fewer protected images are used.
  • Canary generalization: Five different canaries achieve 88.0–99.2% TPR@1% FPR on Pokemon. Our main experiments also use the same fixed canary across five datasets, achieving 92.4% average TPR@1% FPR without dataset-specific canary selection.
  • Downstream utility: Protected images show performance changes within 0.3 percentage points in fashion retrieval and satellite object detection using frozen task models.
  • Video extension: SpectralCanary achieves 80.0% per-video TPR@1% FPR on Fashion-Text2Video using Wan2.1-T2V-1.3B.
  • Adaptive sanitization attack: Our preliminary verifier-feedback attack shows that reducing verifier scores on sanitized images does not necessarily improve evasion after fine-tuning (Appx. G.2).

We also improved the manuscript's organization, clarified attribution to prior work, and explicitly discussed limitations concerning canary selection, partial collection use, and calibration.

3. Reviewer feedback during rebuttal

  • Reviewer 98oA: Found the additional experiments convincing and stated that the main concerns were adequately addressed. The reviewer particularly acknowledged the global calibration, dataset dilution, and adaptive sanitization experiments and increased the rating.
  • Reviewer ZfPW: Acknowledged that our rebuttal addressed most questions and continued to question novelty, arguing that our method does not intervene during fine-tuning and that the trace design is unrelated to downstream training. We clarified that non-intervention is an intentional requirement of dataset-use auditing, since dataset owners cannot control unauthorized fine-tuning. We also highlighted that our controlled frequency-band experiments (Sec. 4.5) directly measure trace transfer through fine-tuning and provide empirical support for our design. We have not received further feedback following these clarifications.
  • Reviewer UC2c: Recognized low-frequency trace transfer through T2I fine-tuning as a new empirical observation and acknowledged our additional clarifications and revised limitations. The reviewer maintained the rating due to concerns about methodological novelty. We clarified that our experiment evaluation is conducted on existing auditing framework, while our main contribution lies in identifying and extensively validating effective low-frequency trace transfer for dataset-use auditing.

We believe that our original results, additional experiments, and manuscript revisions demonstrate a meaningful and practical contribution to dataset-use auditing. While we acknowledge and respect the reviewer's perspective on methodological novelty, we hope the Area Chair will consider the new findings, the practical effectiveness of our method, and the extensive evaluation in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.