acceptodds
Under review as a conference paper at ICLR 2027

SemantiTrace: Stealthy Copyright Tracing in Black-Box Multimodal RAG via Semantic Parasitism

Abstract

Commercial Multimodal Retrieval-Augmented Generation (RAG) platforms ingest web-scale corpora behind proprietary APIs, making visual copyright protection reliant on verifiable black-box provenance tracing. Prior defenses bridge the cross-modal gap with synthetic visual canaries featuring bizarre compositions or arbitrary acronyms. These globally anomalous patterns contradict natural data distributions, leaving them vulnerable to pre-ingestion out-of-distribution sanitization. To resolve this trade-off between stealth and auditable decodability, we introduce SemantiTrace, which replaces de novo canary synthesis with semantic parasitism. SemantiTrace embeds contextually coherent, micro-scale forensic signatures in benign hosts by mutating existing typography or synthesizing geometrically natural, branded scene elements. To survive black-box retrieval and trigger explicit textual generation, Cross-modal Dual-Guided Latent Diffusion couples visual retrieval pull with teacher-forced textual decoding on the latent flow. Off-canvas pixel drift is eliminated by a support-preserving output projection; Dual-Space Soft Latent Blending and spatially gated gradients remove hard mask-edge discontinuities and control boundary energy within physically plausible supports. SemantiTrace passes pre-ingestion anomaly filters with 1.8% rejection on DENSE-10K and MMQA, and 2.2% on WebQA. On WebQA, our parasite canaries achieve 81.70% query success after global filtering; after corpus-wide adversarial repair and index reconstruction on a 400,000-item gallery, they retain 68.80% query success and All- anchors, with a -point loss in benign QA utility. Under zero-clean-baseline audits with private multi-candidate assignment and hidden OCR serving, the selected design requires 11 black-box queries for at least 95% fixed-population detection coverage under complete ingestion, with an exact private-assignment test at and simultaneous 99% empirical false-alarm upper bounds below 1% for its evaluated budgets.

Primary area:
generative models
Keywords:
Multimodal Retrieval-Augmented Generation, Data Provenance and Copyright Protection, Controllable Latent Diffusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZKLaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

SemantiTrace aims to detect whether a multimodal RAG service has ingested a data owner’s images. The owner changes a few visible letters or adds a labeled object, then asks questions that describe the scene without revealing the special text. If the service’s answers reproduce that text, it provides evidence of ingestion. The method optimizes the image edits for retrieval, text extraction, and survival under sanitization, and includes a private randomized-signature audit that does not require a clean reference service.

Critical strengths
  • Using small edits to existing images is a sensible way to make visual canaries less conspicuous than synthetic scenes. The evaluation includes several retrieval interfaces and generators, as well as corpus-wide repair and reindexing.
  • The appendix contains substantial controls, including fully guided and independently tuned AnyText2 baselines. The private-assignment test is also useful: it gives a clearly stated conditional null without requiring access to a matched clean service.
Critical weaknesses
  1. The paper makes the basic task much harder to understand than it needs to be. The reader has to unpack “semantic parasitism,” “dual-guided latent diffusion,” several audit endpoints, and multiple deployment variants before seeing a complete example of what the owner releases and what they actually ask the service. Figure 1 gives the intuition, but the concrete queries are buried in Appendix I. The main method describes a four-query paired audit, while the abstract highlights a one-query-per-host private audit developed much later in the appendix. I struggled to identify the intended default workflow. One worked example and a clear separation of the two audit protocols would help considerably.

  2. I could not reconstruct the main RAG serving interface from the paper. Section 4.1 specifies top-5 retrieval, but the probe templates and qualitative answers repeatedly refer to “this image.” Does the generator receive all five retrieved images, in their retrieval order, or is the target image selected once it appears in the top five? What instruction tells it which image or region to inspect? Appendix B.5 also separates the retrieval query from the generation query in a diagnostic; it is unclear how these correspond to the single request available to an output-only auditor. These details matter because finding an image in the top five and reading a chosen image are different from answering a query against a mixed retrieved context. The exact request and context construction should be shown.

  3. The gradient-stability comparison does not fully justify the optimization approximation. Table 8 compares exact backpropagation, a stop-gradient approximation, and that approximation with normalization, spatial gating, and warm-up. Their maximum gradient RMS values are 142.50, 18.20, and 1.24, respectively. The poor convergence of the first row could therefore partly reflect update scaling rather than a need to discard the velocity Jacobian. The later global-clipping controls are useful for assessing the spatial gate, but do not isolate the Jacobian approximation. I would like to see exact-gradient guidance with matched normalization and a tuned step size, including its runtime. Appendix B.4 gives a sufficient descent condition, but does not establish that the approximation error satisfies it in the actual runs.

Questions for authors
  1. What is the default deployment and audit protocol? Please walk through one host image, its released edit, the auditor’s complete request, the retrieved context, the returned answer, and the final decision.
  2. Are all top-5 images passed to the visual generator without target-aware selection? How are they ordered and packed, and what system prompt is used? Does the auditor submit one query or separately control retrieval and generation queries?
  3. How does normalized, step-size-tuned exact-gradient guidance compare with the proposed approximation? Also, why does removing retrieval guidance produce zero top-5 hits on all 2,000 probes in each dataset, while the Naive baseline has mean ranks of 2.62 and 2.92? Are the hosts, queries, and rank definitions identical?
Reviewer_MgCuOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a copyright tracing framework for multimodal RAG. The major idea is to leverage semantic parasitism to improve the stealthiness of watermarking. The experiments have been conducted on the MMQA and WebQA datasets, and the method is compared with two major baselines, AQUA and AnyText2.

Critical strengths
  • The idea of semantic parasitism is natural and seems promising for stealthy copyright tracing.

  • The evaluation considers the ablation and robustness of the proposed method.

Critical weaknesses
  • Only OOD rejection is considered as a metric to measure stealthiness, which could be limited in real-world cases.

  • The considered scenario is limited. RAG knowledge bases may be curated from multiple sources, and the applicability of the proposed method to multi-source tracing is important.

Questions for authors
  1. Only OOD rejection is considered as a metric to measure stealthiness, which could be limited in real-world cases. What if the adversary leverages other detection methods to detect the modified images? For example, there are a lot of works on detecting edited images, and it would be interesting and important to investigate the performance against these detection methods. In addition, adaptive detection could be possible. Since the adversary knows the watermarking process, could they simulate the process and generate data to train a detector?

  2. The considered scenario is limited. RAG knowledge bases may be curated from multiple different sources, and the applicability of the proposed method to multi-source tracing is important. Specifically, can the proposed method be used in cases where only a small portion of the data has been watermarked? Can the proposed method distinguish copyrighted data belonging to different owners?

  3. Figure 1 is overcomplicated and hard to follow. One possible error: "case 1: watermarked" should be "case 2: watermarked".

1 comment

Thank you for the additional experiments. The detector evaluations and multi-owner tracing results address my main concerns. I am raising my score.

Reviewer_G1oEOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the problem of dataset provenance and copyright protection. Verifying in a black-box setting whether an unauthorized party has crawled a proprietary image dataset and integrated it into their Multimodal Retrieval-Augmented Generation (MM-RAG) system. Prior watermarking approaches (such as AQUA) rely on inserting overtly synthetic, out-of-distribution (OOD) canary images (e.g., floating acronyms or unnatural object compositions) to guarantee easy retrieval and response generation. However, such synthetic artifacts are easily filtered out by basic pre-ingestion OOD sanitization before even entering the RAG database. To overcome this, the authors propose SemantiTrace, a method that stays within the natural image manifold by applying localized diffusion inpainting to introduce subtle semantic mutation, such as minor typographical edits on existing street signs or contextually natural small objects. Crucially, rather than performing naive image editing, the method steers the diffusion sampling steps using gradients from both a retriever and a surrogate vision-language generator, effectively baking a post-hoc spurious correlation between a target query and the injected visual mutation so that the watermark survives into the final black-box text response.

Critical strengths
  • Practical and Well-Motivated Problem Setup: The paper addresses an important real-world data safety issue by recognizing that data scrapers do not blindly ingest raw crawled data, but instead apply anomaly filtering and sanitization pipelines prior to building their RAG index.
  • Non-Trivial Post-Hoc Optimization of Spurious Correlations: Forging a reliable spurious correlation between a query and a subtle visual mutation in a post-hoc manner is technically non-trivial. If the inpainting is purely optimized for visual naturalness, the downstream VLM will either ignore the tiny object or auto-correct the subtle typo during generation. Conversely, pushing the generator gradient too aggressively introduces high-frequency adversarial artifacts that trigger OOD filters. (However, as discussed later, this algorithm already exists)
Critical weaknesses
  • Missing Citations and Conceptual Overlap with Semantic/Physical Backdoor Literature: The core idea of bypassing outlier/anomaly detection by replacing unnatural patch noise with natural-looking objects or visual features to induce a spurious correlation has been extensively studied in the vision security community under Semantic Backdoors and Physical Backdoors (e.g., Bagdasaryan et al., 2020; Wenger et al., CVPR 2021), as well as recent diffusion-inpainting backdoor attacks. Furthermore, repurposing such backdoor correlations for dataset ownership verification (Backdoor-based Dataset Watermarking, e.g., Li et al., NeurIPS 2022) is an established paradigm. However, the paper completely omits citations and discussions of this closely related backdoor and poisoning literature, presenting the in-manifold mutation concept as if it were entirely new.
Questions for authors

It seems that proposed algorithms are highly overlapped (personally, same), and I think the author should referenced the algorithm in this research. But why did the author not referenced the papers?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.