acceptodds
Under review as a conference paper at ICLR 2027

Yours or Mine? Misleading Latent Diffusion Model Attribution via Cross-Model Reconstruction

Abstract

Reconstruction error-based attribution infers an image's source model from reconstruction errors under candidate autoencoders. We find that distinct Stable Diffusion models can produce identical errors for the same image, making source attribution ambiguous. Comparing autoencoder architectures and parameters reveals that this ambiguity arises from shared autoencoders. We therefore test whether reconstructing an image with another model's autoencoder can redirect attribution after generation. We introduce Cross-Model Reconstruction (CMR), which passes an image generated by the original model through the attack model 's encoder and decoder once. This reconstruction pass can induce false attribution to with minimal perceptual distortion and requires neither diffusion sampling nor attribution queries. Models that share an autoencoder can still have different denoisers. We propose READ (Responses Evidence for Attribution from Denoisers), which learns attribution from multiple denoisers' relative responses to controlled latent perturbations, using clean images for calibration. Across 12 generative models, CMR raises mean attack-model acceptance to 93.5-99.8% for AEDR, AEROBLADE, and LatentTracer. By comparing denoising responses, READ distinguishes source models even when they share an autoencoder. On the same attacked images, it reduces mean attack-model acceptance to 13.6% while improving acceptance of the original source model relative to these baselines. The source code of CMR and READ are included in the supplementary material.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Model attribution, Re-encoding attacks, Latent diffusion models, Denoising consistency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sqqYOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The manuscript investigates source attribution for images generated by diffusion models and proposes a Cross-Model Reconstruction (CMR) attack that induces attribution methods to misclassify a target image as having been generated by a designated target model. The authors further introduce READ, which improves the reliability of model-source identification by comparing the relative denoising responses of different diffusion models under controlled latent-space perturbations. Experimental results demonstrate the effectiveness of the CMR attack and show that READ can reduce the false acceptance rate of incorrect models.

Critical strengths
  1. The research problem addressed in the manuscript is important and well motivated.

  2. The proposed CMR and READ methods are simple, efficient, and interpretable.

  3. The manuscript provides extensive experiments that validate the effectiveness of the proposed methods.

Critical weaknesses
  1. The authors should further demonstrate whether the performance improvement of READ stems from the unique information provided by the Denoiser or merely from the self-supervised learning process.

  2. Although the authors acknowledge that the improvement in AUROC may arise from differences between processing conditions, this does not necessarily indicate successful discrimination of the generation source.

  3. The authors should provide a more detailed analysis of computational overhead, particularly how the cost scales with the number of models.

  4. READ exhibits a relatively high rejection rate, and the authors should further clarify the reasons for this behavior.

Questions for authors

Please refer to Weaknesses

Reviewer_x1AvOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows that passing an image once through another generator's autoencoder (CMR) is enough to fool reconstruction-based attribution methods like AEDR, AEROBLADE and LatentTracer into assigning it to the wrong model. This is tested on 12 generators, including ones that share an autoencoder, and also works on natural images. As a defense, READ uses relative denoising responses from several observer models, calibrated on clean images only. It cuts false attribution a lot, but only recovers the true source about half the time.

Critical strengths

1.The attack is very simple, and the paper does a good job explaining why it works. The parameter comparison of shared autoencoders, the JPEG/blur controls in Table 4 and the natural image results make it clear that the reconstruction evidence is acquired, not original.

2.The defense evaluation is honest. Appendix D reports joint outcomes, recovery on the baselines' failure cases, clustered bootstrap intervals and adaptive attacks, instead of just one acceptance number.

Critical weaknesses

1.Table 3 isn't a fair comparison with the extra baselines. Per Appendix C.5, Model-Fingerprint, LIDA and FSD are run on their own datasets (their public subset, GenImage, Synthbuster) and both classes are reconstructed, while READ uses the 12-model benchmark where only positives are reconstructed. Yet Table 3 shows point differences against READ. The gaps mix up method, data and protocol.

2.READ mostly rejects rather than recovers. In Table 12, READ picks only the true source for 48.12% of images and rejects both candidates for 38.27%. On the baselines' failure cases, recovery is about 45% to 47%. Since every method uses its own default threshold, it's hard to see the actual tradeoff between rejecting and recovering.

Questions for authors

1.Could READ and the extra baselines be compared on the same dataset with the same reconstruction protocol? (suggestion)

2.Prompts are shared across all generators. Are calibration and test splits disjoint by prompt and source natural image across generators? (clarification)

2.Could you show true-source recovery vs false attribution at matched clean false acceptance rates, including for the shared-autoencoder pairs? (suggestion)

Reviewer_kLKfOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the reliability of reconstruction error-based source attribution for latent diffusion models. It identifies two related vulnerabilities: distinct generators may share the same autoencoder and therefore yield identical reconstruction errors, and post-generation reconstruction can introduce evidence favoring a model that did not originally generate the image. The authors propose Cross-Model Reconstruction (CMR), which passes an image through a target model’s encoder and decoder once to induce false attribution while largely preserving visual content, without diffusion sampling or attribution queries. They also introduce READ, an attribution method that uses multiple denoisers’ relative recovery responses to controlled latent perturbations, with classifiers calibrated on clean images. Evaluation covers 13,000 images, 12 generative models, and 132 directed model pairs, supplemented by autoencoder comparisons and JPEG and blur controls. The reported results show that CMR raises mean target-model acceptance to 93.5–99.8% for three reconstruction-based baselines, whereas READ reduces it to 13.61% and improves recognition of the original source, including among models sharing an autoencoder.

Critical strengths
  1. A simple attack exposes a consequential limitation of reconstruction-based attribution. CMR requires only a single encoding and decoding pass through the target model's autoencoder, with no diffusion sampling, no iterative optimization, and no attribution queries. Its success against three training-free baselines, AEDR, AEROBLADE and LatentTracer, shows that these methods can be redirected by processing applied after generation. Paired JPEG and Gaussian blur controls, matched to CMR in LPIPS, rule out generic quality loss as the explanation: for AEDR, JPEG and CMR leave acceptance of the original source nearly equal, 45.44 versus 46.86 percent, but differ sharply on acceptance of the attack model, 30.46 versus 99.87 percent. Together with the architecture and weight comparison across 12 checkpoints, this makes the mechanism, rather than any single number, the transferable contribution.
  2. The proposed defence targets the identified ambiguity with a signal that reconstruction-based methods do not use. READ compares the relative denoising responses of a candidate denoiser and several reference denoisers to controlled symmetric latent perturbations, which lets it separate generators that share an autoencoder, a case where reconstruction errors are identical by construction. Normalisers, reference sets, classifiers and thresholds are fitted on clean calibration images only and frozen before attack evaluation. On the same CMR-attacked images, READ lowers mean attack-model acceptance to 13.61 percent, from the 91.7 to 99.9 percent reported for the reconstruction baselines, while raising acceptance of the original source to 54.24 percent, above the 34.93 to 45.76 percent of the baselines. The improvement is relative to these baselines, and is obtained with a larger access budget, which I take up in the weaknesses.
Critical weaknesses
  1. The evaluation does not sufficiently isolate the contribution of READ’s response representation from its stronger supervision and model-access requirements. READ uses seven observer models per candidate, requires access to their encoders and denoisers, and trains candidate-specific classifiers using 500 labeled clean images per registered source. In contrast, the reconstruction baselines use autoencoders and calibrated reconstruction scores without training an attribution classifier. Although the paper includes reference-count ablations, it lacks a corresponding supervised reconstruction-feature baseline using the same calibration data and classifier family. The results therefore demonstrate the effectiveness of the complete READ pipeline, but do not establish how much of its advantage comes from the proposed representation rather than the additional supervision and access. Moreover, natural-image rejection does not establish rejection of unseen generators: the classifiers are trained against the other registered models, and performance on unregistered generative sources remains unclear. These limitations restrict the demonstrated scope of the defense.
  2. The necessity of symmetric perturbations is not established. Appendix C.6 states that the component ablations reuse cached symmetric predictions, so none compares READ against ordinary denoising recovery error without the additional perturbation. Proposition 2 shows that symmetric averaging cancels the first-order perturbation term; consequently, under sufficiently small perturbations and controlled local curvature, the resulting prediction may remain close to the unperturbed prediction. This raises the question of whether the perturbations provide useful attribution information beyond standard denoising responses. For continuous-latent observers, the current construction requires 12 denoiser evaluations per image, compared with three for an unperturbed response at the same noise levels. Both an unperturbed variant and an equal-compute variant using additional noise samples are needed to determine whether symmetric perturbations improve attribution and robustness sufficiently to justify their computational cost.
Questions for authors
  1. How much of READ’s improvement comes from its denoising-response representation rather than its additional supervision and model access? Can you provide a supervised reconstruction-feature baseline using the same clean calibration images, classifier family, and threshold-selection procedure? Comparing clean and post-CMR AUROC and joint acceptance outcomes would help establish whether denoising responses provide information that reconstruction features cannot recover under comparable supervision.
  2. How does READ perform when the symmetric predictions in Equation (7) are replaced by the unperturbed prediction fk,t(zt)f_{k,t}(z_t), keeping the observer sets, noise levels, normalization procedure, and classifier settings unchanged? For continuous-latent observers, can you also compare against an equal-compute variant using four independent noise samples per noise level? These comparisons would distinguish the contribution of symmetric perturbations from ordinary denoising recovery error and additional computation.
  3. Does READ reliably reject images from generators absent from its calibration negatives, both before and after CMR? A leave-one-generator-out evaluation on the existing benchmark, excluding the held-out source’s images from classifier fitting and reference selection, would help answer this question. Please report false acceptance alongside acceptance of registered sources, so that improved rejection can be distinguished from a general loss of source recognition. If such evidence is unavailable, should the demonstrated scope be restricted to discrimination among registered generators and rejection of natural images?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.