acceptodds
Under review as a conference paper at ICLR 2027

Cross-Layer Relational Representation Modeling for Few-Shot AI-Generated Image Detection

Abstract

AI-generated images have become increasingly realistic, raising urgent concerns about content authenticity and misuse, making reliable detection under unseen generators increasingly important. Since collecting large-scale training data for every emerging generator is impractical, this challenge motivates few-shot AI-generated image (AIGI) detection, where a detector must adapt to novel generators from only a handful of labeled examples. In this work, we identify generator-dependent cross-layer relational patterns in pretrained CLIP representations as a generalizable cue for AIGI detection. Specifically, real images and images from different generators exhibit distinct cross-layer behaviors in token representations and class-patch relational patterns. Motivated by this analysis, we propose a Cross-Layer Relational Module (CLRM), which constructs relational representations through bidirectional cross-attention between class tokens and patch tokens from intermediate and final layers. CLRM then transforms such semantic-spatial correspondence patterns across layers into metric-space embeddings and combines them with prototypical learning for few-shot adaptation. Extensive experiments demonstrate that CLRM achieves state-of-the-art performance in few-shot classification and generator source attribution, while providing strong generalization and robustness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-Generated Image Detection, Deepfake Detection, AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jwpnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript proposes a cross-layer representation modeling for the problem of few-shot AI-generated image detection and generator attribution. It combines information from intermediate and final layers of a frozen image encoder through bidirectional cross-attention. A prototypical network applied as a standard application for few-shot image classification problem. Experiments evaluate transfer to unseen generators, different image encoders, and the perturbation condition.

Critical strengths
  1. Evidence of generalization across generators and model architectures. The method performs well on unseen generators and improves results with diferent architectural encoders. The prototype-based design also allows adaptation using a small labeled support set without retraining at test time. However, the contribution of prototypical learning itself is not separately established.

  2. An interesting observation about intermediate and final features. The observation of using cross-layer behaviour between final and intermediate layers as a measurement for AI generated image detector is interesting

Critical weaknesses
  1. Limited few-shot evaluation settings and missing recent baselines . The main training setup is 3-way 5-shot. Although several more classification tasks are evaluated, the results do not show how the method behaves with fewer or more shots settings. Furthermore, there are some recent baselines seems to be missing such as [1,2]

[1] Diversity over Uniformity: Rethinking Representation in Generated Image, CVPR 2026

[2] Few-Shot Synthetic Image Attribution: Identifying Unseen Generators with Limited Samples, ECCV 2026

  1. The specific benefit of cross-layer modeling is not fully developed.
    FSD already uses prototype-based few-shot learning, so the main new contribution in this paper lies in the representation module. It would be more interesting to test CLRM as an add-on to other detection frameworks, because it would further show whether its benefits extend beyond this current prototype-based setup. The ablation also does not fully isolated the improved performance comes from cross-layer modeling or prototypical networkmodeling
Questions for authors
  1. How does the proposed method deal with the problem of imbalanced data in binary classification setting, where the number of fake samples should be much more than the number of real samples?

  2. Could the authors report results with variant number of shots such as (1, 5, 10)?

  3. Does the method also work when comparing two intermediate layers instead of comparing with the final layer? A small experiment might be adequate for this. It would be interesting to see if this method could also work.

4 comments
AuthorsOpenReview ↗

Rebuttal #1

We thank the reviewer for the valuable suggestions concerning shot sensitivity, recent related work, transferability to other detection frameworks, class imbalance, and layer selection. We have added experiments addressing these points. Throughout, M, G, A, S, V, and B denote Midjourney, GLIDE, ADM, Stable Diffusion, VQDM, and BigGAN.

1. Shot ablation study

We report 1-, 3-, 5-, and 10-shot results for both binary classification and source attribution.

1-1. Binary classification accuracy (%)

Method1 shot3 shots5 shots10 shots
UFD81.991.092.894.0
NPR77.685.887.789.1
RINE86.592.993.493.9
FSD72.079.782.584.1
CLRM87.094.395.996.4

1-2. Generator source attribution accuracy (%)

Method1 shot3 shots5 shots10 shots
UFD50.967.671.775.1
NPR68.578.581.083.0
RINE67.374.576.177.6
FSD61.372.675.477.6
CLRM74.586.888.890.7

CLRM achieves the highest accuracy at every shot count.


2. Recent baselines

We appreciate the references. Diversity over Uniformity uses anti-feature-collapse learning to preserve diverse forgery cues for cross-generator detection, whereas we construct cross-layer class-patch correspondence representations for few-shot adaptation. Few-Shot Synthetic Image Attribution introduces the OmniFake dataset and the OmniDFA framework, which jointly addresses authenticity detection and few-shot source attribution with an emphasis on unseen generators.

We trained OmniDFA on GenImage under our leave-one-generator-out protocol. With 5-shot prototype-based evaluation, OmniDFA representations obtain 85.2% binary accuracy, and OmniDFA obtains 85.8% with its own zero-shot binary detector. Because OmniDFA's main strength comes from large-scale training on OmniFake, which our GenImage setting does not provide, these numbers should not be read as a direct comparison with Table 1.

We further integrated CLRM into OmniDFA. In each TwinNeXt branch, CLRM relates a middle ConvNeXt stage to the last stage, with the global-average-pooled feature as the class token and the spatial positions as patch tokens; the concatenated outputs replace the pooled backbone features before OmniDFA's projection head, and OmniDFA's training losses are unchanged. This improves the few-shot and zero-shot binary accuracy to 86.6% and 87.6% (+1.4 / +1.8), indicating that CLRM is complementary to OmniDFA.


3. CLRM with other detectors

We agree that this is an important test of generality. We therefore applied CLRM-style cross-layer relational modeling to three different feature-extraction frameworks: FSD (ResNet-based), DINOv2, and DRCT. The results below are leave-one-generator-out 5-shot average accuracies on GenImage. DINOv2 rows are from Table 5 of the paper.

For FSD (ResNet-50), we use the outputs of the 2nd and 4th residual stages. The globally pooled feature of each stage serves as its class token and the spatial positions of its feature map serve as patch tokens. Because the two stages have different channel dimensions, we project them to a common dimension with a 1×1 convolution before cross-attention.

For DRCT, we use its CLIP version (a frozen CLIP ViT-L/14 followed by a trainable 1024-d embedding layer) and retrain it under our few-shot protocol: DRCT's binary cross-entropy is replaced by the prototypical loss, while its contrastive loss on diffusion-reconstructed images (α = 0.3). Reconstructions are generated online with the Stable Diffusion VAE. DRCT + CLRM differs only in the input to this embedding layer: instead of CLIP's projected class token, it receives the CLRM output computed between the outputs of block 12 (intermediate) and block 24 (final) of the frozen CLIP ViT-L/14.

BinaryMGASVBAvg.
FSD76.297.375.385.371.789.382.5
FSD + CLRM87.298.184.691.381.193.489.3
DINOv271.481.866.170.181.389.276.7
DINOv2 + CLRM70.996.684.780.292.597.087.0
DRCT86.192.981.984.791.592.588.3
DRCT + CLRM92.796.992.888.097.698.594.4
AttributionMGASVBAvg.
FSD79.578.771.274.669.379.275.4
FSD + CLRM87.386.481.284.479.188.684.5
DINOv255.750.851.553.050.450.652.0
DINOv2 + CLRM84.481.883.681.283.788.983.9
DRCT74.473.274.474.174.272.373.8
DRCT + CLRM92.889.394.695.192.996.193.5

Adding CLRM improves the average accuracy of both tasks for all three feature extractors, with the largest gains in attribution.

AuthorsOpenReview ↗

Rebuttal #2

4. Additional ablation variants

We extend the architectural analysis with MLP alternatives that incorporate patch tokens and with a bidirectional inner-layer attention control. The results are averaged over 6 subsets (M, G, A, S, V, B).

ArchitectureBinary classification (%)Source attribution (%)
(i) MLP (class tokens only), variant (c)92.982.2
(ii) MLP (class tokens + mean-pooled patch tokens)93.485.3
(iii) MLP (class tokens + flattened patch tokens)92.283.3
(iv) Bidirectional inner-layer attention95.387.9
(v) Bidirectional cross-layer attention (CLRM)95.988.8

(ii) receives the class tokens and the mean-pooled patch tokens of both layers. (iii) retains all 196 patch positions after a learned per-token 768→8 projection before MLP fusion. Both underperform CLRM despite having access to the same inputs, so the gains cannot be explained by the mere inclusion of intermediate and final CLIP features.

(iv) The inner-layer control uses Proj([CA(f_cls, f_pat, f_pat); CA(g_cls, g_pat, g_pat)]), maintaining two attention branches but restricting each branch to a single layer. It performs competitively, demonstrating that class–patch attention itself is useful. However, CLRM further improves both binary classification and source attribution. This controlled comparison supports an additional benefit from cross-layer rather than only within-layer correspondence.

To further isolate the effect of directionality from additional parameters, we also trained a two-branch unidirectional control that concatenates CA(f_cls, g_pat, g_pat; θ₁) and CA(f_cls, g_pat, g_pat; θ₂) with independent parameters, matching CLRM’s two-branch capacity.

ArchitectureBinary Classification (%)Source Attribution (%)
Single-direction cross-layer attention - variant (e)94.185.3
Two parallel copies of the same direction94.887.7
Opposite-direction branches (CLRM)95.988.8

The parallel control improves upon the single branch but remains 1.1 points below CLRM on both tasks. Therefore, the improvement is not solely attributable to doubling the number of attention branches. The reverse-direction relationship contributes additional information.


5. Class imbalance

Our episodic training samples an equal number of support and query examples per selected class, even when the available training pool contains many more fake images than real images. This prevents the episode-level learning objective from being dominated by the more frequent class. To assess sensitivity to the size of the underlying class pools, we further reduced either the real-image pool or the fake-image pool to 1% of its original size, while keeping the other pool unchanged.

Real training pool retainedFake training pool retainedBinary acc. (%)Attribution acc. (%)
100%100%95.988.8
1%100%95.988.3
100%1%94.884.9

CLRM is robust to training-pool imbalance; reducing the fake pool mainly lowers attribution accuracy, as fewer images per generator remain.


6. Alternative two intermediate layers

We replaced the original pair (block 5 + final block) with blocks 3 and 9, and retrained CLRM with all other settings unchanged.

Layer configurationBinary acc. (%)Attribution acc. (%)
Final-layer class token only92.871.7
CLRM: blocks 3 and 994.986.4
CLRM: block 5 and final block95.988.8

Using two intermediate layers substantially outperforms the final-layer-only baseline. The original block-5/final configuration remains strongest, but the intermediate/intermediate result shows that CLRM does not require the final block and can model informative class–patch correspondence between two intermediate depths of a frozen encoder.


We thank the reviewer again for the constructive comments. We will carefully incorporate them, together with the additional results above, into the revised manuscript.

AuthorsOpenReview ↗

Further Analysis

Why cross-layer pairing helps

We would like to share an additional analysis on the benefits of cross-layer modeling. In our earlier response, the bidirectional inner-layer attention control (variant (iv) of the additional ablation table) reached 95.3% / 87.9% in binary classification / source attribution, close to CLRM's 95.9% / 88.8%. This control is identical to CLRM in architecture, parameter count (4.7M), and training, and the only difference is whether each class token attends to the patch tokens of its own layer or of the other layer. It therefore isolates the effect of cross-layer pairing. To examine whether this effect is specific to CLIP, we repeated the comparison on DINOv2 (ViT-B/14), following the setting of Sec. 5.3 of the paper.

Binary / attribution accuracy (%), 5-shot, averaged over the six leave-one-generator-out splits. | Backbone | Bidirectional inner-layer attention | Cross-layer attention (CLRM) | Gain from cross-layer pairing | |---|---|---|---| | CLIP ViT-B/16 | 95.3 / 87.9 | 95.9 / 88.8 | +0.6 / +0.9 | | DINOv2 ViT-B/14 | 85.6 / 82.2 | 87.0 / 83.9 | +1.4 / +1.7 |

Cross-layer pairing improves both tasks on both backbones, and the gain on DINOv2 is about twice that on CLIP.

Why cross-layer pairing helps. Cross-layer pairing exposes the mismatch between depths, which Fig. 1 shows to be generator-dependent. CLRM is designed around this: its attention (Eqs. 3-4) relates the class token of one layer to the patch tokens of the other, whereas the inner-layer control relates tokens within the same layer. Within a layer, the class token and the patch tokens are entangled. Specifically, at every block, ViT self-attention relates the class token to the patch tokens of the same depth, and pre-training optimizes this relation jointly with the tokens themselves. The within-layer correspondence therefore largely follows the encoder's own aggregation and captures less of the mismatch, and the inner-layer control combines the two depths only after aggregating each separately. In contrast, the encoder's self-attention never relates tokens of different depths, so the cross-layer correspondence reflects how the image changes between depths. This reasoning applies to ViT encoders in general, in line with the DINOv2 result. We believe this is why cross-layer pairing adds a consistent gain on top of class–patch attention: attention provides the mechanism for selecting evidence, and cross-layer pairing determines what that selection measures.


We hope this additional analysis further clarifies the contribution of cross-layer modeling, and we would be glad to answer any further questions.

Thank you again for your review and valuable feedback.

We have provided detailed responses to your comments and uploaded a revised manuscript incorporating the corresponding changes.

We would greatly appreciate it if you could take a moment to review our responses and the updated manuscript. Please let us know if you have any remaining questions or concerns, and we would be happy to address them.

Thank you for your time and consideration.

Reviewer_Xpw9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Cross-Layer Relational Module (CLRM) for few-shot AI-generated image detection. The authors observe that real images and images from different generators exhibit different feature transitions across CLIP layers. Based on this observation, CLRM uses bidirectional cross-attention between class tokens and patch tokens from intermediate and final layers to construct relational features. The resulting features are used in a prototypical learning framework for few-shot binary detection and generator source attribution. Experiments on GenImage and other datasets show improvements over existing methods. The authors also provide architectural ablations and evaluations under different backbones and image perturbations.

Critical strengths
  1. The cross-layer feature analysis is interesting and provides a reasonable motivation for the proposed method. The idea of combining intermediate spatial information with final-layer semantic features is relevant to AI-generated image detection.

  2. The experimental results are convincing, especially for generator source attribution. CLRM improves the average accuracy from 81.0% to 88.8% over the strongest baseline. The architectural ablations also show clear improvements over simpler feature fusion and unidirectional cross-attention. Results with different backbones further support the effectiveness of the method.

Critical weaknesses
  1. The methodological novelty is limited. The proposed CLRM mainly combines bidirectional cross-attention with prototypical learning, both of which are well-established techniques. Although the method improves classification accuracy, I am not fully convinced that the experiments demonstrate the claimed generator-dependent cross-layer relational patterns, rather than simply benefiting from complementary features at different layers.

  2. Some important experimental comparisons are still missing.The bidirectional design is compared with a unidirectional version, but the additional feature branch and model capacity are not fully controlled. It is therefore unclear how much of the improvement comes from bidirectional modeling itself. Also, the paper mainly reports 5-shot results, without a detailed analysis under different shot numbers. This limits the evaluation of the method's few-shot generalization ability.

Questions for authors

See weakness.

6 comments
AuthorsOpenReview ↗

Rebuttal #1

We appreciate the reviewer’s concern about whether the reported gains are caused by relational modeling or by access to complementary features and increased model capacity.

We have added controlled experiments to test whether our method explains the gains.


1. Distinguishing cross-layer relational modeling from generic feature fusion

We control for the input features. We trained MLP baselines using (i) class tokens from both (variant (c) of Table 7), (ii) class tokens and mean-pooled patch tokens of both layers, and (iii) class tokens and all 196 patch tokens of both layers after a per-token 768→8 projection. The same episodic protocol was used throughout.

RepresentationBinary acc. (%)Attribution acc. (%)
(i) MLP (class tokens only)92.982.2
(ii) MLP (class + pooled patch tokens)93.485.3
(iii) MLP (class + flattened patch tokens)92.283.3
CLRM95.988.8

Controls (ii) and (iii) receive the class and patch tokens of the same two layers but cannot learn class-conditioned cross-layer patch selection. They are not limited by capacity: (ii) has 11.8M trainable parameters, 2.5× CLRM's 4.7M, and (iii) has 4.2M. Their lower performance, particularly in attribution, indicates that generic multi-layer fusion alone does not reproduce CLRM's gains.


2. Further analyses

We performed the following analyses on the trained models, using the same 5-shot leave-one-generator-out evaluation protocol as in Table 1.

2-1. Decomposing CLRM: the complementary-feature path vs. the relation path

The output of CLRM combines two things: the ‘content’ of the other layer's patches (through the value and output projections) and the ‘relation’, i.e., which patches are selected by the cross-layer class-patch correspondence (through the attention). We isolate each path on the trained models below.

(i) Complementary-feature path only (uniform attention). At test time we replace every attention distribution with the uniform distribution (1/196) and keep all trained weights. The embedding then retains the projected features of both layers but no class-conditioned selection. (ii) Relation path only (attention-only embedding). We use the attention pattern itself as the embedding: the projected cosine similarities between the query and the 196 keys for the 8 heads of both directions (3,136 numbers per image), with no value vectors and therefore no information about what the patches look like. We also report the same map computed without training. The frozen CLIP final-layer class token is included as a reference global representation.

EmbeddingBinary acc.Binary AUROCAttribution acc.Held-out generator recall in attribution
CLRM (full)95.90.99088.875.0
(i) Complementary-feature path only (uniform attention)91.10.95980.967.9
(ii) Relation path only: projected query–key cosine (3,136-d)77.10.81750.251.9
Untrained relation: cos⁡(fcls,gpat,i)\cos(f_{cls}, g_{pat,i}), cos⁡(gcls,fpat,i)\cos(g_{cls}, f_{pat,i})64.00.66432.534.3
Frozen CLIP final-layer class token77.70.84946.748.5

"Held-out generator recall": fraction of the held-out generator's images assigned to the correct source among the seven classes, given only its five support images (chance 14.3%).

  • (i) Complementary features are necessary but not sufficient. Without the learned selection, CLRM drops by 4.8 points in binary accuracy and 7.9 points in attribution.
  • (ii) The relation alone is informative. The attention pattern, without any patch content, reaches 77.1% / 50.2%, on par with the frozen CLIP class token (77.7% / 46.7%), and it attributes images to one of seven sources at 3.5× chance. The same map without training is much weaker (64.0% / 32.5%), so the discriminative correspondence pattern is learned. It also transfers: the held-out generator, represented only by its five support images, is attributed to the correct source among seven for 51.9% of its images from the attention pattern alone (34.3% for the untrained map), and only 12.7% of its images are mistaken for real ones.

These findings suggest that learned correspondence scores carry discriminative information in addition to the underlying patch content.

AuthorsOpenReview ↗

Rebuttal #2

2-2. Evidence for generator-dependent cross-layer relational patterns

To test whether this pattern is a property of the generator rather than of the individual image, we replace the key/value patch tokens with those of another image at test time, while keeping each image's own class tokens. We test two settings: (i) another image of the same class in the episode, and (ii) for queries, an image of the other class of the binary episode (the label stays that of the query's own class token).

Key/value patch sourceBinary acc. (%)AUROCAttribution acc. (%)
Original CLRM95.90.99088.8
(i) Different image, same class96.50.99289.7
(ii) Different image, opposite class (queries only)13.60.066–

Replacing patches with those of another image from the same class does not degrade performance, whereas using patches from the opposite class changes the majority of binary predictions. The relation CLRM uses is therefore shared by the images of a generator: it is how a generator's class tokens correspond to its patch evidence across layers. This is what we mean by a generator-dependent cross-layer relational pattern, as opposed to an image-specific cue.


3. Additional architectural designs

3-1. Inner-layer attention Proj([CA(f_cls, f_pat, f_pat); CA(g_cls, g_pat, g_pat)])

ArchitectureBinary classification (%)Source attribution (%)
Bidirectional inner-layer attention95.387.9
Bidirectional cross-layer attention (CLRM)95.988.8

The inner-layer control maintains two attention branches but restricts each branch to a single layer. It performs competitively, demonstrating that class–patch attention itself is useful. However, CLRM further improves both binary classification and source attribution. This controlled comparison supports an additional benefit from cross-layer rather than only within-layer correspondence.

3-2. Controlling branch capacity

To test whether the gain comes from the reverse attention direction rather than simply additional capacity, we replaced the second direction with an independently parameterized copy of the first direction: z_parallel = Proj([CA(f_cls, g_pat, g_pat; θ₁); CA(f_cls, g_pat, g_pat; θ₂)])

ArchitectureBinary acc. (%)Attribution acc. (%)
Single-direction cross-layer attention, variant (e)94.185.3
Two parallel copies of the same direction94.887.7
Opposite-direction branches (CLRM)95.988.8

The parallel control improves upon the single branch but remains 1.1 points below CLRM on both tasks. Therefore, the improvement is not solely attributable to doubling the number of attention branches. The reverse-direction relationship contributes additional information.


4. Shot ablation

We report 1-, 3-, 5-, and 10-shot results for both binary classification and source attribution.

4.1. Binary classification accuracy (%)

Method1 shot3 shots5 shots10 shots
UFD81.991.092.894.0
NPR77.685.887.789.1
RINE86.592.993.493.9
FSD72.079.782.584.1
CLRM87.094.395.996.4

4.2. Generator source attribution accuracy (%)

Method1 shot3 shots5 shots10 shots
UFD50.967.671.775.1
NPR68.578.581.083.0
RINE67.374.576.177.6
FSD61.372.675.477.6
CLRM74.586.888.890.7

CLRM achieves the highest accuracy at every evaluated shot count. We will add the analysis to the revised manuscript.


We thank the reviewer again for the constructive comments. We will carefully incorporate them, together with the additional results above, into the revised manuscript.

Thanks for your explanation. The authors have addressed my concern. I have no further questions about 1 and 2.1

Reviewer_u2nLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a few-shot AI-generated image detection method. The authors observe that the difference between the final layer and intermediate layers extracted by CLIP varies across different image generators. Based on this observation, this paper proposes a "Cross-Layer Relational Module" which computes cross-attention between the final and intermediate layer features. By few-shot episodic training, the model learns to clasify images produced by unseen image generators. Experiments demonstrate the effectiveness of the proposed approach.

Critical strengths
  1. Clear problem statement. Detecting AI-generated images is a significant problem. As the number of new models continues to grow, the challenge of distinguishing the specific source model used to generate an image has become an interesting problem.
  2. Simple but effective method with supportive observation. The solution proposed in this paper stems from a simple observation: the relation between the final layer and the intermediate layers differs for real and synthetic images. Based on this, this paper introduces a dedicated module designed to leverage this characteristic, improving the ability to distinguish AI-generated images.
  3. Convincing performance and comprehensive experiment. The experimental results demonstrate clear performance improvement over baseline methods.
Critical weaknesses
  1. Core mechanisms are not sufficiently explained. This paper claims the CLRM as a semantic-visual consistency tester. However, the actual operation is to compute attention by using the class token from one layer as Q and the patch tokens form another layer as K,V. My concern is that, the attention mechanism itself does not explicitly models such "inconsistency", also does not directly mesure any layer difference. Therefore, the analysis presented in the introduction and Figure 1 does not fully algin with the method design.
  2. Lack ablation variants. The ablation study shows comparison of different architectural variants. However, It is unclear why are these variants chosen. The following variants should also be considered:
    1. z=MLP([fC,fP,gC,gP])z=\mathrm{MLP}([f_C,f_P,g_C,g_P]). This measures whether the cross-attention calculation (variant f) performs better than pure MLP under the same amount of input information (fC,fP,gC,gP)(f_C,f_P,g_C,g_P).
z=Proj[CA(fC,cP,fP) ; CA(gC,gP,gP)].z=\mathrm{Proj}\big[ \mathrm{CA}(f_C,c_P,f_P)\ ;\ \mathrm{CA}(g_C,g_P,g_P) \big].

This measures whether the cross-layer relation modeling (variant f) performs better than inner-layer relation modeling. 3. The authors mentioned that "in ablation studies where we vary ˜K to analyze the impact of support set size". However, I did not find any ablation regarding different number of shots in the paper.

Questions for authors
  1. How the values in Figure 1 are computed, including the features extraction, exact similarity metrics, and how the heatmaps are constructed? The caption outlines the analysis but leaves these detailed implementation unclear.
4 comments
AuthorsOpenReview ↗

Rebuttal #1

We thank the reviewer for the valuable comments. We have conducted additional analyses and ablations to clarify the mechanism, isolate the contribution of cross-layer relational modeling, and examine the effect of support-set size. Throughout, M, G, A, S, V, and B denote Midjourney, GLIDE, ADM, Stable Diffusion, VQDM, and BigGAN.

1. Additional analyses

To clarify what CLRM encodes, we analyzed the trained models under the 5-shot leave-one-generator-out protocol of Table 1.

1-1. The cross-layer attention pattern alone carries real/fake and source information

We compare the full CLRM with four representations:

(i) attention only, where we discard the value aggregation and use the projected query–key similarities themselves (2 directions × 8 heads × 196 patches = 3,136 dimensions) as the embedding.

(ii) uniform attention, where we replace the attention distributions of the trained CLRM with uniform weights (1/196) and keep all other trained parameters.

(iii) the same query–key similarities without training.

(iv) the frozen CLIP final-layer class token.

Results are averaged over the six leave-one-out splits.

RepresentationBinary acc. (%)Binary AUROCAttribution acc. (%)Held-out generator recall
Full CLRM95.90.99088.875.0
(i) Projected query–key attention only77.10.81750.251.9
(ii) CLRM with uniform attention91.10.95980.967.9
(iii) Query–key similarity without training64.00.66432.534.3
(iv) Frozen CLIP final-layer class token77.70.84946.748.5

(i) The projected query-key similarities contain no value vectors or raw patch-token embeddings, yet reach 77.1% binary accuracy and 50.2% source attribution accuracy. These results demonstrate that the correspondence scores are themselves discriminative, rather than merely serving as weights for combining patch features.

(ii) Replacing the learned attention with uniform aggregation reduces accuracy by 4.8 and 7.9 percentage points in binary classification and source attribution, respectively. This indicates that class-conditioned patch selection contributes to the representation.

(iii) Without training, the same similarities are much weaker, so the discriminative pattern is learned.

Here, “held-out generator recall” denotes the percentage of samples from the excluded generator assigned to their correct source among seven classes, using only five support examples of that generator.

1-2. CLRM captures a generator-level relation rather than a per-image consistency score

We replace the key/value patch tokens with those of another image, while keeping each image's own class tokens. We test two settings: i) the patch tokens of another image of the same class in the episode, (ii) for each query, the patch tokens of an image from the other class of the binary episode, with the label remaining that of the query's own class token.

Key/value patch sourceBinary acc. (%)AUROCAttribution acc. (%)
Original image95.90.99088.8
(i) Different image, same class96.50.99289.7
(ii) Different image, opposite class (queries only)13.60.066–

Replacing patches with those of another image from the same class does not degrade performance, whereas using patches from the opposite class flips 86% of the binary predictions. These observations show that the patch evidence supplied to CLRM contains strong source-associated information and is compatible across different images of the same source.

With the attention-only and uniform-attention analyses, this supports our interpretation of CLRM as learning generator-relevant cross-layer correspondence rather than computing an image-wise consistency score.

AuthorsOpenReview ↗

Rebuttal #2

2. Additional ablation variants

We extend this analysis with the two controls the reviewer suggested: MLP alternatives that receive patch tokens, and a bidirectional inner-layer attention control. Results are averaged over the six splits (M, G, A, S, V, B).

ArchitectureBinary classification (%)Source attribution (%)
(i) MLP (class tokens only), variant (c)92.982.2
(ii) MLP (class tokens + mean-pooled patch tokens)93.485.3
(iii) MLP (class tokens + flattened patch tokens)92.283.3
(iv) Bidirectional inner-layer attention95.387.9
(v) CLRM95.988.8

(ii) receives the class tokens and the mean-pooled patch tokens of both layers. (iii) retains all 196 patch positions after a learned per-token 768→8 projection before MLP fusion. Both underperform CLRM despite having access to the same inputs, so the gains cannot be explained by the mere inclusion of intermediate and final CLIP features.

(iv) The inner-layer control uses Proj([CA(f_cls, f_pat, f_pat); CA(g_cls, g_pat, g_pat)]), maintaining two attention branches but restricting each branch to a single layer. It performs competitively, demonstrating that class-patch attention itself is useful. However, CLRM further improves both binary classification and source attribution. This controlled comparison supports an additional benefit from cross-layer rather than only within-layer correspondence.


3. Support-set-size ablation

We apologize for referring to a varying-shot ablation that was not included. We now report 1-, 3-, 5-, and 10-shot evaluations in both binary classification and source attribution.

3-1. Binary classification accuracy (%)

Method1 shot3 shots5 shots10 shots
UFD81.991.092.894.0
NPR77.685.887.789.1
RINE86.592.993.493.9
FSD72.079.782.584.1
CLRM87.094.395.996.4

3-2. Generator source attribution accuracy (%)

Method1 shot3 shots5 shots10 shots
UFD50.967.671.775.1
NPR68.578.581.083.0
RINE67.374.576.177.6
FSD61.372.675.477.6
CLRM74.586.888.890.7

CLRM achieves the highest accuracy at every shot count.


4. How was Figure 1 constructed?

4-1. Fig 1. (a)

Fig. 1(a) uses the frozen backbone and the two layers (6th block and last block). For each image, we take the patch tokens of both layers and compute, for every patch p, the sum over the 768 channels of their difference, dp=Σk(fp,kp−gp,kp)d_p = Σ_k (f^p_{p,k} − g^p_{p,k}). The 196 values are reshaped to the 14×14 patch grid, min–max normalized per image to [0, 1], bilinearly upsampled to 224×224, and rendered with the viridis colormap. The examples are [randomly sampled training crops / selected images] of each class.

4-2. Fig 1. (b)

Tokens are read directly from the residual stream at the block outputs. For each image, we compute the cosine similarity between the intermediate and final class tokens, and each bar is the mean over all sampled images of that class (real, or one generator).

4-3. Fig. 1 (c)

We use the final-layer class token fcls\mathbf{f}^{\text{cls}} as a fixed semantic reference and measure how its correspondence with each spatial position changes between the two layers.

  1. Class-patch similarity in each layer. For every patch p∈{1,…,196}p \in \{1,\dots,196\}:

apmid=cos⁡(fcls,gppat),apfin=cos⁡(fcls,fppat).a^{\text{mid}}_p = \cos\left(\mathbf{f}^{\text{cls}}, \mathbf{g}^{\text{pat}}_p\right), \qquad a^{\text{fin}}_p = \cos\left(\mathbf{f}^{\text{cls}}, \mathbf{f}^{\text{pat}}_p\right).

  1. Per-patch transition.

Δp=apfin−apmid.\Delta_p = a^{\text{fin}}_p - a^{\text{mid}}_p.

  1. Spatial variance of the transition. For each image, we take the population variance of Δp\Delta_p across the P=196P = 196 patches:

v(x)=1P∑p=1P(Δp−Δˉ)2,Δˉ=1P∑p=1PΔp.v(\mathbf{x}) = \frac{1}{P}\sum_{p=1}^{P}\left(\Delta_p - \bar{\Delta}\right)^2, \qquad \bar{\Delta} = \frac{1}{P}\sum_{p=1}^{P}\Delta_p.

  1. Per-class aggregation. Each bar is the mean of v(x)v(\mathbf{x}) over all sampled images of that class.

Because the reference vector is identical for both layers, Δp\Delta_p reflects only how the patch tokens move relative to the same semantic summary. Note that apmida^{\text{mid}}_p is the untrained counterpart of the CLRM attention logit in Eq. 3 (final class token as query, intermediate patch tokens as keys).


We thank the reviewer again for the constructive comments. We will carefully incorporate them, together with the additional results above, into the revised manuscript.

AuthorsOpenReview ↗

Further Analysis

Why cross-layer pairing helps

We would like to share an additional analysis on the benefits of cross-layer modeling. In our earlier response, the bidirectional inner-layer attention control (variant (iv) of the additional ablation table) reached 95.3% / 87.9% in binary classification / source attribution, close to CLRM's 95.9% / 88.8%. This control is identical to CLRM in architecture, parameter count (4.7M), and training, and the only difference is whether each class token attends to the patch tokens of its own layer or of the other layer. It therefore isolates the effect of cross-layer pairing. To examine whether this effect is specific to CLIP, we repeated the comparison on DINOv2 (ViT-B/14), following the setting of Sec. 5.3 of the paper.

Binary / attribution accuracy (%), 5-shot, averaged over the six leave-one-generator-out splits. | Backbone | Bidirectional inner-layer attention | Cross-layer attention (CLRM) | Gain from cross-layer pairing | |---|---|---|---| | CLIP ViT-B/16 | 95.3 / 87.9 | 95.9 / 88.8 | +0.6 / +0.9 | | DINOv2 ViT-B/14 | 85.6 / 82.2 | 87.0 / 83.9 | +1.4 / +1.7 |

Cross-layer pairing improves both tasks on both backbones, and the gain on DINOv2 is about twice that on CLIP.

Why cross-layer pairing helps. Cross-layer pairing exposes the mismatch between depths, which Fig. 1 shows to be generator-dependent. CLRM is designed around this: its attention (Eqs. 3-4) relates the class token of one layer to the patch tokens of the other, whereas the inner-layer control relates tokens within the same layer. Within a layer, the class token and the patch tokens are entangled. Specifically, at every block, ViT self-attention relates the class token to the patch tokens of the same depth, and pre-training optimizes this relation jointly with the tokens themselves. The within-layer correspondence therefore largely follows the encoder's own aggregation and captures less of the mismatch, and the inner-layer control combines the two depths only after aggregating each separately. In contrast, the encoder's self-attention never relates tokens of different depths, so the cross-layer correspondence reflects how the image changes between depths. This reasoning applies to ViT encoders in general, in line with the DINOv2 result. We believe this is why cross-layer pairing adds a consistent gain on top of class–patch attention: attention provides the mechanism for selecting evidence, and cross-layer pairing determines what that selection measures.


We hope this additional analysis further clarifies the contribution of cross-layer modeling, and we would be glad to answer any further questions.

Thank you again for your review and valuable feedback.

We have provided detailed responses to your comments and uploaded a revised manuscript incorporating the corresponding changes.

We would greatly appreciate it if you could take a moment to review our responses and the updated manuscript. Please let us know if you have any remaining questions or concerns, and we would be happy to address them.

Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the rebuttal

Dear Reviewers, ACs, SACs, and PCs,

We sincerely appreciate the continued effort.

To assist with the evaluation, we have compiled a summary of the reviewers' main concerns along with our corresponding responses. All new or modified content in the revised manuscript is highlighted in blue.


1. What CLRM Actually Learns [u2nL, Xpw9]

  • What CLRM computes: The CLRM output combines two parts: the 'relation', i.e., which patches of the other layer the class token attends to, and the 'content' of those patches. We analyzed the trained models to see what each part contributes. (Appendix C.5)
  • The relation itself is discriminative: Using only the learned attention pattern as the embedding, with no patch content at all, performs on par with the frozen CLIP class token. It also attributes the held-out generator correctly 3.6× above chance. The same similarities computed without training are much weaker, so this pattern is learned rather than inherited from CLIP.
  • The content alone is not enough: Replacing the learned attention with uniform attention keeps all patch content but removes the class-conditioned selection, and accuracy drops by 4.8%p / 7.9%p. Retrained MLPs over the same tokens show the same trend, so CLRM's gain comes from which evidence is selected across layers.
  • Generator-level, not image-level: We keep each image's own class tokens and replace its patch tokens with those of another image. Patches from another image of the same generator keep performance, while patches from the opposite class cause binary accuracy to collapse to about 14%. CLRM therefore captures a cross-layer relation shared by the images of a generator, rather than an image-specific cue.

2. Why Cross-Layer Relation Modeling Matters [u2nL, Xpw9, jwpn]

  • Cross-layer vs. inner-layer: Against a bidirectional inner-layer control, cross-layer pairing consistently improves both tasks on CLIP (+0.6%p / +0.9%p) and on DINOv2 (+1.4%p / +1.7%p). (Section 5.4)
  • Insight: ViT self-attention already relates tokens within a layer but never across depths, so cross-layer correspondence captures how the representation changes between depths, which is generator-dependent.
  • Not just more features or capacity: MLPs given the same class and patch tokens, even with 2.5× more parameters, and two parallel same-direction branches all remain below CLRM. (Appendix C.6)

3. Beyond the Prototypical Setup: Plug-and-Play [jwpn]

  • Isolating the representation: All baselines share the same prototypical objective and episodes, so the comparisons isolate the representation module. (Section 4.1)
  • Plug-and-play: Adding CLRM improves FSD (ResNet-50) by +6.8%p / +9.1%p and DRCT (contrastive CLIP detector) by +6.1%p / +19.7%p in binary / attribution accuracy, showing that CLRM extends to CNN features and other training objectives. (Section 5.5, Appendix C.8)

4. Support Set Size [u2nL, Xpw9, jwpn]

  • 1/3/5/10-shot evaluation: CLRM achieves the best accuracy at every shot count in both tasks. (Appendix C.4)
  • Data efficiency: CLRM with only 3 shots outperforms all baselines with 10 shots.

5. Additional Analyses [jwpn]

  • Class imbalance: Episodic sampling and class-mean prototypes keep the decision rule unbiased. Reducing the real training pool to 1% causes no drop in binary accuracy. (Appendix C.10)
  • Two intermediate layers: Relating blocks 3 and 9 still outperforms the final-layer baseline by +14.7%p in attribution, so CLRM does not depend on the final layer. (Appendix C.9)

6. Related Work [jwpn] and Details of Fig. 1 [u2nL]

  • Related work: We added Diversity over Uniformity (CVPR 2026) and Few-Shot Synthetic Image Attribution (ECCV 2026) and clarified their differences from CLRM. (Section 2)
  • Fig. 1: We added the full procedure, formulas, and pseudo-code, and its link to the attention in CLRM. (Appendix B)

The revised paper has been uploaded, and detailed responses to each reviewer are provided below. We believe the added analyses and clarifications substantially strengthen both the motivation and the empirical support of our work, and we kindly request a careful consideration in light of these improvements.

Sincerely,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.