acceptodds
Under review as a conference paper at ICLR 2027

SCORE: Semantic-Conditioned Orthogonal Refinement for Generalizable AI-Generated Image Detection

Abstract

Generalizable forgery detection requires forensic evidence that remains reliable under shifts in content, acquisition conditions, and generative mechanisms. A central challenge is that the diagnostic meaning of many forensic cues depends on the visual context in which they occur: local texture, boundary, and frequency responses can vary with identity, pose, expression, scene structure, and image quality. This motivates a controlled use of semantic context when forming and refining detection decisions. We propose SCORE, a Semantic-Conditioned Orthogonal Refinement framework that organizes detection into a semantic-conditioned primary stage and a conservative refinement stage. SCORE first uses the current visual state to parameterize a gated residual and forms an independently supervised primary prediction. It then constructs a sample-conditioned low-dimensional basis and applies soft orthogonal attenuation to obtain a structured refinement view that preserves part of the basis-aligned information. A conditional refinement head operates on this detached representation, while uncertainty derived from the detached primary prediction controls the strength of a bounded logit correction. SCORE achieves an average video AUC of across five cross-dataset targets, an average video AUC of over eight unseen manipulations in DF40, and mAP and mAcc across 19 UniversalFakeDetect (UFD) test sources. These results support structured semantic conditioning and controlled refinement as an effective approach to forgery detection under diverse distribution shifts.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
AI-generated image detection, deepfake detection, cross-domain generalization, semantic-conditioned learning, conditional orthogonal decomposition

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qtBoOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCORE, a semantic-conditioned refinement framework for generalizable AI-generated image detection. The method addresses the challenge that forensic cues may have different discriminative meanings depending on the visual context. SCORE consists of two stages: Semantic-Conditioned Primary Prediction (SCPP), which uses feature gating to produce a context-dependent primary prediction, and Instance-Conditioned Soft-Orthogonal Refinement (ISOR), which constructs a sample-specific orthogonal subspace to refine forensic representations. An uncertainty-aware mechanism then applies a bounded correction to the primary prediction. Experiments on cross-dataset face forgery detection and UniversalFakeDetect demonstrate improved generalization performance.

Critical strengths

The proposed method is evaluated across different forensic scenarios, including AI-generated image detection and deepfake detection, demonstrating its applicability to diverse forgery detection tasks.

Critical weaknesses
  1. The paper does not clearly explain whether the orthogonal subspace learned by ISOR represents semantic information or forensic cues, making its underlying mechanism difficult to understand. Moreover, the soft-orthogonal attenuation only removes 25% of the projected component. Why is this ratio chosen, and what advantages does it offer over directly using the original features?
  2. Why does Baseline + SCPP perform worse than the baseline, while combining SCPP with ISOR improves performance on UFD? The paper lacks a convincing analysis of the interaction and complementarity between these two modules.
  3. The current ablation studies are insufficient to justify the necessity of the various auxiliary loss functions and hyperparameter choices. More detailed analysis is needed to clarify their individual contributions.
  4. The presentation of experimental results should be more consistent. Some tables report values as percentages, while others use decimals. A unified reporting format would improve readability.
Questions for authors

Please refer to the Weaknesses section above for detailed questions and concerns, particularly regarding the interpretation of the ISOR subspace, the interaction between SCPP and ISOR, and the justification of auxiliary losses and hyperparameter choices.

1 comment
AuthorsOpenReview ↗

Clarifications on soft attenuation and module interaction

Thank you for the constructive questions. On the fixed 34,400-image UFD list, SCORE achieves 95.23% mAcc and 99.55% mAP, improving over our internal Baseline by 4.74 and 0.71 percentage points, respectively. ISOR-only provides a 3.14-point mAcc gain, and adding SCPP yields a further 1.60-point gain.

1. Subspace interpretation and soft attenuation. For an image feature zz, ISOR constructs an orthonormal basis QQ and projector P=QQ⊤P=QQ^\top. The feature decomposes into PzPz and (I−P)z(I-P)z. The projection-energy objective constrains their relative magnitudes without assigning semantic or forensic labels to the basis directions. We therefore interpret the subspace as an image-conditioned geometric refinement view, rather than an explicitly identified semantic or forensic subspace.

The refinement head receives

r=(I−P)z+(1−α)Pz.r=(I-P)z+(1-\alpha)Pz.

With α=0.25\alpha=0.25, this preserves the orthogonal complement and 75% of the projected amplitude. The rationale is to change the relative weighting of the two components while keeping information from both available to the refinement head. The separate 0.25-logit correction cap constrains the resulting adjustment to the primary prediction.

We use α=0.25\alpha=0.25 as a shared setting across Face and UFD. This is a conservative design choice; the current component ablations evaluate the complete refinement module and do not establish that 0.25 is optimal or isolate its advantage over using the original feature (α=0\alpha=0). We will clarify this distinction between design rationale and experimental evidence.

2. Interaction between SCPP and ISOR. The matched evaluation on the fixed UFD list gives:

ConfigurationmAcc (%) ↑mAP (%) ↑ΔmAcc (pp)ΔmAP (pp)
Baseline (A0)90.4998.84——
SCPP-only (A1)87.5298.00−2.97−0.84
ISOR-only (A2)93.6399.02+3.14+0.18
SCORE (A3)95.2399.55+4.74+0.71

All configurations were evaluated on the same frozen, balanced 34,400-image list, with a fixed threshold of 0.5 and equal weighting across 19 sources. Differences are percentage points, computed before rounding.

SCPP changes the primary prediction and therefore also changes the primary uncertainty used to weight ISOR's correction. ISOR then uses the softly attenuated feature view to refine the detached primary logits. This provides a concrete interaction between the two stages: SCPP affects both the prediction being refined and the strength of the subsequent correction.

Adding SCPP to ISOR improves mAcc/mAP by 1.60/0.53 points on the fixed list and 1.59/0.47 points on the submitted subset, using the same trained checkpoints across both evaluations. These results support SCPP's contribution within the integrated UFD model, despite its weaker standalone performance. They do not by themselves establish why SCPP alone underperforms, so we will avoid attributing that behavior to an unverified mechanism or implying uniform complementarity across protocols.

3. Auxiliary objectives and hyperparameters. The range term directly supervises the Basis MLP by constraining projected energy. The conditional-logit and correction penalties regularize refinement outputs; the directional objective supervises the correction direction; and the preservation term discourages changes to confident primary predictions. Appendix A.5 and Table 8 report the module settings and activation schedules, while Appendix B.1 provides task-specific training details.

These objectives define the intended constraints, but the current module-level ablations do not separately quantify each loss's contribution. We will distinguish their design rationale from the empirical evidence and clarify the gradient pathways. The module settings are shared across Face and UFD rather than separately configured for individual target datasets; this does not, by itself, establish their optimality.

4. Consistent reporting. We will standardize AUC, AP, accuracy, and EER as percentages throughout the tables and text, and report absolute differences in percentage points. Aggregate values will be computed from unrounded measurements before display.

Reviewer_iYNNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCORE for generalizable AI-generated image and face-forgery detection. It builds on an EFFORT-style spectral-residual CLIP encoder and introduces two components. Semantic-Conditioned Primary Prediction (SCPP) uses an instance-dependent gate to produce a residual adjustment to the base fake logit. Instance-Conditioned Soft-Orthogonal Refinement (ISOR) constructs a sample-dependent orthonormal basis, attenuates the corresponding projected features, and applies an uncertainty-aware bounded correction. Experiments cover cross-dataset face-forgery detection, eight unseen manipulations in DF40, and 19 UniversalFakeDetect (UFD) test sources.

Critical strengths
  1. The paper addresses a relevant problem. Forensic cues may depend on image content, capture conditions, and post-processing, so studying context-dependent evidence under distribution shifts is valuable.

  2. The distinction between primary prediction and conservative refinement is clearly formulated. The stop-gradient paths, independent supervision, and bound on the final logit correction make the design relatively easy to analyze.

  3. The evaluation spans several generalization settings. The paired analysis also reports newly introduced errors rather than presenting only successful corrections, which is useful for assessing the method.

Critical weaknesses
  1. The conditional-basis network does not have a sufficiently justified learning objective. In Eq. (16), the refinement representation is detached before it reaches the conditional head. Appendix A.2 (Table 7) indicates that the final classification loss does not directly update the Basis MLP; its direct training signal comes from the projection-energy range loss. That loss merely encourages the projected energy ratio to remain between 0.04 and 0.96. Why should a subspace satisfying this broad constraint capture semantic, domain-specific, or forgery-relevant directions? Orthogonality by itself does not establish their forensic meaning. Could the authors explain this?

  2. The semantic-conditioning claim is not directly validated. SCPP forms its gate from the shared CLIP feature using a bottleneck MLP (Eq. (7)), which also describes a generic instance-adaptive channel-reweighting mechanism. The paper does not isolate semantic information or show that gate responses track the pose, expression, identity, scene, or content factors motivating the method. Consequently, it is unclear whether the claimed semantic–forensic interaction is learned rather than ordinary feature statistics.

  3. The ablations do not establish the necessity of the main design choices. The current Baseline, SCPP-only, ISOR-only, and SCORE configurations show component-level effects but do not isolate orthogonality, sample conditioning, soft attenuation, gradient isolation, or uncertainty control. Important controls include random orthogonal bases, a shared learnable basis, non-orthogonal low-rank projections, removal of QR or attenuation, and a parameter-matched residual MLP without the basis construction. The authors should also ablate the stop-gradient operations, uncertainty weighting, and bounded correction.

  4. The reported component results do not consistently support the complementarity of SCPP and ISOR. In Table 5, SCPP-only reduces UFD mAcc from 90.82% to 87.72% and mAP from 99.00% to 98.13%. The full SCORE model attains 0.8984 cross-dataset AUC versus 0.8986 for ISOR-only, and both attain 0.9723 average DF40 AUC at the reported precision. SCORE does outperform ISOR-only on UFD, so SCPP may be helpful in that setting, but the evidence for a broadly beneficial two-stage combination is mixed.

  5. The paper does not directly measure the refinement effect within a single SCORE model. Figure 3 compares separately trained Baseline and SCORE models; the resulting score changes conflate differences in the encoder and primary prediction with the effect of ISOR. A more diagnostic comparison would use the same SCORE checkpoint and report errors corrected and introduced between its primary and final predictions, stratified by primary confidence. This is especially important because the final correction is bounded in magnitude by 0.25 logit units and therefore cannot reverse sufficiently confident primary decisions.

  6. External comparisons are not fully controlled. The internal Baseline follows EFFORT-style adaptation but excludes the orthogonality and singular-value preservation regularizers used in the published EFFORT method. On DF40, this internal Baseline already achieves 0.9661 average AUC, compared with 0.940 for the published EFFORT result, whereas SCORE improves the internal Baseline only to 0.9723. The source of this large gap should be explained before attributing the external advantage to SCORE. The UFD tables also combine locally sampled SCORE subsets with published results obtained under their original sampling and threshold settings. Hope the author can clarify the detailes?

Questions for authors

Please see weaknesses above.

1 comment
AuthorsOpenReview ↗

Clarifications on basis supervision and refinement analysis

Thank you for the detailed analysis, particularly the distinction between training SCORE and the immediate effect of its final correction.

1. Basis supervision and interpretation. The Basis MLP is directly supervised by the projection-energy-ratio objective, while the stop-gradient operation prevents the final classification loss from updating it through the refinement head. QR ensures an orthonormal basis, and the energy-range constraint discourages near-zero or near-total projection of the input feature.

The basis is used to construct an image-conditioned feature view, rather than to identify a labeled semantic or forensic subspace. Soft attenuation changes the relative weighting of the projected component and its orthogonal complement while retaining information from both. The refinement head is then trained to use this view for classification. Thus, the basis objective controls the geometry of the refinement input, while the classification objective trains the prediction made from that input.

2. Meaning of semantic conditioning. In SCPP, “semantic conditioning” refers to conditioning on the shared CLIP visual representation, which contains both semantic and forensic information. The gate performs instance-adaptive channel reweighting; individual channels are not assumed to correspond to identified factors such as pose or expression. Our motivation concerns context-dependent evidence weighting, while the experiments evaluate detection performance rather than directly identify those factors. We will clarify this distinction.

3–4. Design controls and component contributions. Table 5 compares independently trained module configurations and establishes component-level effects. The proposed random/shared-basis, non-orthogonal, parameter-matched, and gradient-control experiments address individual design choices. We will distinguish these levels of evidence rather than present module comparisons as proof that every choice is necessary.

The contribution is protocol-dependent. ISOR-only and SCORE achieve 0.8986 and 0.8984 cross-dataset AUC, respectively, and both reach 0.9723 on DF40 at the reported precision. On UFD, adding SCPP to ISOR improves mAcc/mAP by 1.59/0.47 percentage points on the submitted subset and 1.60/0.53 points on the new frozen 34,400-image list. Both lists use the same trained checkpoints. These results support SCPP's contribution within the integrated UFD model, with comparable Face performance to ISOR-only.

On the fixed list, SCORE achieves 95.23% mAcc / 99.55% mAP, improving over the internal Baseline by 4.74/0.71 points. On the complete 88,353-image inventory, it achieves 95.27% mean balanced accuracy / 99.63% mean AP, equally averaged over 19 sources. These are checkpoint evaluations, not independent training runs.

5. Within-model refinement. Figure 3 compares separately trained Baseline and SCORE detectors, so its score changes reflect the complete models rather than the final correction alone.

We additionally compared the primary and final outputs of the same SCORE checkpoint on the evaluated test data at threshold 0.5. At frame level, the correction resolved 21 errors and introduced 17 errors. These counts describe decision changes attributable to the final correction, separately from Figure 3's comparison of independently trained models.

For binary softmax with a fake-logit correction satisfying ∣Δℓ∣≤0.25|\Delta\ell|\leq0.25, a threshold-0.5 decision can flip only when the primary fake probability is approximately within [0.438, 0.562]. Thus, the correction adjusts decisions near the primary boundary. We will distinguish this within-model analysis from Figure 3's complete-model comparison.

6. External comparisons. On DF40, SCORE improves the internal Baseline from 96.61% to 97.23% AUC (+0.62 percentage points). This measures the gain over our implemented baseline under the internal evaluation protocol. The Baseline uses EFFORT-style adaptation but excludes its orthogonality and singular-value-preservation regularizers, so it is not an exact reproduction of official EFFORT. This difference alone does not explain the gap from the published 94.0% result; that gap cannot be attributed to SCORE.

For UFD, the original table captions and Appendix B.1 disclose that external results retain their published sampling and threshold protocols. We will make this distinction more prominent, present the new fixed-list evaluation separately, and reserve matched-evaluation improvement claims for the internal A0–A3 comparison.

Reviewer_vnRxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SCORE (Semantic-Conditioned Orthogonal Refinement) targets generalized AI-generated image/face forgery detection. Its central premise is that the diagnostic value of forensic cues—such as texture, boundaries, and high-frequency responses—depends on the semantic context in which they appear. For example, mouth texture varies with facial expression, boundary responses change with pose, hair, and background, and high-frequency patterns shift with compression and image quality. The authors formulate this problem as controlled semantic conditioning: evidence should be organized according to the sample’s semantics, while the influence of contextual information on cross-domain prediction must be constrained, so as to avoid learning domain-specific shortcuts related to identity, scene, or acquisition conditions.

Critical strengths

The proposed method outperforms multiple cutting-edge deepfake detection methods and achieves state-of-the-art performance on DF40 and UniversalFakeDetect. In section 5.4, The authors proactively report the 99 newly introduced errors (including 94 false positives on real samples) and design a transfer-gap diagnosis based on oracle-threshold vs. fixed-threshold, rather than only showcasing the missed detections “recovered” in Fig. 1. This kind of bidirectional reporting is uncommon in similar papers.

Critical weaknesses
  1. The evaluation datasets are excessively outdated. All image generators used in the test sets were developed 4 to 10 years ago and lag far behind the rapid progress of image generation techniques. It is therefore unclear how the proposed method would perform on more recent image generators such as GPT-Image and Nano-Banana.
  2. The methodological novelty is weak: the paper is largely a reassembly of existing components, while key related work is missing. Looking at the components one by one: the encoder is essentially unchanged from EFFORT (the authors themselves acknowledge that the baseline is EFFORT with two regularizers removed); the gating in Eq. 7 corresponds to squeeze-excitation in SENet/FiLM; per-sample basis generation resembles HyperNetworks/Involution; the QR orthogonal basis plus z - αQQᵀz is a softened, instance-conditioned version of concept erasure as in LEACE/INLP, and is also highly structurally similar to ORL (Orthogonal Residual Learning) and GPM/OWM; logit residual addition is analogous to gradient boosting / Cascaded Refinement; uncertainty gating is related to Focal Loss / confidence-thresholded pseudo-labeling; and L_pres is essentially self-distillation KL. The issue is that none of these directly corresponding methods—LEACE, INLP, ORL, GPM, SENet, FiLM, HyperNetworks, Focal Loss, KD—are cited. Even C2P-CLIP, the closest work in the same task setting (also using CLIP-based semantic conditioning for AIGI detection), is not discussed. This makes it impossible to assess the originality of “controlled semantic conditioning”—reviewers cannot tell whether the authors are even aware of these prior works.
  3. The ablation study does not support the claim that semantic conditioning is the key contribution. In Table 5, ISOR-only performs as well as or slightly better than full SCORE on the two face-forgery protocols, while SCPP-only performs worse than the baseline on UFD. This suggests that most of the gains come from ISOR rather than the semantic-conditioning module. The paper does not explain this inconsistency or provide significance tests or multi-seed results.
  4. The method relies on many hand-tuned hyperparameters, which raises concerns about reproducibility and generalization. The paper includes more than twenty tunable settings but provides no sensitivity analysis or tuning details. Some values also appear dataset-specific, so their stability across protocols is unclear. In addition, the UFD evaluation is based on a self-constructed local subset, while competing methods use published EFFORT results, making the comparisons in Tables 1 and 2 not strictly fair despite the strong claims in the main text.
Questions for authors

None

1 comment
AuthorsOpenReview ↗

Clarifications on novelty, recent generators, and evaluation

Thank you for the constructive comments and for recognizing our bidirectional error analysis. We address the four concerns below.

1. Recent generators. We evaluated the original ProGAN-trained SCORE checkpoint on GPT Image 2 and Nano Banana 2 without fine-tuning, using 300 generated images per source and the same 300 real images. SCORE achieves 98.35% AUC / 98.44% AP on GPT Image 2 and 72.17% AUC on Nano Banana 2. The GPT Image 2 Hybrid and Texture strata achieve 98.83% and 99.26% AUC, respectively, each with 100 real and 100 generated images. These results extend evaluation beyond UFD's generators. We will document the evaluation scale and data construction and limit generalization claims to tested sources.

2. Novelty and related work. Our contribution is the design and evaluation of semantic-conditioned primary prediction followed by controlled, instance-conditioned refinement. SCPP forms an independently supervised primary prediction. ISOR constructs a sample-conditioned basis for soft feature attenuation; its refinement head receives detached features, and its uncertainty-weighted correction is bounded by 0.25 logit units per image. The contribution concerns how these stages are coupled to organize evidence and constrain correction.

Sec. 3.2 already credits the EFFORT-derived encoder. We will expand the discussion of channel conditioning (SENet/FiLM), conditional parameter generation (HyperNetworks/Involution), projection methods (including ORL), residual refinement, confidence weighting, and distillation, with appropriate citations.

C2P-CLIP is closely related: it uses category-common text prompts during training to transfer category information to the image encoder. SCORE instead conditions prediction and refinement on the current image's visual features. INLP/LEACE remove specified concepts; SCORE learns an image-conditioned basis without concept labels or a concept-erasure guarantee. GPM/OWM constrain parameter updates, while SCORE attenuates image features.

3. Component contributions. The results indicate protocol-dependent contributions. On Face, ISOR provides the main improvement: ISOR-only and SCORE obtain cross-dataset AUCs of 0.8986 and 0.8984, respectively, and both reach 0.9723 on DF40 at the reported precision. On UFD, adding SCPP to ISOR improves mAcc by 1.59 points and mAP by 0.47 points on the submitted subset.

Using the same trained checkpoints on a new frozen, balanced 34,400-image list, the corresponding gains are 1.60 and 0.53 points. Thus, SCPP's contribution is supported within the integrated UFD model, even though SCPP alone does not improve over Baseline. We will clarify this complementary, protocol-dependent behavior.

4. Hyperparameters and comparison fairness. Appendix A.5/Table 8 already report fixed module settings and activation schedules; Appendix B.1 provides optimization, preprocessing, sampling, and seeds. Face and UFD share the module settings: k=8k=8, attenuation α=0.25\alpha=0.25, uncertainty exponent 1.25, maximum SCPP residual scale 0.15, refinement scale 0.035, and correction bound 0.25. These settings are not separately configured for individual target datasets. We will make their location and the task-specific training protocols clearer. Controlled sensitivity and loss-removal retraining remain to be completed.

For the internal UFD comparison, we reevaluated all configurations on the same frozen, balanced 34,400-image list, averaging equally over 19 sources at threshold 0.5:

ConfigurationmAP (%)mAcc (%)
A0: Baseline98.8490.49
A1: SCPP only98.0087.52
A2: ISOR only99.0293.63
A3: SCORE99.5595.23

SCORE improves over Baseline by 4.74 points in mAcc and 0.71 points in mAP under this matched evaluation. On the full 88,353-image inventory, the same SCORE checkpoint achieves 99.63% mean AP / 95.27% mean balanced accuracy, equally averaged over 19 sources.

The submitted table captions and Appendix B.1 disclose that external rows retain their published protocols. We will reflect this distinction more clearly in the claims, present the new evaluation separately from the submitted subset, and reserve matched-evaluation improvement claims for the internal A0–A3 comparison. The new results use the existing single-seed checkpoints.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.