- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies cross-generator generalization for AI-generated image detection. Two practical pain points are addressed: 1) multiple LoRA experts tend to learn redundant shortcut cues instead of complementary forensic evidence; 2) high local response scores do not always imply forgery, since real images can also produce strong local activations. The authors propose a two-stage pipeline: multi-source evidence-aware training with generator-consensus loss and evidence-diversity loss; followed by offline real-image-only response calibration after freezing network weights.
Critical strengths
- It identifies two under-discussed failure modes in multi-expert detection models: expert redundancy and false alarms from normal real-image local responses.
- The proposed evidence diversity regularizer measures expert complementarity via leave-one-out loss increment, instead of naive feature space orthogonality. The real-image-only calibration stage is a practical attempt to suppress false positives.
- Strong cross dataset results are reported on challenging benchmarks. Cumulative ablation studies validate the contribution of each main component.
Critical weaknesses
- Evidence learning builds entirely upon underlying DINOv3 features. The evidence oriented losses cannot repair bad shortcut representations learned by the backbone. If base features capture spurious correlations, even well-regularized experts cannot produce valid forensic evidence. This risk lacks dedicated ablation.
- Calibration is purely post-processing. It only rescales output scores and cannot fix erroneous representations inside network layers. Calibration performance heavily relies on distribution compatibility between held-out real images and test data. Degradation under real image distribution shift is not studied.
- The pipeline is relatively heavy with many hyper parameters (expert number, quantile q, etc.). The evidence diversity loss involves determinant computation, which raises potential numerical stability concerns during training.
- Contributions are mostly loss design and post processing tricks built on existing LoRA expert framework.
Questions for authors
- Evidence learning builds entirely upon underlying DINOv3 features. The evidence oriented losses cannot repair bad shortcut representations learned by the backbone. If base features capture spurious correlations, even well-regularized experts cannot produce valid forensic evidence. This risk lacks dedicated ablation.
- Calibration is purely post-processing. It only rescales output scores and cannot fix erroneous representations inside network layers. Calibration performance heavily relies on distribution compatibility between held-out real images and test data. Degradation under real image distribution shift is not studied.
- The pipeline is relatively heavy with many hyper parameters (expert number, quantile q, etc.). The evidence diversity loss involves determinant computation, which raises potential numerical stability concerns during training.
- Contributions are mostly loss design and post processing tricks built on existing LoRA expert framework.
4 comments
Q3 (a): The pipeline is relatively heavy with many hyper parameters.
A: Thank you for raising this concern. EviCal does introduce additional hyperparameters, including the number of expert and calibration settings you mentioned. However, the quantities appearing in the calibration procedure are not all independently tuned hyperparameters: the response medians and spreads, regression coefficients, residual-tail offsets, and final decision threshold are estimated from held-out source-real images. Once fitted, the same calibration model and threshold are used across all test datasets, without dataset-specific refitting. The remaining settings are shared across datasets and explicitly specified in the implementation details. This does not eliminate the additional configuration burden, but distinguishes the settings that require specification from the quantities estimated automatically.
Meanwhile, Table 3 of the paper reports sensitivity to the number of experts (K) and calibration quantile (q). Four experts achieve the highest mean accuracy across the three benchmarks. For (q=0.95)–(0.98), mean accuracy varies by only 0.35%, while (q=0.99) reduces cross-dataset accuracy. We also distinguish configuration selection from calibration fitting. The main configuration, (K=4,q=0.97), was selected using mean accuracy across the three benchmarks, so the target-benchmark results informed this choice. We additionally report a configuration selected using GenImage alone, (K=6,q=0.99), which achieves 88.28% on Chameleon and 94.03% on AIGI-Now Pixel. For either configuration, calibration fitting uses only held-out source-real images, without dataset-specific refitting.
Q3 (b): Potential numerical stability concerns during training.
A: Thanks for you attention on this concern. Our implementation evaluates the combined consensus–diversity objective through an algebraically equivalent reuse-weighted Gram formulation, consistent with Eq. (7). We add diagonal regularization of 1e-3 to this weighted matrix to regularize rank-deficient contribution profiles. We compute its log-determinant using the standard PyTorch function torch.linalg.slogdet and use a denominator floor of 1e-4 during profile normalization. Furthermore, we did not observe any non-finite objectives or gradients in our additional loss-ablation experiments, providing further empirical support for the stability of the implementation under the evaluated configuration. These observations support the numerical stability of the implementation under the evaluated settings.
Q4: Contributions are mostly loss design and post processing tricks built on existing LoRA expert framework.
A: Thank you for raising this important question about the methodological contributions. Our contribution is not LoRA itself, but how complementary evidence is defined and learned. EviCal measures each expert’s utility by its marginal reduction in detector loss, then uses this same utility to encourage contributions across generators and on different images. Source-wise centering prevents differences in average generator preference alone from being treated as evidence diversity. This targets useful predictive contributions rather than merely different parameters or features. Calibration also goes beyond score rescaling or threshold adjustment. It models the local response expected from the global and expert state on real images, then removes this predictable component before aggregation. The same local score can therefore be interpreted differently depending on its context, potentially changing image rankings. The methodological contribution lies in contribution-guided evidence learning and conditional interpretation of evidence, with LoRA serving as the implementation framework.
Finally, we sincerely appreciate your positive and thoughtful review which encourages us a lot. We hope our responses have adequately addressed your concerns. If anything remains unclear or requires further clarification, we are willing to provide additional explanations and welcome any further feedback.
Dear Reviewer k7BG,
Thank you for your thoughtful comments. Since the discussion deadline is approaching, we just wanted to check whether there are any remaining issues that would benefit from further clarification. If so, we would be happy to provide additional experiments or explanations before the discussion concludes.
We truly appreciate your time and consideration.
Best regards,
Authors
Dear Reviewer k7BG,
Thank you for reviewing our paper and sharing your suggestions. Your feedback has helped us improve the paper.
As there is little time left in the rebuttal period, we just wanted to check whether our responses have addressed your concerns. If anything remains unclear or unresolved, please let us know, and we will do our best to address it in the remaining time.
Thank you for your time and help!
Best regards,
Authors
A: Thank you for raising this concern. It is important to distinguish between the information encoded by a backbone and the information predominantly used for prediction. A model can rely heavily on spurious correlations while its representations still retain useful alternative cues. This distinction has been demonstrated by Kirichenko et al. [1], who showed that retraining the final classifier can improve shortcut robustness by changing how existing features are used. Therefore, the presence of spurious correlations does not by itself imply that downstream evidence learning cannot improve generalization.
EviCal addresses this feature-utilization problem by learning how shared token information is transformed into evidence responses. Although the base DINOv3 weights are frozen, heterogeneous LoRA updates modify operators inside the Transformer, allowing experts to learn different feature transformations from their shared inputs. The evidence objectives directly optimize these updates, shaping the resulting representations rather than merely regularizing classifiers attached to identical, fixed final embeddings. Generator consensus evaluates an expert’s marginal contribution separately across source generators, favoring usefulness that extends beyond an individual source. Evidence diversity then discourages repeated sample-wise contribution patterns among experts. Together, these objectives are designed to identify complementary discriminative directions supported by multiple sources.
Our claim is therefore about improving the use of potentially transferable information, not recovering information absent from the representation or guaranteeing a universally shortcut-free backbone. We examined the capacity for such improvement in a matched same-backbone control: compared to the original baseline, Evical improves the performance from 82.21% to 86.26% in Chameleon and from 87.63% to 90.26% in AIGI-Now. These results demonstrate that the backbone does leave room for improved transfer through adaptation.
[1] Last layer re-training is sufficient for robustness to spurious correlations. ICLR 2023.
Thank you for your constructive comment. Our calibration operates after evidence extraction, but it does more than uniformly rescale the outputs. It models how local responses relate to global evidence on source-real images and removes the predictable component before aggregation. The same local score can therefore receive different interpretations depending on the image's global response, helping distinguish unexpected local evidence from high responses that also occur naturally in real images.
We agree that calibration cannot recover information absent from the learned representation. Representational adaptation is handled by the trainable evidence experts, whereas calibration addresses how their responses should be interpreted against real-image behavior. These two components therefore address different aspects of detection.
Regarding distribution compatibility, both the calibration statistics and decision threshold are fitted exclusively on held-out source-real images and remain fixed across all evaluation datasets, without target-based refitting or dataset-specific thresholds. The real-image collections also differ: GenImage uses ImageNet images, Chameleon collects photographs from platforms such as Unsplash, and AIGI-Now Pixel uses COCO test images. Cross-dataset evaluation therefore changes the real-image source as well as the fake generators.
For better show the detailed performance under cross-dataset setting, we test the AUROC and FPR under target Chameleon and AIGI-Now Pixel with source GenImage. The fixed source-fitted calibration retains high real-image accuracy alongside effective fake-image detection on both cross-dataset benchmarks. These results support Evical can keep a good ability transfer across the evaluated real-image collections.
Table 1. Cross-dataset performance with fixed source-fitted calibration. The source FPR target is set to 3%.