- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on class-level and domain-level imbalance in deepfake detection. Authors propose a Bi-level Balanced (BB) Loss that combines class-aware and domain-aware logit adjustment with a top-k loss aggregation strategy to emphasize difficult samples. Experiments on five deepfake detection datasets show improvements in classification accuracy and balanced performance under different imbalance settings.
Critical strengths
1 Considering domain-level imbalance in addition to conventional real/fake class imbalance is well motivated.
2 The proposed loss can be easily used on existing deepfake detection frameworks.
3 The evaluation covers multiple datasets, model architectures, imbalance ratios, and robustness settings is soild and demonstrates improvements of proposed method. But implementation details need further explanation.
Critical weaknesses
Major comments:
1 The experiments primarily vary the real-to-fake ratio without independently controlling domain-level imbalance. I suggest evaluating different fake-domain proportions while keeping the class ratio fixed.
2 The ablation study in Table 6 is useful, but it is conducted on the original FF++ training distribution, where performance is already close to saturation. I suggest repeating the component ablations under severe imbalance setting.
Minor comments:
1 Table 6 reports percentages, whereas most other tables use values between 0 and 1. Please use a consistent format.
2 Implementation details like per-domain sample counts, the procedure used to construct imbalanced datasets, and random seeds would be good for reproducibility.
Questions for authors
1 Theorem 1 claims an equivalence between Eq. (5) and proposed domain-level surrogate. However, the original domain-balanced objective penalizes binary classification errors, whereas the proposed surrogate also penalizes incorrect predictions among different fake domains. Can authors explain how the claimed equivalence holds?
3 comments
Thank you to the authors for their detailed responses and additional experiments. These have clarified some of my concerns. I will maintain my rating.
Thank you for your positive feedback and continued support of our work. We sincerely appreciate your time and effort in reviewing our responses and additional experiments.
WEAKNESSES
MAJOR COMMENTS
W1
In our original experiments, changing the class ratio also changes the relative proportions of the real and fake domains, and AI-Face is naturally imbalanced across generator families. Per your request, we conduct an additional experiment below. To isolate domain-level imbalance, we fix the class ratio at 1:1 and vary the share of one fake domain (Deepfakes, selected uniformly at random before running any experiments), which makes up only 1%, 5%, or 10% of the fake samples, with the remaining fakes split equally among the other four domains. All methods are trained on identical data.
Overall, under these extreme settings, our approach still performs best.
W2
The original FF++ training distribution is already imbalanced (17,218 real : 84,346 fake), so Table 6 was not run on balanced data. To test the components under more severe imbalance, we added a 1:10 real:fake setting.
The full objective is best on all three metrics, and the largest gain is on DegreeD (Variant 5 → BB: 94.83 → 96.53, with smaller standard deviations).
MINOR COMMENTS
C1
Thank you for pointing this out. We will convert Table 6 to the same [0, 1] scale used in the other tables, so all results are reported in a consistent format. We will include this in the final manuscript.
C2
Thank you for this suggestion. We will add to Appendix D the per-domain sample counts for every training set, the procedure used to construct the imbalanced datasets (random subsampling of the real and fake training images to reach each target real:fake ratio, with a fixed sampling seed), and the random seeds used for each run. We will also release the scripts that construct these datasets. We will include this in the final manuscript.
QUESTIONS
Q1
Thank you for this question. Theorem 1 concerns the relationship between two surrogate formulations, not pointwise equality with the original objective in Eq. (3). Specifically, circle ① is replaced by the class surrogate in Eq. (4). For circle ②, we introduce the indicator-ratio decomposition and replace 1{g^=g}/p(g) with a domain-classification surrogate, yielding Eq. (5). Thus, Eq. (5) is already a surrogate. The argument for the claimed equivalence with the final domain surrogate is presented in Appendix C.1 through four cases, according to whether the true and predicted domains are real or fake. We will clarify these separate steps in the manuscript.