- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Fake-in-Facext, a framework for fine-grained explainable face-forgery analysis. It uses a hierarchical facial concept tree to generate region-specific explanations and artifact masks, producing 1.38 million QA pairs. FiFa-VLM combines a face-pretrained encoder, box prompts, and a multi-task decoder to support detection, localization, and explanations interleaved with segmentation masks.
Critical strengths
-
Linking individual explanation phrases to masks is a useful extension beyond image-level explanations. Region and box queries also provide a practical way to inspect specific facial areas.
-
The facial concept hierarchy gives the annotations a consistent spatial vocabulary. The resulting dataset supports several related tasks within one framework.
-
The experiments include annotation-source comparisons, architectural ablations, and a trainable-vision GLaMM baseline. These controls provide evidence that the proposed supervision and face-specific representation improve performance on FiFa-Bench.
Critical weaknesses
-
The artifact ground truth is not sufficiently justified. Section 3.2 defines artifact masks using the top 5% of pixel differences between real and manipulated images. This identifies large changes, but does not establish that they are visible forensic artifacts. It may miss subtle defects or include differences caused by alignment and compression. The fixed percentile and concept-selection thresholds are not validated against independent human annotations. The treatment of fully synthesized faces, which lack a corresponding original image, also needs clarification.
-
The evidence for reduced hallucination is partly circular. Table 5 evaluates concept precision and recall against labels produced by the same rules used to construct the training annotations. This tests agreement with the annotation pipeline, not whether claims about lighting, texture, or geometry are correct. Likewise, METEOR and mask IoU are reported separately, leaving the central phrase-to-evidence connection insufficiently evaluated. A model could identify the right facial region while giving an incorrect explanation.
-
The annotation prompt can encourage unsupported explanations. Appendix A.3 requires every selected concept to be discussed as a tampered region, without an explicit option to report insufficient evidence. It also lists gender-associated hairstyles and makeup as possible forgery cues. These characteristics are not inherently evidence of manipulation and could introduce systematic bias. More precise region names do not, by themselves, make the accompanying claims more reliable.
-
The dataset’s effective diversity and split independence are unclear. The reported 1.38 million examples are QA pairs, with many region and box queries derived from the same images. Table 16 reports task-level counts but does not establish the number of unique images or whether all related queries, frames, and identities remain within one split. In addition, augmented boxes are retained only when they contain at least three artifact-existing concepts. This leaves uncertainty about performance on clean regions, very small edits, and arbitrary user-selected boxes.
-
Generalization claims are stronger than the supporting results. Table 6 evaluates FiFa-Annotator+CLIP rather than FiFa-VLM’s grounded explanations. Its AUC gains over FFTG+CLIP are modest: 0.23, 0.44, and 0.55 percentage points on the three cross-domain datasets, without uncertainty estimates. Table 8 similarly reduces cross-domain ACER from 29.32% to 28.57%. These results suggest some benefit, but do not establish robust transfer of explanation and localization capabilities.
-
The architectural contribution is not fully separated from face-specific pretraining and additional supervision. Table 7 includes a useful trainable GLaMM control, but replacing its visual encoders also changes the pretraining domain and encoder structure. The single-task comparison matches task-specific sample counts, while the multi-task model receives additional supervision from other tasks. This supports the usefulness of joint training, but does not isolate architectural gains or establish an advantage under equal total training budgets.
Questions for authors
-
How accurately do the difference-based masks match independently annotated visible artifacts? How sensitive are the results to the 5% threshold and the concept-selection thresholds?
-
How are artifact masks constructed for fully synthesized faces without paired originals? What alignment and compression controls are applied before computing image differences?
-
Can you provide a blinded human audit of the explanations, checking whether each claimed defect is actually visible? Does allowing the annotator to report insufficient evidence reduce hallucinations?
-
How are generated phrases matched to ground-truth masks during AGE evaluation? Can you measure whether both the region and the associated artifact description are correct?
-
How many unique images and identities underlie the QA pairs, and are all related examples kept within the same split? How does the model perform on boxes containing no artifacts or only one small artifact?
-
Can FiFa-VLM retain its explanation and localization performance on unseen manipulation methods without retraining? Please distinguish these results from the CLIP-based annotation-transfer experiment and report variation across training runs.