- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies how deepfake-related information can be spread across the layers of the fine-tuned pretrained self-supervised speech encoders. It applies the final linear classifier to every intermediate layer. From this logit-lens style probe, it argues that discriminability of bonafide and spoof appears only in the last layer. It then proposes the method DWLS, which alternates final-layer fine-tuning with phases where a shared pooling and classifier head is applied to all layers. The per-layer losses are weighted by a cosine schedule that moves from deep to shallow layers. Experiments use XLS-R trained on ASVspoof 2019 LA and evaluated on ten in-domain and out-of-domain sets, with linear, MLP, and KAN heads. The paper also includes neural collapse (NC1/NC3) and CKA analyses.
Critical strengths
None
Critical weaknesses
Main argument
- The layer-wise analysis does not support the claim that "discriminability emerges late."
In Section 3, the author applies the final-layer classifier to every layer. was trained only on . Fig. 2 and Fig. 4(d–e) also show that intermediate layers are nearly orthogonal to the final layer (AngDist ≈ 0.5, cosine ≤ 0.05). So, high EER at early layers mainly shows that does not fit those layers' coordinate systems. It does not show that the information is missing.
The paper's own numbers point to this. XLS-R-2B reaches 82.62% EER at layer 11. Flipping the sign of the score would give about 17% EER, which means the layer carries a strong discriminative signal. If it is really "uninformative", the EER should be around 50% and close to random selection. A per-layer trained linear probe is the standard tool for this question, and prior layer-wise studies cited by the paper (El Kheir et al., 2025; Pasad et al.) use such probes. Without it, the motivation for the method is not established, and this is a very serious issue.
- The headline layer-wise improvement is close to being true by construction.
In Fig. 4(c), DWLS trains the shared head on all layers, while the baseline head never sees intermediate layers. A lower logit-lens EER at layer 7 under DWLS (4.39% versus the baseline, described as "43.5% improvement") is therefore expected. It is not evidence that the representations themselves became better. A minimal example makes this issue clear. Suppose the baseline is:
and it is trained only with . There is therefore no direct training constraint requiring to be discriminative. By contrast, DWLS optimizes an objective of the form . DWLS has an inherent advantage because was explicitly part of its training objective, whereas the baseline was never trained to make the same classifier work at layer 7. A fair comparison would train a probe per layer for both models, or evaluate both with a head trained the same way.
- Novelty and positioning.
The core novelty of the paper is not new. Applying a shared or auxiliary classifier to intermediate layers is deep supervision. There are a lot of references about deep supervision. Examples include Deeply-Supervised Nets [1], GoogLeNet auxiliary heads [2], intermediate CTC in speech [3], and early-exit networks [4]. None of these are discussed. The claim in Section 2 that existing methods "all address how to aggregate intermediate features; none changes how the layers are trained" clearly overstates the gap. I suppose the new parts are the cosine depth schedule and the alternation, but neither is ablated against simpler options:
- a constant uniform multi-layer loss,
- progressive phases only, with no alternation,
- a SUPERB-style learned weighted layer sum with the same linear head,
- a per-layer linear probe baseline.
- The empirical evidence is weak.
-
In Table 2, Linear-DWLS is worse than its own baseline on 19LA, 21DF, 25DF, and LSV. KAN-DWLS is worse on 21LA (+1.41) and 25DF (+0.86). The average gains (0.26–1.17 EER) only come from a few sets.
-
The Linear-DWLS numbers in Table 2 match the D→S, N=300 row of Table 1. That row was the best on ITW. This suggests the direction and N were chosen using out-of-domain test results, even though the paper says checkpoints are selected on the 19LA dev set. Please clarify how the final configuration was chosen.
- Some internal contradictions.
- Eq. 11 and Fig. 1 define a deep-to-shallow schedule. Section 6.1 then concludes that feature collapse "requires progressive shallow-to-deep optimization," while Section 6.3 says deep-to-shallow generalizes better. Which schedule is the proposed method?
- Section 6.1 says NC3 is "rapidly established," which reads as good alignment. Section 7 says "NC3 remains persistently high … the linear classifier does not align with the class-mean axis." These two statements conflict.
- The conclusion says DWLS is "a more efficient route … than increasing back-end complexity." But back-end parameters are tiny next to the ~315M backbone, and all systems have about the same total size. The extra training cost of DWLS (pooling and a head on 25 layers) is not reported.
I tend to reject the paper at the current stage for two main reasons: (1) The motivating analysis and the layer-wise results are confounded. (2) The method is a form of deep supervision with auxiliary losses. That idea is well known, and the paper does not cite or compare against it. On top of that, the empirical gains are mixed.
[1] Lee, Chen-Yu, et al. "Deeply-supervised nets." Artificial intelligence and statistics. PMLR, 2015.
[2] Szegedy, Christian, et al. "Going deeper with convolutions." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
[3] Lee, Jaesong, and Shinji Watanabe. "Intermediate loss regularization for CTC-based speech recognition." ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021.
[4] Teerapittayanon, Surat, Bradley McDanel, and Hsiang-Tsung Kung. "Branchynet: Fast inference via early exiting from deep neural networks." 2016 23rd international conference on pattern recognition (ICPR). IEEE, 2016.
Questions for authors
- Can you report per-layer trained linear probe EER for the baseline and DWLS, both in Section 3 and in Fig. 4(c)?
- Why are mean-pooled intermediate features almost exactly orthogonal to the final layer (cosine ≤ 0.05)? Could a few large-norm dimensions dominate the cosine and angular metrics?
- Table 1 reports AUC, but Table 2 does not. Please be consistent.
- Fig. 2 and Fig. 4 axis labels are too small to read.
- The limitations section should include single-seed results and a single backbone.