- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how much performance can be recovered by adapting the scoring stage of a frozen encoder. FREEZER examines task-specific procedures based on established statistical operations: centering and covariance reweighting followed by cosine similarity for verification, and regularized readouts with optional original-score blending for deepfake detection. A separate collective verification procedure uses unlabeled evaluation embeddings and neighboring comparisons. On Common Voice 26, speaker-verification macro EER decreases from 3.03% to 2.17%, with improvements in 49 of 51 locales. For speech deepfake detection, adaptation using separate MLAAD and In-the-Wild data reduces EER on the full CoSG release from 20.79% to 10.05%.
The main contribution is an empirical investigation of which scoring adjustments produce these gains and how their benefits transfer. Component studies attribute most of the speaker-verification improvement to centering and show that the fitted speech-detection readout preserves nearly all the improvement without score blending. Image experiments demonstrate population-dependent blending tradeoffs. The paper also examines source composition, extraction procedures, comparison-graph access, and low-false-positive operating points, documenting cases where improved EER does not translate into better performance at the operating point of interest.
Critical strengths
-
The component studies provide meaningful evidence about what actually supplies the improvements. I found the centering result particularly striking: optimized centering alone reaches 2.2045% EER on Common Voice, close to the selected full transformation’s 2.1678%. This is a useful finding for anyone deciding how much adaptation machinery is necessary. The paper separates ordinary mean subtraction, supervised centering, and covariance adaptation, and uses direction-randomization controls to investigate whether fitted directions matter. For speech detection, head-only scoring retains almost all the CoSG improvement, achieving 10.28% EER compared with 10.05% for the blend and 20.79% for the original detector. Shared-validation comparisons between fitting populations and readout families further help distinguish adaptation-data effects from scoring choices. These experiments give the paper value as an analysis of frozen-representation adaptation.
-
The failure analysis and transparent reporting strengthen the empirical contribution. I particularly appreciated the CPLFW example, where a better EER accompanies a substantial deterioration at low FAR. It makes the limitations of the aggregate metric concrete. The authors investigate the extreme impostor score affecting the threshold and evaluate corrections that mitigate the problem while retaining important limitations. They also distinguish independent pair scoring from collective verification with additional evaluation-population access, report population-dependent failures of image-score blending, and separate extraction-related gains from head adaptation. The disclosure of exploratory follow-ups, threshold-transfer failures, and conditional uncertainty estimates helps the reader interpret the positive results. This is considerably more informative than reporting favorable benchmark averages alone.
Critical weaknesses
-
The speaker-verification comparison omits a particularly relevant discriminative backend, leaving the practical value of the cosine-preserving choice incompletely established. Cumani et al., Analysis of the ABC Classification Backends for NIST SRE24 (Interspeech 2025), report strong PSVM results under mismatch. FREEZER cites this work but does not evaluate a PSVM backend using its frozen embeddings and available training identities. The moments-based Gaussian control in Appendix B is useful, but it does not cover this discriminatively trained alternative. Since FREEZER’s optimized center already uses supervised genuine/impostor pairs, a pairwise discriminative backend is directly relevant to assessing how effectively the same supervision can be used without changing the encoder. Preserving cosine is an attractive design choice; what I am missing is a matched comparison showing its accuracy and computational tradeoff against this alternative. An embedding-only PSVM would provide a suitable starting point, with any metadata-dependent variants reported separately. I do not assume that PSVM will outperform FREEZER on Common Voice; the comparison would resolve that empirical question.
-
The methodological contribution is limited, and the incremental value of the more elaborate scoring choices remains insufficiently resolved. Centering, covariance normalization, ridge/LDA readouts, score blending, and ITQ are established components, as the manuscript acknowledges. Along the reported ablation sequence, supervised centering supplies 95.7% of the raw-to-final speaker EER reduction. In Appendix H, the image Gaussian-family control selects shared-covariance LDA and reproduces the ridge result under the matched absolute-EER selection objective, consistent with the equivalence derived in Appendix A. I find the simplicity of the effective adaptations appealing. However, these results make it especially important to establish when the additional components offer a reliable benefit. The covariance increment beyond the optimized center is small, while blending can produce substantial gains or regressions depending on the evaluation population. The paper documents these outcomes carefully, but provides limited guidance for anticipating which choice will transfer successfully. I recognize the empirical contribution across detection and collective verification independently of the missing speaker backend comparison. Taken together, however, the limited methodological novelty and unresolved comparative and incremental benefits leave this a borderline submission for me.
Questions for authors
-
How does supervised centering plus covariance adaptation compare with an embedding-only PSVM under matched conditions? Please use the same frozen ReDimNet2 features, available training identities, locale information, development data, and strict-test trials, with justified tuning budgets. Any use of additional metadata, such as duration, should be identified and controlled separately. Reporting fitting and scoring costs alongside EER and low-FAR performance would clarify whether FREEZER offers an accuracy advantage or a compelling efficiency tradeoff. This comparison would materially inform my assessment.
-
How robust is the incremental benefit of covariance adaptation beyond the optimized center? I appreciate that the manuscript explicitly describes the additional value of correct identity grouping as unresolved. The strict-test EER difference between center-only and the selected full transformation is approximately 0.037 percentage points, and total or shuffled-label covariance approximately ties the selected covariance once the supervised center is fixed. The uncertainty evidence for improvement over raw cosine does not establish the reliability of this smaller increment. Could you provide paired uncertainty for center-plus-metric versus center-only, preferably including refitting variability, and indicate whether the benefit is consistent across locales or operating points? This would clarify when the additional metric estimation is worthwhile.
-
Does the speech readout comparison in Table 2 remain stable under family-specific regularization tuning? Appendix B.5 notes that identical absolute penalties can have different effective strengths across losses. The existing comparisons control several important factors, but matching candidate counts alone does not establish that the search ranges are equally suitable for ridge and logistic regression. Would expanded or independently justified regularization ranges preserve the observed ridge advantage? For images, the manuscript already shows that the relative ordering varies across destinations; my question concerns the robustness of the speech result and the extent to which it supports a preference for ridge in that setting.
Thank you for your detailed suggestions. We’ve added the three comparisons you requested to the manuscript.
Q1. How does PSVM compare with adapted cosine?
When trained separately by locale, diagonal PSVM and adapted cosine performed similarly (macro EER 2.174% and 2.168%). Combining a shared PSVM with a locale-specific PSVM lowered macro EER to 1.877% and improved on adapted cosine in 50 of 51 locales. At one false accept per 10,000 impostor trials, adapted cosine accepted more genuine trials (67.58% versus 61.48%).
Both used the same frozen ReDimNet2 embeddings, training identities and 41.3 million test trials. Development macro EER selected each configuration. Locale was the only metadata. On three locales, fitting one adapted-cosine configuration took 2.54–2.74 seconds per locale on an H100. Across the full PSVM search, local fits used 2.75 GPU-hours and pooled fits took 30 minutes. Scoring the cached trials took 0.304 seconds for adapted cosine and 0.325 for PSVM on the same H100 (Table 2 and Appendix I).
Q2. How much does covariance add beyond centering?
Covariance reweighting lowered macro EER from 2.205% to 2.168%. EER fell in 34 locales, tied in three and rose in 14. With fits fixed, the gain was 0.037 percentage points (paired 95% interval 0.019–0.073). Resampling both training and evaluation speakers across 100 refits gave a mean gain of 0.024 points (0.003–0.056). At FAR 10⁻⁴, the interval for the gain in genuine accepts included zero. Shuffling speaker labels gave similar EER, leaving the benefit of correct speaker grouping uncertain (Section 5.1 and Appendix I).
Q3. Does ridge still lead after separate tuning?
Yes. Ridge had lower EER on all four speech datasets. We used the same validation folds and blend-selection rule for both methods. We scaled the logistic penalty grid by ¼ to account for its curvature and widened either grid if validation favored an endpoint. On full CoSG, ridge reached 10.05% EER and logistic regression 11.78%. Separate tuning narrowed the gap from 2.09 to 1.73 points (paired 95% interval 0.81–2.81). Table 3 and Appendix B.5 give the full comparison.
Compared with the fitted head alone, blending in the original CLIP detector’s score lowered EER on CNN and Diffusion but raised it on Synthbuster (Section 5.2).