acceptodds
Under review as a conference paper at ICLR 2027

Adaptation or Recalibration? Continual Test-Time Adaptation for Audio Deepfake Detection

Abstract

Audio deepfake detectors meet new attacks after deployment: a detector trained on today's synthesis systems must run against generators and recording conditions that are not known in advance. We study this as source-free, label-free continual test-time adaptation (CTTA) and build a controlled, strictly prequential benchmark for audio deepfake detection, in which every utterance is scored before it can affect an update. The benchmark has five streams and 266,400 scored positions, with source training on ASVspoof 2019 LA and target streams pairing MLAAD spoofed speech with M-AILABS bona fide speech. It separates two failures that a single accuracy number hides: discrimination fails when the score ordering gets worse, and the operating point fails when the scores move relative to a threshold fixed before deployment. Five prediction-driven CTTA methods, evaluated on XLS-R and WavLM, move the operating point far more than they repair discrimination. On the stream of 52 sequential generators, Tent lowers per-block minimum detection cost (minDCF) by 0.010 and 0.013 on the two held-out checkpoints, against 0.100–0.112 for a labeled update of the same parameters. Recalibrating the frozen detector removes 22–77% of its calibration loss, depending on the checkpoint, without changing the ordering. An utterance's own score cannot say which of two utterances is ranked wrongly; relations between utterances can. We introduce CAST (Consensus-Anchored Score Transfer), which smooths frozen scores over reciprocal nearest neighbors and distills the resulting order into the adapting detector with a pairwise ranking loss. On the two held-out checkpoints of that stream, CAST lowers minDCF by 0.065 adapting only the LayerNorm affine parameters and by 0.103 adapting the full transformer encoder, where EER reaches 11.3%, against 0.011 for the strongest baseline. The gain holds on all three controlled target streams and both backbones. Smoothed scores improve discrimination even with no parameter update, while CAST's loss reads only score differences and leaves the operating point unresolved. Label-free continual detection therefore needs to solve two problems, and repairing discrimination solves only one.

Primary area:
datasets and benchmarks
Keywords:
continual test-time adaptation, audio deepfake detection, distribution shift, entropy minimization, calibration, discrimination, gradient alignment, benchmark

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_a1N1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies source-free, label-free continual test-time adaptation for audio deepfake detection. The goal is to let a detector adapt during inference when source data and target labels are unavailable and the time of environmental change is unknown.

Benchmark (Section 3). The source model is trained on ASVspoof 2019 LA. On the target side, spoofed speech from MLAAD is paired with bona fide speech from M-AILABS. The benchmark contains five streams and 266,400 scored positions. Evaluation is strictly prequential: each batch of 16 utterances is scored and recorded before an update is allowed, and recorded scores are never revised. To reduce corpus identity as a classification shortcut, the authors remove leading and trailing silence with energy-based VAD, low-pass filter below 7.7 kHz at the native sampling rate before resampling, and then audit sample rate, edge silence, signal level, duration, and padding. The strongest remaining audited cue after these controls has an EER of 42.7--42.9%.

Diagnosis (Section 4). The paper decomposes delivered cost as actDCF=minDCF+(actDCF−minDCF)\mathrm{actDCF}=\mathrm{minDCF}+(\mathrm{actDCF}-\mathrm{minDCF}), corresponding respectively to discrimination loss and calibration loss, and evaluates five prediction-driven CTTA methods: Tent, EATA-rd, SAR, RoTTA, and RDumb. The conclusion is that these methods mainly move the operating point and barely improve discrimination. As a reference, replacing the objective with cross-entropy using true labels, on the same parameter set, lowers minDCF by 0.100--0.112, whereas Tent lowers it by only 0.010--0.013.

Method (Section 5). CAST uses a frozen copy of the source detector to produce layer-4 embeddings and frozen scores. It smooths frozen scores over reciprocal nearest-neighbor utterances (Equation 2), and then distills that ordering into the adapting detector with a RankNet pairwise ranking loss (Equation 3). It uses neither class pseudo-labels nor a target threshold or class-prior estimate. On the held-out GENERATOR checkpoints, CAST-LN lowers minDCF by 0.065 and CAST-FT by 0.103.

Critical strengths

1. The diagnostic tool is novel and useful in this context. The detection-cost decomposition separates “the detector got worse” into two measurable components, and the authors use it to evaluate a family of existing methods. This yields a concrete empirical conclusion: the effects of these methods are concentrated in (actDCF−minDCF)(\mathrm{actDCF}-\mathrm{minDCF}). The combination of Figure 2 and Table 2 prevents this conclusion from being hidden by a single metric. This is the paper’s most valuable contribution, and it does not depend on whether CAST itself succeeds.

2. The benchmark construction is more rigorous than that of comparable work. The authors do not ignore the threat that corpus identity predicts the class; they actively address and quantify it. VAD removes leading and trailing silence (applied to every partition, including source training data), and native-rate low-pass filtering below 7.7 kHz is applied before resampling (the two corpora use 22.05 kHz and 16 kHz, respectively). Six attributes are then audited, and the strongest remaining cue after control is reported as 42.7--42.9% EER. The authors state that these two controls are not standard for this corpus pair and were adopted based on their own measurements.

3. The paper directly checks whether the baselines are under-tuned or compared on unequal adaptation surfaces. Appendix C.4 tests whether the parameter-count difference (110,594 versus 108,544) explains the Tent--CAST comparison. Appendix B.4 tunes Tent over 16 configurations on checkpoint 1234 and obtains ΔminDCF=+0.022\Delta\mathrm{minDCF}=+0.022 on held-out checkpoints, worse than the published configuration. These are necessary controls, and the authors provide them.

4. The authors are unusually candid about unfavorable evidence. Appendix D.2 includes probe experiments showing that 99.9% of reciprocal neighbors on the bona fide side share a speaker (versus 26.4% for a causal-pool-matched random sample), and that after holding out complete books, a layer-4 linear probe still distinguishes the two corpora with accuracy 0.980. The authors explicitly state that they cannot rule out dependence on speaker structure. Appendix E.1 also reports that direct smoothing outperforms CAST-LN. These measurements are unfavorable to the main method, but they are still reported.

5. The problem setting is clearly positioned. Section 1 lays out the assumptions of few-shot learning, domain generalization, continual unsupervised domain adaptation, and MLOps drift monitoring, and explains how the setting studied here is weaker than each of them.

Critical weaknesses

Weakness 1 : the paper’s central method claim is contradicted by the authors’ own experiment

The title, the latter half of the abstract, and Sections 5--6 present CAST as the main contribution. However, Appendix E.1 shows that direct neighborhood smoothing, which updates no parameters and simply uses the pseudo-scores from Equation 2 as scores, outperforms CAST-LN on every target stream at the two held-out checkpoints:

StreamDirect smoothingCAST-LN
STATIONARY-0.082-0.069
GENERATOR-0.100-0.065
PRIOR-0.091-0.023
LOCALE-0.033-0.012

Thus, the source of the discrimination repair may simply be the parameter-free smoothing operator, while the ranking-distillation step contributes negatively in the target domain.

The paper gives two reasons for distillation. First, direct smoothing fails on the source regime: it worsens 117 of 118 SOURCE windows, increases mean minDCF by 0.112, and raises EER from 6.3% to 12.4% (CAST-LN changes SOURCE minDCF by -0.017). This is a valid reason to prefer distillation in a source-retention trade-off, but it establishes that distillation exchanges target-domain performance for source stability; it does not establish that distillation is necessary. The second issue is discussed under Weakness 2.

This point cannot be fixed by clarification or a supplementary experiment: direct smoothing outperforming CAST-LN is already an observed result, and the result cannot be removed by clarification.

In addition, the paper tests only a score-space constraint (the score-anchoring penalty in Appendix E.4, which penalizes movement of the adapting detector’s score location away from the frozen detector), not a parameter-space constraint such as λ∥ϕ−ϕ0∥2\lambda\|\phi-\phi_0\|^2. The latter also does not require source data. EATA’s Fisher term requires source data because Fisher information must be estimated from source samples; parameter anchoring itself does not. CAST already has access to the frozen copy’s parameters. This is a concrete gap in the method comparison.

Weakness 2: discrimination and operating point are evaluated with inconsistent standards

The paper uses actDCF to criticize baselines for “only moving the threshold,” and also uses it to reject direct smoothing, but Table 2 reports only minDCF and EER. Both of these are offline, post hoc metrics obtained by sweeping all thresholds using true labels within each block or window. In the label-free deployment setting, the system does not know the optimal threshold. actDCF appears only in the appendices.

The asymmetry is as follows. Appendix E.1 rejects direct smoothing using actDCF:

“After the smoothed scorer and frozen detector are independently recalibrated using causally estimated label-free thresholds, the smoothed scorer delivers lower cost on only one of the three XLS-R checkpoints on each main target stream.”

Appendix E.4 shows that score-anchored CAST also delivers lower actDCF in only three of the nine stream--checkpoint cells across the three main target streams, all from the same checkpoint. Under the same standard, the comparison is 3/9 versus 3/9. Appendix E.4 further shows that applying MLLS recalibration directly to CAST fails: on SOURCE, MLLS estimates the spoofed fraction as 0.2--0.5 when the true value is approximately 0.9, and FAR remains at 46--77%.

The paper uses actDCF to reject direct smoothing, while praising CAST-LN in the main table using minDCF. This asymmetry requires explanation.

To be clear, the underlying claim that discrimination and operating point are independent is sound, and does not depend on CAST. Figure 2 and Table 2 show that the five prediction-driven methods (most of which predate this paper) mainly move the operating point. MLLS removes 22--77% of calibration loss without updating any parameters or changing the ordering. Appendix E.3 shows that a monotone transformation can preserve ordering exactly within a window while substantially changing actDCF (decreasing it by 0.60--0.79 on SOURCE and increasing it by 0.02--0.39 on target streams). The concern is about the inconsistent evaluation standard, not this diagnostic conclusion.

Weakness 3: only half of the gain is attributable, and the bona fide-side speaker structure does not match the application scenario

The paper does not explain why the same neighborhood operation improves ordering on target streams but harms ordering in the source regime. Appendix E.1 reports the phenomenon but not a mechanism. Without such an explanation, the hypotheses “neighborhoods contain generalizable ordering information” and “neighborhoods happen to work under this particular distribution structure” cannot be distinguished.

The authors’ own probe evidence supports the latter possibility (Appendix D.2):

  • 99.9% of reciprocal neighbors on the bona fide side share a speaker (versus 26.4% for a causal-pool-matched random sample), because only four M-AILABS English speakers recur throughout the streams;
  • after holding out complete books, a layer-4 linear probe still distinguishes the two corpora with accuracy 0.980, while k-NN corpus-probe accuracy falls from 0.965 to 0.831;
  • after projecting out only two corpus-predictive directions, direct-smoothing gains retain 84% on GENERATOR but only 45% on STATIONARY and 52% on PRIOR. The authors themselves write that “this control does not establish corpus independence.”

Taken together, more than half of the gain on two of the three main target streams claimed as effective depends on a cross-corpus bias that is linearly decodable.

The key point is that the corpus pairing is a design choice, not an unavoidable property of the corpora. Appendix C.3 states:

“A development stream disjoint from all evaluation data would provide stronger separation. The available corpus pairing does not permit such a construction. M-AILABS English contains only four bonafide speakers, and all four occur in the evaluation streams.”

This statement is literally correct: MLAAD was synthesized from M-AILABS, so a same-source pairing necessarily inherits those four speakers. But the requirement that bona fide and spoofed speech be same-source is itself a design goal chosen by the authors. A corpus with many speakers could be used instead (for example, a multi-speaker read-speech corpus paired with open-source TTS synthesis, or a public deepfake corpus with substantially more than four speakers) to construct a stream in which the two classes share a recording pipeline while bona fide speakers do not recur. The paper does not discuss such alternatives.

The authors identify this as the immediate test for future work in Section 7:

“The immediate test for any successor is a target stream with many bonafide speakers, which would show whether relational signals of this kind survive without repeated-speaker structure.”

However, the application scenarios introduced in Section 1 are forensic laboratories and fraud-screening services, whose bona fide traffic would normally contain many non-repeating speakers. For a paper whose contribution includes a controlled benchmark, the authors’ admission that the benchmark cannot test the source of its core method gain weakens the benchmark’s support for the paper’s own method claim (the benchmark may still be useful as an evaluation tool).

Weakness 4: the method degrades under the most imbalanced prior, which is also the paper’s chosen application prior

Appendix D.3 reports that in the R=0.05R=0.05 block of PRIOR, CAST-LN changes local minDCF relative to the frozen model by +0.035, +0.052, and +0.004 on the three checkpoints. Approximately 24% of spoofed utterances have no reciprocal neighbor (versus about 4% under the balanced prior), and 12.6% of the retained reciprocal neighbors are bona fide (versus about 5% under the balanced prior).

The paper fixes the cost-function prior at πspf=0.05\pi_{\mathrm{spf}}=0.05, explicitly describing it on page 4 as a realistic application prior. All three checkpoint changes in this block are positive; although +0.004 is at noise level, the direction is consistent. The authors appropriately report this as a limitation and note that CAST-FT still improves in the same block, so they do not claim a causal explanation. However, they do not discuss the coincidence that the weakest result occurs exactly at the realistic prior selected in the paper, or what this implies for sparse attack traffic in practice.

Weakness 5: two relevant prior works are not cited

Section 2 states:

“We evaluate strictly prequential CTTA, in which no source data, target labels, or domain boundaries are available... We are not aware of previous work on audio deepfake detection in this regime.”

Under the paper’s own definition (no source data, no target labels, no domain boundaries, a threshold fixed on source data, and separate reporting of discrimination and calibration loss), the narrow claim appears defensible: T²A concerns vision and is not continual; CDDB is labeled continual learning; RF-Prompt is labeled continual learning; and Oiso et al. require labeled target examples.

However, the following peer-reviewed works were published before the stated submission deadline (September 16, 2026) and are not cited:

  • T²A: Think Twice before Adaptation (IJCAI-25). Its abstract describes “an online test-time adaptation method that enhances the adaptability of detectors during inference without requiring access to source training data or labels.” It is source-free and label-free online TTA for deepfake detection, although it targets vision rather than audio.
  • CDDB: A Continual Deepfake Detection Benchmark (WACV 2023). It considers “a stream of likely heterogeneous deepfakes [that] appear time by time rather than at once,” which is structurally related to the streams constructed in Section 3. Its information setting is labeled continual learning (each new task has full annotations and exemplar replay), so it does not invalidate the paper’s exact setting, but it is a prior benchmark for continual deepfake detection. The generator, prior, and locale sequences in this paper also resemble its EASY/HARD/LONG sequence design.

There is also Ranked Entropy Minimization for Continual Test-Time Adaptation (ICML 2025), which uses a ranking objective in continual TTA. Its ranking is over the entropy of the same sample under different difficulty conditions, rather than over score relations between different samples. It is related but operates on a different signal.

The paper cites Oiso et al. (Interspeech 2024) and correctly classifies it as requiring labeled target samples. The two missing citations are not, by themselves, grounds for rejection, but the positioning should be expanded.

Questions for authors

1. Please explain why ranking distillation is needed instead of directly using the smoothed scores from Equation 2. Appendix E.1 shows that direct smoothing gives better discrimination on every target stream. The paper’s stated reasons are that direct smoothing collapses on SOURCE and has poor delivered cost after recalibration. However, Appendix E.4 shows that anchored CAST also delivers lower cost in only 3 of 9 cells. In what measurable sense is distillation necessary? If the answer is source retention, please give a way to verify that property without target labels.

2. Have you tried parameter-space anchoring? Appendix E.4 tests score-space anchoring (a penalty on score-location departure), but no experiment with a parameter displacement penalty λ∥ϕ−ϕ0∥2\lambda\|\phi-\phi_0\|^2 is reported. This constraint does not require source data, and CAST already has a frozen copy of the parameters. If you tried it, please report the result; if not, please explain why.

3. Can you provide a target stream with non-repeating bona fide speakers? Section 7 identifies this as the immediate future test. Even a smaller stream showing how much of the CAST and direct-smoothing gains remain would substantially change the assessment of Weakness 3.

4. What mechanism makes the neighborhood signal fail in the source regime but work after target shift? Equation 2 raises EER from 6.3% to 12.4% on SOURCE, while improving ordering on all three target streams. What measurable property determines whether smoothing improves or damages an ordering?

5. Can you add actDCF to the main table, or provide an operational metric computable without target labels? Both minDCF and EER require true labels to sweep thresholds. If CAST is intended for strict prequential deployment, its fixed-threshold or causally estimated-threshold behavior should appear alongside minDCF.

6. Please clarify the relationship to the following works: T²A (IJCAI-25, source-free and label-free online TTA for deepfake detection), CDDB (WACV 2023, continual deepfake detection benchmark), and Ranked Entropy Minimization (ICML 2025). In particular, why do T²A and CDDB not affect the claim in Section 2 that there is no prior audio deepfake detection work under this exact regime, and how does CAST’s ranking target differ from Ranked Entropy Minimization?

7. CAST-FT still improves in the R=0.05R=0.05 block while using the same neighborhoods (Appendix D.3). Does this suggest that the limitation is the capacity of CAST-LN rather than the neighborhood signal itself?

1 comment

We thank the reviewer for a detailed reading and for granting that the Section 2 regime claim is defensible. Three things are new since submission: a stream in which bona fide and spoofed speech share corpus and recording conditions, a second with 1,086 bona fide speakers, and the Q2 anchoring experiment. All corrections below go into the revision.

W1, Q1. Direct smoothing beats CAST-LN; is distillation necessary. Your table is correct and stands. Distillation is necessary in two measurable senses.

First, the ranking target matters, not only the smoothing. Appendix E.2 holds the pipeline fixed and changes only the target: soft class targets gain 0.031 to 0.041 on the held-out Generator checkpoints, against 0.062 to 0.067 for the ranking target. Hard class targets are worse than frozen in eight of nine comparisons.

Second, the ordering between the two is not stable once the corpus pairing is removed. Mean per-block change in minDCF against frozen on the new ASVspoof 5 stream:

methodXLS-RWavLM
smoother−0.060−0.107
CAST-LN−0.066−0.088

No paired interval between the methods was computed, so the claim is narrow: the smoother no longer leads on both backbones.

Source retention needs no target labels. It is measured on the labelled ASVspoof 2019 LA evaluation set, which is used for evaluation only and never for adaptation. There the smoother raises minDCF by 0.112 and EER from 6.3 to 12.4%, where CAST-LN lowers minDCF by 0.017.

W3, Q3. Attribution; a stream with non-repeating speakers. The pairing is a design choice, as you say, and both new streams drop it.

SpoofCeleb answers Q3. It has ten blocks of 1,200 items and 1,086 bona fide speakers, with speaker identity at 50.00% EER. On WavLM, CAST-LN lowers mean per-block minDCF by 0.037 across all ten blocks, with intervals below zero at every checkpoint, against 0.041 for a labelled update. The smoother's many-speaker result is in the table above.

ASVspoof 5 carries 367 speakers at 49.5% EER and is class-balanced, so random pairing would give 50% same-class neighbours. Reciprocal neighbours share an item's class 90.1% of the time, against a permuted-label floor of 49.9%, and its speaker only 11.1%. Among neighbours from a different speaker, 81.5% still share the class, so the signal does not run through speaker identity.

W2, Q5. Inconsistent standards; actDCF in the main table. Agreed, and actDCF joins Table 2. Under one rule applied to both methods, a 95% paired interval below zero, the count is three of nine each way, as you state.

The operating-point loss has since become partly recoverable. The adapted scores are mapped onto the frozen score scale by the causal monotone transformation of Appendix E.3, then recalibrated with MLLS. Against the recalibrated frozen detector, the combination lowers actDCF in 31 of 36 stream-and-checkpoint comparisons, with per-block minDCF unchanged.

actDCF needs labels, being built from false acceptance and false rejection. What can be label-free is the threshold, which the appendix estimates causally.

Q2. Parameter-space anchoring. We tested it: six strengths of an L2 penalty on the LayerNorm parameters against the frozen copy. None met the rule that selected the score-space anchor: false acceptance and false rejection at the fixed threshold on Source must stay within 5 points of frozen. At matched displacement the penalty keeps 87 to 89% of the Generator gain, against 40 to 46% for the score-space form. It moves Source false acceptance by 66 to 69 points, against 1.7 to 3.3. Neither form holds both.

Q4. Why smoothing fails on Source. The measurable property is how much ordering error is left to repair: the frozen detector's AUC is 0.985 on Source against 0.91 to 0.93 on the target streams. The share of neighbours with the same class does not explain it: Locale is lowest at 0.88 and still gains. Centring within stream and checkpoint weakens the correlation from −0.776 to −0.223, so most of the correlation lies between streams rather than within them.

W4, Q7. Degradation at R = 0.05; is it capacity. The numbers are right, and the revision discusses the coincidence with the application prior. CAST-LN is worse than frozen by 0.035, 0.052, and 0.004, while CAST-FT is better by 0.042, 0.069, and 0.047. Both use the same utterances and neighbourhoods at every checkpoint, so the neighbourhood signal does not separate them. Capacity is the remaining difference, though they also differ in which parameters can move.

W5, Q6. Uncited works. All three will be cited in Section 2 of the revision. T²A works on images, adapts to one domain at a time, and carries no threshold across conditions: the three conditions our claim rests on. CDDB is supervised with replay, and is the precedent for our streams. Ranked Entropy Minimization ranks a sample against its own masked versions, so no second sample enters its loss; CAST ranks different utterances.

Reviewer_PsAcOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies continual test-time adaptation for audio deepfake detection under a challenging deployment setting where source data, target labels, and domain boundaries are unavailable. The paper makes an important distinction between discrimination failure, where the ranking between bona fide and spoofed samples deteriorates, and operating-point failure, where scores shift relative to a fixed deployment threshold. Through a controlled benchmark, the authors show that several existing prediction-driven CTTA methods primarily affect the operating point while providing limited improvement in discrimination. Motivated by this observation, they propose CAST, which improves discrimination over existing CTTA baselines across multiple streams and two backbones, although it does not resolve operating-point calibration.

Critical strengths
  • The idea and analysis is well motivated.
  • The proposed method, CAST, is conceptually simple and well connected to the diagnosis.
  • The paper considers several types of distribution shift, five CTTA baselines, two SSL backbones, labeled reference updates, direct neighborhood smoothing, and class-target alternatives. The improvements over prediction-driven CTTA are substantial rather than marginal.
Critical weaknesses
  • Bona fide and spoofed target samples originate from different corpora. Although the authors make quite a lot of efforts to remove obvious corpus cues, their own analysis shows that corpus information remains in the representation, and the CAST gain drops considerably after removing corpus directions on STATIONARY and PRIOR. This makes it difficult to determine how much of the improvement comes from genuine deepfake-related structure rather than dataset identity.
  • The paper reports that 99.9% of reciprocal neighbors for bona fide samples share the same speaker, with only four M-AILABS voices recurring in the streams. This is important because CAST’s main contribution relies precisely on neighborhood structure. It is therefore unclear whether the method would retain its advantage in a realistic stream containing many unseen speakers.
  • CAST can substantially improve minDCF while severely degrading performance at the fixed deployment threshold; the paper reports FAR reaching 84–89% for CAST-LN on SOURCE in some settings. While the authors correctly frame operating-point estimation as a separate problem, this limits the practical value of CAST as a standalone adaptation method.
  • The experiments are all conducted on the proposed method. There are no evaluations on other datasets or distribution shift method such as [1], [2], which makes it hard to validate the generalization of the proposed method.

[1] Li, Yuang, et al. "Cross-domain audio deepfake detection: Dataset and analysis." Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

[2] Li, Xiang, Pin-Yu Chen, and Wenqi Wei. "Measuring the Robustness of Audio Deepfake Detection under Real-World Corruption." Proceedings of the Sixteenth ACM Conference on Data and Application Security and Privacy. 2026

Questions for authors
  1. How much of CAST’s gain would remain on a target dataset where bona fide and spoofed speech share the same recording pipeline and corpus? Given the remaining corpus separability, I think this is the most important question for establishing whether the neighborhood signal captures deepfake characteristics rather than dataset identity.
  2. Can CAST be evaluated on a stream with much more bona fide speakers and limited speaker repetition? The extremely high same-speaker rate among reciprocal neighbors raises the possibility that speaker identity is an important source of the useful neighborhood structure.
  3. Could the authors explore a joint solution for ranking adaptation and online recalibration? CAST improves discrimination but can simultaneously make the deployed threshold unusable. Even a simple combination with the paper’s label-free recalibration baseline would help clarify how much of CAST’s discrimination improvement can translate into better delivered detection cost.
2 comments

We thank the reviewer for three questions, which we address with three new experiments. We built two new target streams. In the first, bona fide and spoofed speech share corpus and recording conditions; the second carries 1,086 bona fide speakers. We also implemented the joint ranking-and-recalibration method asked for in Q3. The five published baselines were rerun on both streams, and all of this goes into the revision.

W1, Q1. Different corpora; how much of the gain survives a shared one. Most of the gain survives, and the new stream does not need the projection control. It is drawn from ASVspoof 5. Bona fide and spoofed speech come from the same Multilingual LibriSpeech partition, and both pass through the same five codec conditions. This unties the class label from the source corpus in the data, rather than by removing corpus directions afterwards. Mean per-block change in minDCF against the frozen detector, averaged over three checkpoints, where negative is an improvement:

methodXLS-RWavLM
Tent−0.008+0.001
EATA-rd−0.002−0.002
RDumb−0.004−0.000
SAR−0.000−0.000
RoTTA+0.048+0.114
CAST-Score−0.060−0.107
CAST-FT−0.060−0.094
CAST-LN−0.066−0.088
Labeled-LN (uses labels)−0.080−0.158

CAST-LN gains 0.066 and 0.088 here, against 0.065 on the published Generator stream. All six 95% bootstrap intervals, one per backbone and checkpoint, lie below zero. A shared corpus does not rule out every class-correlated processing difference, but the gain does not require the corpus pairing.

W2, Q2. Repeated speakers; a stream with many more. The ASVspoof 5 stream carries 367 bona fide speakers in equal numbers of each class, so random pairing would give 50% same-class neighbours. Reciprocal neighbours share an item's class 90.1% of the time, against a permuted-label floor of 49.9%, and share its speaker only 11.1% of the time. Among neighbours drawn from a different speaker, 81.5% still share the class, against approximately 0% on the published stream. The neighbourhood signal therefore carries class information without carrying speaker identity.

The second stream, from SpoofCeleb, carries 1,086 bona fide speakers, with speaker identity predicting the class at 50.00% EER. Its results are in W4.

W3, Q3. Fixed-threshold cost; a joint solution. We agree, and the combination you suggest does translate part of the discrimination gain into delivered cost. The adapted scores are mapped onto the frozen detector's score scale by the causal monotone transformation of Appendix E.3, then recalibrated with MLLS. Measured against the recalibrated frozen detector, the combination lowers actDCF in 31 of 36 stream-and-checkpoint comparisons on the target streams. MLLS applied to the adapted scores alone wins 20, at a similar mean reduction of 0.037. The map is monotone, so per-block minDCF is unchanged everywhere we measured, and the discrimination gain is kept whole.

On Source, where the 84–89% figure comes from, false acceptance at the unmoved threshold falls from 84.2, 84.9, and 88.9% to 6.2, 9.9, and 9.2% across the three checkpoints. These rates are below the frozen detector's own 16.8, 17.9, and 18.1% at the same threshold, at a false-rejection rate of 4.0, 2.3, and 2.5%. The worst 600-utterance window falls from 100% to 19.6, 23.4, and 22.2%.

On the target streams the frozen detector at that threshold already accepts about 80% of spoofs, because the threshold is fitted on ASVspoof 2019 LA development data and the target streams are a different corpus. The combination improves on the frozen detector in every stream and checkpoint we measured, and its worst case is 68.9% false acceptance.

W4. No evaluation on other datasets or shift types. Both new streams are outside the published benchmark. The SpoofCeleb stream has ten blocks of 1,200 items and 1,086 bona fide speakers. On WavLM, CAST-LN lowers mean per-block minDCF by 0.037 across all ten blocks, with intervals below zero at every checkpoint; a labelled update of the same parameters reaches 0.041. The published baselines move it by 0.004 or less, except RoTTA, which degrades it by 0.237. On XLS-R the frozen detector is already strong on this corpus, at 8.7 to 9.6% EER, which leaves little to repair. Over the six blocks with room to improve, the gain is 0.010. WavLM was fixed as this stream's primary backbone before the methods ran.

The ASVspoof 5 stream also adds a shift the benchmark did not have: five codec conditions vary alongside the generator. We thank the reviewer for the two benchmarks named. We spent the discussion period building these two streams, and the revision discusses both works in its positioning.

Thank you for the response and additional experiments. The authors have substantially addressed my major concerns, although some practical limitations remain. I therefore raise my rating from 2 to 3.

Reviewer_Bu6kOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies continual test-time adaptation for audio deepfake detection. It shows that existing prediction-based adaptation methods fails discrimination. The authors further propose CAST, which smooths frozen scores over reciprocal nearest neighbors and distills the resulting order into the adapting detector with a pairwise ranking loss. Experiments demonstrate the improvements of CAST over several existing methods.

Critical strengths
  1. The paper studies a practical and important problem in audio deepfake detection.
  2. Separating discrimination and calibration provides useful insights into existing test-time adaptation methods.
  3. The experiments include several baselines, two backbones, and detailed analysis of different adaptation settings.
Critical weaknesses
  1. The target real and fake audio come from different datasets. Although the authors apply some preprocessing, the model may still exploit dataset-specific artifacts.
  2. CAST mainly improves discrimination. The false acceptance rate is very high, which limits its practical usefulness.
  3. The limited number of bona fide speakers and the fixed generator ordering may affect the generalization of the conclusions.
Questions for authors
  1. Can the authors evaluate CAST on a dataset where real and fake audio share the same recording conditions?
  2. How does CAST perform when the test stream contains more speakers and more frequent distribution changes?
  3. Can the authors provide a practical solution to the operating-point shift introduced by CAST?
1 comment

We thank the reviewer for putting the dataset and stream-structure concerns in a form we could test. We built a target stream in which bona fide and spoofed speech share corpus and recording conditions, added a second corpus with 1,086 bona fide speakers, and rebuilt the Generator stream under alternative orderings and block lengths. The five published baselines were rerun on each. All of this goes into the revision.

W1, Q1. Real and fake audio from different datasets. The new stream is drawn from ASVspoof 5. Bona fide and spoofed speech come from the same Multilingual LibriSpeech partition, and both pass through the same five codec conditions, so the class label is no longer tied to the source dataset. We report mean per-block change in minDCF against the frozen detector, averaged over three checkpoints, where negative is an improvement:

methodXLS-RWavLM
Tent−0.008+0.001
EATA-rd−0.002−0.002
RDumb−0.004−0.000
SAR−0.000−0.000
RoTTA+0.048+0.114
CAST-Score−0.060−0.107
CAST-FT−0.060−0.094
CAST-LN−0.066−0.088
Labeled-LN (uses labels)−0.080−0.158

CAST-LN gains 0.066 and 0.088 here, against 0.065 on the published Generator stream. All six 95% bootstrap intervals, one per backbone and checkpoint, lie below zero. The published baselines move minDCF by 0.008 or less, except RoTTA, which degrades it.

W3, Q2 (speakers). Few bona fide speakers; more speakers in the stream. The ASVspoof 5 stream carries 367 bona fide speakers, and speaker identity predicts the class at 49.5% EER, which is chance, so speakers carry no class information. A second stream built from SpoofCeleb carries 1,086 speakers, with speaker identity at 50.00% EER. On WavLM, CAST-LN lowers mean per-block minDCF by 0.037 across all ten blocks, with intervals below zero at every checkpoint; a labelled update of the same parameters reaches 0.041. On XLS-R the frozen detector is already strong on this corpus, at 8.7 to 9.6% EER, which leaves little to repair. Over the six blocks with room to improve, the gain is 0.010.

W3, Q2 (ordering). Fixed generator order; more frequent change. We rebuilt the Generator stream under two further orderings: one weakening the correlation between block position and frozen error to −0.20, and one reversing it to −1.00. CAST-LN lowers mean per-block minDCF by 0.065 under the published order, 0.043 under the first, and 0.051 under the second. Every interval lies below zero, and the gain therefore does not depend on the published ordering. Reordering does change which blocks gain most. Under the reversed order the largest gains fall where the frozen detector errs least, and the revision withdraws the reading that CAST helps most where the frozen detector is weakest.

For more frequent change we re-blocked the same utterances, traversing one fixed generator order k times so that each generator contributes one chunk per pass. The table below gives CAST-LN's reduction in minDCF against the frozen detector, averaged over three checkpoints:

utterances per blockblockswhole streamper generator
1,200520.0470.043
4001560.0320.038
2402600.0370.042
807800.0490.053

The gain holds at every block length, and the shortest blocks, where change is most frequent, give the largest. Shortening the blocks also makes each generator recur, so block length and recurrence vary together. All four constructions use the reordered stream above, which holds the ordering fixed.

W2, Q3. High false acceptance; a practical solution. We agree that fixed-threshold behaviour limits practical use. CAST's ranking loss sees only score differences, so nothing in it holds the operating point.

The adapted scores are mapped onto the frozen detector's score scale by the causal monotone transformation of Appendix E.3, then recalibrated with MLLS. The map is monotone, so per-block minDCF is unchanged everywhere we measure.

On Source, where the 84–89% figure comes from, false acceptance at the unmoved threshold falls from 84.2, 84.9, and 88.9% to 6.2, 9.9, and 9.2% across the three checkpoints. These rates are below the frozen detector's own 16.8, 17.9, and 18.1% at the same threshold, at a false-rejection rate of 4.0, 2.3, and 2.5%. The worst 600-utterance window falls from 100% to 19.6, 23.4, and 22.2%.

On the target streams the frozen detector at that threshold already accepts about 80% of spoofs, because the threshold is fitted on ASVspoof 2019 LA development data and the target streams are a different corpus. The combined method improves on the frozen detector in every stream and checkpoint we measured. Its worst case is 68.9% false acceptance, so the repair is real on Source but not yet a deployable operating point on every stream.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal revision

Revised PDF uploaded

A revised PDF is now uploaded. All additions and changes are marked in blue. The main text is within the page limit; most new material is in the appendix.

New experiments

AdditionWhere
Two corpora outside the benchmark: ASVspoof 5 (one corpus, 367 speakers on both class sides) and SpoofCeleb (1,086 bona fide speakers). All five baselines, both labeled references, the smoothed scorer and both CAST arms, on both backbones§5, Table 3; App. A.7
Construction, preprocessing, shortcut audit, pretraining overlap and frozen levels of both new streamsApp. A.7
Two alternative generator orderings and four block lengthsApp. A.4
Parameter-space anchoring, six penalty strengthsApp. E.4
Monotone score map composed with label-free recalibrationApp. E.5
Why direct smoothing fails in the source regime (frozen AUC, neighbour-purity controls)App. E.1
Neighbourhood statistics on ASVspoof 5, including neighbours from a different speakerApp. C.2
CAST-FT in the 5%-spoof block, and the application prior in that blockApp. C.3
Five further references, including ranking as a continual TTA objective and the closest label-free image setting§2

Corrections to the submitted version

CorrectionWhere
The low-pass control was described as 7.7 kHz applied at the native rate before resampling. The filter applied is a zero-phase eighth-order Butterworth at 7 kHz after resampling to 16 kHz§3
Scope statements saying several arms were not run on WavLM now say this applies to the five benchmark streams; every arm runs on both backbones on the two new streamsApp. B.2, C.5, F.5
Corrected cross-reference for the k=0 and direction-reversal ablations§5, App. F
A missing operator in the normalized DCF equation, and a stray sentence fragment in §6§3, §6

Table 1 now reports change in actDCF and EER alongside change in minDCF, with the frozen detector's own levels. §6 was restructured to incorporate the new experiments.

We are glad to answer further questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.