- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AOI, an all-in-one AudioLLM framework for deepfake detection across speech, music, and environmental audio. The method first trains three type-specific experts using evidence-grounded rationales constructed with privileged acoustic measurements, while the detectors themselves only receive raw audio. These experts are then distilled into a single model using on-policy distillation with reward extrapolation (ExOPD). AOI performs five-way classification over genuine, synthetic, edited, codec-reconstructed, and vocoder-reconstructed audio. The final model achieves 97.95% weighted five-way accuracy, approaching the 99.33% performance of three separately routed label-only experts while producing explanatory rationales with a single checkpoint.
Critical strengths
- Unifying multiple audio types into a single detector without requiring an explicit type router is a useful direction, particularly as audio generation increasingly extends beyond speech.
- The evidence-grounded rationale construction is well motivated
- The paper separately investigates the cost of rationale training versus generative decoding and attempt to quantitatively evaluate rationale grounding. This makes the empirical analysis more informative than simply comparing final classification numbers
Critical weaknesses
- The benefit of the proposed Stage-2 distillation appears relatively modest. ExOPD improves weighted five-way accuracy from 97.71% to 97.95%. While statistically significant on a very large test set, the practical magnitude is small, making it unclear whether the additional complexity of training three experts followed by on-policy distillation is sufficiently justified.
- The paper does not convincingly isolate the contribution of ExOPD itself. The main comparison is essentially mixed SFT versus ExOPD. Important controls such as standard OPD (λ=1), alternative distillation strategies, and a sweep over the extrapolation coefficient are missing.
- The rationale evaluation remains somewhat weak. The main evidence relies on LLM-as-a-judge evaluation, and the grounding analysis is performed on only 500 examples and four acoustic descriptors. Moreover, correlation between values mentioned in generated explanations and measured descriptors does not necessarily establish that those explanations faithfully reflect the evidence actually used for classification.
- The experiments use path-disjoint held-out splits, but the paper’s motivation emphasizes robustness across audio types and forgery algorithms. Evaluation on genuinely unseen generators, codecs, datasets, or recording conditions would provide stronger evidence that AOI learns transferable forensic cues rather than dataset-specific characteristics.
Questions for authors
- How much of the Stage-2 improvement is actually attributable to reward extrapolation?
- Does AOI generalize to completely unseen forgery methods or datasets? Given that cross-generator generalization is one of the central challenges in audio deepfake detection, an out-of-distribution evaluation is necessary.
- How faithful are the generated rationales to the model’s actual decision process? The descriptor correlations show that the student can estimate certain acoustic properties, but do not establish that those properties causally influence its prediction.
- Given the relatively small improvement from ExOPD (97.71% → 97.95%), what is the computational cost of Stage 2, including training three experts and performing on-policy distillation?
2 comments
Thank you for the response and clarifications. However, my main concerns remain unresolved. The contribution of reward extrapolation cannot be isolated without a standard OPD baseline, particularly given the modest Stage-2 improvement and unquantified training cost. Moreover, the evaluation demonstrates unification across audio types but does not establish generalization to unseen generators or external datasets. The faithfulness of the generated rationales to the model's actual decisions also remains unverified. These issues concern the paper's central methodological claims and empirical support. I therefore maintain my original rating.
Thank you for the careful distinction between the utility of unification, the Stage-2 gain, and the evidence needed for stronger generalization and faithfulness claims.
Due to limited GPU resources, we are unable to conduct additional experiments during the rebuttal period. We focus on clarifying the method (§3) and existing results (§4) and providing implementation details in Appendices F and G. We welcome further discussion of AOI's design and experimental setting.
1. Magnitude and attribution of the Stage-2 gain. Weighted five-way accuracy improves from 97.71% to 97.95%, or 0.24 percentage points (§4.3, Table 1). The reported 26% recovery uses the 0.92-point gap between mixed SFT (97.71%) and the rationale-producing per-type experts (98.63%):
0.24 / 0.92 ≈ 26%.Section 4.3 reports a 2.12-point gain on speech codec reconstruction, and Appendix E describes 127 speech codec clips corrected from genuine to codec-reconstruction. These observations locate the gains in codec-reconstruction detection. The comparison evaluates the Stage-2 procedure as a whole; a λ = 1 control is not included. The λ = 1.25 choice follows the original work's recommended setting, and the reported checkpoint uses 14% of the planned schedule, at which point training was stopped to stay within the available compute budget (Appendix G.2).
2. Training cost and whether Stage 2 is justified. AOI (§§3.3–3.4) requires training three type-specific rationale experts and a mixed SFT model, followed by on-policy student distillation. Each ExOPD example entails student generation and scoring by the relevant frozen expert and fixed reference, in addition to the student update. These operations account for the additional Stage-2 training cost. The practical motivation is consolidating specialized knowledge into one rationale-producing checkpoint.
3. Unseen forgery methods and datasets. Our experiments evaluate unification across speech, music, and environmental audio (§4.1). The reported path-disjoint splits and shortcut controls define this evaluation setting (§§3.3, 4.2; Appendix B). A dedicated unseen-generator or external-dataset evaluation is not included.
4. Faithfulness of the rationales. The 500-clip descriptor study (Appendix D) tests agreement with signal measurements across four acoustic features. The judge study (§4.4) provides complementary preference evidence. Together, these results support partial acoustic grounding and the ability to inspect a model's stated rationale. Causal attribution of the model's decision was not tested.