acceptodds
Under review as a conference paper at ICLR 2027

SRAM: Learning Spectral Relation-Aware Representations for Environmental Sound Deepfake Detection

Abstract

Environmental sound deepfake detection (ESDD) models are critical for countering fabricated environmental audio, which can mislead users of audio-based assistive technologies and trigger unintended smart-home responses. A reliable ESDD method requires accurate detection even when the source datasets and generators are unseen in training. Existing ESDD methods broadly follow two paradigms: task-specific discriminative models and generative methods based on audio large language models (ALLMs). We find that both approaches have significant limitations: (i) discriminative methods are highly dependent on the seen domain conditions of the training data, and their classification performance drops significantly when the test audio comes from unseen domains; (ii) generative methods generally underperform and incur higher per-sample inference latency and deployment costs. We conduct a large-scale evaluation of 15 ALLMs, spanning five model families, on the ESDD task. The results show that their zero-shot performance is close to random guessing, and unlike the gains reported in speech deepfake detection, their performance remains significantly inferior to that of strong task-specific discriminative models even after fine-tuning. Further analysis shows that a classifier built on an ALLM's audio encoder can outperform the full generative model, while local spectro-temporal residuals and time-frequency geometry reveal differences between real and AI-generated audio. Motivated by these findings, we propose a novel detection framework, called Spectral-Relational Artifact Modeling (SRAM). SRAM combines audio representation learning with spectral relation modeling. It builds on the Qwen3-Omni audio encoder, using cross-layer trajectory refinement to integrate information from multiple encoder depths and sparse evidence pooling to focus on informative temporal tokens. To complement these audio representations, we use a spectrogram-residual branch to capture local spectral differences and a structure-tensor branch to describe local time-frequency geometry. Extensive experiments demonstrate SRAM’s strong detection performance, generalization to unseen domains, and robustness to unseen perturbations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
deepfake detection, environmental sound, audio

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3L5QOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SRAM, a spectral relation-aware framework for environmental sound deepfake detection (ESDD). The method builds on the Qwen3-Omni audio encoder and combines cross-layer trajectory refinement, sparse evidence pooling, spectrogram residual modeling, and structure-tensor-based geometric features. The paper also presents a systematic evaluation of 15 Audio Large Language Models (ALLMs) under zero-shot and supervised fine-tuning settings. Experiments on EnvSDD demonstrate strong detection performance across seen and unseen source datasets and generation models.

Critical strengths
  1. The paper addresses an important and relatively underexplored problem in environmental sound deepfake detection, with particular emphasis on generalization to unseen domains and generators.

  2. The systematic evaluation of 15 ALLMs provides useful empirical insights into the limitations of generative audio models for deepfake detection. The observation that an audio encoder with a simple classifier can outperform the full generative model is particularly interesting.

  3. The proposed method achieves strong results on EnvSDD, especially under unseen source and generator conditions. The paper also includes component ablations, robustness evaluations, and additional analyses that help characterize the proposed framework.

Critical weaknesses
  1. The comparison with recent state-of-the-art ESDD methods is incomplete.

Although the paper compares SRAM with several representative discriminative and generative approaches, it does not include strong recent systems from the ESDD 2026 Challenge, particularly the top-performing challenge solutions. Given the relevance of these methods to environmental sound deepfake detection, a more comprehensive comparison would be valuable. While differences in datasets and evaluation protocols may prevent direct numerical comparisons, the authors should either evaluate representative challenge systems under a consistent protocol or explicitly discuss their relationship to SRAM. This would provide stronger evidence for the claimed state-of-the-art performance.

  1. The generalization evaluation is primarily limited to a single benchmark.

Although EnvSDD provides comprehensive combinations of seen and unseen source datasets and generators, all primary detection experiments are conducted within this benchmark and its predefined splits. Such evaluations demonstrate generalization under the EnvSDD protocol but do not fully establish robustness across independently constructed datasets with different collection procedures, synthesis systems, and acoustic characteristics. Evaluation on an additional external ESDD benchmark, such as Fakesound or other sound deepfake dataset, would substantially strengthen the generalization claims.

  1. The contribution of the proposed architectural components is not sufficiently isolated from the strong pretrained audio encoder.

The authors evaluate SRAM with several audio backbones, including Qwen2-Audio, Qwen2.5-Omni, Qwen3-Omni, and Gemma-3n, in Table 10. These results provide useful evidence of backbone generality. However, Figure 9 evaluates the Encoder + MLP baseline only with Qwen2-Audio and Qwen2.5-Omni, while the main SRAM model uses Qwen3-Omni. A directly matched Qwen3-Omni Encoder + MLP baseline is therefore missing. Furthermore, although Table 4 supports the effectiveness of the proposed components, the individual contributions of the structure-tensor branch and sparse pooling are relatively modest. A controlled comparison between a fine-tuned Qwen3-Omni encoder with a simple classifier and the complete SRAM framework would help clarify whether the additional architectural complexity is justified by substantial performance gains.

Questions for authors
  1. The comparison with recent state-of-the-art ESDD methods is incomplete.

Although the paper compares SRAM with several representative discriminative and generative approaches, it does not include strong recent systems from the ESDD 2026 Challenge, particularly the top-performing challenge solutions. Given the relevance of these methods to environmental sound deepfake detection, a more comprehensive comparison would be valuable. While differences in datasets and evaluation protocols may prevent direct numerical comparisons, the authors should either evaluate representative challenge systems under a consistent protocol or explicitly discuss their relationship to SRAM. This would provide stronger evidence for the claimed state-of-the-art performance.

  1. The generalization evaluation is primarily limited to a single benchmark.

Although EnvSDD provides comprehensive combinations of seen and unseen source datasets and generators, all primary detection experiments are conducted within this benchmark and its predefined splits. Such evaluations demonstrate generalization under the EnvSDD protocol but do not fully establish robustness across independently constructed datasets with different collection procedures, synthesis systems, and acoustic characteristics. Evaluation on an additional external ESDD benchmark, such as Fakesound or other sound deepfake dataset, would substantially strengthen the generalization claims.

  1. The contribution of the proposed architectural components is not sufficiently isolated from the strong pretrained audio encoder.

The authors evaluate SRAM with several audio backbones, including Qwen2-Audio, Qwen2.5-Omni, Qwen3-Omni, and Gemma-3n, in Table 10. These results provide useful evidence of backbone generality. However, Figure 9 evaluates the Encoder + MLP baseline only with Qwen2-Audio and Qwen2.5-Omni, while the main SRAM model uses Qwen3-Omni. A directly matched Qwen3-Omni Encoder + MLP baseline is therefore missing. Furthermore, although Table 4 supports the effectiveness of the proposed components, the individual contributions of the structure-tensor branch and sparse pooling are relatively modest. A controlled comparison between a fine-tuned Qwen3-Omni encoder with a simple classifier and the complete SRAM framework would help clarify whether the additional architectural complexity is justified by substantial performance gains.

Reviewer_cfSQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SRAM, a framework for environmental sound deepfake detection that combines pretrained audio representations with spectral residual modeling, structure-tensor features, cross-layer trajectory refinement, and sparse evidence pooling. The authors also systematically evaluate 15 audio large language models (ALLMs), revealing their limitations on this task. Experiments on EnvSDD demonstrate strong detection performance, achieving 99.30% average AUC and 2.04% average EER. While the results are promising, several concerns remain regarding backbone selection, baseline coverage, robustness evaluation, and generalization.

Critical strengths

Comprehensive ALLM evaluation. The systematic evaluation of 15 ALLMs provides valuable insights into the limitations of general-purpose audio models for environmental sound deepfake detection. Well-motivated framework. The integration of local spectral residuals, time-frequency geometry, and cross-layer representations is technically reasonable, and the ablation studies demonstrate the effectiveness of the proposed components. Strong empirical performance. SRAM achieves strong performance across seen and unseen domains, supported by extensive ablation studies, robustness evaluations, and efficiency comparisons.

Critical weaknesses

Insufficient justification for backbone selection. Although several ALLM audio encoders are evaluated, the study does not investigate widely used pretrained audio models such as BEATs and W2V2 within the SRAM framework. This limits the evidence supporting the choice of Qwen3-Omni as the primary backbone. Limited coverage of recent detection methods. While the paper includes several representative baselines, the comparison could be strengthened by incorporating more recent approaches, such as AASIST2 and the ESDD-specific BEAT2AASIST. Evaluating these methods under the same protocol would provide a more convincing assessment of SRAM's competitiveness. Limited robustness evaluation. The robustness experiments cover only four types of perturbations, each at two severity levels. Common real-world distortions, such as audio compression and reverberation, are not considered. Consequently, the current evaluation may not fully demonstrate the robustness of SRAM under practical deployment conditions. Limited generalization validation. The experiments are primarily conducted on EnvSDD, and performance remains noticeably lower under the most challenging jointly unseen setting. Additional evaluation on independent datasets would further strengthen the generalization claims.

Questions for authors

Could the authors evaluate alternative pretrained audio backbones, such as BEATs and W2V2, within the SRAM framework to further justify the backbone selection? Could the authors include more recent detection methods, particularly AASIST2 and BEAT2AASIST, under the same EnvSDD evaluation protocol? Could the authors extend the robustness evaluation to include more realistic audio distortions, such as compression and reverberation? Could the authors provide additional cross-dataset evaluations to further validate SRAM's generalization to unseen acoustic domains and generators?

Reviewer_51kAOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies environmental sound deepfake detection (ESDD) and presents an evaluation of 15 audio large language models (ALLMs). The authors claim that existing ALLMs perform poorly in the zero-shot setting and generally remain behind strong task-specific discriminative models after supervised fine-tuning. Motivated by the observation that pretrained audio encoders still contain useful detection signals, the paper proposes SRAM, which combines representations from a Qwen3-Omni audio encoder with explicit spectral-relation features. Experiments on EnvSDD show strong overall performance, particularly under unseen-domain and audio-perturbation settings.

Critical strengths
  • The large-scale evaluation provides an interesting empirical finding: strong general-purpose audio models do not necessarily translate into strong environmental deepfake detectors (although it makes sense for zero-shot evaluation).
  • The proposed SRAM achieves strong empirical results, especially on unseen source datasets and generators.
  • The method is reasonably well supported by ablations. Removing spectral-relation modeling or cross-layer refinement leads to clear degradation, particularly on the more challenging test subsets.
Critical weaknesses
  • I have concerns about the fairness of the ALLM comparison, particularly the supervised fine-tuning setup, and thus the corresponding claim. For most ALLMs, the audio encoder is frozen and LoRA is applied only to the LLM component. This seems potentially suboptimal for a low-level forensic task whose discriminative cues are primarily acoustic. In contrast, SRAM fine-tunes the Qwen3-Omni audio encoder end-to-end. Therefore, the reported gap may reflect the adaptation strategy as much as an inherent limitation of ALLMs. This concern is reinforced by Figure 9, where fully fine-tuning the audio encoder substantially improves several ALLMs.
  • Similarly, the paper makes a relatively strong claim that task-specific discriminative approaches outperform fine-tuned ALLMs, but the ALLM adaptation space is not explored sufficiently to support such a general conclusion. The authors use one common LoRA configuration (rank 64, α=16, LR 5\times10^{-5}) across heterogeneous model families and largely inherit the protocol from prior work. It is unclear whether these settings are close to optimal for each model.
  • The empirical evaluation is heavily centered on EnvSDD. Although its seen/unseen partitions are useful, the near-saturated performance (99.30% average AUC) raises questions about how well the conclusions transfer to genuinely different datasets and newer generators. Cross-dataset evaluation would make the generalization claim substantially stronger.
  • SRAM contains several specialized components—two spectral branches, trajectory refinement, gating, and sparse pooling. While the ablations establish that these components help, the paper provides less evidence that the particular design is necessary compared with simpler ways of combining a fully fine-tuned pretrained encoder with conventional spectral features. The improvement over strong baselines should therefore be weighed against the added complexity.
Questions for authors
  1. Could the authors clarify the rationale for primarily fine-tuning the LLM while freezing the audio encoder for the ALLM baselines? For ESDD, the relevant information appears to be low-level acoustic artifacts, so adapting the audio encoder seems particularly important. Figure 9 indeed suggests substantial gains from doing so. Would the main conclusion about ALLMs still hold if all models were compared under carefully tuned full/partial audio-encoder adaptation?
  2. Were the LoRA rank, learning rate, number of epochs, and other fine-tuning hyperparameters tuned independently for each ALLM family? Using one configuration across substantially different architectures may disadvantage some models and makes it difficult to distinguish limitations of ALLMs from limitations of the adaptation recipe.
  3. How well does SRAM generalize across datasets, rather than only across splits of EnvSDD? In particular, can the authors evaluate a detector trained on EnvSDD directly on an independently collected ESDD dataset or audio from newly released generators?
  4. Could the authors provide a simpler controlled baseline using the same fully fine-tuned Qwen3-Omni audio encoder as SRAM, followed by a standard pooling/classification head?
  5. Can the authors provide more clarification regarding the adoption of Qwen3-Omni as the backbone of the proposed method instead of the encoders from other ALLMs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.