acceptodds
Under review as a conference paper at ICLR 2027

BASS :Benchmarking AI-Synthetic Song Detection under Evolving Generators and Audio Corruptions

Abstract

Rapid updates to AI music generators challenge synthetic song detectors trained on fixed collections, while common audio processing can further degrade their performance. We introduce **BASS** (**B**enchmarking **A**I-**S**ynthetic **S**ong Detection under Evolving Generators and Real-World Corruptions), a benchmark containing 47,332 synthetic songs spanning 12 Suno and Mureka versions, together with rich generation metadata and 23 audio corruption conditions for robustness evaluation. We further propose **CARLA** (**C**orruption-**A**ware **R**eal-Density **L**earning and **A**daptation), a two-stage framework that learns a corruption-aware real-music density from human songs and adapts to each emerging generator through a lightweight LoRA adapter trained with at most 100 synthetic support songs. Experiments on BASS show that CARLA achieves state-of-the-art performance while remaining robust compared with baseline models. These results highlight the complementary importance of adapting to new generators and modeling robustness to audio corruption. Code and data are available at https://anonymous.4open.science/r/BASS-DAF5/.

Primary area:
datasets and benchmarks
Keywords:
Synthetic song detection, Benchmark, robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GLqnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tackles the problems of real-world corruptions and fast-improving synthetic song generators by proposing a new dataset BASS, which contains 23 audio corruptions. They also propose CARLA a 2 stage framework that learns a real-music density from human songs and adapts to new generators using a LoRA adaptor trained with fewer samples (at most 100). The main motivation is the low performance of existing SOTA models on unseen generators and corruptions. They combine 4 frozen audio encoders and try to model the density of real audio in stage 1 using projectors and a normalising flow, utilising both real and corrupted songs. To capture unseen generators, they add LoRA modules per generator to learn generator-specific features using few-shot samples.

Critical strengths
  1. The Problem is well motivated, interesting and useful, providing people an opportunity to build upon this dataset especially in deployment where new models arise commonly.

  2. The 2-stage workflow is interesting and useful in real-world scenarios.

  3. Modelling the real audio as a density and then adding adaptors for new generators is a different and interesting approach compared to what others in this domain have used/explored.

  4. The synthetic dataset is mostly well-built (see weaknesses), and the controlled perturbations are an added benefit.

Critical weaknesses
  1. The model is extremely inefficient and only slightly better than SpecTTTra (SONICS paper), CARLA requires passing the audio through 4 audio encoders, which are complex foundational models, causing much more compute, while only being slightly better than the other compared models.

  2. Ablations are not there for selecting foundational models and contribution of individual backbones; why weren't other models like Whisper (and variants of whisper) and other newer models tried?

  3. The human songs come from MTG-Jamendo, while synthetic songs come from Suno and Mureka, and the real and fake songs are also not matched by genre, so it could be that the trained model is learning that. And the real audio are all 30-second clips and the model operates at 60 seconds; this doesn't make sense

  4. No evaluation on MoM (Batra et al., 2025) as well SONICS (Rahman et al., 2025)

  5. Different versions of Suno and Mureka dont really count as unseen as they would have the signatures of their respective platforms, so the authors cant claim that they generalise to unseen generators.

  6. No real life corruptions or recordings through microphones; all robustness testing is controlled and does not make a good claim regarding the method.

  7. Sensitivity to few-shot samples is too random i think as performance decreases as we go from 100 to 200, not sufficiently explained using embedding based selection or multiple seeds

Questions for authors
  1. Why were four audio encoders required? There are no ablations showing the contribution of each encoder individually.

  2. Can the authors report inference time, FLOPs, memory consumption, and throughput compared to SpecTTTra?

  3. Can the authors evaluate alternative pretrained audio encoders, such as different Whisper versions or more recent encoders?

  4. How do the authors ensure that CARLA is not learning dataset-specific differences, such as 30-second clip lengths or unmatched genres?

  5. How are different audio durations handled, given that the model takes 1-minute audio inputs?

  6. Can the authors evaluate the effects of changing corruption parameters or applying multiple corruptions simultaneously?

  7. Can the authors evaluate CARLA on a completely unseen generator, such as Udio from MoM?

  8. Can the authors report results across multiple random seeds for support-sample selection and investigate why performance decreases when the number of support samples increases from 100 to 200? This is surprising and may be related to seed variability.

  9. Can the authors compare CARLA against simpler baselines, such as frozen pretrained models with few-shot adapters? Some straightforward baselines appear to be missing.

Reviewer_9UeTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper seeks to address two challenges: synthetic song detectors may not be able to handle new generators and audio processing methods can degrade detector performance. So the authors proposed a benchmark (named BASS) from recent song generators (from Suno and Mureka) including 23 audio corruptions. They also proposed a framework (named CARLA) that claims to learn corruption-aware real sample probability density and utilize few-shot training with LoRA per generator (with a routing mechanism) to adapt to new generators while keeping performance consistent. The experimental results demonstrate that the proposed CARLA method performs better across generators and corruptions in the BASS benchmark.

Critical strengths
  1. The paper seeks to address how to efficiently adapt to new generators while maintaining existing detector performance, which is a practical problem in synthetic song detection. The experiments on Table 3 show that this method has lower EER than related works under the same 100 song budget, which makes this adaptation a practical and useful contribution.
  2. The proposed BASS benchmark includes 11 identified generators (with version-level coverage) and 23 corruption conditions, which should be a valuable resource for future research.
Critical weaknesses

CARLA is a “SOFIA-inspired network” that follows “the class-conditional formulation of MusicDET”. Table 3 performance indicates that class-conditional MusicDET does not perform well, so it becomes important to isolate the CARLA’s gains from SOFIA-style networks. Ablation studies on SOFIA or SOFIA-based networks can potentially explain the drastic change in EER compared to class-conditional MusicDET and CARLA. This is important to properly understand the novelty of CARLA and why this is performing well in the BASS relative to other works

Questions for authors

Experiments:

  1. Can the authors use the SOFIA encoders with a simplified classifier and compare the performance against CARLA under the same 100 song budget?

Clarifications:

  1. In Table 3, class-conditioned MusicDET has 50.07% EER, whereas in Table 1, it gets 49.67% EER in BASS. Could the authors give some explanation on the lack of change in performance, specially since for SpecTTTra-alpha, few-shot training improved performance?
Reviewer_kVZWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AI music generators like Suno and Mureka keep releasing new versions, and detectors trained on older songs stop working on them. The paper presents BASS, a collection of 47,332 AI songs from 12 generator versions with 23 kinds of audio damage added for testing. It also proposes CARLA, which first learns what real music looks like from human songs and then learns each new generator from only 100 of its songs using a small add-on module.

Critical strengths

Collecting 47k Suno and Mureka songs with version labels, creator-grouped splits and 23 fixed corruptions is useful work.

Critical weaknesses
  1. The real samples are from MTG-Jamendo raw 30-second clips and the synthetic ones are full songs averaging 229 s. Since the model window is 60 s, I am confused: how are 30-s clips fed to a 60-s window? Padding or looping should not be used. Please use full-length Jamendo tracks or another length-matched real source.

  2. My main worry is that the paper claims “evolving generators” but I could not see proper structured results and analysis about “evolving generators” claim.

  3. The method is SOFIA features plus the MusicDET flow plus a LoRA bank - this is not novel at all.

  4. There are several typos - a few examples are: “We We demonstrate”, “basline”, “remaining robustness compared with”, “to enables”.

  5. No human evaluation for the benchmark.

Questions for authors

In addition to above questions/points:

  1. Several EERs in Table 3 are multople of 1/22 (4.55, 9.09, 13.64 all are multiples of 1/22). The v4.5-all and v7.5-all test sets have about 22 songs. Please report per-category test counts.

  2. Was augmentation applied identically to baselines? Do baselines see the same real data?

  3. There is no unseen-generator evaluation. Every test generator has its own adapter trained on its own data. Can we not do leave-one-generator-out testing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.