acceptodds
Under review as a conference paper at ICLR 2027

Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoning

Abstract

Large Audio Language models (LALMs) have made rapid progress on acoustic understanding, yet they still struggle with fine-grained audio reasoning (e.g., recognizing event order, repetitions and duration). Existing post-training methods heavily rely on expensive external labels or provide only coarse semantic signals. To bridge this gap, we introduce Audio-Zero, the first label-free self-evolution framework in the field of LALMs that improves fine-grained auditory perception and reasoning. Audio-Zero constructs an auditory self-play game from unlabeled audio contrast pairs: most players hear a reference audio, while one odd listener hears a subtle variant. The model first generates clues describing what it hears and then identifies the odd listener by reasoning over inconsistencies among clues. Since the odd listener is known by construction, the game provides verifiable rewards without any annotated answers. Experiments with Qwen2-Audio-7B-Instruct and Qwen2.5-Omni-7B on TREA, MMAU Test-mini and MMAR show that Audio-Zero improves fine-grained audio reasoning while preserving broad audio understanding. Evolutionary and diagnostic analyses further reveal that increasingly fine-grained auditory descriptions emerge naturally from game pressure.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Audio Language Models, Label-free Self-evolution, Auditory Self-play, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FT2JOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Audio-Zero, a label-free self-evolution framework designed to enhance fine-grained audio perception and reasoning in large audio language models. The approach builds a multi-player self-play game based on audio contrast pairs, in which the model produces auditory clues and locates the odd listener who hears a different recording. The two stages are optimized alternately with GRPO using rule-based, verifiable rewards. Experiments with Qwen2-Audio-7B-Instruct and Qwen2.5-Omni-7B show gains on TREA, MMAU Test-mini, and MMAR while using only 2,000 audio contrast pairs.

Critical strengths

The paper presents an interesting self-play mechanism that turns audio contrast pairs into verifiable training signals, eliminating the need for explicit audio annotations or QA labels. This offers a promising direction for lowering annotation costs in fine-grained audio reasoning. And it delivers notable improvements on TREA, MMAU Test-mini, and MMAR across two different LALM backbones, using only 2,000 audio contrast pairs. Ablation studies further corroborate the effectiveness of the proposed dual-stage optimization framework.

Critical weaknesses
  1. The proposed method requires audio contrast pairs from Audio-Alpaca. If we want to scale up training data from unlabeled wild audio, how can we construct such pairs automatically?
  2. In section 2.3, every candidate in a group of GG shares the same role ii, and the penalty in Eq. (6) depends only on ii (since the votes all come from one fixed clue set ZZ). Therefore, it is constant across the group and cancels out in Eq. (8) with Uj=Rutilityi(zi,jr)U_j=R^i_{\mathrm{utility}}(z^r_{i,j}), meaning the vote term does not affect the gradient. Could you clarify whether viv_i is in fact calculated on a per-candidate-clue basis?
Questions for authors

Please refer to the issue mentioned in Weaknesses.

Reviewer_SXqdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Audio-Zero, a self-play method in reinforcement learning to improve the reasoning capability of the audio language model. By comparing multiple rollouts, the model is designed to tell the audio clip with nuanced differences and then learn the detailed capability for audio understanding.

Critical strengths

(1) The self-play method is a novel one: by providing a similar pair of audio clips, the model automatically learns how to find the nuances within the audio. The method looks like a scalable approach even though the exact experiments are of limited size. (2) The proposed method also shows clear improvement on those audio understanding benchmarks. (3) The paper also provides a good ablation study to explain how the game is designed and the impact of each component in the design.

Critical weaknesses

(1) The model is trained on 2k examples of Audio-Alpaca, which are synthesized audio clips rather than natural audio, so the concern is whether the method is still applicable to natural audio. (2) I would recommend that the authors add a dedicated paragraph to compare this work with prior Vision-zero work to explain the major methodological difference, besides the modality (vision vs. audio). The paper presents a good method with good results. But if it heavily overlaps with prior work, we should reconsider its value.

Questions for authors

(1) I'm curious if we can make the whole process more scalable, especially about how to curate the training data. The current 2k samples are synthetic, but can we use a simple method like "two consecutive audio clips" or "two clips with very high CLAP value"? (2) To make the taks to be harder, can we generalize the formulation from "distinguishing N different samples among M samples", where current setup is N=1.

Reviewer_WxtROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Analogous to Vision-Zero, Audio-Zero trains audio-language models through a social-deduction game. Three players hear the same recording and one hears a different recording. They give clues over two rounds, and a judge identifies the odd listener. The known odd-player assignment supplies a reward for judging. Clue generation gets a hand-designed content reward and a penalty based on the judge's votes. GRPO alternates between training the two roles. The paper reports gains on TREA, MMAU test-mini, and MMAR for Qwen2-Audio-7B-Instruct and Qwen2.5-Omni-7B. The Qwen2.5-Omni gains are 4.00, 8.00, and 10.00 percentage points, respectively.

Critical strengths
  1. The odd-player assignment gives a known target without needing a new QA label for every training example. Both models improve on all three benchmark averages, including TREA's order, count, and duration tasks. The MMAR gains of 7.9 and 10.0 points are substantial and make the method worth studying.

  2. Table 2 includes listening-only, judging-only, sequential-training, and no-vote-feedback versions. The full method is strongest across the reported model and benchmark combinations. This gives useful evidence that the stages work together. TREA also directly tests the fine-grained temporal skills that motivate the method.

Critical weaknesses
  1. I cannot tell whether the game is worth the extra training work. Vision-Zero already uses much of this setup, and I would like a clearer account of what needed to change for audio. More directly, the paper does not explain the task, data, or reward used for "Vanilla GRPO." Its averages match R1-AQA exactly on both models, so I could not tell whether this was a separate experiment. A simpler task using the same audio pairs would be a useful comparison. The authors also do not report the hardware, batch size, or GPU-hours. Figure 3 shows progress over iterations, but gives no comparison at equal compute. On Qwen2.5-Omni, the gains over Audio-Thinker are 1.7 points on MMAU and 2.5 on MMAR. The training cost matters when judging those gains.

  2. I am less convinced by the claim that the clues become more accurate. The reward encourages words about events, timing, quantity, and sound properties (Section 2.3; Appendix A.1.3). Figure 4 then counts similar words. A clue can use these words and still describe the recording incorrectly. The odd listener is also rewarded for avoiding detection; in Table 3, it leaves out the event order that would give it away. There is a possible shortcut in the data as well. The chosen Audio-Alpaca clip always serves as the reference, and the rejected clip as the variant. These clips can differ in recording quality. Could that difference help identify the odd listener? I would want a check of clue accuracy and an experiment that controls for recording quality before accepting the explanation for why the method works. Audio-Alpaca construction in Tango 2. (https://arxiv.org/html/2404.09956v1)

Questions for authors
  1. What is "Vanilla GRPO" trained to do, and what reward does it use? Its scores match R1-AQA exactly. Is this a separate experiment? How would a simpler task that compares the recordings perform with the same pairs and compute?

  2. How much training compute does Audio-Zero use compared with the baselines? I could not find the hardware, batch size, or total GPU-hours. What does one iteration include? Would the gains hold if a simpler baseline received the same compute budget?

  3. Have you tried randomly swapping which clip is the reference, or otherwise controlling for differences in recording quality? I also wondered whether you checked the events, counts, and timing in the clues against the recordings. The word counts in Figure 4 do not tell me whether those details are correct.

  4. Are the judge's votes recomputed for each candidate clue? Section 2.4 says the decisions are held fixed during a listening update. If every candidate gets the same penalty, wouldn't it cancel when GRPO subtracts the group mean? A small example would help me follow this and understand why V=G×N.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.