- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Audio-Zero, a label-free self-evolution framework designed to enhance fine-grained audio perception and reasoning in large audio language models. The approach builds a multi-player self-play game based on audio contrast pairs, in which the model produces auditory clues and locates the odd listener who hears a different recording. The two stages are optimized alternately with GRPO using rule-based, verifiable rewards. Experiments with Qwen2-Audio-7B-Instruct and Qwen2.5-Omni-7B show gains on TREA, MMAU Test-mini, and MMAR while using only 2,000 audio contrast pairs.
Critical strengths
The paper presents an interesting self-play mechanism that turns audio contrast pairs into verifiable training signals, eliminating the need for explicit audio annotations or QA labels. This offers a promising direction for lowering annotation costs in fine-grained audio reasoning. And it delivers notable improvements on TREA, MMAU Test-mini, and MMAR across two different LALM backbones, using only 2,000 audio contrast pairs. Ablation studies further corroborate the effectiveness of the proposed dual-stage optimization framework.
Critical weaknesses
- The proposed method requires audio contrast pairs from Audio-Alpaca. If we want to scale up training data from unlabeled wild audio, how can we construct such pairs automatically?
- In section 2.3, every candidate in a group of shares the same role , and the penalty in Eq. (6) depends only on (since the votes all come from one fixed clue set ). Therefore, it is constant across the group and cancels out in Eq. (8) with , meaning the vote term does not affect the gradient. Could you clarify whether is in fact calculated on a per-candidate-clue basis?
Questions for authors
Please refer to the issue mentioned in Weaknesses.