- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
MEL is a pipeline for audio QA. The model indexes the recording, lists what the question needs, selects up to three segments, and answers using the cropped segments plus the full audio. Indexing and planning are LoRA-tuned on 1,247 teacher-generated demonstrations, and the answerer keeps its original weights. On MMAR and MMAU with Qwen2.5-Omni-7B and Qwen3-Omni-30B, MEL improves over direct answering, with ablations and a correction/regression analysis.
Critical strengths
Re-listening to question-relevant segments while keeping the full recording is simple, and it improves over Direct in all four settings, most clearly on MMAU. The SFT-Direct control shows that fine-tuning alone does not help, so the gain comes from the re-listening step.
Critical weaknesses
(1) The paper’s main contribution is not tested. Sections 3.2–3.3 center on a decision model that generates an applicability explanation z and is trained with Eq. 3. However, Appendix B.2 says the answerer uses the original weights, and Table 3 contains only indexing and planning data. As a result, z is never generated, and the “acoustic tools” reduce to audio cropping. What is evaluated is segment selection plus re-listening, close to PlanRAG-Audio (Someki et al., 2026) and Audio-Mind (Wang et al., 2026).
(2) The gains are small and unverified. On MMAR, MEL leads the best baseline by 0.3 and 0.5 points, from a single run. McNemar’s test on the authors’ own Figure 3 counts gives p ≈ 0.32 (30B) and p ≈ 0.06 (7B) for MEL vs. w/o evidence. For 30B MMAR, removing decomposition or SFT scores higher than the full method.
References
Masao Someki, Chien-yu Huang, Siddhant Arora, Samuele Cornell, Markus Müller, Nathan Susanj, Rupak Vignesh Swaminathan, Grant Strimel, Jing Liu, and Shinji Watanabe. PlanRAG-audio: Planning and retrieval augmented generation for long-form audio understanding. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 26167–26183, 2026.
Yucheng Wang, Jing Peng, Hanqi Li, Chenghao Wang, Wenming Tu, Yu Xi, Zhaokai Sun, Kai Yu, and Shuai Wang. Audio-Mind: An auditable agentic framework for audio understanding. arXiv:2605.28480, 2026.
Quinn McNemar. Note on the sampling distribution of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, 1947.
Questions for authors
(1) Is z generated or trained in any reported experiment? If not, Sections 3.2–3.3 need revision or new results.
(2) Does a control with random segments of equal total duration perform similarly? This would show whether question-guided selection matters or whether extra audio alone explains the gain.
(3) Can you add significance tests and multiple seeds?
2 comments
Response Part 2/2
3. Can the gains be explained by additional listening rather than question-guided localization?
Following your suggestion, we fix the number of segments per question, the actual frame count of each segment, cumulative playback duration, the answerer, and input order, changing only segment positions. The complete controls for the original segment-based experiments are:
| Position configuration | 7B MMAR | 7B MMAU | 30B MMAR | 30B MMAU |
|---|---|---|---|---|
| Direct from the same evaluation batch | 55.0 | 67.4 | 62.5 | 70.9 |
| Question-guided segments | 57.1 | 72.8 | 64.0 | 73.8 |
| Uniform segments | 56.8 | 71.3 | 66.3 | 73.3 |
| Random positions, seed17 | 56.7 | 70.4 | 63.6 | 72.9 |
| Random positions, seed42 | 56.6 | 70.7 | 63.9 | 73.3 |
| Random positions, seed73 | 55.8 | 70.2 | 64.3 | 72.7 |
Question-guided selection outperforms these positional controls on MMAU. Its advantage is less consistent on MMAR, where uniform segments perform better for 30B. Thus, additional listening accounts for part of the gain, while the added value of question-guided localization depends on the task.
4. What statistical evidence do the independent training runs provide?
To assess training stability, we added seeds 17 and 73 and trained the indexing/segment-selection model through the full procedure for each seed, alongside the original seed42 run:
| Original segment-based method | Direct from the same evaluation batch | Mean ± standard deviation across three training seeds |
|---|---|---|
| 7B MMAR | 55.0 | 57.37 ± 0.31 |
| 7B MMAU | 67.4 | 72.20 ± 0.53 |
| 30B MMAR | 62.5 | 64.00 ± 0.90 |
| 30B MMAU | 70.9 | 73.77 ± 0.25 |
All four settings outperform their respective Direct baselines under each of the three training seeds.
The new complete tool-report method also includes three answerer training seeds, evaluated using shared reports:
| Setting | SFT without reports, mean ± standard deviation | Full MEL, mean ± standard deviation | Mean report gain (percentage points) |
|---|---|---|---|
| 7B MMSU | 66.21 ± 0.08 | 70.08 ± 0.22 | +3.87 |
| 7B AudioMarathon | 75.87 ± 0.35 | 76.39 ± 0.25 | +0.52 |
| 30B MMSU | 72.59 ± 0.10 | 76.63 ± 0.16 | +4.04 |
| 30B AudioMarathon | 70.33 ± 0.16 | 71.46 ± 0.14 | +1.14 |
For every training seed, providing reports outperforms withholding them from the same trained model. Training variability is reported separately through the standard deviation. Gains are computed from unrounded per-question results, and the three seeds share the same evaluation questions.
In summary, we will present the gains and limitations of the original segment-based evidence-acquisition method separately from the new tool-report learning experiments. The actual training configurations, matched playback controls, and multi-seed statistics jointly support the revised method. The new results support the contribution of reports to final decisions, while explicit explanation supervision is treated as a separate analysis.
Response Part 1/2
We thank the reviewer for the specific analysis of the learning objective, positional controls, and statistical reliability. We have verified the actual training configurations, completed duration-matched uniform/random segment controls and independent training runs, and added full tool-report and answerer-learning experiments on MMSU and AudioMarathon. We address each concern below.
1. How will Sections 3.2-3.3 be aligned with the actual learning objective and implementation?
The original description of explanation z and the actual supervision targets was not sufficiently precise. In the MMAR/MMAU segment-based method, the 1,247 demonstrations and 78 LoRA training steps are used for indexing and planning/segment selection, with the corresponding demonstrations as supervision; the final answerer retains its original weights. We will retain these as supplementary segment-based evidence-acquisition experiments in the appendix and organize the main text around the new complete tool-report experiments.
The new main experiments follow this workflow: a fixed 7B planner identifies information requirements, objects, and scopes; specialized tools perform measurements; the program preserves the actual observations and adds comparative relations; and an SFT-trained 7B or 30B answerer combines them with the original audio to answer. During training, the planner and tools remain fixed. The inputs are the original audio, question, and reports under different conditions, and supervision covers only the final answer y and end markers.
The revised main objective is report-conditioned answer learning:
Lans=−E[∑tlogpθ(yt∣x,q,r,y<t)].
Explanation z primarily supports interpretability analysis. We have separately completed supervision controls using Ming-generated and rule-based explanations, which will be presented as appendix analyses.
This implementation shares the motivation of active evidence acquisition with PlanRAG-Audio and Audio-Mind, but focuses on a different aspect. PlanRAG-Audio primarily plans queries to retrieve relevant information from a structured database of long audio. Audio-Mind emphasizes deciding whether to invoke tools based on evidence gaps while retaining the audio frontend's judgment. MEL further binds information requirements to explicit attributes, objects, and scopes, programmatically organizes verifiable measurement relations, and trains the answerer under report-conditioned inputs. The new experiments examine how measurements and their expression support the final decision.
2. What do the new full-set results and module ablations show?
We completed the following comparisons on 5,000 MMSU questions and 6,364 AudioMarathon questions, using the same audio input order and evaluation protocol while retaining each method's own prompts and invocation budgets. Values are accuracy (%).
The 7B and 30B backbones are Qwen2.5-Omni-7B and Qwen3-Omni-30B-A3B-Instruct, respectively. Since we could not obtain publicly released author code for EChO-Agent, we reproduced it according to the paper.
Both model scales obtain net gains from tool reports on both datasets, and SFT further improves results on both 7B datasets and on 30B MMSU. The configuration without SFT remains slightly better on 30B AudioMarathon; we will retain and discuss this finding. The numerical gains from decomposition are modest, with positive changes for both backbones on MMSU and approximately neutral results for 30B on AudioMarathon. Decomposition also preserves inspectable information requirements and tool bindings, allowing researchers to trace attribute mismatches, mixed objects and scopes, and insufficient measurement granularity. This provides interpretability beyond accuracy.