acceptodds
Withdrawn from ICLR 2027

From Probes to Decisions: Multimodal Evidence Learning for Trustworthy Audio Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Audio-language models can understand sound events and context in complex recordings, but being able to describe a recording does not imply having the evidence needed to answer a specific question. Question-guided acoustic measurements can provide targeted observations; however, even accurate observations may be insufficient to support the judgment at hand. We propose Multimodal Evidence Learning (MEL), a framework for trustworthy audio reasoning that connects question-relevant acoustic evidence acquisition with task-oriented evidence use. MEL decomposes a question's decision requirements into textual probes, guides acoustic tools to obtain observations with source context and scope, and combines these observations with the original audio and question for task decisions. The framework emphasizes what observations support and where their inferential limits lie, allowing local evidence to complement holistic audio understanding rather than be equated with task conclusions. We incorporate supervised learning into this process to strengthen the connection between a question's information needs and evidence acquisition, allowing local acoustic observations and holistic audio understanding to jointly inform task decisions. Experiments on MMAR and MMAU with two audio-language backbones of different sizes show that MEL improves overall accuracy over direct answering. Ablations and per-question analyses further reveal the contributions of evidence acquisition and learning, as well as how answer corrections and regressions jointly determine the net gains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio-Language Models, Audio Reasoning, Multimodal Evidence Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_1VTgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MEL is a pipeline for audio QA. The model indexes the recording, lists what the question needs, selects up to three segments, and answers using the cropped segments plus the full audio. Indexing and planning are LoRA-tuned on 1,247 teacher-generated demonstrations, and the answerer keeps its original weights. On MMAR and MMAU with Qwen2.5-Omni-7B and Qwen3-Omni-30B, MEL improves over direct answering, with ablations and a correction/regression analysis.

Critical strengths

Re-listening to question-relevant segments while keeping the full recording is simple, and it improves over Direct in all four settings, most clearly on MMAU. The SFT-Direct control shows that fine-tuning alone does not help, so the gain comes from the re-listening step.

Critical weaknesses

(1) The paper’s main contribution is not tested. Sections 3.2–3.3 center on a decision model that generates an applicability explanation z and is trained with Eq. 3. However, Appendix B.2 says the answerer uses the original weights, and Table 3 contains only indexing and planning data. As a result, z is never generated, and the “acoustic tools” reduce to audio cropping. What is evaluated is segment selection plus re-listening, close to PlanRAG-Audio (Someki et al., 2026) and Audio-Mind (Wang et al., 2026).

(2) The gains are small and unverified. On MMAR, MEL leads the best baseline by 0.3 and 0.5 points, from a single run. McNemar’s test on the authors’ own Figure 3 counts gives p ≈ 0.32 (30B) and p ≈ 0.06 (7B) for MEL vs. w/o evidence. For 30B MMAR, removing decomposition or SFT scores higher than the full method.

References

Masao Someki, Chien-yu Huang, Siddhant Arora, Samuele Cornell, Markus Müller, Nathan Susanj, Rupak Vignesh Swaminathan, Grant Strimel, Jing Liu, and Shinji Watanabe. PlanRAG-audio: Planning and retrieval augmented generation for long-form audio understanding. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 26167–26183, 2026.

Yucheng Wang, Jing Peng, Hanqi Li, Chenghao Wang, Wenming Tu, Yu Xi, Zhaokai Sun, Kai Yu, and Shuai Wang. Audio-Mind: An auditable agentic framework for audio understanding. arXiv:2605.28480, 2026.

Quinn McNemar. Note on the sampling distribution of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, 1947.

Questions for authors

(1) Is z generated or trained in any reported experiment? If not, Sections 3.2–3.3 need revision or new results.

(2) Does a control with random segments of equal total duration perform similarly? This would show whether question-guided selection matters or whether extra audio alone explains the gain.

(3) Can you add significance tests and multiple seeds?

2 comments
AuthorsOpenReview ↗

Response Part 1/2

We thank the reviewer for the specific analysis of the learning objective, positional controls, and statistical reliability. We have verified the actual training configurations, completed duration-matched uniform/random segment controls and independent training runs, and added full tool-report and answerer-learning experiments on MMSU and AudioMarathon. We address each concern below.

1. How will Sections 3.2-3.3 be aligned with the actual learning objective and implementation?

The original description of explanation z and the actual supervision targets was not sufficiently precise. In the MMAR/MMAU segment-based method, the 1,247 demonstrations and 78 LoRA training steps are used for indexing and planning/segment selection, with the corresponding demonstrations as supervision; the final answerer retains its original weights. We will retain these as supplementary segment-based evidence-acquisition experiments in the appendix and organize the main text around the new complete tool-report experiments.

The new main experiments follow this workflow: a fixed 7B planner identifies information requirements, objects, and scopes; specialized tools perform measurements; the program preserves the actual observations and adds comparative relations; and an SFT-trained 7B or 30B answerer combines them with the original audio to answer. During training, the planner and tools remain fixed. The inputs are the original audio, question, and reports under different conditions, and supervision covers only the final answer y and end markers.

The revised main objective is report-conditioned answer learning:

Lans=−E[∑tlog⁡pθ(yt∣x,q,r,y<t)].\mathcal L_{\mathrm{ans}}=-\mathbb E\left[\sum_t\log p_\theta(y_t\mid x,q,r,y_{ < t})\right].

Explanation zz primarily supports interpretability analysis. We have separately completed supervision controls using Ming-generated and rule-based explanations, which will be presented as appendix analyses.

This implementation shares the motivation of active evidence acquisition with PlanRAG-Audio and Audio-Mind, but focuses on a different aspect. PlanRAG-Audio primarily plans queries to retrieve relevant information from a structured database of long audio. Audio-Mind emphasizes deciding whether to invoke tools based on evidence gaps while retaining the audio frontend's judgment. MEL further binds information requirements to explicit attributes, objects, and scopes, programmatically organizes verifiable measurement relations, and trains the answerer under report-conditioned inputs. The new experiments examine how measurements and their expression support the final decision.

2. What do the new full-set results and module ablations show?

We completed the following comparisons on 5,000 MMSU questions and 6,364 AudioMarathon questions, using the same audio input order and evaluation protocol while retaining each method's own prompts and invocation budgets. Values are accuracy (%).

Method7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Direct64.7073.9572.1869.80
Free-CoT63.9868.7071.3062.63
MATCH64.8871.3971.3268.48
MATA61.7460.6468.9065.32
Self-Refine59.3266.3972.1867.49
TwS57.2464.7565.9067.96
EChO-Agent (reproduced from the paper)63.5868.2965.4864.97
AGR64.0456.0268.9262.41
MEL70.1276.2676.7871.50

The 7B and 30B backbones are Qwen2.5-Omni-7B and Qwen3-Omni-30B-A3B-Instruct, respectively. Since we could not obtain publicly released author code for EChO-Agent, we reproduced it according to the paper.

Ablation7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Without decomposition: select tools directly from the original question69.4476.0476.4871.51
Without reports: the same SFT answerer receives no report66.2075.7772.4870.40
Without SFT: the original answerer receives the same report67.8074.3976.3471.92
Full method70.1276.2676.7871.50

Both model scales obtain net gains from tool reports on both datasets, and SFT further improves results on both 7B datasets and on 30B MMSU. The configuration without SFT remains slightly better on 30B AudioMarathon; we will retain and discuss this finding. The numerical gains from decomposition are modest, with positive changes for both backbones on MMSU and approximately neutral results for 30B on AudioMarathon. Decomposition also preserves inspectable information requirements and tool bindings, allowing researchers to trace attribute mismatches, mixed objects and scopes, and insufficient measurement granularity. This provides interpretability beyond accuracy.


AuthorsOpenReview ↗

Response Part 2/2

3. Can the gains be explained by additional listening rather than question-guided localization?

Following your suggestion, we fix the number of segments per question, the actual frame count of each segment, cumulative playback duration, the answerer, and input order, changing only segment positions. The complete controls for the original segment-based experiments are:

Position configuration7B MMAR7B MMAU30B MMAR30B MMAU
Direct from the same evaluation batch55.067.462.570.9
Question-guided segments57.172.864.073.8
Uniform segments56.871.366.373.3
Random positions, seed1756.770.463.672.9
Random positions, seed4256.670.763.973.3
Random positions, seed7355.870.264.372.7

Question-guided selection outperforms these positional controls on MMAU. Its advantage is less consistent on MMAR, where uniform segments perform better for 30B. Thus, additional listening accounts for part of the gain, while the added value of question-guided localization depends on the task.

4. What statistical evidence do the independent training runs provide?

To assess training stability, we added seeds 17 and 73 and trained the indexing/segment-selection model through the full procedure for each seed, alongside the original seed42 run:

Original segment-based methodDirect from the same evaluation batchMean ± standard deviation across three training seeds
7B MMAR55.057.37 ± 0.31
7B MMAU67.472.20 ± 0.53
30B MMAR62.564.00 ± 0.90
30B MMAU70.973.77 ± 0.25

All four settings outperform their respective Direct baselines under each of the three training seeds.

The new complete tool-report method also includes three answerer training seeds, evaluated using shared reports:

SettingSFT without reports, mean ± standard deviationFull MEL, mean ± standard deviationMean report gain (percentage points)
7B MMSU66.21 ± 0.0870.08 ± 0.22+3.87
7B AudioMarathon75.87 ± 0.3576.39 ± 0.25+0.52
30B MMSU72.59 ± 0.1076.63 ± 0.16+4.04
30B AudioMarathon70.33 ± 0.1671.46 ± 0.14+1.14

For every training seed, providing reports outperforms withholding them from the same trained model. Training variability is reported separately through the standard deviation. Gains are computed from unrounded per-question results, and the three seeds share the same evaluation questions.

In summary, we will present the gains and limitations of the original segment-based evidence-acquisition method separately from the new tool-report learning experiments. The actual training configurations, matched playback controls, and multi-seed statistics jointly support the revised method. The new results support the contribution of reports to final decisions, while explicit explanation supervision is treated as a separate analysis.

Reviewer_UKJ5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, MEL index an audio to break the Q in various needed information "audio segments". These segments, as well as the full recording, are supplied to an audio language model to generate the answer. The authors demo superior performance on answers generation on the MMAR and MMAU sets.

Critical strengths
  • Its an interesting idea to let the model select audio segments for listening closely to the parts of the audio segments relevant to the Q.
  • The training seems lightweight, authors use only 78 LORA training steps.
Critical weaknesses
  • The main description of the paper does not match the evaluation, as the main method includes generating and training an evidence explanation. But in the appendix, it lists only indexing & planning demos, they also say that the final answer retains the original weights.
  • The eval for explanation quality or trustworthiness is missing. The evidence is only context expanded audio segments.
  • Eval scores: performance improvement over the best baseline is low on MMAR. Additionally, ablation for no decomposition and no SFT is better than the main model for the 30B version. Please explain these details.
  • The baseline should also include uniform crops or random crops to establish that the question guided segment selection is needed. How are the identified overlaps and source linked matches obtained for MMAR/ MMAR?
Questions for authors
  • Please align the method details with the implementations (see weakness above)
  • Multiple seed results are important to compare MEL performance, additionally, please include MEL scores with random/unform crops of similar duration.
  • How do you check the training audio overlap with MMAR and MMAU (see weakness above)
3 comments
AuthorsOpenReview ↗

Response Part 1/3

We thank the reviewer for the specific suggestions on implementation alignment, duration-matched segments, multiple seeds, and source checks. We have completed these controls and added full tool-report and answerer-training experiments on MMSU and AudioMarathon. The revision will distinguish the two implementations. We address each concern below.

1. How will the method description be aligned with the actual training?

The original text did not clearly distinguish the two roles of training, particularly in its description of explanation z and answer supervision. We will move the original MMAR/MMAU segment-based experiments to the appendix as supplementary experiments and use the new complete tool-report experiments as the main empirical evaluation. Sections 3.2-3.3, the method figure, and Appendix B.2 will be updated according to the following configurations:

ExperimentInputs and training targetsActual supervisionFinal answering
MMAR/MMAU segment-based evidence acquisition944 blind-listening index demonstrations and 303 planning demonstrations; indexing/segment-selection model trained for 78 stepsIndexing and evidence-acquisition plansThe original answerer combines the segments with the original audio
MMSU/AudioMarathon tool reportsFixed 7B planner and tools; the answerer receives the original audio, question, and different report conditionsFinal answer y and end markersShort answers from the SFT-trained 7B or 30B answerer

The new answerer first undergoes 60 warm-up steps on 960 training rows, followed by two passes over 4,540 distinct input conditions, totaling 9,088 rows and 568 answer-training steps. The input conditions include no reports, matching reports, scope-mismatched reports, and mixed reports.

Thus, the learning objective in the new main experiments is report-conditioned answer prediction, while explanation z is retained for interpretability analysis. We have separately completed explicit explanation-supervision experiments, which will be analyzed in the appendix.

2. What do the new full-set results and module ablations show?

The new evidence is no longer limited to audio crops; it includes transcripts, event observations, speech activity, pitch, and energy measurements. The workflow explicitly preserves the correspondence between requirements, tools, objects and scopes, and actual observations, and then provides computable comparative relations to the answerer.

The main results on the two additional full evaluation sets are shown below. Values are accuracy (%); the 7B and 30B backbones are Qwen2.5-Omni-7B and Qwen3-Omni-30B-A3B-Instruct, respectively.

Method7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Direct64.7073.9572.1869.80
Free-CoT63.9868.7071.3062.63
MATCH64.8871.3971.3268.48
MATA61.7460.6468.9065.32
Self-Refine59.3266.3972.1867.49
TwS57.2464.7565.9067.96
EChO-Agent63.5868.2965.4864.97
AGR64.0456.0268.9262.41
MEL70.1276.2676.7871.50

Within each dataset, all methods use the same audio input order, while retaining their own prompts and invocation budgets. MMSU contains 5,000 questions and AudioMarathon contains 6,364 questions. Since we could not obtain publicly released author code for EChO-Agent, we reproduced it according to the paper. All configurations with four completed evaluations are assessed under a unified protocol.

Ablation7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Without decomposition: select tools directly from the original question69.4476.0476.4871.51
Without reports: the same SFT answerer receives no report66.2075.7772.4870.40
Without SFT: the original answerer receives the same report67.8074.3976.3471.92
Full method70.1276.2676.7871.50

The full method outperforms these ablations on both 7B datasets and on 30B MMSU. On 30B AudioMarathon, decomposition is approximately neutral, and the configuration without SFT remains slightly better. We therefore discuss component contributions separately by task and backbone. Another practical contribution of decomposition is the inspectable record of information requirements. These records allow us to identify errors such as measuring over a mixed scope when the question requires comparing different objects, or returning only a recording-wide average when the question concerns temporal changes. Its accuracy contribution is modest, but it improves the interpretability of evidence acquisition.

AuthorsOpenReview ↗

Response Part 2/3

3. Do question-guided segments outperform duration-matched uniform and random segments?

We completed the requested positional controls for the original segment-based method. For each question, we fix the number of segments, the actual frame count of each segment, cumulative playback duration, the answerer, and input order, changing only segment positions:

Position configuration7B MMAR7B MMAU30B MMAR30B MMAU
Direct from the same evaluation batch55.067.462.570.9
Question-guided segments57.172.864.073.8
Uniform segments56.871.366.373.3
Random positions, seed1756.770.463.672.9
Random positions, seed4256.670.763.973.3
Random positions, seed7355.870.264.372.7

Each column contains 1,000 questions, and all random seeds are retained. Question-guided selection performs better on MMAU: 7B exceeds uniform positions by 1.5 percentage points and random positions by 2.1-2.6 percentage points; 30B also slightly outperforms the uniform and random controls. The positional advantage is weaker on MMAR, where uniform segments outperform question-guided selection for 30B.

These results indicate that re-listening itself accounts for part of the gain, while the additional value of requirement-guided localization depends on the task. We will include this conclusion and the positional controls in the appendix experiments for the original segment-based method. These experiments retain the original question-first protocol and are evaluated separately from the audio-first protocol used in the new tool-report experiments.

4. Are the gains consistent across independent training seeds?

For the original segment-based method, we added training seeds 17 and 73, using the same demonstrations, 78 training steps, and fixed answerer as the original seed42 run. The table below summarizes three independent fine-tuning runs.

Original segment-based methodDirect from the same evaluation batchMean ± standard deviation across three training seeds
7B MMAR55.057.37 ± 0.31
7B MMAU67.472.20 ± 0.53
30B MMAR62.564.00 ± 0.90
30B MMAU70.973.77 ± 0.25

All four settings outperform their respective Direct baselines under each of the three seeds. The new main tool-report experiments also include separate answerer training runs with seeds 17, 42, and 73, evaluated using shared reports:

SettingSFT without reports, mean ± standard deviationFull MEL, mean ± standard deviationMean report gain (percentage points)
7B MMSU66.21 ± 0.0870.08 ± 0.22+3.87
7B AudioMarathon75.87 ± 0.3576.39 ± 0.25+0.52
30B MMSU72.59 ± 0.1076.63 ± 0.16+4.04
30B AudioMarathon70.33 ± 0.1671.46 ± 0.14+1.14

For every training seed, providing reports outperforms withholding them from the same trained model.

AuthorsOpenReview ↗

Response Part 3/3

5. What do the analyses show about explanations and evidence support?

We added per-question analyses of answer changes with and without reports, together with checks of the actual measurements. With seed42, providing reports to the same answerer corrects 256, 100, 273, and 153 previously incorrect answers on 7B MMSU, 7B AudioMarathon, 30B MMSU, and 30B AudioMarathon, respectively, while changing 60, 69, 58, and 83 correct answers to incorrect ones. All four settings show a net gain.

We also examine the relationship between observations, scopes, and conclusions. For example, RMS values from three localized intervals can support their relative amplitude ordering within a recording, and successful cases have also been checked by recomputing the measurements from the waveform. In contrast, observations concerning only some objects cannot support a conclusion shared by all objects. The audit of a fixed set of 64 natural questions and these cases will be included in the appendix, distinguishing actual measurements, proxy cues, and conclusions that remain insufficiently supported.

6. How are training sources separated from the evaluation sets?

We verify and describe exact file duplication and known source associations separately. Preparation of the original segment-based training data excludes exact matches using the SHA-256 hashes of public evaluation audio, and uses known parent sources for training deduplication and separation from internal validation data.

For the new answerer training, we rechecked 1,046 training audio files and traced the original files and parent sources of 480 constituent ESC clips. TRAIN and DEV contain 611 and 240 parent-source IDs, respectively, with no overlap. The hashes of the training media and registered constituent media do not overlap with those of the available evaluation media in MMAR, MMAU, MMSU, or AudioMarathon. This checks both the actual training files and the original components of synthesized audio, rather than only the newly concatenated files. The appendix will separately document the source-processing procedures and corresponding records for the original segment-based experiments and the new answerer experiments.

Reviewer_qq5rOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies audio reasoning and proposes a multimodal evidence learning method for trustworthy audio reasoning. The proposed method decomposes a question’s decision requirements into textual probes, guides acoustic tools to obtain observations with source context and scope, and combines these observations with the original audio and question for task decisions. This paper also incorporates supervised learning to strengthen the reasoning capability of the proposed method. The performance on MMAR and MMAU shows improvement over baselines.

Critical strengths
  1. The research problem is important.
  2. The writing is easy to follow.
  3. The experiment is comprehensive, while some important work is missing.
Critical weaknesses
  1. My major concern is that the proposed pipeline largely combines established techniques, including question decomposition, audio segment selection, and evidence-augmented reasoning. The paper does not clearly establish a novel technical mechanism beyond existing tool-augmented audio reasoning approaches such as Audio-Mind and EChO-Agent.
  2. Important audio reasoning methods are missing, e.g., Audio-DeepThinker, which outperforms the proposed method by a large margin, especially on MMAR, and was released on 20 Apr 2026.
  3. The experiments mainly evaluate QA accuracy. They do not directly measure whether MEL improves evidence sufficiency assessment, prevents unsupported conclusions, or distinguishes local observations from globally justified judgments.
  4. Removing decomposition or SFT improves accuracy for the 30B backbone on MMAR, raising questions about the necessity and generalizability of these components.

References:

  1. He, Xiang, Chenxing Li, Jinting Wang, Yan Rong, Tianxin Xie, Wenfu Wang, Li Liu, and Dong Yu. "Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models." arXiv preprint arXiv:2604.18187 (2026).
  2. Wang, Yucheng, Jing Peng, Hanqi Li, Chenghao Wang, Wenming Tu, Yu Xi, Zhaokai Sun, Kai Yu, and Shuai Wang. "Audio-Mind: An Auditable Agentic Framework for Audio Understanding." arXiv preprint arXiv:2605.28480 (2026).
  3. Zhang, Siyuan, Jian Zong, Junyu Wang, Peiyuan Jiang, Jiahao Yan, Jingyu Zhang, Tianrui Wang, Xiaobao Wang, Longbiao Wang, and Jianwu Dang. "EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning." arXiv preprint arXiv:2606.15141 (2026).
Questions for authors

Section 3.3 formulates evidence applicability learning as supervised generation of evidence-support explanations and task answers. However, Appendix B.2 indicates that the adapted weights are used only for indexing and evidence acquisition planning, while the final decision model retains its original weights. It is therefore unclear how the proposed evidence applicability learning objective is actually implemented and validated. The authors should explain this discrepancy.

3 comments
AuthorsOpenReview ↗

Response Part 1/3

We thank the reviewer for the specific suggestions regarding the technical contribution, evidence support, and training implementation. We have added full tool-report experiments, module ablations, multiple training seeds, and per-question evidence analyses on MMSU and AudioMarathon. We will update the method and experimental sections accordingly and address each concern below.

1. What is MEL's technical contribution relative to Audio-Mind and EChO-Agent?

MEL focuses not simply on acquiring more information, but on translating the information required by a question into executable, inspectable measurements and training the answerer to use them. The complete workflow in the new experiments is as follows: an audio model identifies information requirements and the relevant objects and scopes; tools perform the corresponding measurements; the program preserves the raw observations and organizes verifiable relations; and an SFT-trained answerer combines the original audio with the reports to make a decision.

Audio-Mind emphasizes using a text planner to decide whether to acquire additional evidence or re-listen, while retaining the audio frontend's original judgment. EChO-Agent selects predefined tool combinations by question type, uses DeepSeek-V3 to filter and integrate observations, and then combines audio-model answering with text-side verification. MEL uses a locally deployed audio backbone to generate information requirements, connects them to tools through explicit attributes, objects, and scopes, and programmatically constructs comparative relations that can be independently recomputed. At test time, it does not rely on an external text model to write or integrate evidence. The two answerer backbones share a fixed 7B planner and the same reports; no stronger teacher provides plans for test questions.

This design consists of three interconnected components:

  1. Inspectable information requirements. Questions are translated into specifications of which attribute to measure, which object to examine, and which scope to cover, making the rationale for tool selection explicit. For example, comparing sound intensity across three stages requires separate measurements for the three intervals, rather than only the average energy of the entire recording.
  2. Measurement relations with provenance and scope. We retain the actual tool outputs, temporal scopes, and limitations, and then programmatically compute relations such as rankings and differences. This distinguishes measurement results from model-generated explanations.
  3. Report-conditioned answerer learning. The original audio remains available throughout. Answer supervision under no-report, matching-report, scope-mismatched-report, and mixed-report conditions trains the model to answer under different evidence conditions.

We have completed the following comparisons on the two additional datasets. The 7B and 30B backbones are Qwen2.5-Omni-7B and Qwen3-Omni-30B-A3B-Instruct, respectively. All values are accuracy (%).

Method7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Direct64.7073.9572.1869.80
Free-CoT63.9868.7071.3062.63
MATCH64.8871.3971.3268.48
MATA61.7460.6468.9065.32
Self-Refine59.3266.3972.1867.49
TwS57.2464.7565.9067.96
EChO-Agent63.5868.2965.4864.97
AGR64.0456.0268.9262.41
MEL70.1276.2676.7871.50

MMSU contains 5,000 questions and AudioMarathon contains 6,364 questions. All methods use the same audio input order and evaluation protocol, while retaining their method-specific prompts and invocation budgets. Since we could not obtain publicly released author code for EChO-Agent, we reproduced it according to the paper. The table includes configurations for which all four evaluations are complete.

The full method outperforms Direct and all completed comparison methods in the table across these four settings. The module ablations and evidence analyses below further examine which design choices contribute to these gains.


AuthorsOpenReview ↗

Response Part 2/3

2. How does MEL relate to Audio-DeepThinker, and how are the comparison conditions handled?

We will add Audio-DeepThinker to the related work. It strengthens the model's internal chain-of-thought reasoning through two-stage reinforcement learning and reasoning-quality rewards. MEL instead focuses on aligning explicit measurements with information requirements and enabling the answerer to use tool evidence alongside the original audio. The two approaches therefore have different technical emphases.

The authors report scores of 74.00, 78.50, and 77.26 on MMAR, MMAU-test-mini, and MMSU, respectively. Their Qwen3-Omni-Instruct baseline on MMSU is 76.86, whereas the corresponding baseline under our unified evaluation is 72.18, indicating that the execution conditions underlying the two sets of results are not yet aligned. We have not obtained the authors' trained weights and public implementation needed for evaluation under the same protocol. We will therefore present these as author-reported results, separately from our unified evaluation table.

3. Do the reports provide evidence that supports correct decisions?

Following your suggestion, we added per-question comparisons in which the same report is provided to or withheld from the same answerer. Model weights, original audio, questions, prompts, and decoding configurations remain unchanged. We first examine which originally incorrect answers are corrected when the report is provided:

SettingAccuracy without reportsAccuracy with reportsPreviously incorrect answers correctedNet increase in correct answers
7B MMSU66.2070.12256+196
7B AudioMarathon75.7776.26100+31
30B MMSU72.4876.78273+215
30B AudioMarathon70.4071.50153+70

Grouping questions by the actual report content, the 557 MMSU questions containing relational measurements yield a net increase of 154 correct answers for both 7B and 30B, showing that gains are concentrated in this category. To further examine the contribution of relation expression itself, we fix the measurement values and remove only the specified comparison-relation fields. Accuracy decreases from 70.12 to 66.84 for 7B MMSU and from 76.78 to 74.36 for 30B, supporting the role of organizing measurements into explicit relations.

To illustrate what a report supports, we include traceable successful cases. For example, in an AudioMarathon event-recognition case, a report spanning multiple windows provides relevant cues for the speech category, changing the model's answer from "alarm" to "speech." Such reports supplement the model's original judgment, but support only event cues within the measured scope.

Misuse and insufficient support are analyzed separately. In the four settings above, 60, 69, 58, and 83 originally correct answers become incorrect, respectively; all remain included in the statistics. We also conducted a record-level audit of a fixed set of 64 natural questions. Among the 48 questions with reports, 10 provide directly relevant text, 26 provide relevant proxy measurements, 2 contain relevant but incompletely delivered content, and 10 exhibit a mismatch between the measured attribute and the question. The 16 questions without reports are considered separately. For example, transcripts describing only some hotels cannot support a claim that all four hotels share an attribute, and the duration of speech in a recording cannot answer a question about historical elapsed time described in that speech. These analyses distinguish reports' decision-level benefits, measurement grounding, and scope of support. We will accordingly characterize "trustworthiness" through traceable observations, verifiable relations, and explicit boundaries of support.

AuthorsOpenReview ↗

Response Part 3/3

4. What do the ablations show about the roles of decomposition and SFT across tasks?

We added module ablations for the complete tool-report workflow:

Configuration7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Without decomposition: select tools directly from the original question69.4476.0476.4871.51
Without reports: the same SFT answerer receives no report66.2075.7772.4870.40
Without SFT: the original answerer receives the same report67.8074.3976.3471.92
Full method70.1276.2676.7871.50

The accuracy gains from the decomposition branch are modest: they are positive for both backbones on MMSU and approximately neutral for 30B on AudioMarathon. This comparison reflects the combined contribution of requirement planning and its associated measurement organization. At the same time, the recorded information requirements, tool choices, and scopes allow researchers to inspect attribute mismatches, mixed comparison objects, and insufficient measurement granularity. This provides interpretability beyond accuracy and a basis for targeted improvements to the tools.

SFT improves performance on both 7B datasets and on 30B MMSU; on 30B AudioMarathon, the configuration without SFT remains slightly better. The full method still outperforms Direct in that setting. We will retain this result and discuss its applicability in relation to different report types.

5. How does Section 3.3 correspond to the actual supervision objective?

The original description of evidence explanations and the training objective was not sufficiently precise. We will update Section 3.3, the equation, the method figure, and Appendix B.2 to align them with the actual implementation.

In the MMAR and MMAU segment-based experiments, SFT is applied to indexing and planning/segment selection, with indexing and segment-selection demonstrations as the supervision targets; the final answerer retains its original weights. Joint supervision of the explanation z and answer y is a separately evaluated configuration. We will present the segment-based experiments as supplementary evidence-acquisition experiments in the appendix. The new MMSU and AudioMarathon experiments cover requirement planning, tool measurement, relation organization, and answerer learning, and will serve as the main empirical evaluation.

The new main experiments fix the planner and tools and apply report-conditioned SFT to the answerer. Inputs consist of the original audio, the question, and reports under different conditions. The supervision targets are the final answer y and end markers, with the following loss:

Lans=−E(x,q,r,y)∑tlog⁡pθ(yt∣x,q,r,y<t).\mathcal L_{\mathrm{ans}}=-\mathbb E_{(x,q,r,y)}\sum_t\log p_\theta(y_t\mid x,q,r,y_{ < t}).

Here, r may be an empty, matching, scope-mismatched, or mixed report. The explanation z supports interpretability analysis and is not a supervision target in the main method. We have also completed explicit explanation-supervision controls, which will be analyzed in the appendix. The revised description will clearly distinguish information requirements, actual tool observations, analytical explanations, and answer learning.

Reviewer_Laa3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MEL for audio question answering. It uses audio indexing and question decomposition to select relevant segments, then provides these segments together with the original recording to an audio language model. Experiments on MMAR and MMAU with two Qwen Omni backbones report improvements over direct answering, with ablations examining the roles of evidence acquisition and supervised training.

Critical strengths

The results show that selected audio segments can improve accuracy while keeping the final answering model unchanged. This provides useful practical evidence for supplementing the complete recording with targeted observations. I also appreciate that the ablations report cases where training or decomposition does not help, which makes the contribution of these components easier to assess.

Critical weaknesses

My main concern is novelty. Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning (ICLR 2026) already trains an audio model to locate informative segments and listen to them again during reasoning. EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning (2026) also organises question relevant observations into evidence and combines them with the original audio for answering. I understand that MEL follows a different workflow, but it is not clear to me what limitation of these methods the proposed design addresses. In particular, I could not find a discussion or comparison with Echo. The additional indexing and planning steps may be useful, but their methodological novelty remains limited to me.

I am also not sure whether the experiments implement the main learning contribution as described. Section 3.3 defines supervision over an evidence applicability explanation followed by the answer. However, Appendix B.2 states that the adapted weights are used only for indexing and acquisition planning, while final answering retains the original weights. This distinction matters because learning what evidence can justify appears to be the main proposed advance beyond evidence acquisition. As presented, the experiments do not clearly establish that this capability has been learned.

With that being said, the improvements are useful. My rejection leaning assessment comes mainly from the unclear contribution beyond existing methods and the unresolved connection between the proposed objective and the reported implementation.

Questions for authors
  1. How does MEL address a limitation of Echo beyond using a different acquisition workflow? Please clarify what is technically new and which experimental result supports that distinction.

  2. Was the objective in Section 3.3 used in the reported experiments? If final answering retains the original weights, where is evidence applicability supervision applied, and which result isolates its contribution? Clarifying this would materially affect my assessment.

2 comments
AuthorsOpenReview ↗

Response Part 1/2

We thank the reviewer for highlighting the need to clarify our methodological contribution. We have added experiments on tool reports and answerer training, and distinguish MEL from related work in three aspects: evidence acquisition, evidence organization, and the learning mechanism. We address each concern below.

1. How does MEL differ technically from Echo and EChO-Agent, and what evidence supports these differences?

Compared with Echo, MEL addresses not only “where to listen again,” but also “what to measure and how to use the measurements.” Echo primarily uses audio-interleaved reasoning to locate and re-listen to relevant segments. MEL instead translates the question into specific information requirements, identifying the attributes, objects, and temporal scopes to examine before invoking the corresponding tools. For example, when comparing changes in sound intensity, we provide not only relevant audio but also energy measurements over the corresponding intervals and their comparative relations, making the basis for the subsequent decision inspectable.

Compared with EChO-Agent, MEL places greater emphasis on the audio model’s own requirement-planning capability, rule-based evidence organization, and evidence-conditioned answerer training. EChO-Agent selects predefined tool combinations according to question type and relies on external text models such as DeepSeek-V3 to filter and organize tool outputs and check candidate answers, with an audio model producing the answer. Its evidence processing is primarily implemented through inference-time orchestration by text models.

MEL follows a different approach: the audio backbone identifies information requirements from the original audio and question, and these requirements are mapped to concrete measurements. Raw tool observations are retained, while fixed rules organize the objects, scopes, and computable comparative relations. The audio answerer is then trained to make decisions using this evidence. At test time, no additional text model is required to rewrite, integrate, or adjudicate the evidence.

The distinction is therefore not simply an additional decomposition step, but an explicit connection between information requirements, actual measurements, verifiable relations, and model decisions. For example, the average energy of an entire recording cannot directly answer a comparison between different stages, and speech activity duration cannot substitute for historical time described in speech. This representation allows us to inspect whether a tool has supplied the information the question actually requires.

We have completed supplementary comparisons on the full MMSU and AudioMarathon evaluation sets. The following table reports accuracy (%), with MEL trained using the fixed seed 42:

Method7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Direct64.7073.9572.1869.80
EChO-Agent63.5868.2965.4864.97
MEL70.1276.2676.7871.50

Since EChO-Agent does not provide public code, we reproduced it according to the paper. All methods use the same evaluation sets, corresponding audio backbones, and input order, while retaining their respective method-specific prompts and invocation procedures.

We also conducted module ablations under matched conditions:

Configuration7B MMSU7B AudioMarathon30B MMSU30B AudioMarathon
Without SFT: original weights + full report67.8074.3976.3471.92
Without reports: SFT answerer receives no report66.2075.7772.4870.40
Without decomposition: select tools directly from the original question69.4476.0476.4871.51
Full method70.1276.2676.7871.50

AuthorsOpenReview ↗

Response Part 2/2

Providing reports improves performance over the no-report condition with the same weights across both backbones and datasets, indicating that tool observations supply useful additional information. A supplementary control on MMSU further shows that, with the same measurement values, objects, and scopes, adding explicit comparisons, rankings, and differences improves 7B accuracy from 66.84 to 70.12 and 30B accuracy from 74.36 to 76.78. This supports organizing tool measurements into readable, verifiable relations.

The accuracy gains from the decomposition branch are relatively modest: they are positive for both backbones on MMSU and approximately neutral for 30B on AudioMarathon. However, decomposition also provides important interpretability by recording why particular information is needed, why a tool is selected, and which scope a measurement concerns. These records allow us to identify specific problems, including attribute mismatches, mixed comparison objects, and insufficient measurement granularity, and thereby locate the stages that need improvement. Decomposition thus contributes not only to final accuracy, but also to making evidence acquisition inspectable and its errors traceable.

We will accordingly emphasize three interconnected technical components of MEL: question decomposition into measurement requirements, evidence organization that preserves objects and scopes, and report-conditioned answerer learning.

2. How does Section 3.3 align with the actual training, and how is evidence learning validated?

Thank you for pointing out this inconsistency in our description. The original text summarized different experimental configurations under a single explanation-and-answer learning objective, which was not sufficiently precise. We have rechecked the training and inference configurations at each stage and will explicitly distinguish evidence-acquisition training from answerer training in the revision.

The segment-based experiments on MMAR and MMAU train the indexing and evidence-acquisition planning modules, while the final answerer retains its original weights. We plan to move these experiments to the appendix as supplementary investigations using an audio-cropping tool, and make the new tool-report experiments on MMSU and AudioMarathon the main evaluation. These experiments more fully demonstrate the complete workflow of requirement decomposition, tool-based evidence acquisition, evidence organization, and answerer learning.

The new experiments directly train the answerer, with the final answer yy as the supervised target. During training, the planner, tools, and report-generation rules are fixed. The answerer receives the original audio, the question, and a matching, inapplicable, mixed, or empty report. Through answer supervision, the model learns to make decisions under these different conditions rather than simply copying a tool’s conclusion. The loss covers only the answer and the template end markers:

Lans=−E[∑tlog⁡pθ(yt∣x,q,E,y<t)].\mathcal L_{\mathrm{ans}}=-\mathbb E\left[\sum_t\log p_\theta(y_t\mid x,q,E,y_{ < t})\right].

Parameter updates are applied through LoRA to the q/k/v/o projections in the answerer’s text Thinker, while the audio modules and output head remain frozen.

The explanation zz in the original text will be clearly distinguished from the actual learning objective: explanatory content is used to present and analyze the scope of evidential support; it is neither a supervision target in the main experiments nor a required intermediate output of the final answerer. The main method’s interpretability comes primarily from inspectable records of information requirements, tool choices, measurement objects, temporal scopes, and comparative relations. The final answerer only needs to produce an answer based on the original audio and evidence.

The ablations above separately evaluate the contributions of report input and answerer training. For example, SFT improves 7B accuracy from 67.80 to 70.12 on MMSU and from 74.39 to 76.26 on AudioMarathon when reports are provided. Removing the reports from the trained answerer yields 66.20 and 75.77, respectively. These results show that the new method benefits from combining tool evidence with a trained answerer, without requiring explicit explanation generation.

We will update Section 3.3, the training-objective equation, the method figure, and the appendix accordingly. The main text will describe evidence-conditioned learning with answer supervision, the appendix will specify the weights and training scope of each stage, and explanation generation will be presented as a separate analysis, aligning the methodological description with the actual experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.