acceptodds
Under review as a conference paper at ICLR 2027

Separating What from When: Stage-Specialised Credit Assignment for Auditable Foreground Sound Evidence

Abstract

Audio–language models can recognise a foreground sound while failing to show when it occurs. A tool-assisted system can also change a correct class label even when its timing tool supplies no new class evidence. We study these as separate fields of structured audio prediction. Our fixed protocol has three auditable stages: the model proposes a class, a frozen label-free tool inspects the waveform and returns an interval, and a finalisation rule retains the proposed class unless the new observation contains class evidence. A proposal-only adapter changes semantic behaviour while the temporal tool and the finalisation stage remain fixed. Across source-disjoint evaluation splits, stage specialisation improves answer exact match from to on the primary split without changing any predicted interval. The direction recurs on a later split and on new recordings of the classes used for adaptation. It reverses for an unseen class, which makes the transfer boundary explicit. Holding the semantic stage fixed, a frozen spectral-contrast localiser improves temporal IoU from to over an energy-based tool. Comparisons with three external audio–language models and the unadapted backbone use a shared tool and answer contract, so they measure semantic recognition under the evaluated protocol rather than native end-to-end system performance. Independent retraining preserves the direction of the semantic effect, whereas learned routing does not improve localisation. The evidence supports stage-specific credit assignment within this protocol and identifies low-SNR localisation and unseen-class transfer as open limits. It does not establish general state-of-the-art performance.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Counterfactual Acoustic Editing, Faithful Reasoning, Temporal Evidence, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_F7N6OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a three-stage pipeline that separates semantic sound classification from temporal localization using stage-specific adaptation and a frozen audio localization tool.

Critical strengths

None.

Critical weaknesses

The experimental evaluation is quite limited, with only three classes per dataset (Alarm, Dog, Speech.), a small number of independent audio sources, and mostly synthetic audios. It is unclear how well the proposed approach would generalize to a larger set of sound events or real-world audio. The method also shows a drop in performance on an unseen class, which further raises concerns about generalization.

The paper is missing some obvious and much simpler baselines. For example, why not use an existing audio onset/offset detection model to identify the foreground event and then apply a pretrained audio classifier to that segment? This would help establish an baseline for how hard detecting the class and onset/offset sound event is. This would also establish the need for such complex methods.

Questions for authors

Why is the evaluation limited to only four distinct sound classes? Why not use a realistic dataset like Audioset-strong to evaluate the proposed model. Why is a simple onset/offset detection followed by pretrained audio classification pipeline not included as a baseline?

Reviewer_UeXpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

the paper centers on sound event detection (SED) using LALM. Instead of producing the output end-to-end, the paper proposes to split it into stages: the first stage proposes the class label and a rough time interval, and the second stage proposes a more accurate time interval. The claim made in the paper is that separating the system into these stages makes it more interpretable and allows evaluation of the impact of each stage independently. The interventions are evaluated against a small set of synthetic mixtures with a continuous background and a single foreground sound event. The paper claims that the two-stage breakdown results in improved performance over the baseline LLM without the two-stage breakdown.

Critical strengths

None

Critical weaknesses

The three primary weaknesses of the paper are (1) method, (2) evaluation, and (3) very hard-to-parse writing.

Method: The proposed method is extremely simplistic. In particular, the stage for determining the timing interval is based on very basic onset detection on top of spectral flux or RMS energy (i.e., signal processing approaches that have been around for decades). These approaches are known to be brittle and do not generalize to complex mixtures. I think the only reason the paper reports any positive results using these methods is because the evaluation is based on highly simplified and artificial data of a very small scale (see evaluation). The paper also motivates breaking down the end-to-end process into stages, stating that otherwise it wouldn't be possible to evaluate model performance. Along the different axes of correct labeling and correct time interval prediction, however, it is absolutely possible to evaluate an end-to-end system along these axes simply by breaking down the metrics into classification metrics (time agnostic) and interval prediction metrics (class agnostic).

Evaluation: The interventions are evaluated against extremely small synthetic evaluation sets. The eval sets are made using a very low number of source material files on the order of tens to low hundreds, and the resulting mixtures are highly simplistic consisting of a single background and foreground sound. As such, it is highly questionable whether the reported results would generalize to real-world recordings which typically have more complex sound mixtures and many many more sound categories. I would have expected to see an evaluation on a dataset of real world data such as TACOS to help determine whether the proposed method generalizes to realistic data.

There is also no comparison to frontier models such as Gemini 3 Pro or Qwen3-Omni-30B, which do a pretty good job at both localizing and labeling sound events. While the paper clearly states the goal is not to produce a SOTA method, it is hard to determine whether the presented ablations provide generalizable insight with respect to LALMs that perform strongly on this task.

Writing: I found the paper extremely hard to read. The paper is packed with custom terminology and dense sentences that are unnecessary when things could have been described in much plainer language. Here are just a couple of examples, but there are many more throughout the paper:

  • "These observations motivate separating the semantic and protocol objectives; they do not by themselves establish proposal drift as the sole semantic bottleneck."
  • "The stage-specialised proposal adapter is trained on PROPOSE states only, with an activity-inspection curriculum and a coarse full-interval target, and is active only for the first action."

The structure of the paper is also very hard to follow - some results are mentioned as early as the introduction section when there is very little context to parse them properly. Others are interleaved in the methodology section. The evaluation sets are not summarized in a table, and are referred to with names that are hard to track. The formulas, for example equations 2 and 3, contain multiple undefined symbols.

Questions for authors
  • Why was no real world dataset like TACOs used for evaluation?
  • Why is the evaluation scenario constrained to such a simplistic mixing setup of single foreground sound and a single background sound?
  • Is there any evidence that a simplistic DSP-based interval detector, based on flux or RMS, could generalize to real-world recordings with multiple overlapping sounds?
  • What evidence is there that the results reported in this paper would be applicable to and benefit state-of-the-art LALMs that can do sound event detection?
Reviewer_sKHpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper try to address foreground sound location detection.

Critical strengths

None

Critical weaknesses

There is no problem statement, motivation, novelty in the paper. So single reference in the Introduction section. Figure 1 not convincing. Methodology is unclear, hard to follow the paragraphs, sentences are not related. random words used in the paper.

Questions for authors

Authors should define the problem that they are trying to address, motivation, proposed methodology, how authors plan to evaluate their proposed methodology. What is the research question that authors are trying to address in this paper?

Authors should include Experimental setup, datasets/benchmarks, results and conclusions. As such this paper does not include any of above mentioned details. Looks like this study is at early stage.

Reviewer_nSktOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a fundamental problem in evaluating tool-augmented audio-language models: when a model uses an external tool, it is often unclear whether improvements come from better sound recognition, more accurate event timing, or simply changes in how the answer is expressed. The authors propose a controlled, three-stage framework that separates these functions, making it possible to measure each contribution independently.

The framework first uses an audio-language model to identify a sound event and estimate its timing. A separate signal-processing tool then locates the event more precisely in the audio. Finally, the system combines the original event classification with the tool's timing estimate, preventing the tool from changing a classification without new evidence. By independently modifying the recognition and timing components, the authors can determine which part of the system is responsible for each improvement.

Experiments on synthetic soundscapes and real-world audio demonstrate that specialized recognition training improves classification accuracy, while a simple, fixed signal-processing tool improves temporal localization. The study also finds that more complex learned tool-selection strategies do not necessarily offer additional benefits.

The paper's central contribution is an auditable evaluation framework that separates semantic recognition from temporal localization in tool-augmented audio reasoning. It demonstrates how external tools can provide measurable improvements while preserving correct model decisions, and offers a principled way to distinguish genuine gains in audio understanding from changes introduced by the tool itself.

Critical strengths
  1. A principled, well-controlled attribution methodology. The combination of (a) paired single-stage interventions with the unchanged field treated as a control, (b) one frozen, label-free tool attached identically to every compared system so cross-model comparisons are not contaminated by differing front-ends, and (c) the no-rewrite contract, is a genuine contribution to how tool-augmented audio reasoning is evaluated. It directly answers a real failure of aggregate scoring and makes negative results interpretable. Execution is unusually disciplined: source-disjoint sealed splits, manifests frozen before evaluation, clustered bootstrap CIs, per-clip records.

  2. Exemplary calibration and honesty. The paper reports the router negative result, the Cat (unseen-class) reversal, abstention following the prompt rather than audio, low-SNR localization failure, and the freezing cost on long real events, and explicitly disclaims SOTA/coverage. This makes the evidence trustworthy and the method's boundaries legible, rare and valuable.

Critical weaknesses
  1. The empirical base is very small and the central effect is marginal, undercutting the claim that the effect is robust. The core positive result is a +0.097 EM difference on n=72 (12 source clusters) with CI nearly touching zero, one backbone, three classes, and only 48 adaptation clips. All data are synthetic mixtures, including the ESC-50 "real foreground" set, whose backgrounds and mixing are still synthetic, so external validity is thin. Crucially, the split closest to realistic conditions (ESC-50 cross-corpus) yields a non-significant contrast (+0.053 [-0.017,+0.117]); the "direction recurs" narrative rests mainly on two DESED splits sharing the same renderer and background pool. For a top venue the demonstrated capability is narrow and the strongest realistic test is inconclusive.

  2. Part of the headline "separation" is definitional, and the external-baseline table is largely disclaimed, so the apparent contribution is inflated relative to what is empirically shown. By Eq. 2, C_nr forces c_f=c_p and τ_f=τ_t for every tool output; "all 72 intervals unchanged under the semantic intervention" is therefore guaranteed by the contract, not an empirical finding, the only empirical content of the semantic result is the adapter's EM gain. Separately, the four-row external comparison (0.05–0.32 EM) is explicitly acknowledged to measure behavior under a non-native prompt/contract, so it is not a fair baseline; the only meaningful semantic comparison is ours-vs-single-adapter on the same backbone.

Questions for authors
  1. How much of the attribution is empirical vs. definitional? Since C_nr mechanically fixes the interval under the semantic intervention and the class under the temporal intervention, what is the non-trivial content of the "unchanged field is a control" claim beyond the EM delta?

  2. The primary effect CI is [+0.014,+0.194] on 12 clusters and the ESC-50 contrast includes zero. Can you report an aggregate significance across the seeds/splits jointly (not just "same sign"), and state whether the real-foreground result alone supports the robustness claim?

  3. Does the +0.097 persist if the single-adapter baseline is given the same 48 proposal clips / matched training budget, i.e., is the gain from stage specialization per se or from additional class-focused supervision?

Future work (would not change assessment):

  • Generalization beyond one backbone and three classes; non-synthetic backgrounds/mixing; a localizer whose window adapts to event extent (the 1 s cap bounds ESC-50 IoU at 0.841).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.