acceptodds
Under review as a conference paper at ICLR 2027

Region-Prototype Attention for Calibrated Few-Shot Multi-Label Audio Tagging

Abstract

Few-shot multi-label audio tagging, the task of assigning several co-occurring sound labels to a clip from only a handful of examples per novel class, has been studied only in narrow, non-standardized settings, and we find that standard prototypical models expose a subtle but serious failure: while their ranking of labels is strong, their decisions are badly miscalibrated, with models over-predicting and assigning almost every candidate label to each query. We make three contributions. (i) We establish a few-shot multi-label benchmark on FSD50K and AudioSet-Balanced with a disjoint base/novel class split, a chance floor, and identically-trained baselines in single-class and compositional (LC-Protonets-style) prototypical networks, a frozen AudioSet Transformer, and a post-hoc temperature-scaling calibration baseline. (ii) We show that over-prediction and miscalibration are pervasive across convolutional, compositional, and transformer backbones; a large AudioSet-pretrained Transformer attains high recall only by predicting nearly all labels, with an expected calibration error (ECE) far above that of metric-based models. (iii) We introduce RAFTa, whose region-prototype readout lets class prototypes attend over a query's time–frequency regions before matching. RAFTa is the best-calibrated model in every setting, with the lowest ECE among all baselines and the predicted label-cardinality closest to the truth, while remaining competitive on ranking (mAP), and its calibration edge is not matched by post-hoc temperature scaling of a strong baseline. Code and benchmark splits are released via an anonymized repository.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
few-shot learning, multi-label audio tagging, calibration, prototypical networks, region-prototype attention, sound event detection, expected calibration error, graph attention networks, episodic learning, FSD50K

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VVPDOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces a new method for few-shot multi-label audio tagging that addresses systematic over-prediction calibration failures. It presents a small benchmark with given splits to evaluate generalization to unseen classes. The work frames few-shot multi-label audio tagging around decision quality rather than ranking performance alone.

Critical strengths
  • efficient and elegant method translated from vision to audio.
  • the topic is relevant to the field
  • a small scale benchmark is introduced for better future evaluation
Critical weaknesses

Missing citations. Especially the introduction contains many claims that are not backed up by citations at all. This creates a false impression of novelty. For example, "rare annotations" or "inherently multilabel" is mentioned but no related work that backs this claim up. Bioacoustic monitoring is mentioned as an example but e.g. no birdset dataset/benchmark that underlines this argument. Additionally, AudioSet or FSD50k are introduced as standard datasets without citing them properly or citing papers that use these as the standard setting. This makes the field appear completely novel. All citations are not properly formatted with brackets.

Order of definitions. For example, the paper suddenly introduces terms like prototype. There are a lot of different prototype definitions and the explanation came pretty late after using the term in this work. Same for "pretrain-then-meta-train recipe", this should be introduced earlier so that it's understandable.

Incorrect statements and incomplete related work. There are also statements like: "prior multilabel fewshot tagging centers on singlelabel" without any citations. Moreover, this is not true, especially in bioacoustics that is mentioned as a use case. For example these works do few-shot multilabel classification:

  • "Simulated soundscapes and transfer learning boost the performance of acoustic classifiers under data scarcity" (Weldy et al.)
  • "Can Masked Autoencoders Also Listen to Birds?" (Rauch et al.)
  • "Domain-Invariant Representation Learning of Bird Sounds" (Moummad et al.)
  • this and a lot of others are missing in realted work as well

There are also SSL pretraining methods missing in related work (very random and barebones). Current SOTA models in audio such as Audio-MAE, EAT, SSLAM, BAT, AudioMosaic are all missing (and none of them is used in the benchmark).

Limited data and model scope. The FSD50k/AudioSet benchmark for example does not contain any bioacoustic datasets even though this is mentioned a few times as a motivation. There are a lot of bioacoustic datasets available that could, at least partially such as Birdset, be used for this. Also there is no benchmark engagement with current SOTA SSL models even though they dominate current SOTA on AudioSet (see the models mentioned above). A more complete picture would include modern SSL approaches as well. Also the margins of the results are pretty small and only one seed was used, questioning the statistical significance of the results.

Lack of method ablations. While there are ablations on shot count and computational cost, there are no ablations about what makes the method work (only the GAT in the results table as an additional ablation).

Questions for authors

Please address the weaknesses mentioned above.

Reviewer_bT8oOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of few-shot multi-label audio labeling, in which a model must predict multiple co-occurring audio classes based on a small support set. The core conclusion is that, while existing metric-based few-shot models and frozen AST backbone networks achieve reasonable results in mAP rankings, their decision quality is poor: they overpredict labels, exhibit high ECE, and predict a number of labels far exceeding the actual number of active labels. These shortcomings are present in CNNs, composite prototypes, and Transformer backbone networks, and are particularly severe in the frozen AST.

Critical strengths

It is practically significant to focus on calibration and overprediction issues beyond ranking performance. By having class prototypes focus on different time-frequency regions, the design is intuitive and aligns with the characteristics of tasks involving multiple coexisting sounds.

The study compares multi-class baselines across two datasets and two sampling protocols, reporting calibration errors and the number of predicted labels, thereby providing a comprehensive evaluation.

Critical weaknesses
  1. The method proposed by the authors may not necessarily be a good option for solving the problem they claim to address; comparisons with multimodal large models such as Qwen-Audio need to be added.
  2. Some conclusions are stronger than the results suggest. When the number of true labels is approximately 1 or 2, RAFTa still predicts an average of approximately 3.5–4.2 labels. The results support a relative improvement, but the statement that the model ‘approaches the number of true labels’ is not entirely accurate.
Questions for authors

What are the advantages of the current method compared with multimodal large models such as Qwen-Audio?

Reviewer_odzHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper adapts prototype-region attention from few-shot image classification to multi-label audio tagging and introduces graph attention smoothing over the region representations. While there have been other multi-label audio classificaiotn models, their aim is to create better calibrated outputs. They evaluate their model (with and without the GAT) on their own class splits of AudioSet and FSD50k, but find that their method is not statistically significantly better than other models on FSD50k according to mAP (shown in supplementary material) and likely not statistically significantly better than other models on AudioSet according to mAP (overlapped confdience intervals, but no statistical test, not even in supplementary material). However, the model does have better calibrated outputs according to ECE. Unfortanatly, this advantage largely disappears once a simple post-hoc model-wide temperature scaling is fit and completely reverses one of the main findings of the paper which states that transformer architectures are poorly calibrated (the supplementary material shows that with a simple post-hoc temperature scaling, the transformer based model becomes the best calibrated model according to ECE). Their GAT module generally decreases performance.

Critical strengths

This paper presents an adaptation from image classification that intuitively makes sense for audio tagging as comparing region encodings and prototypes may mitigate source overlap, leading to better calibrated outputs.

Critical weaknesses
  1. Novelty of the contribution is limited. The proposed method is an adptation from image classification with the addition of the GAT, but the GAT was detrimental to performance (and the region prototype method has inconclusive gains). Additionaly, no code was included or linked in submission. Without released code, the baselines and benchmark splits are difficult to verify or reuse, which limits their value as contributions.

  2. The results are inconclusive, and several claims in the main text are not supported by the supplementary results. The model does not clearly perform better on tagging performance or calibration. The main body of the paper shows that the model performs on par with other models in regards to tagging performance. The paper acknowledges this but notes that its calibration results are better. However, the supplementary material shows that fitting a single post-hoc temperature parameter largely removes this advantage. As a result, several statements in the main text overstate the findings. For example:

    • Section 1.2 - "and we show this edge is not reproduced by temperaturescaling a strong baseline" - The supplementary material shows that is not the case.
    • Section 1.2 - "graph-attention stage is well-conditioned but contributes modestly," - It improves one metric by 0.001 and lowers the others, so “contributes modestly” overstates its effect.
    • Section 6 - "The transformer is the worst-calibrated." This omits that, after post-hoc temperature scaling, the transformer becomes the best-calibrated model (supplementary material). The claim should be qualified or removed.

    Furthermore, in the ECE results in Table 3 should also be using the per-class average like in the supplementary. material so that it is not affected by class distribution. F1 on the other hand is macro-averaged. Using different averaging schemes for different metrics makes the comparisons hard to interpret. Please report both micro- and macro-averaged values for ECE and F1 in Table 3. You have plenty of space in the table to be transparent and report both additional numbers. Lastly, the supplementary material does statistical analysis on FSD50k but not audioset. Please add the same analysis for AudioSet or explain why it was omitted.

  3. (Related to the above point) Proper baselines and analysis are not included to isolate the model contributions. Given that the focus of the paper is on the calibration, there should be much more analysis about this, e.g., the additional analysis for the post-hoc temperature scaling in the supplementary material should be in the main body. And there should be more analyses. For example, what about a bias term? A bias term would improve the F1 score on other models as well. What about per-class terms instead of global ones (e.g., fit on leave-on-out support). What about fitting the class logit in the other models as you do in RAFta? How does the models perform when classes have challenging class overlap / polyphony? In general, several baselines needed to isolate the contribution are either confined to the supplement or missing. This makes it hard to evaluate the model, its components, and minimizes any potential insights or contributions.

  4. Writing is inconsistent and unclear at times. Some parts of the paper included details that were not consistent in other parts of the paper. For example:

    • Section 1.2 - "a post-hoc temperature-scaling calibration baseline Guo et al. (2017)" - This was not included in the body of the paper (thus is also not a contribution).
    • Section 2 - "we use AST as a representative frozen transformer and leave foundation-model backbones to future work." - This is confusing as it makes it seem that your model uses AST, but it uses PANNs CNN architecture. After reading again, I realize that you meant that for the representative frozen transformer that you did use to compare against, you used AST. Clarify.
    • Section 3.4 - It is never clearly stated how the region prototypes are encoded. Are these the original region representations before or after the GAT are you taking the mean of all of them? What projection are you referring to?
    • Section 5.3 - The random and multi-label protocol were introduced but not used in the analysis as was stated in Section 5.3. Reporting this is critical since part of the claim is that prototype regions are better for overlapping sounds. Furthermore, I would suggest going even further and breaking it down if labels temporally overlap or not (i.e., use AudioSet Strong to do this).
    • Section 5.4 - CNN+Protonet+TS was introduced here but not reported in the main paper body.
  5. References are all 2 years or older. I suggest included more recent work in background and potentially as baseline comparisons. I have not read them, but a quick search yielded the following recent papers that may be of interest:

    1. Li et al, "AudioCAN: Enhanced Few-Shot Audio Classification via Energy-Guided Temporal Cross Attention", IEEE TASLP, 2026.
    2. Zhang et al, "Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypes", IEEE SPL, 2026.
Questions for authors
  1. The temperature-scaling results and statistical tests in the supplementary material bear directly on the paper’s main claims about calibration and tagging performance. Could you move these into the main body, or explain why they were placed in the supplement?
  2. Several claims in Section 1.2 and Section 6 appear to conflict with the supplementary results, notably the claim that the calibration advantage is not reproduced by temperature scaling and the claim that the transformer is the worst-calibrated model. Could you reconcile these claims with the supplementary results, or revise them?
  3. Does the calibration advantage hold against stronger post-hoc baselines than a single global temperature, such as per-class temperatures, a bias term, or fitting the class logit in the other models as you do in RAFta?
  4. How do the models compare on clips with high polyphony or overlapping classes? The random and multi-label protocols from Section 5.3 seem suited to this, and AudioSet Strong would allow separating temporally overlapping from non-overlapping labels. Since the motivation for region prototypes is mitigating source overlap, this seems central to the paper’s argument.
  5. Could you report statistical tests for AudioSet, as you did for FSD50k?
  6. Given that the GAT lowers most metrics, what is the case for including it in the final model?
  7. How exactly are the region prototypes encoded in Section 3.4? Are they computed from region representations before or after the GAT, and what projection is applied?
  8. What are the CNN+ProtoNet+TS results introduced in Section 5.4?
  9. Will code, class splits, and baseline implementations be released?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.