acceptodds
Under review as a conference paper at ICLR 2027

Prior-Centric Support Allocation for Few-Shot Audio Recognition with Audio-Language Models

Abstract

Few-shot audio recognition based on audio-language models requires the collaboration of transferable semantic priors derived from class texts and task-specific evidence from support sets. Nevertheless, existing methods tend to over-emphasize support memories or matching pathways, resulting in the entanglement of sparse support-set evidence with already strong semantic priors and leaving the adaptation mechanism insufficiently explicit. To address this issue, this paper formulates few-shot audio adaptation as a prior-centric support allocation problem, and accordingly proposes a Prior-Centric Support Allocation (PCSA) framework, which treats the support set as a control signal for episode-level inference rather than as a competing memory pathway. The framework consists of two coupled components: Prior-Centric Semantic Scaffold (PriorCore) and Support-Aware Transductive Refinement (SATR). Specifically, PriorCore constructs a stable cross-granular semantic scaffold from coarse and fine priors, while SATR converts support evidence into reliability signals and uses them to regulate transductive refinement over the current episode. Furthermore, this paper designs an explicit reliability-controlled prediction rule that delineates the functional boundaries between semantic priors and support evidence, making the division of labor explicit and analyzable. Extensive experiments on several few-shot audio recognition benchmarks illustrate that our method achieves strong performance under the adopted evaluation protocols, and comprehensive ablation and diagnostic analyses further confirm the effectiveness and robustness of the proposed framework.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
few-shot learning, few-shot audio recognition, audio-language models, transductive inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fdA6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Motivated by the fact that we now have strong joint audio-language embedding models and captioning models, this paper presents a method for few-shot audio classification aims to leverage support informatino from these two modalities as needed. They contribute two modules: Prior-Centric Semantic Scaffold (PriorCore) and Support-Aware Transductive Refinement (SATR). The first aims to fuse semantic (class text) and support information (caption text + audio) at both temporally fine and coarse grains, this is then converted into a reliability signal that informs how reliable audio and caption support is over the target classes. In parallel audo and caption features are input into a label propagation network (SATR), the outputs of which are merged with that of the direct support evidence, modulated by the support reliability signal. They compare this approach with existing few-shot methods on ESC-50, US8k, and FSDKaggle2018, along with ablation to help understand the impact of their contributions.

Critical strengths
  1. The paper identifies an interesting problem that has arisen in the past few years but hasn't been adequately addressed, i.e., "how do we leverage both audio-language models and few-shot support for low-data regime audio classification?"
  2. The solution appears to be sound and performs well over competing methods and baselines.
Critical weaknesses
  1. Clarity. First of all, the introduction was clear. I had no problem with that. The methods were mostly clear but should be improved. Section 3 should do a better job communicating how the different modules relate to the overall motivation of the paper. For me, the initial confusion was between the caption information and the class text information. I suggest clearly showing that the text class information is only used in PriorCore to relate the support information to the classes, and that the "support" is inclusive of both the audio feature and the audio caption text features. The experiments were not clear. In Section 4.2, it's unclear from the paper how classes are handled in the experiments, e.g., how the novel classes are held out. Rather than point to MGCL, please state it in the paper, as this is important for understanding the validity of the experiments. Also, while MGCL paper contains ESC-50 and FSDKaggle2018, it doesn't include US8k. So, when the paper states that US8k follows the protocol where twelve-way vocabulary repeats two of the classes, this is confusing since it is not in MGCL paper. Furthermore, US8k only has 10 classes. So, what is the training data for US8k in a 12-way setup? Maybe the whole model is trained on the aggregation of all the datasets? If so, that isn't communicated in the paper and needs to be. Related, the statement "The rows other than PCSA in Table 1 are as reported under the MGCL protocol (Zhao et al., 2024)" - what aspects of the MGCL paper is this referring to? Please clarify.
  2. The experiments don't adequately isolate the different proposed modules, which lowers the informativeness of the paper. Ablations results are muddied by the addition of the cache module that wasn't even introduced in this paper (and in general it seems the cache hurts performance). At the moment, it is unclear where the real gains come from, especially as it relates to the motivation, e.g.,
    • Caption (i.e., text-mediated) features vs audio features
    • SATR vs PriorCore (Or even a full analysis of PiorCore vs SATR + PriorCore, rather than just a few episodes)
    • Fine vs Coarse
  3. The experiments and analysis don't adequately answer the motivation of this paper, which lowers the informativeness of the paper. I would think that with audio examples you would gain specificity while leveraging class prior information, e.g., you want to detect certain instances of a class, rather than all instances of a class, like a specific type of engine, power tool, drum, or even an individual one, like in a few-shot individual id scenario or even few-shot audio-moment retrieval. But the current evaluation setup does not investigate this. In its current form, it is unclear what advantage there is by combining these two support types and if both are necessary or not. This requires more than just new ablations, but a new evaluation setup as well. At the very least the experiments should dive deeper into when one source of information is needed more than the other. Section 4.6 notes that the benefit of of SATR for instance varies a lot by episode, but it doesn't investigate the charcteristics of those episodes, i.e., why was it helpful in certain episodes but not others.
Questions for authors
  1. Were the models trained on each dataset separately or in aggregate?
  2. How were classes held out for evaluation? In particular, for US8k, which only has 10 classes.
  3. How should we interpret the US8k results when two of the classes are repeated?
  4. What aspects of the MGCL protocol does the paper adopt
  5. Why was CLAP-Base not included for US8k?
  6. What is the performance comparison of caption (i.e., text-mediated) features vs audio features?
  7. What is the performance comparison SATR vs PriorCore?
  8. What is the performance comparison of fine vs coarse?
  9. How would you augment/modify the experimental design or validation tasks to more clearly demonstrate the benefit of the proposed method?
  10. In what scenarios does the model benefit from semantic prior vs. support information?
Reviewer_WNETOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PCSA for few-shot audio classification on a frozen MS-CLAP encoder: PriorCore keeps MGCL's coarse and fine Sinkhorn-OT prior logits, sums them and drops MGCL's cache branches; SATR computes a scalar support reliability r , runs clamped label propagation on a top-k cosine support+query graph, and blends with αr=clip(1−βr,0,1)\alpha_r = clip(1 − \beta r, 0, 1) , so αr∈[0.65,1]\alpha_r \in [0.65, 1] for β=0.35\beta = 0.35. On ESC50 / UrbanSound8K / FSDKaggle2018 (5/12-way, 1/10-shot), PCSA reports the best Table 1 averages (99.43 / 90.46 / 93.53 vs. MGCL's 99.03 / 86.14 / 91.74) and is the maximum in all twelve cells; ablations, seeds, hyperparameters and diagnostics are on FSDKaggle2018 only.

Critical strengths
  • Clean, simple inference rule. Eq. 9 is a convex mixture of prior and propagated distributions; Eqs. 12-14 are correct, making the method transparent and the requested ablations easy to run.
  • Within-pipeline ablation with some statistical effort and candid scoping. "Prior + cache + SATR" 92.36 vs. PCSA 93.53 (Table 2; +0.50 / +0.56 / +1.61 / +2.02), paired t-tests (Sec. 4.4), four seeds (std 0.15), fold intervals (Sec. 4.3); the transductive confound and App. C.3's negative result are disclosed.
Critical weaknesses
  • Persistent cross-episode training of fusion layers on evaluation-fold support (potentially fatal; needs clarification). Sec. 3.1 (l. 205-207), 3.2 (l. 222-223), 4.2 (l. 355-356): layers "persist across episodes" over a fixed 12-class per-fold vocabulary (100 tasks per fold), seeing up to 100 × 12 × 10 = 12,000 labeled support draws (with repetition) from the ~12 test classes at 12-way 10-shot; reset policy and support/query disjointness are unstated; the size of any accumulation cannot be diagnosed from the tables because Table 1's MGCL rows come from different episodes (cf. Weakness 2), so only a per-episode re-initialised run would settle it. If unresolved, Tables 1-3 are in question.
  • Headline comparison is not like-for-like, and the transductive control exists for one dataset only. All non-PCSA Table 1 rows are copied, inductive MGCL-protocol numbers (l. 352-354). The only transduction-controlled comparison is Table 2's "Prior + cache + SATR" row (the authors' MGCL re-implementation plus SATR, l. 374-376), on FSDKaggle2018 only (+1.17 vs. +1.79 against copied MGCL) and under the persistent-layer protocol of Weakness 1; TPN, LaplacianShot, TIM, Martin et al. (CVPR 2024) and TransCLIP (NeurIPS 2024) are cited but not run. For ESC50 and UrbanSound8K (the largest gain, +4.32), the +0.40 / +4.32 gains cannot be separated from transduction or protocol, so the cross-dataset claim is only partially supported.
Questions for authors
  • Controller validation: The main contribution is tested only against α=1\alpha = 1. Fixed-weight and K-only baselines are needed to show that the reliability estimate adds value.

  • Incomplete ablation: Table 2 omits full PriorCore without SATR, leaving SATR’s contribution unclear. Appendix C.3 also reports a slightly negative 12-way 10-shot gain without specifying the comparison clearly.

  • Evaluation protocol: Fusion layers retain updates across episodes, but reset boundaries and possible data overlap are unclear. Appendix B also justifies parameter choices using the reported evaluation results, raising concerns about test-set tuning.

  • Baseline comparisons: PCSA uses query-set information, while the published baselines are inductive. Without matched transductive baselines and paired tests, the gains cannot be attributed specifically to the proposed method.

Reviewer_MYkWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Prior-Centric Sup-port Allocation (PCSA), which builds on the previously published MGCL approach ("Multi-grained Correspondence Learning of Audio-language Models for Few-shot Audio Recognition") for few-shot audio classification. The main changes in PCSA with respect to MGCL, as I understand them, are:

  • remove the KV cache (an ablation that was evaluated in the original MGCL paper)
  • add "Support-Aware Transductive Refinement" (SATR), which (1) switches from inductive inference to transductive inference and (2) weights the impact of the transductive refinement based on the support set via a reliability estimate of the support set.

Evaluation against three datasets shows moderate (0.5, 2, or 4 point) improvements compared to the MGCL baseline (weaker baselines are also evaluated, but since the approach builds on MGCL, I think it is the only relevant baseline.). An ablation on one of the sets shows that most of the 2 point gain comes from the switch from inductive to transductive inference.

Critical strengths

The paper shows that switching from an inductive to transductive inference in a few-shot audio classification setting can lead to marginal to moderate gains.

Critical weaknesses

Over-claimed novelty: The paper presents its prior-centric framing (centering the approach design on the prior "scaffold", i.e. the prior from the frozen audio-language model) as a significant contribution. However, when read in close contrast to the MGCL paper, it becomes evident that this framing is not novel, as the latter already evaluated a cache-free version of MGCL as an ablation and showed that most of the gains indeed come from the multi-grained cross-modal contrast (i.e. the prior) and not from the support set cache. Most of the components in PCSA's "PriorCore" are adopted directly from MGCL. As such, I think the only significant novelty is the SATR step. The transductive refinement applied in that step is also adopted from the literature, rather than being a new technique. However, its application for few-shot audio classification appears to be novel. The reliability condition in this step is new, however the ablation experiments suggest that its impact is marginal. Putting it all together, it amounts to the novel contributions of the paper being notably overstated. While the authors disclose in different parts of the paper which elements are based on prior work, it requires some detective work to piece it together into a clear picture of what's actually novel in the paper with respect to MGCL, and it's notably thinner compared to what's stated in the paper's introduction.

Evaluation validity and strength. The authors evaluate on 3 datasets, ESC50, UrbanSound8K, and FSDKaggle2018, including 5-way and 12-way setups. This caught my attention, given that UrbanSound8K only has 10 classes, which doesn't support a 12-way few-shot setup. The paper explains that "UrbanSound8K follows the five-class and twelve-slot protocols defined by MGCL, whose twelve-slot vocabulary repeats two of the ten official classes in each fold.". MGCL didn't evaluate on UrbanSound8K (US8K) at all. It's fine to evaluate 12-way on ESC50 and FSD2018 since they have more than 12 classes, but UrbanSound8K doesn't. Repeating two of the 10 classes to simulate 12-way few-shot classification doesn't make sense, as 4 of the classes (ways) are now no longer unique. In my mind, this invalidates the results on US8K, where the largest gains of the proposed method (4 points) are observed. The gains on ESC50 (0.5) are marginal, and on FSD2018 (2 points) they are moderate. Of these 2 points, 1.76 can be achieved with a fixed alpha, meaning the contribution of the reliability controller is marginal. Furthermore, the 95% confidence interval for these two sets is reported as ±1.3, placing the gains on ESC50 inside the interval and FSD2018 only just outside of it.

Combining all of the above, I think a fair representation would be to state the primary contribution as showing that, compared to MGCL, switching from inductive inference to a standard transductive one based on prior literature leads to 1.76 point improvement on DSF20218 and marginal improvement on ESC50. As such, it's a fairly limited contribution.

Questions for authors
  • Do you agree with my characterization of the delta proposed in this paper compared to MGCL? If not, why not?
  • What is your reasoning behind using a dataset with 10 classes (US8K) to evaluate 12-way few-shot classification? How can repeating 2 of the 10 classes be a valid experimental design?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.