- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper provides an alternative to fine-tuning and in-context learning for bioacoustic classification in a single particular case: marmoset vocalization classification with 2 other datasets used in experimentation: dogs and watkins from the larger suite BEANS.
Critical strengths
Strong method to steer an audio LLM towards a particular task and dataset without fine-tuning or complex in-context learning. I found the experiments concerening predicting gradients from acoustic features very interesting proving that the model learns relationships between these features and the direction of the steering gradients
Critical weaknesses
The dataset suite is somehow limited. There are multiple datasets in BEANS and there are other bioacoustic datasets (BirdSet has a real drift between train and test) but somehow only 2 from BEANS were chosen in addition to the main one on which probably the main development and experiments were conducted. It looks somehow cherry-picked but it might be that the results were not ready for the deadline.
The paper has some readability problems, I find many paragraphs very compressed and hard to grasp, unusual terms which have more established alternatives, bad captions (if you present results mention the dataset for which the result is presented) .
The main evaluation issue is that the paper uses different sometimes not consistent experimental design for different experiments which makes it hard when actually trying to understand if the proofs support the claims. It would be better to have some clear experiments across multiple datasets (use all datasets for all main experiments), without doubtful conditions like the LP-experiments and a series of ablations (including the many kinds of drifts).
Questions for authors
The protocols are somehow confusing, tbh I have never seen this experiment design pre-LLM era in AI audio related papers. These sound like ablation or different condition testing. Comparators/evaluators are just "metrics" in the ML idiom. Why use "Exploratory diagnostics" and not just "Results". Going into the actual text I found the same issue with multiple paragraph, it makes it a bit hard to read, like a compressed version of a longer, more comprehensive explanation. Why use these terms when there are established ones? I hope I am wrong but I guess the experiment design and the way it's described for readers are largely LLM-driven?
I am not convinced by the low pass filtering experiments: a classifier may decide to focus less on high frequency content and for particular datasets like dogs there is not much above 8KHz. For marmosets may indeed matter but then low pass may introduce biases e.g. different classes have consistently higher frequency than the others.
It's not clear why that one particular model was used. NatureLM-audio is a bioacoustic specialized model so I wonder if the results generalize to it.
The main alternative to AKR is ICR and the paper lacks a main AKR-versus-ICL comparison across the dataset. In addition, ridge seems far better than AKR. AKR itself improves the internal representations only in the LP ablations. Not sure if this is enough.