- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposed a way to use LLMs in speaker recognition. Rather than finetuning a speech-LLM to assess whether the pair of recordings in question comes from the same or different speakers, it uses SOTA speaker embedding extractors, their cosine similarity (and other embedding-based measures named "comparison feature"), a transformer mapper for the embeddings to map them to the LLM space, as well as 18 measurements of the two recordings and teaches the LLM to first provide an explanation of the differences between the two recordings. The decision is obtained by teacher-forcing the "same" or "different" tokens and forming the log-likelihood ratio (LLR) as their difference in the logit domain. To train the model, the authors created the VoxReason dataset, where "each example is a pair of recordings with four things attached: the two utterances, their extracted acoustic measurements, the speaker label, and a comparison report whose numerical claims have been checked against those measurements". An LLM (Gemini 2.5 Flash-Lite) writes each report (given the labels), and a numerical validator and an LLM judge filter these reports.
Critical strengths
(a) Interesting idea on using LLMs for the problem of speaker recognition. Also realistic, in the sense that the embeddings are extracted by a dedicated extractor, rather than computed by the LLM.
(b) Rich and informative ablations. The text vs soft tokens, and frozen vs. unfrozen verifier, provide useful information.
(c) Honest reporting of negative results, showing that although the LLMs own reports are grounded, they are not decision-selective.
Critical weaknesses
(a) The explanations do not improve the overall performance in terms of EER, compared to using the embeddings and their cosine similarity. (b) The comparisons with the performance of other speech-LLMs are not convincing, since speech LLMs do not use a dedicated speaker embedding model. (c) The explanations are mainly verbalizations of the 18 acoustic measurements, which does not justify the use of such a pipeline, as it can be accomplished with other means (e.g. a more standard LLM). (d) In Figure 1, the explanations contain statistical tests, such as Fisher's method. I found not explanation about how are these tests are performed. Are pythonic tools called by the LLM? If not, then the LLM is performing these test using their own (inadequate) computational skills?
Questions for authors
(a) Why does the LM verdict underperforms the cosine similarity on the same encoder (0.99% vs 0.34% in EER)? Have you considered using the head for the verdict, and the LM for the report, given the verdict? (b) Are tools used to perform statistical test, such as Fisher's method?
6 comments
I thank the authors for their response. Some questions: (a) Regarding the new classifier experiment, which embedding model was used (if any), which text segments were kept, and how was the jackknife split performed (by speaker or by pair)? (b) Please also state how the statistical test sentence is produced (by the model itself, by Python code at inference). It is still very unclear to me.
We thank the reviewer for responding!
The embeddings were generated by Gemma as metioned above. Also, as mentioned earlier, the entire text explanation minus the confidence scores and final verdicts was used. The jackknife was by speaker.
The statistical significance numbers are not computed by the LLM, but by a separate call to a library, the same one used to generate training labels.
We thank the reviewer for the constructive feedback. We agree that an encoder-based cascade is an essential comparison and thank the reviewer for highlighting this point. Alongside verification accuracy, an important question is whether joint training produces explanations containing more speaker-discriminative information than reports generated separately by an LLM.
To investigate this, we used Gemma to obtain embeddings from the explanation text alone, after removing explicit verdicts and confidence labels. We then trained a binary classifier to predict the same-speaker/different-speaker label from these embeddings, using a five-way jackknife evaluation.
The classifier achieved an EER of 1.66% with CoLMbo-SV explanations, compared with 13.1% with Qwen explanations. These results show that CoLMbo-SV’s explanations contain substantially more information useful for distinguishing same-speaker from different-speaker pairs, even without their explicit conclusions. This supports the discriminative value of the explanations, although it does not by itself establish causal faithfulness to the verifier’s decision. We will include this analysis and its evaluation protocol in the revised manuscript.
Regarding the statistical tests, the results are computed using Python code; the LLM does not invoke external tools. We will clarify this computation procedure in the revised manuscript.
We hope these additional results and clarifications address the reviewer’s concerns and would appreciate their consideration of a higher score.