- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
SepAdapter adds an activity-conditioned residual module before an ASR encoder to help recognize a target speaker in overlapping speech. It uses target and competitor activity together with temporal context. The authors implement it in FastConformer, Zipformer, and Whisper. Paired FastConformer runs show a modest but consistent gain over framewise conditioning, mainly on overlapping words. System comparisons are promising, but use different recipes and show mixed gains.
Critical strengths
-
The parameter-matched comparison across paired training runs gives credible evidence that temporal conditioning helps
-
The design is tested in several ASR backbones, with oracle and estimated activity reported separately.
Critical weaknesses
-
LibriSpeechMix uses 30.4 s of diarizer buffering, excluded from the advertised ASR lookaheads. The buffering preset was also selected using evaluation-set activity references. These choices leave the practical latency and held-out system comparison unresolved, even though the shared preset preserves the conditional DP-FW comparison.
-
The reliable result is a 0.356-point cpWER reduction over framewise conditioning. A parameter-matched TCN is slightly better in the single reported run, while the larger gains over SSA use different training and diarization recipes. Thus the evidence supports temporal correction, but does not establish a compelling advantage for the particular dual-path design.
Questions for authors
Do you have results with the diarizer preset selected only on development data and the complete buffering delay included? Does the advantage over SSA remain at a comparable total latency?
What evidence supports the dual-path design over the TCN alternative, and have the dropout and positional-parameter differences in the framewise control been checked as explanations for part of the gain?
2 comments
Thank you again for your careful review. We have uploaded a revised PDF and abstract addressing your concerns.
Appendix B reports the dropout/position control, Appendix C includes the fixed 1.04-s diarizer evaluation, and Section 6.1 with Appendix E presents the shared-protocol SSA comparison. We also clarify the distinction between ASR lookahead, diarizer buffering, and unmeasured end-to-end latency.
We appreciate your consideration.
Thank you for the careful reading.
Diarizer buffering and preset choice. The 80–1120 ms values are ASR lookaheads; the original diarizer preset buffers 30.4 s. As stated in the appendix, this preset was chosen by DER on the evaluation-set references. Our focus was the ASR module, so the preset was fixed once, shared by all models and lookaheads, and never chosen by recognition error. To test robustness with a fixed, lower-buffering preset, we re-scored the submitted seed-43 DP/FW checkpoints, without retraining, using the published 1.04-s setting of Sortformer v2.1 (model-card values, not tuned). DP/FW obtain 14.429/14.819% at 1120 ms (−0.389 pp [−0.511, −0.269]) and 16.218/16.584% at 80 ms (−0.366 pp [−0.483, −0.250]).
Comparison with SSA at the same buffering. We retrained SSA in our pipeline under a revised post-submission recipe with shared backbone initialization, data, training schedule and activity corruption. SepAdapter's history window and competitor routing were aligned with streaming inference; the recipe was developed using diagnostics on the existing evaluation benchmark. Both models use the same activity from Nemotron-3-Diarization in its published 1.04-s setting (model-card values, not tuned) and the same ASR lookahead. SepAdapter obtains 12.93% versus 13.30% for SSA (−0.371 pp [−0.494, −0.246]), and the interval excludes zero at every lookahead. SepAdapter uses past cached states and the current chunk, without future audio beyond the ASR configuration, and costs 1.4–1.9 ms more computation per step (real-time factor 0.107 vs 0.090 at 80 ms, one stream). We did not directly measure the full end-to-end output latency.
Dual-path versus TCN. The temporal operator inside SepAdapter can be swapped. On FastConformer (seed 42), TCN is 0.102 pp better than DP (CI of TCN minus DP [−0.179, −0.024]). On Whisper (one seed, development sets, different context lengths), DP is 0.934 pp better than TCN at 25k steps (CI [0.208, 1.597]). We keep DP as the common implementation and treat TCN as a tested alternative for streaming. Our module-level results do not depend on DP being the best operator.
Dropout and positional parameters. We retrained a stricter seed-43 DP/FW pair without adapter dropout and without learned positions inside chunks (3,764,416 and 3,764,432 parameters). DP/FW obtain 14.059/14.473% (−0.415 pp [−0.501, −0.328]), with the interval excluding zero at every lookahead; the original seed-43 pair gives −0.368 pp. The gain remains when both factors are removed.