- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper identifies phonological interference: multilingual phoneme-level models infer one language from the whole input and impose its phonology, dropping phonemes the other language lacks. The authors measure this via unshared phoneme recall on code-switched and unseen-language speech, trace the cause to a low-dimensional language subspace whose causal editing reproduces the failure, and propose WLE, an inference-time repair that replaces subspace coordinates with estimates from a short local window. WLE removes 34–58% of interference (69% for TTS with span windows) at almost no cost to monolingual performance.
Critical strengths
- Complete causal story: the subspace is built from training centroids, editing it reproduces the failure, and three controls (random subspace, erasure, third-language edit) rule out trivial explanations. Careful interpretability work.
- WLE is practical—no language labels, no external LID, no retraining—and beats the naive sliding window, which costs 1–3 pp PER while WLE stays within ±0.35 pp.
- The unshared/shared recall contrast cleanly separates interference from general accuracy drops, and the phenomenon is shown in recognition, synthesis, and linked to Whisper's language confusion, broadening impact.
Critical weaknesses
- The unseen-language evidence is purely observational: recordings are compared by probe confidence, with no intervention, and WLE is not applied there—precisely the setting the paper motivates as central for universal phone recognizers.
- Narrow code-switching setting: all utterances switch exactly once, both languages are seen in training, SwitchLingua has only 288 utterances, and Spliced FLEURS uses spliced audio with unnatural boundaries.
- References are derived from orthographic text via eSpeak NG; G2P noise may contaminate the unshared-phoneme judgments the metric relies on. This is not analyzed.
- Hand-picked hyperparameters (edit layers 6–8/14–18/14–22, 2 s window) with little sensitivity guidance; WLE's double encoding cost is never quantified.
Questions for authors
- What is the runtime/memory overhead of the second windowed encoding pass?
- Why does WLE help POWSM less (36%) than PhoneticXeus (58%)—the
<unk>tag, CTC head, or encoder architecture? - Can the centroids/subspace be extended to unseen languages, e.g., by interpolating from the nearest training languages? This seems the natural next step.
2 comments
Follow-up to Reviewer HZCy
We thank the reviewer again for the careful reading and positive evaluation. With a few days left in the discussion period, we would like to ask whether our response addresses the reviewer's questions, including the WLE overhead, the window width sensitivity, and G2P noise.
We are happy to answer any remaining questions.
Response to reviewer HZCy
We thank the reviewer for the careful reading and the positive evaluation of our causal argument. We address each point below.
W1. The unseen-language evidence is observational, and WLE is not applied there
We refer the reviewer to the global response.
W2. Narrow code-switching setting
We acknowledge this in the limitations and have expanded the discussion. The single-switch setting is deliberate: the SEP/TOG design needs a span that can be processed with and without the other language.
W3. G2P noise in eSpeak NG references
While G2P may include errors, our findings do not depend on G2P accuracy:
W4 and Q1. Hyperparameters and WLE overhead
Hyperparameters. Rank 15 is the full span of the 16 centroids. WLE reduces interference at every window width from 1 to 4 s, with small monolingual cost at every width (Appendix H), and 2 s was simply the middle choice. We now summarize this in the main text.
Overhead. WLE encodes the audio a second time in 2 s windows, which are independent and are encoded in one batch, and the decoder runs once as usual. On both recognizers this takes about 3× the unmodified encoder's time at any utterance length, and adds at most 11% to peak GPU memory.
Q2. Why does WLE help POWSM less than PhoneticXeus?
A plausible cause is that POWSM is trained to generate a language tag, which may spread language identity across more directions than the centroid span. Finding which architectural and data properties affect interference is an interesting direction for future work, which we discuss in the limitations section.
Q3. Extending the centroids to unseen languages
We agree this is the natural next step. Table 16 shows that the basis must contain the pair's languages to repair code-switching. This suggests that for an unseen language, a target centroid estimated from the nearest training centroids, weighted by the probe's scores, could be substituted.