Multilingual Latent Representation Distortion Drives Generative Drift and Bounds Latent-Channel Communication Capacity
Abstract
Multilingual large language models (LLMs) internalize conceptual reasoning within an English-centric latent frame, while low-resource languages (LRLs) occupy activation subspaces that are compact relative to English ( under mean pooling; – per token excluding the BOS position). Across seven LLM backbones, Belebele configurations and FLORES+ varieties, we show that this single geometric property is closely associated with two failures previously studied in isolation: (i) intra-model language drift during free-form generation, and (ii) the limited capacity of continuous latent channels for inter-agent communication. First, contrastive Singular Value Decomposition isolates a low-rank language surface subspace from an orthogonal reasoning subspace that is cross-lingually isomorphic to English in a model-invariant way (debiased CKA, Procrustes, and RSA; Spearman's - across six model lineages). Building on this geometry, we design a training-free, depth-stratified and magnitude-normalized steering method that bounds relative layer-wise perturbation. Our method reduces Involuntary Fidelity Loss by on low-resource scripts (Macro IFL on SEA-LION-v3-8B) and by - relative across six backbones, while leaving safety-classification accuracy within confidence intervals. Second, we introduce a specificity protocol for continuous multi-agent latent channels across nine languages, and a Universal Latent Hub that extends continuous communication to heterogeneous model pools with mismatched hidden dimensions. We establish the token-budget crossover where continuous latents outperform literal text on high-fertility scripts. Finally, we examine the unifying bridge: on the six languages shared by both programs, activation distortion and tokenizer fertility are collinear ( under mean pooling), suggesting a single resource-scarcity axis associated with both single-model generative drift and multi-agent channel capacity, a pattern consistent with fertility and accuracy trends across Belebele configurations.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.