- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies multilingual representation geometry and its connection to language drift and latent communication. The authors use contrastive SVD to separate language-specific directions from a shared reasoning subspace, and propose training-free steering with depth scheduling and magnitude normalization to reduce drift. They also introduce a latent hub connecting models with different hidden dimensions and controls that distinguish content transfer from fixed prefix cues. Experiments across models and languages examine these methods and relate representation distortion to tokenization and communication performance.
Critical strengths
-
Meaningful research scope: The paper studies multilingual representations in both single-model generation and multi-agent communication. It examines their connection to language drift and information transfer. I think the controlled prefix experiments are useful for distinguishing instance-specific content transfer from input-independent cues. These controls help clarify whether apparent communication gains reflect actual information transfer.
-
Broad empirical coverage: The evaluation covers languages with varying resource levels and multiple model families. It also uses complementary geometric and behavioral metrics. This breadth helps assess whether the observed patterns extend beyond a particular language, model, or measurement.
Critical weaknesses
-
Validation could be more complete: The geometric analysis shows associations, but provides limited evidence that distortion explains both failures. The steering results also leave some uncertainty, as the full method reduces drift but hurts accuracy on some models, while the gated variant preserves accuracy but lacks corresponding drift results in Table 2. A more direct evaluation of these claims would strengthen the paper.
-
The two research directions could be better connected: The steering and latent communication components are interesting, but their connection is mainly supported by cross-language correlations. Testing whether the proposed subspaces affect communication would help clarify the value of the unified geometric explanation.
Questions for authors
-
Effect of pretraining language distribution: Could the observed geometry depend on the backbone’s pretraining language mix? For example, Qwen may have greater exposure to Chinese than some other backbones. Would using Chinese or another well-supported language as the reference change the distortion patterns or steering results?
-
Interpretation of the reasoning subspace: What evidence distinguishes reasoning-specific functionality in from broader language-agnostic semantic processing? The alignment and drift experiments seem compatible with both interpretations. Are there targeted interventions showing that this subspace specifically supports reasoning?
-
Evaluation of baselines and components: Could the authors report drift and task accuracy together for the gated variant and tuned steering baselines? It would also help to evaluate the G/M/P ablations on free-generation drift, since the current factorial measures teacher-forced accuracy. This would clarify which components drive drift reduction and what performance trade-offs they introduce.
I would be willing to raise my score if the authors could clarify these points and provide stronger evidence for the main claims.