Preserving What Matters: Human-Relevant Manifold Geometry of Speech Representations Under Acoustic Intervention

Robust representations should preserve what matters across perturbations. Yet invariance alone cannot determine whether a representation has isolated task-relevant identity or simply discarded information required downstream. Here, we introduce functional geometry under intervention, a framework for testing when representational geometry is functionally meaningful. We hold sentence identity fixed while manipulating acoustic realization through clean speech, noise, and two speech-enhancement interventions that produce opposite effects on human comprehension. We characterize sentence manifolds across 429 frozen model-layer representations and test whether their geometry preserves sentence identity under an acoustic condition excluded from representation construction. We find that late Whisper encoders develop a coordinated geometric regime with greater inter-sentence separability and reduced within-sentence intervention radius and effective dimension. Across seven direct-audio encoders, these geometric properties reliably track leave-one-condition-out identity retrieval, and the same pattern is reproduced by mean reciprocal rank. Pairwise transfer further reveals an important distinction hidden by aggregate robustness measures: some representations preserve identity only between acoustically neighboring conditions, whereas others support identity across the full intervention space. Human fMRI provides an independent constraint. Beneficial enhancement selectively shifts Inferior Frontal Gyrus (IFG) representations toward the geometry evoked by clean speech, while cross-intervention identity preservation and cortical predictivity remain related but non-equivalent. Together, these findings show that robust representations cannot be identified from invariance alone. A representation is functionally robust when the object survives an unseen intervention and remains available in a form useful to a downstream system.