Refraction: Beyond Axis-Aligned Interpretable Subspaces in Visual Representations

Neuron explanation (NE) is a crucial task for opening the “black box” of deep neural networks, linking neurons to human-understandable textual concepts. Existing research predominantly focuses on single-neuron explanation (SNE), while multi-neuron explanation (MNE) has received less attention, despite evidence that neurons jointly construct various subspaces to encode concepts. In this paper, we aim to discover interpretable subspaces more comprehensively. We first show that axis-aligned discovery in existing MNE methods can overlook interpretable subspaces. These methods rely on highly co-activated neurons, whereas meaningful concepts can also be encoded by weighted combinations of neurons with individually moderate activations, a phenomenon we term mesosemanticity. Motivated by this finding, we propose Refraction, which adaptively constructs new axes as weighted combinations of the original neuron axes to discover and explain multi-neuron subspaces. Specifically, Refraction uses generalized eigendecomposition to contrast representative images within a neuron-activation cluster with similar images outside it, identifying directions with high target-to-background response energy. We also prove that the classical MNE method FALCON is a special case of Refraction. To evaluate MNE, we extend Concept Synthesis (CoSy) to multi-neuron subspaces, termed MN-CoSy, which measures the alignment between subspaces and their textual explanations. We also introduce Effective Interpretable Subspace Count (EISC) to measure the quality and efficiency of discovered subspaces. Extensive experiments demonstrate substantial improvements over existing NE methods. Further analyses provide evidence for mesosemanticity by revealing interpretable subspaces missed by axis-aligned discovery and showing how the learned axes combine contributions from multiple neurons to represent visual concepts.