- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces Minimal Viable Subspaces (MVS), a method for identifying low-dimensional activation subspaces that are sufficient to preserve performance on a target “logical inference” task.
Using complementary retention and ablation interventions on Qwen3-8B and Gemma-4-12B, the authors identify subspaces associated with logical inference that exhibit functional dissociations from several other model capabilities.
Their contribution in this reviewer’s opinion appears two-fold:
- They propose a method to extract task relevant subspaces using an unsupervised learning approach, where the subspace is learned by minimizing the KL loss between the output distributions of original and intervened models.
- They apply their method to logical inference, design a synthetic dataset to evaluate this capability, and demonstrate the causal relevance of the identified subspace to the target behavior through retention and ablation interventions.
Critical strengths
-
They sufficiently and clearly motivate the problem and discuss the parallels with human cognition that is understandable to readers in both cognitive science and computer science. Also the positioning relative to previous studies seems well-justified.
-
The paper is generally well-written and understandable with clear explanations and coherent structure. The formulations are clear and the figures are intuitive, clean and understandable and help the paper comprehension tremendously.
-
Retention and ablation experiments provide stronger evidence for functional localization than correlational probing alone (Section 5.2, Figure 4). The effects generalize and their diverse control tasks support functional dissociation (Figures 2 and 4) supporting their narrative. The paper experiments across two model families, which shows the generalizability of the mechanism beyond a single model.
Critical weaknesses
- Novelty seems moderate; their causal subspace approach is not particularly novel. Their approach’s novelty compared to DAS (Geiger et al) and Yuan & Su is not sufficiently established.
- The method’s presentation (Section 3) seems somewhat rushed and would benefit from more detailed preliminaries and explanations. In particular, the authors could better explain the intuition behind the projection operation, and how the learned subspace is optimized and the minimum sufficient rank is selected.
- While the paper mentions their advantage of DAS is in the “unsupervised”ness of their method, the paper lacks direct comparison with the alternatives (especially DAS) in terms of performance. If we ignore the lack of labels problem, does DAS outperform MVS otherwise? This was never sufficiently discussed.
- Despite using multiple seeds during optimization, the paper lacks uncertainty estimates for its main accuracy results and causal intervention effects, making it difficult to assess their statistical robustness.
- Figure 6 is not explicitly referenced or adequately discussed in the main text. They mention in Figures caption that “the activation for Qwen3-8B are linearly separable” but this needs more elaboration, stating what it signifies in the context of the paper.
- Limited validation of logical inference; The primary evaluation relies on a single synthetic benchmark involving one-step, binary conditional inference, limiting evidence that the identified subspace captures logical reasoning more generally. The held-out tests vary vocabulary and logical formulations but do not sufficiently extend reasoning complexity (Section 4.1). This ties closely to their mixed results on PrOntoQA. Notably, both retaining and ablating the identified subspace impair PrOntoQA performance (Figure 4), yet this important observation is insufficiently discussed. As a result, the findings may establish localization of task-specific representations rather than a general logical inference capability. As a final point, it appears that the extracted subspace corresponds specifically to their synthetic dataset of simple, single-step inferences. Extending their experiments beyond this limited setting would significantly strengthen their claims about isolating logical inference capabilities. As of now, this seems like an overclaim. The performance drop on PrOntoQA when ablating (S) (Figure 4) potentially suggests some support for general logical inference behavior in the subspace, but this needs to be investigated further.
Questions for authors
-
Beyond its unsupervised nature, what distinguishes MVS from DAS and Yuan & Su (2026)? Could the authors provide direct experimental comparisons?
-
Could the authors clarify the intuition behind the projection operation, the choice of neutral-corpus mean, and how the minimum sufficient rank is determined?
-
How consistent are the main results and intervention effects across optimization seeds? Could the authors report uncertainty estimates?
-
Can the authors evaluate MVS on more diverse logical reasoning benchmarks, particularly those involving multi-step inference and nonbinary outputs?
-
Why do both retention and ablation impair PrOntoQA performance (Figure 4), and what does this imply about the generality of the identified logic subspace?