- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether prominent spectral modes in neural network training are causally responsible for feature learning or merely correlate with it. The authors propose an identification-first framework that requires spectral modes to satisfy separation, stability, and feature-specificity criteria before performing targeted interventions. In a teacher-student model, suppressing the identified Hessian direction significantly slows feature learning compared with matched controls. However, in deeper networks, prominent Hessian modes do not necessarily correspond to task-relevant features. The authors show that a function-space input-sensitivity operator can identify a causally important direction in a parity MLP, while no compact identifiable mode is found in the tested modular-addition transformers.
Critical strengths
-
The paper goes beyond observing spectral modes by directly suppressing them during training to test their causal role in feature learning.
-
The authors compare targeted suppression against random directions, Hessian bulk directions, and energy-matched controls, helping isolate direction-specific effects.
-
The framework explicitly evaluates spectral separation, stability, and feature specificity before intervention.
Critical weaknesses
-
I think the paper only partially answers the question in its title. The authors propose criteria to identify spectral directions and then test their causal role through interventions, but these criteria do not necessarily characterize when a spectral mode is causal. In particular, a direction could fail the identification criteria and still play a causal role in learning. Therefore, I see the contribution more as a framework for testing causality rather than an explanation of when spectral modes are causal.
-
The strongest causal evidence comes from controlled teacher–student and sparse-parity settings. In the modular-addition transformers, no compact spectral subspace satisfies all identification criteria under the tested operators. Consequently, the applicability of the proposed framework to larger, more complex networks and realistic learning tasks remains uncertain.
-
Some operational details are not sufficiently explicit in the main text, particularly the definition of grokking onset, the selection of Hessian bulk-control directions, and the precise conditions triggering prospective interventions. Although additional details are referenced in the supplement, clearer descriptions would improve the reproducibility and interpretation of the experiments.
-
In the function-space experiments, spectral modes are identified using the average gradient outer product (AGOP), whereas the intervention direction is constructed from the parameter gradient of mean directional input sensitivity. The relationship between these two quantities is not fully established, making it unclear whether the intervention directly controls the identified spectral mode or a related feature-dependent direction
-
The framework relies on predefined thresholds for spectral separation, stability, and feature specificity. Although the authors provide calibration and sensitivity analyses, it remains unclear how these thresholds should be determined in practice when applying the framework to a new learning problem. In particular, the paper would benefit from clearer guidance on how to calibrate the identification criteria without relying on task-specific knowledge or extensive preliminary experiments.
Questions for authors
How exactly is the grokking transition defined?
Why is the candidate rank range restricted to 2–50 in the transformer analysis? Could relevant causal structure exist at higher ranks?
How are Hessian bulk directions selected? Are they sampled randomly from the bulk or chosen according to a specific criterion?
How can feature specificity be evaluated in real-world networks where the ground-truth feature direction is unknown?
Can you elaborate in how do you answer the question in your title?