- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes to study how a hidden layer representation forms during training. At each checkpoint, the representation on a fixed evaluation set is summarized by the centered, trace normalized feature covariance, treated as a density matrix like state . Movement between checkpoints is measured with the Bures distance, and tortuosity (path length divided by initial to final displacement) summarizes how direct the training trajectory is. The quantum Fisher information (QFI), which is the infinitesimal Bures metric, quantifies the local sensitivity of the state to controlled perturbations of known input factors. Pulling this metric back to parameter space yields a representation natural gradient (RepNG).
Empirically, the paper reports four findings. (i) After training, QFI concentrates on task relevant factors, and retargeting the task on dSprites reorganizes the sensitivity accordingly. (ii) Tortuosity increases with the interaction order of the target computation. This holds for sign pattern and parity tasks with MLPs on Gaussian inputs, under matched accuracy and matched duration controls, checkpoint subsampling, and alternative distances, and it transfers to a small residual CNN on dSprites and to XOR tasks over animal/vehicle labels on CIFAR-10 composites (5 seeds). (iii) In a fresh 30 seed sweep with interaction order as a fixed effect, tortuosity predicts representational drift over 30 further epochs () and sensitivity to random weight perturbations (), whereas initial to final displacement predicts neither. (iv) RepNG yields lower tortuosity than Adam and SGD at matched accuracy in a 24 dimensional random subspace of a 6-16-8-4 MLP, while its difference from output natural gradient is not statistically resolved at matched performance ().
Critical strengths
-
A clean, reusable measurement framework, applied with unusual care. Defining the state as a trace normalized covariance gives a bounded object with a principled local counterpart (QFI is the infinitesimal Bures metric), so local sensitivity and global movement share one geometry. The controls are strong for this genre: fixed evaluation sets, within seed pairing, hyperparameters selected on disjoint pilot seeds, permutation tests, checkpoint subsampling (Table 5), alternative distances (App. E.6), and a function preserving reparameterization audit (Table 4). The main ordering reproduces across MLPs, a small residual CNN, and CIFAR-10 composites. This makes the descriptive finding credible and the tool reusable, including potentially on longitudinal neural recordings, where covariance states of population activity are directly measurable.
-
A potentially new prognostic use of training histories (Sec. 4.3, App. G). Within a fixed task, a path statistic predicts later drift () and perturbation sensitivity (), using an appropriate design: fresh seeds, task fixed effects, seed clustered errors, and permutation inference. If this survives stronger baselines (see Weakness 1b), it would justify treating formation history as an object of study in its own right, which is the paper's core aspiration.
Critical weaknesses
-
The two central interpretations are not separated from simpler explanations. (a) Interaction order is confounded with optimization difficulty. Parity targets provide no low order signal and are known to induce long plateaus with hidden progress (e.g., Barak et al., 2022). Higher also means far more time at high loss: Table 7 reports 1.9 versus 175.3 epochs to threshold for versus . Matched accuracy leaves duration unmatched, and matched duration leaves progress unmatched; neither matches difficulty itself. Some comparisons are near degenerate: parity stops after about 2 epochs with ; in Table 2 (CIFAR, seed 4, ), and , apparently a single segment trajectory; checkpoint counts per trajectory are not reported, and CIFAR uses only 5 seeds. Because and is near saturated in the CNN and CIFAR runs (about 1.1 to 1.3), there is essentially path length, so the finding partly reduces to "harder tasks move the representation more overall," which is less surprising. Intermediate orders are weakly separated in the matched duration control (Table 6: vs. at 60 updates, vs. at 40 updates, both inverted). No trajectory level analysis supports the "intermediate construction and reorganization" account; Figs. 1 and 5 are schematics. (b) The claim that the path carries information the endpoint does not is tested only against initial to final displacement, a quantity with no a priori reason to predict drift. Natural competitors are not examined: path length alone, recent step size or velocity near the end of training, final loss, gradient norm, a sharpness proxy, and spectral properties of the final state . Continued motion of a not yet converged representation would produce the observed associations without any role for path shape. This weakness is consequential because, if these simpler explanations hold, both headline claims reduce to known phenomena.
-
The intervention, theory, and fit to the chosen area are limited. RepNG is tested only in a 24 to 48 dimensional random subspace of a roughly 284 parameter MLP stopped at 72% accuracy. At matched performance it is indistinguishable from output natural gradient (), and the headline 29% reduction is relative to Adam, whose per coordinate normalization is arbitrary in a randomly rotated subspace. Since RepNG explicitly minimizes per step representational movement, shorter paths are largely built into the method, and its smaller endpoint displacement ( vs. for Adam, Table 8) is consistent with simply moving less overall. The paper never shows that lower tortuosity buys anything, not even the stability that Sec. 4.3 associates with it. Theorems 1 and 2 and Proposition 3 are standard results from the cited literature. The claimed local to global unification remains formal: the QFI analysis and the path analysis are never connected empirically. Finally, for the selected primary area (neuroscience and cognitive science), the paper contains no neural or behavioral data and does not engage with directly relevant work such as representational drift, mathematical theories of learning dynamics (e.g., Saxe et al., 2019), or classic category learning paradigms that manipulate interaction structure (Shepard, Hovland & Jenkins, 1961).
Questions for authors
-
Difficulty matched control. Can a lower order task be made as hard to learn as a higher order one, for example with tuned label or input noise so that loss curves match , and can same tasks be run at different difficulties? Does then track interaction order or difficulty?
-
Stronger baselines for Sec. 4.3. Does remain predictive after adding , recent step size, final loss, gradient norm, a sharpness proxy, and spectral properties of as covariates? Please report coefficients with confidence intervals and per scatter plots.
-
Trajectory composition. How many checkpoints enter each CIFAR and dSprites CNN trajectory? Is Table 2, seed 4, a single segment? Do the CIFAR results hold at matched duration, at sub epoch resolution, and with more seeds?
-
Mechanism. Can you plot distance to the final state and step size over training, to show whether the extra path length comes from plateau wandering or from genuine reorganization events?
-
Intervention. Do RepNG trained networks show lower subsequent drift or perturbation sensitivity, as Sec. 4.3 would predict? How many runs were excluded as not "successful matched runs" (App. A.6)? Given that the optimizer alone shifts by about 0.6 to 0.8 (Table 8), comparable to the whole range in Table 5, does the ordering hold across optimizers, learning rates, and batch sizes?
-
Which layer defines the state in each MLP experiment, and what sample sizes are used for the tasks in App. A.4?
-
On the unit trace slice, the geodesic length is the Bures angle rather than the chord , so a geodesic trajectory at the values in Tables 1 and 2 already has to . Does any conclusion change if the angle is used instead of the chord?
If Q1 and Q2 are addressed and the claims survive, I would raise it to 3. Suggestions for future work: consider stimulus indexed representations (Gram or shape metrics), since is invariant to which stimulus maps to which feature direction; consider the Shepard, Hovland, and Jenkins category types, which dissociate interaction order from the number of relevant dimensions and come with a known human difficulty ordering; and apply the method to neural population recordings collected during learning, where representational drift is directly observable.