acceptodds
Withdrawn from ICLR 2027

Representations have Trajectories: Characterizing Temporal Representation Dynamics in Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent work has shown that deep neural networks exhibit an implicit low-rank simplicity bias. In this paper, we investigate the questions that naturally arise from this observation. What are the temporal consequences of low-rank simplicity bias during optimization? How does representation geometry evolve throughout optimization rather than only at convergence? Do different neural architectures exhibit distinct geometric optimization behaviors? To answer these questions, we conduct a large-scale empirical study tracking representational geometry throughout training. We find that neural networks follow three recurring geometric trajectories: persistent high-rank, collapse-and-recovery, and persistent low-rank. We further identify Destructive Representation Collapse (DRC) as a pathological regime in which representations collapse to rank-one and fail to recover, leading to a sustained loss of representational capacity. While collapse-and-recovery is consistently associated with successful optimization in controlled settings, experiments on real-world models show that the relationship between geometric trajectories and optimization outcomes is configuration dependent. Overall, our findings suggest that representation geometry evolves dynamically during training and provide a new perspective on how deep neural networks learn.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation dynamics, representation collapse, low-rank simplicity bias, neural collapse, effective rank, stable rank, optimization trajectories, deep learning theory, residual connections, empirical study

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_DmtZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In this work, the authors present an empirical analysis which suggests three possible trajectories of the rank of weight matrices. The authors also identify what they name as Destructive Representation Collapse (DRC), namely a persistent low-rank state which do not increment during training. The authors claim that identification of the category of rank trajectory during training can be used as an effective indicator for the success of training. The core of the work is given by a large series of numerical experiments, mainly concentrated on the exploratory side of the rank dynamics.

Critical strengths

The problem investigated by the authors, namely feature learning and representational rank, is an important problem which deserves attention because of the consequences on our understanding about how deep networks act on intermediate representations.

Critical weaknesses
  • The presentation is hasty and unclear, in my opinion far from being acceptable at ICLR. I believe this level of presentation could already be sufficient to justify immediate rejection, as it would require a complete restructuring to be brought at an acceptable state.
  • The main model proposed is a linear teacher and a general student (is it general? no details are given). This assumption can already explain some part of the results, as linear networks can be exactly represented by certain kinds of architectures (as residual networks) and not by others. No discussion of this kind is included in the work.
  • The main corpus of the work contains a disproportionate amount of technical details on the experimental setting, without any substantial substance on the research question, results and hypothesis to explain what is observed.
  • Literature review is fairly shallow: there is an enormous body of work studying rank collapse in deep feedforward and residual networks which has been almost completely ignored by the authors.

The main criticisms I want to move regarding this work is that, on its entirety, it reads overall as a very large collection of numerical results without a specific aim or a specific conclusion

Questions for authors

See "Critical weaknesses".

Reviewer_2Ep2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies low-rank simplicity bias during the learning trajectory of deep neural networks. Trajectories are classified according to three archetypes: persistent high rank, collapse and recover, and persistent low rank including one-dimensional collapse. A statistical link is shown between the collapse-and-recover archetype and training convergence, which is not a direct consequence of architecture of optimizer used. Results have been tested for synthetic teacher-student data, and CIFAR-10.

Critical strengths

The study is rather extensive within its scope, with over 4000 synthetic runs and 300 CIFAR-10 runs.

Critical weaknesses

My biggest concern with this paper is that I find the motivation for it underdeveloped. The vast majority of the text is dedicated to describing the observations, but there is not a lot explaining why these observations are important. Simplicity bias is mentioned in the introduction, but the paper adds no additional theory to the topic, and mainly contains results on training loss, not test loss, so generalization is not really studied here anyway. The paper mentions trajectory forecasting, which I could see as a potential valuable application, but the results on this are rather limited in scope, i.e. only synthetic MLPs and few stuck runs for vision transformers on CIFAR. I'm also not entirely clear on why DRC is novel, given the extensive literature on representational collapse.

The paper would be stronger if it claimed a specific mechanism linking the collapse-and-recovery observation to performance, and then showed the mechanism holds up to different, potentially falsifiable, experiments. Currently it is hard to rule out the link is not confounded by some less interesting explanation (e.g. models which don't get stuck in learning tend to change their weights more and therefore show more change in rank).

Questions for authors

How does DRC differ from the prior observed representational collapse? What is unique here and what is the significance of this?

What motivates classifying specifically into these 3 architypes? Did they fall out of an unsupervised analysis? And what is the significance?

In figure 1a, archetype 1 is a persistent high rank but also a much smaller drop in loss compared to archetype 2 and 3. Did you plot any other metrics to test if it still learns the task here, and did you try training for longer?

Reviewer_1M7cOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper tracks the stable rank (SR) and effective rank (ER) of layer activations during training.

Critical strengths

Scale and candour. The synthetic grid is large and replicated over three seeds, and the thresholds are explicit. The authors are commendably transparent about limits: archetypes use the final epoch, so the associations are not prospective; DRC is present at epoch 1; and the DRC thresholds were tuned on the same CIFAR sweep used to check them. The paper also includes two useful cautionary examples: the dead-gradient SIREN CNNs, which have high rank but are untrained, and the plain CNN with SR ≈ 1.05 but ER ≥ 10.4 and 90% accuracy (Table S.7). Both show that a single spectral statistic can mislead. This is the main positive I see; it is not enough to carry the paper's central claims.

Critical weaknesses
  1. The central trajectory–outcome associations are largely implied by the definitions or by initialization, so they do not support the paper's interpretation.
    • (a) DRC → failure is near-tautological. Fitting a rank-rr teacher (or separating 10 CIFAR classes) requires every layer — including the min-rank bottleneck layer used to define the archetypes — to carry rank ≥ rr. A network whose bottleneck is effectively rank-one at the final epoch (DRC) therefore cannot succeed, except for r=1r=1 teachers. The paper itself calls chance accuracy "inevitable" for a rank-one representation. So the 0.9% DRC success rate and the p=2.67×10−54p=2.67\times10^{-54} Fisher test largely restate the definition.
    • (b) A2 selects for effective training. A2 requires SR to rise substantially during training, i.e., optimization actually moved the representation. This mechanically correlates with reaching low loss.
    • (c) DRC is an initialization and architecture property, not a trajectory. DRC is present one full-batch step from initialization. It is strongly driven by the rank-one initialization used in half the grid: W=AB/rdinW=AB/\sqrt{r d_{in}} with r=1r=1 makes every weight matrix rank-one at init (DRC 54.4% vs 29.3% under default init). Under default init it reflects the well-known rank collapse of deep unnormalized networks (Poole et al. 2016; Schoenholz et al. 2017; Daneshmand et al. 2020; Noci et al. 2022).
    • (d) Statistical issues. The p-values treat deterministic, designed grid cells as i.i.d. samples, and the cells are pooled across four heterogeneous sweeps (2,268 + 24 + 448 + 1,344 runs). Including archetype — a post-treatment variable — in the regression also biases the configuration coefficients.
    • (e) Possible metric artifact. The paper does not say whether activations are mean-centred before computing SR/ER. If they are not, the dominant mean direction of ReLU-family activations would push SR toward 1 independently of representational collapse. This may explain Table S.7, and it could change the A1/A2/A3 split substantially.
  2. The forecasting and practical claims lack the obvious baselines, and many "failures" reflect untuned setups.
    • Little added information. Configuration alone already gives AUC 0.946. Adding the full trajectory raises it only to 0.966, and that uses final-epoch information.
    • Unadjusted early-milestone ORs. The early-milestone ORs (6.3–12.1) are not adjusted for configuration. No comparison is made with the simplest early-warning signal: training loss, or its slope, at the same milestone. That signal would very plausibly predict "train loss < 1e-3 at epoch 1000" at least as well. Without showing that early rank categories add information beyond configuration and early loss, the "prospective value" claim is unsupported.
    • Untuned failures. Plain SGD uses the same lr = 1e-3 as Adam (adjusted OR 0.0015). SIREN (ω0=30\omega_0=30) is run with default or rank-one init rather than the initialization it requires (Sitzmann et al., 2020).
    • Known findings. Several headline "factors" are well known: Adam > SGD at a fixed small lr, residual connections help, and depth hurts unnormalized plain nets.
Questions for authors
  1. Are activation matrices mean-centred before computing SR/ER? If not, please recompute the archetype assignment with centred activations, and report how the A1/A2/A3/DRC proportions and the success associations change. This could change the paper's main results.
  2. Please report cross-validated prediction of eventual success from:
    • (i) configuration + training loss at the 25% milestone;
    • (ii) (i) + the early rank category. Only an incremental gain of (ii) over (i) would support the forecasting claim.
  3. How do the main associations change if you:
    • (i) exclude rank-one initialization;
    • (ii) exclude SIREN and plain SGD;
    • (iii) tune the learning rate per optimizer? Stratified by teacher rank: for r=1r=1 teachers, where a rank-one bottleneck suffices, does DRC still predict failure?
  4. What are the teacher singular values, and how is the absolute loss threshold made comparable across task ranks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.