acceptodds
Withdrawn from ICLR 2027

Trajectory-Induced Modes: Decomposing Layerwise Representation Flow in Deep Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Deep neural networks progressively transform internal representations across depth, yet how representational structure persists through this process remains poorly understood. Existing analyses characterize layerwise content, cross-layer similarity, or output decodability, but do not directly characterize persistence as a property of representation flow. We introduce Trajectory-Induced Modes (TIMs), which models stimulus-specific hidden-state trajectories through a reduced spectral operator and uses eigenvalue magnitude as an operational measure of cross-layer persistence, distinguishing attenuated, preserved, and amplified components. Across 108 frozen pretrained models spanning vision, audio, and language, TIMs reveals prevalent persistent modes with systematic modality-dependent profiles. Language models are dominated by highly persistent components, vision models exhibit broader mixtures of attenuation and amplification, and audio models show an intermediate pattern. Crucially, TIM persistence is not merely a spectral property. Higher-persistence components retain substantially more information aligned with terminal representations, and removing them from intermediate activations produces larger task-loss increases than removing transient or random TIM-subspace components. Higher persistence is also associated with stronger cortical alignment in audio and language, with more limited gains in vision. These findings remain consistent across perturbation, layer subsampling, rank variation, and residual-structure controls. Together, these results establish cross-layer persistence as a measurable and functionally meaningful property of deep representations, offering a new perspective on how information is transformed across network depth.

Primary area:
interpretability and explainable AI
Keywords:
Representation Dynamics, Deep Neural Networks, Model Interpretability, Brain Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bWNwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Trajectory-Induced Modes (TIMs), a technique for decomposing activation trajectories of individual neural network inputs into components that are transient, persistent or amplified through layers in the model. The approach derives "persistence scores" from an eigendecomposition of an affine surrogate fitted on the activation trajectory of a single input - these scores along with their corresponding eigenvectors are used to construct representative activation-space vectors (TIM representatives) at varying levels of persistence. The authors argue that TIM representatives are both predictive of the final hidden state (hence they capture representational retention) and functionally relevant directions (which they demonstrate via ablation). They also explore prevalence of transient, persistent and amplified modes across the modalities of vision, language and audio in 108 pretrained models, and evaluate the alignment of TIM representatives to cortical responses.

Critical strengths

The analysis covers a real breadth of models and modalities - 108 models across vision, language, audio. They also include many of their results broken down per-model in the appendix which is good to see.

The evaluation is also relatively broad, including a comparison of the distribution of eigenvalues between the three modalities and evaluations of TIM representatives for predicting the final hidden state, functional contributions of these directions to task performance, cortical alignment and robustness to perturbation.

The authors are careful to note what each evaluation does and does not establish and they use this to inform the set of experiments they run, which helps form a coherent narrative, e.g:

Spectral persistence alone does not establish... we therefore...

and

Greater terminal-output alignment does not establish... we therefore...

Critical weaknesses

Soundness concerns that affect some central claims of the paper:

  • It seems the transient representative T=zn(0)T = z_n^{(0)} (and perhaps dTd_T, depending on its definition) may not always actually be "transient" - e.g. if a trajectory had ∣λi∣>0.9|\lambda_i| > 0.9 for all ii, it is unclear what zn(0)z_n^{(0)} represents given that all the modes involved are actually "persistent". This seems especially plausible for certain inputs in the case of language or vision, based on Figure 1.
  • The authors discuss the effect of residual connections on spectral distributions but they do not discuss the implications of residual identity on their results in Table 1 and Table 4, where this may have a significant effect (it seems easy to predict layer N+1 if it is equal to layer N + some small update, and this effect is not isolated in the results).

Additionally, the significance or applications of this work are not immediately clear to me; the paper would be significantly strengthened if the authors could describe a research question or use case for which TIMs are better-suited or more informative than existing techniques.

Questions for authors

In addition to the points raised in the "weaknesses" section, I would like to request a few clarifications:

  • What is the exact construction of dTd_T and dPd_P and how do these differ from the zn(0)z_n^{(0)} and zn(1)z_n^{(1)} in the previous section? I think it is important to add some detail here as these vectors are a key part of the ablation experiment.
  • How does this approach deal with models that have layers of varying width, which is probably the case for many/most models being used here? The paper appears to assume a fixed dimensionality dd across all layers of a given model.
  • When computing the representatives, what is the motivation for deriving them from the depth-averaged representation x‾n\overline{x}_n? Why not derive them from the first layer x1x_1 instead? Do the results in section 4.3 (on representational retention) depend on this choice?
Reviewer_YdoQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates the notion of cross-layer “persistence” and whether it can be used to characterize a network’s internal representation across successive layer transformations. The novelty claim is that existing analyses do not use “persistence” as a property of representation flow; other popular cross-layer methods (CKA, SVCCA) quantify changes in layer geometry instead. To this end, it proposes Trajectory-Induced Modes (TIMs), in which stimulus-specific hidden-state trajectories are summarized by an affine map. Cross-layer persistence is then measured via eigenvalue magnitude to differentiate between “attenuated” (also called “transient”), “preserved”, and “amplified” components. The paper tests 108 frozen pretrained models across various domains (vision, audio, language), and identifies different persistence profiles for different domains. An ablation study suggests components have functional significance: removing higher-persistence components from intermediate activations results in higher loss than removing transient or random components. Higher persistence is also shown to be associated with stronger cortical alignment.

Novelty claim: The Related Work section mentions other techniques that decompose transitioning signals into modes associated with persistence (Dynamic Mode Decomposition and Koopman analysis). This study’s main contribution seems to be applying this to layers instead of time, for a given stimulus.

Critical strengths
  1. Interesting idea for characterizing latent space transformations by decomposing each input’s layerwise trajectory into different types of modes, broad evaluation (108 pretrained models).
  2. Sensible ablation in section 4.4 comparing removal of persistent directions vs transient and random directions.
Critical weaknesses
  1. Clarity and definition of persistence. Little justification is provided for the main method -- the paper jumps straight into formulas (“We fit a shared affine surrogate...”). The manuscript could benefit from a paragraph or two on motivation and intuition for why this eigendecomposition is the right thing to do. The same applies to the choice of trajectory representatives. A related issue: persistence is defined as eigenvalue magnitudes of the reduced operator A_r, but I couldn’t find a conceptual/intuitive definition of it. The paper should have opened with this, to motivate the study. In Section 3.2 the authors state “Persistence characterizes the spectral behavior of a trajectory component under the fitted operator, rather than the invariance of its semantic content or activation across individual layers”, so its relevance is entirely dictated by what the eigenvalue magnitude represents. The paper classifies ranges into transient (|\lambda| < 0.1), persistent (between 0.9 and 1.1) and amplified (> 1.2), but this seems arbitrary and leaves out a large portion unclassified (between 0.1 and 0.9 and between 1.1 and 1.2).

  2. Potential issues with eqs. 1 and 2. (i) Although eq. 1 expresses how each layer’s F_l responds to small perturbations around x_l, eq. 2 does not make use of this approximation, and it is never again used in the paper (the Jacobian does not appear anywhere else in the paper). (ii) I might be missing something, but even if we assume that F_l is well approximated by its affine map around x_l, this doesn’t imply that a single shared map can fit all the layer transitions.

Questions for authors
  1. If a single operator is fitted to all layers together, each eigenvalue then summarizes a direction across the full trajectory. I don’t see how this would be able to distinguish between components that exist, say, between two intermediate layers only, vs components with uniform representation across the full network. Can this method be applied to do that?

  2. As far as I understand, the brain-alignment comparison from Table 3 considers aggregations across all layers. However, classic previous work on brain alignment typically considers the single best-performing layer (e.g.: Yamins et al., PNAS, 2014; Kell et al., Neuron, 2018; Schrimpf et al., PNAS, 2021). Could the best single layer be added as a baseline to Table 3?

  3. Section 4.3 describes a check for whether components retain information that is relevant to subsequent computation. However, it is not clear to me how Table 1 and Figure 2 establish this. Table 1 measures similarity between the output and persistence-weighted representative modes. When lambda is near 1 (persistent), shouldn’t they be similar by design?

  4. The profiles by modality shown in Figure 1 are described in the abstract as a main finding, but the paper itself notes that similar profiles are found for fMRI stimuli and Gaussian-random inputs, and concludes that profiles might be reflecting architecture instead. Could the authors please comment on this confound? Comparing the same architecture across modalities would help.

Reviewer_cTv8OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes an interpretability method that models the trajectory of inputs through the layers with a linear operator. Performing spectral decomposition of this operator allows to find directions which are preserved throughout the network. The experimental results show that persistent directions are more aligned with the final representation and with brain representations across modalities.

Critical strengths

The idea to model the trajectory of hidden states with modes and study their properties seems interesting and with potential, though I’m not familiar enough with the relevant literature to have a strong opinion

Critical weaknesses
  • The paper is very hard to follow, due to excessive jargon typical of LLM writing. Some examples, look at how many words connected by hyphen, almost impossible to follow. This is a general trend which substantially lowers the value of the paper for the community:

    • “We formulate cross-layer persistence as a stimulus-specific representation-flow problem”
    • “a trajectory-restricted spectral framework that operationalizes cross-layer persistence through reduced-operator eigenvalues and constructs persistence-conditioned representatives”
  • The paper results feel like a proposed method and set of experiments, but it’s not clear what the research question is and what insights we can gain from the experiments, or what the paper is trying to claim:

    • 4.2: the paper shows that models have some transient, some persistent and some amplified modalities. But what does this tell us? What is the insight?
    • 4.3 shows us that persistent modes are more aligned to the final representation. But isn’t this obvious? By definition they are the directions that keep being present across layers, so they will be represented also in the final representations, while the transient directions by definition get reduced.
    • 4.4 shows that removing the persistent direction hurts performance. Again, I’m expecting this, and I’m not sure what the insight is?
    • 4.5 shows we can predict alignment barely any better than uniform mean, so again not sure what we are gaining here and what is the insight
Questions for authors
  1. What is the problem the proposed approach is trying to solve (or what is the research question)?
  2. How do the experimental results provide supporting evidence that the proposed approach is a good solution to solve the aforementioned problem?
  3. What are the scientific insights we get from the experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.