acceptodds
Under review as a conference paper at ICLR 2027

READING TRAINING DYNAMICS THROUGH REPRESENTATION DEPENDENCE

Abstract

Neural network training continually reshapes how inputs relate to one another in representation space. Tracking these relationships offers a view of learning that prediction error and gradient magnitude alone do not fully describe. We introduce representation dependence (RD), a training signal that aggregates dependence between hidden representations of different inputs. We analyse what RD captures beyond loss and gradient norm, and how changes in training data affect its evolution. In NanoGPT, adding RD features to a baseline combining loss, gradient norms and representation statistics reduces mean absolute error in predicting validation deterioration 250 updates ahead by 13.6%. RD also tracks representation changes under loss-matched label corruption in ViT and separates instruction-tuning fault windows in Pythia. In grokking, changes in RD accompany delayed generalisation after training examples have already been fitted. These findings show that the evolution of cross-example relationships provides useful information for predicting and interpreting training behaviour.

Primary area:
optimization
Keywords:
training dynamics, representation learning, training monitoring, representation dependence, generalization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wjtTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes representation dependence (RD), which is Pearson correlation or mutual information computed from fixed probes, in order to monitor the neural network training. Experimental results demonsrate the effectivenss for forecasting overfitting and accuracy.

Critical strengths

The concept of representation dependence such as Pearson correlation or mutual informatoin is interesting. Using the representation dependence between fixed probes for monitoring the neural network training is the novelty of this paper. Experimental results demonsrate the effectivenss for forecasting overfitting and accuracy.

Critical weaknesses

Discussion and analysis regarding experimental results are insufficient.

  1. There is no result of MI without reason in Figure 4. It would be better to include the result of MI. The effective rank is better than RD_Pearson but there is no reason or description of it.

2.In Table 2, Activation SD (L16) is better than RD-Pearson or RD-MI in Template mixing. The authors should discuss and analyze the result.

  1. In Figure 5, there is no result of gradient norm. It would be better to include the result.

  2. In Figure 5, Mean |r| is better than Mean r, but the reason remains unclear.

  3. Although effective rank worked well in Figure 4, it was not used in other experiments. It remains unclear how the authors select the method in each experiment.

Questions for authors

Please respond to the Weakness I pointed out.

Reviewer_1My8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents representation dependence (RD), a training-time signal that tracks the relationships among hidden representations of input instances. RD has two variants: Pearson-based RD and mutual information-based RD. The paper provided theoretical analysis on RD and empirically evaluated RD on NanoGPT validation deterioration prediction, ViT label corruption, Pythia instruction-tuning fault windows, and grokking.

Critical strengths
  • The experiments indicated that the proposed metric, RD, is useful as an aspect of learning in addition to loss and gradient norm.
  • The experiments covered four different settings: NanoGPT validation deterioration prediction, ViT label corruption, Pythia instruction-tuning fault windows, and grokking.
Critical weaknesses
  • The motivation of introducing RD is not clear. The following aspects should be mentioned in Introduction (section 1).
    • The limitation of loss and gradient norm.
    • Why RD is necessary.
    • More detailed distinctions from CKA, SVCCA, and other representation analysis tools.
  • The theoretical analysis (section 3) uses two-layer networks whereas the experiments use Transformer. There should be sufficient explanation of why the two-layer networks are adopted for the theoretical analysis and the relation between these theoretical results and the actual training dynamics in section 4.
Questions for authors
  • What is the meaning of "fixed probe"? It seems not to refer to a small learned classifier as in mechanistic interpretability but refer to a set of input examples.
  • Why can RD capture the relationships among hidden representations of input instances? For example, Pearson RD does not capture pairwise relationships among instances but averaged centered angular alignment. The term "relationships" is generally broader than the scope in this paper.
Reviewer_d9M2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Representation Dependence (RD), as a new signal to track during the training. Its an aggregation of dependence between representations of different inputs throughout the training process. RD score is an aggregation of two entities i.e. Pearson Correlation (for measuring linear alignment) and a Mutual Information Score from a pretrained network called InfoNet (for measuring non linear dependence).

The paper argues that loss and gradient norm tracking is not sufficient enough to completely track the training progress and RD captures information beyond these two signals. It contributes three theoretical results where they show settings where training loss and gradient-norms are identical with different RD in toy networks, a formula to track changes in RD as a response to data updates and estimating RD convergence from a sample of pairs.

Above theoretical contribution is complemented with four experimental settings i.e. predicting overfitting in NanoGPT, response to wrong labels in ViT, separating faulty fine-tuning periods in Pythia and tracking grokking.

Critical strengths
  1. The Pearson version is simple and cheap and can be computed over all pairs very efficiently.
  2. Proposition 1 shows RD can differ when loss and gradient norms are identical across runs.
  3. The authors are honest to disclose how configurations were selected, seeds and report cases where the other signals beat RD
  4. Experimentation across varied use cases, four architectures and training regimes is a very broad evaluation scope.
Critical weaknesses
  1. The motivation for choosing RD over other measures is unclear. Paper shows that RD is capturing a different signal compared to loss and gradient norm, but never explains why tracking RD is useful. it doesn't say how a practitioner use the extra information from RD, what kind of intervention it powers the practitioner to take.
  2. Same direction of change in RD accompanies both harmful and beneficial events across experiments and corruption of training data moves RD in opposite direction in ViT and Pythia. The interpretation of RD is not consistent and doesn't help the practitioner take any decision about what kind of next steps they should take.
  3. The MI version is unclear. It treats the coordinates of representation vectors as independent random samples, which they aren't. For Pythia, its dependent on unexplained preprocessing choices like aggregation in 32 groups per token. Also, well-understood measures like HSIC and distance correlation are mentioned in related works but never compared as baselines w.r.t InfoNet as scoring samples incurs roughly 6x cost of a training step in NanoGPT, so a comparison showcasing incremental value addition is needed to justify the high cost.
  4. Overall the text is pretty unclear and hard to read. InfoNet is barely explained in the paper, usage of "probe" without explanation in the introduction, section 3.2 mentions "ordinary" and "alternative" data but they are not explained till section 4, W_c's subscript is never explained, D and P has multiple interpretations throughout the paper. On aggregate, the language is not able to portray a flow of the paper and skips explaining lots of important details.
  5. Overall experiments results portray a weak picture. The NanoGPT forecasting results are pretty weak, as results vary hugely across seeds (deviation of 1.4 on a mean of 2.1), simply looking at validation loss predicts much better, so how is RD adding value? For ViT, effective rank, reacts about as strongly as RD, so incremental value for RD is not clear. For Pythia, score only measures how well a monitor distinguishes a known fault period after training is over, but how does it translate to actions for practitioners during training? In Grokking experiments, RD changes at the same time as test accuracy, not before, adding no incremental value compared to accuracy tracking.
  6. Each experiment uses a different version of RD and configurations often chosen on development data. There is no single recommended setup that is tested across all four settings, so a practitioner won't know what to use
Questions for authors
  1. What would a practitioner do differently because of RD? Can you show a usecase where it informs a decision that loss, gradient norm and other basic metrics can’t?
  2. How does RD-MI compare to HSIC or distance correlation on the same coordinate pairs?
  3. In grokking or any other experiment, does RD changes before test accuracy with enough lead time to dictate any intervention and how does it compare to other standard metrics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.