acceptodds
Under review as a conference paper at ICLR 2027

Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks

Abstract

Neural networks are known to develop latent representations that are , namely structurally similar across networks trained with different architectures, training protocols, or training datasets. We study this phenomenon in a controlled and simplified setting, where we train an ensemble of networks on regression and classification tasks using training sets perturbed by independent realizations of a noise process. We show that the signal-to-noise ratio (SNR) and the training sample size influence the alignment in qualitatively similar ways in networks trained on real-world datasets and in an extremely simple network with a single hidden layer, for which the alignment can be estimated analytically. Across linear and nonlinear networks, regression and classification tasks, and both synthetic and real-world data, we consistently observe that alignment varies monotonically with SNR and typically non-monotonically with training sample size. In particular, the alignment is minimal near the interpolation threshold. In the linear theory, alignment collapses at interpolation in the ridgeless limit, whereas explicit regularization prevents this collapse and yields a nonzero floor, as observed in nonlinear networks; moreover, in the regularized linear model, the regularization strength that minimizes the generalization error is exactly the one that maximizes alignment. Finally, we show that task performance is an insufficient proxy for representational informativeness: a network with lower test error can retain less of the local geometry of its input. These findings reveal a non-trivial dependence of representational alignment on the quality and quantity of the training data and on the regularization of the learning problem.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Representational alignment, Signal-to-noise ratio, training sample size, double descent, deep linear network, generalization error, Representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nQMROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper is a mostly theoretical paper tackling the question of why neural networks trained in different regimes like different datasets, architectures, or training algorithms still converge to similar representations. The work mostly focuses on theoretical proof in mostly linear student-teacher setups focusing on the CCE metric. The main insight is that in the student-teacher setup CCE is governed by the signal-to-noise ratio.

Critical strengths
  • The paper is written very clearly and hits the right balance between presenting mathematically rigorous results but not overloading notation. This earns the presentation points.
  • The question asked is very timely and important in times of, e.g., "The Platonic Representation Hypothesis (https://arxiv.org/abs/2405.07987). Their main results on linear networks are interesting.
Critical weaknesses

I think it partly misses the point of the wider question the community has. The authors open with the question of how alignment can be similar for "networks trained with different architectures, training protocols, or training datasets". However, their main model is just about dataset size as well as noise. To me, it is not clear how this extends to the more realistic setup of changing architectures or actually different datasets (that are not just different i.i.d. draws, e.g.). The empirical experiment on CIFAR-10 also does not close this gap. One very surprising result (at least to me) is that a nonlinearity can act as a ridge. However, here no theoretical insight or deeper explanation is offered.

Questions for authors
2 comments

We thank the reviewer for the valuable comments and for finding our work timely and interesting. Our responses are below.

1. Why does adding ReLU act as a ridge at the interpolation threshold?

We proposed the implicit-ridge effect of the ReLU as a hypothesis to explain the non-zero alignment at the interpolation threshold, building on the high-dimensional result that the nonlinear component of the activation acts on the linear component as an implicit ridge [1]. Following Reviewer uh36 (comment 2), we tested it directly: we replaced the ReLU with a leaky ReLU and varied its negative slope aa from 0 (ReLU) to 1 (linear) at α=1\alpha = 1. At a=1a = 1, we recover the linear theory: CCE≈0CCE \approx 0 and the test MSE explodes. As the nonlinearity increases (a→0a \to 0), the collapse disappears continuously: the CCE rises and the test MSE decreases. The nonlinearity thus acts as a regularizer of tunable strength, suppressing the small-eigenvalue instability behind the collapse. Likewise, explicit weight decay on the ReLU networks raises the CCE at α=1\alpha = 1 and lowers the error peak, as in the linear theory. These experiments will be added to the Appendix.

2. In the light of works like [5], can you give a broader discussion of how your setup connects to the general findings that large models converge to similar representations?

The Platonic Representation Hypothesis (PRH) [5] posits that neural networks, regardless of architecture, training data, or modality, converge toward a shared statistical model of reality. This convergence can be driven by the training protocol, the architecture, or the training data; we isolate the role of the data statistics. Although simple, our setting yields two insights:

i. Shared ground truth. The alignment between two models is predicted by the product of each model's cosine similarity with the ground truth, because their dataset-specific components are uncorrelated. Independently trained models thus align because each approximates the same underlying reality. This is consistent with [2], which proves that models trained separately on any number of modalities learn the same representation, up to a single rotation, whenever their paired inputs observe the same scene.

ii. Data quality and quantity. Alignment grows monotonically with data quality and, often non-monotonically, with sample size, providing a theoretical counterpart to the empirically supported "convergence via task generality" argument of [5].

3. Can your model be used to analyze what happens in deep linear networks, at least some empirical results?

At the global minimum we analyze, every hidden layer of a deep linear network is a rescaled or orthogonally rotated version of the others, so the CCE is identical across layers. We therefore did not study depth empirically. We expect depth to matter in the learning dynamics, i.e., in how representations develop during training, which remains an open question, as stated in the paper.

4. What role does overparameterization and superposition play [4]?

Overparameterization: In our theory, overparameterization limits alignment precisely. Below the threshold (α<1\alpha < 1), the gradient-descent solution lies in the span of the network's training inputs, so two networks trained on independent data share only the overlap of their data spans, ETr⁡(PaPb)=n2/d\mathbb{E}\operatorname{Tr}(P_a P_b) = n^2/d, with Pa,PbP_a, P_b the corresponding projectors. The correlation is ρ∗=αSNR/(SNR+11−α)\rho^* = \alpha\mathrm{SNR}/(\mathrm{SNR} + \frac{1}{1-\alpha}), which vanishes as α→0\alpha \to 0. Here the data underdetermine the solution, so the implicit bias of training selects it; this explains why a shared initialization increases alignment at small nn in the CIFAR-10 experiments.

Superposition: Our linear theory cannot capture superposition, since linear networks represent at most one feature per hidden dimension [3]. Still, [4] suggests a bridge: in their toy ReLU models, small datasets lead to storing data points in superposition (memorization) and large datasets to storing features (generalization), with the double-descent peak at the transition. Our linear analysis has the same structure without superposition: below the threshold, the learned direction lies in the span of the network's own training inputs and is dataset-specific; above it, the solution converges to the teacher's features, which are shared across datasets. Representations storing data points should thus not align across networks trained on disjoint data, whereas those storing features should, so the alignment minimum would signal the transition from memorization to generalization. This hypothesis can be tested in the direction of pushing the theory beyond the linear case.

[1] https://arxiv.org/abs/1908.05355

[2] https://arxiv.org/abs/2610.39302

[3] https://transformer-circuits.pub/2022/toy_model/index.html

[4] https://www.anthropic.com/research/superposition-memorization-and-double-descent

[5] https://arxiv.org/abs/2405.07987

AuthorsOpenReview ↗

Summary of revision based on Reviewer nQMR review

We have uploaded a revised version of the paper. It contains the following revisions based on your review:

  1. We added the discussion of the leaky ReLU and weight decay experiment results in Sec. 4.2 (Figs. 10–11).
  2. We added the two insights related to the Platonic Representation Hypothesis to the conclusion (Sec. 5).

Thanks

Reviewer_9vWYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how neural networks embed input data and how embeddings learned by independently trained networks align. It examines the effects of signal-to-noise ratio and training sample size through linear theory and nonlinear experiments. Its motivation is closely related to the Platonic Representation Hypothesis.

I am not an expert in this area, so my assessment of novelty and technical depth is tentative. A proper technical revision would require more time.

Critical strengths
  • The problem is interesting and appears relevant to understanding learned representations.
  • Overall, the paper is well written, and the references appear appropriate and relevant.
Critical weaknesses

I think the main limitation is that the paper is primarily theoretical, yet its analysis is restricted to linear networks. The nonlinear evidence is solely empirical and limited to single-hidden-layer networks and a small CNN, with no evaluation on more modern architectures.

Questions for authors

could you test whether the reported alignment properties hold in a ViT, as used in the Platonic Representation Hypothesis paper?

2 comments

We thank the reviewer for the positive feedback and the valuable suggestion to extend our experiments to ViTs.

We would first like to clarify the scope of our work. Our goal is to go beyond empirical observations of representational alignment by identifying analytically tractable mechanisms for how alignment depends on the quality and quantity of the training data. We therefore derive these dependencies exactly in a two-layer linear model and then test whether the predicted behavior persists in nonlinear networks.

We agree that testing a modern architecture strengthens the empirical generality of our findings, particularly given the connection to the Platonic Representation Hypothesis. Following your suggestion, we repeated the CIFAR-10 experiment, replacing the CNNs with a pair of small ViTs (one transformer block, 12,618 parameters). The two networks were trained on disjoint training samples for 500 epochs using Adam with a batch size of 128. We observe qualitatively the same behavior as for the CNNs: alignment increases monotonically as label noise decreases and varies non-monotonically with the number of training samples, with a minimum near the interpolation threshold. While this model is much smaller than the ViTs studied in the PRH paper, these results indicate that the trends we identify are not specific to convolutional or multilayer perceptron architectures and extend to transformer-based models. We will add these experiments to the Appendix of the revised version.

AuthorsOpenReview ↗

Summary of revision based on Reviewer 9vWY review

We have uploaded a revised version of the paper. It contains the ViT results in Sec. 4.3 (Fig. 6) and the experimental details in the Appendix.

Thanks.

Reviewer_uh36OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how signal-to-noise ratio, training sample size, and regularization affect representational alignment between independently trained neural networks. Using Conditional Copula Entropy (CCE), the authors derive analytical expressions for alignment in a two-layer linear model under a rank-one representation assumption. In a simplified linear setting, the authors connect alignment to double descent, showing that alignment collapses at the unregularized interpolation threshold and that regularization prevents this collapse. Experiments with nonlinear networks show broadly similar trends, though with exceptions. The paper also finds that better task performance does not necessarily imply greater preservation of local input geometry.

Critical strengths

Interesting and timely research question. The paper makes a clear connection between representational alignment and double descent. The correspondence between an error peak and an alignment minimum connects prediction-level behavior with changes in internal representations.

The simplified model produces explicit and interpretable predictions. Beyond empirical correlations, the authors derive how alignment depends on SNR ratio and sample size under stated assumptions. The analysis identifies noise amplification near interpolation as a mechanism for alignment collapse. Although restricted, this setting provides a concrete reference against which more complex behavior can be compared.

The regularization analysis provides a substantive result. The paper provides interesting results by showing how regularization prevents alignment collapse. Within the analyzed model, the effective regularization strength that minimizes generalization error also maximizes alignment. This connection between the two quantities extends the qualitative observation that their curves have corresponding extrema.

Critical weaknesses

The theory relies on a restrictive assumption. The analysis assumes isotropic Gaussian inputs, a scalar linear target, and rank-one hidden representations, and consequently representation alignment reduces to the correlation between one-dimentional projections. This leaves unclear the conclusion will hold in higher-rank representations with multiple features and their geometry.

Analyzing the sensitivity of your findings to different input covariance settings and representation rank would strengthen the contribution. The mechanism for nonlinear networks remains unverified. Section 4.2 shows that regularization in a linear model reproduces the nonzero alignment floor observed in ReLU networks. However, the underlying mechanism is not fully justified with this result, and ablation studies with controlled interventions.

The connection to the interpolation threshold needs stronger empirical evidence. The nonlinear and classification experiments identify alignment minima near peaks in generalization error, but do not explicitly establish the interpolation threshold through training-error measurements. This distinction is especially important for CIFAR-10, where the test-error peak occurs around (n/p=0.4). Moreover, the proposed regularization-like explanation for the nonzero alignment floor in nonlinear networks is not directly validated.

Interpretation of representational informativeness. The conclusion that better task performance does not imply more informative representations (Fig. 4) relies on preserving local input geometry as the notion of informativeness. It does not establish that the representation preserving more geometry is more useful for other tasks or retains more semantically relevant information. Transfer experiments would strengthen the justification of this finding.

Questions for authors

See weaknesses.

2 comments

We thank the reviewer for the very positive evaluation and for finding our work interesting and timely. Below we address each of the listed concerns.

1. Sensitivity to input covariance and representation rank.

We thank the reviewer for this suggestion. Our results naturally extend to inputs x∼N(0,Σ/d)x \sim N(0,\Sigma/d) with arbitrary anisotropic Σ\Sigma, and to a teacher with mm outputs, y=Wx+ϵy=Wx+\epsilon, whose rows wjw_j are i.i.d. N(0,σw2Id)N(0,\sigma_w^2 I_d) and ϵ∼N(0,σϵ2Im)\epsilon \sim N(0,\sigma_\epsilon^2 I_m). The CCE becomes CCE(A→B)=m[−12log⁡(1−ρ∗2)−12ρ∗2],CCE(A \to B)= m [-\frac{1}{2} \log(1-\rho^{*2}) -\frac{1}{2} \rho^{*2}],

where ρ∗=SNR/(SNR+1α−1)\rho^* = SNR/(SNR + \frac{1}{\alpha -1}) for α>1\alpha >1, while for α<1\alpha <1 it depends on the spectrum of Σ\Sigma, with α=n/d\alpha = n/d. For Σ=Id\Sigma = I_d and m=1m=1 we recover the paper's results. The main conclusions remain unchanged. For every invertible Σ\Sigma and every mm, alignment increases monotonically with SNR and non-monotonically with sample size, vanishing at the interpolation threshold (α=1\alpha = 1), and any non-zero weight decay restores a floor. For α>1\alpha> 1, alignment is independent of Σ\Sigma, while for α<1\alpha< 1 anisotropy increases alignment relative to the isotropic case: both networks focus on the same large variance directions, which improves both their overlap with the teacher and their alignment.

Two things change: the CCE grows linearly with the number of shared features, and the ridge δopt=1/SNR\delta_{\rm opt} =1/SNR still minimizes the test error for every Σ\Sigma but no longer maximizes the alignment between networks. Neither effect alters our main conclusion. We will dedicate a paragraph to this extension in the revised version.

2. Mechanism in nonlinear networks.

We agree that Section 4.2 only showed that explicit regularization in the linear model reproduces the nonzero CCE floor of the nonlinear networks. As suggested, we performed two controlled interventions:

i. Strength of the nonlinearity. We replaced the ReLU with a leaky ReLU and varied its negative slope aa from 0 (ReLU) to 1 (linear), at the interpolation threshold of the linear network, α=1\alpha =1. At a=1a=1 we recover the linear theory: CCE≈0CCE \approx 0 and the test MSE explodes. As the nonlinearity increases (a→0a \to 0), the collapse disappears: the CCE rises and the test MSE decreases. The nonlinearity thus acts as a regularizer of tunable strength, suppressing the small-eigenvalue instability responsible for the collapse, consistent with the nonlinear component of the activation acting as an implicit ridge on the linear one [1].

ii. Explicit weight decay. We trained the ReLU networks with layer-wise weight decay ω\omega. The results reproduce the predictions of the regularized linear theory: weight decay raises the CCE at γ=1\gamma =1 and lowers the error peaks. Among the values tested, the one minimizing the test error also maximizes alignment, as δopt=1/SNR\delta_{\rm opt}=1/\mathrm{SNR} does in the linear theory.

Together, these analyses confirm that alignment at the interpolation threshold of nonlinear networks is controlled by regularization, whether explicit or induced by the architecture (e.g., the activation function). We will add these results to the Appendix.

3. Identifying the interpolation threshold.

In our experiments, we fix the network and vary the training sample size, moving from the overparameterized to the underparameterized regime. The interpolation threshold is the sample size at which the network stops fitting the training data exactly: below it, the network can interpolate the training samples (overparameterized); above it, the minimum achievable training error becomes nonzero (underparameterized). We did not include training error curves in the submission because this transition is a standard feature of double descent, and our focus was on the relationship between test error and representational alignment. However, we agree that reporting the training error would make the threshold clearer, particularly for the CIFAR-10 experiments. We will add these curves to the Appendix, allowing a direct comparison with the test error peak and the CCE minimum.

Regarding CIFAR-10, the peak at n/p≈0.4n/p\approx 0.4 is not expected to coincide with the parameter count. As shown by [2], the test error peak occurs where the number of training samples matches the effective model complexity, i.e., the largest number of samples the training procedure can fit to near-zero training error.

4. Interpretation of representational informativeness.

We agree that CCE measures the predictability of local neighborhood geometry rather than semantic usefulness in general. Our existing experiments establish that better task performance does not imply greater geometric consistency of the hidden representations. We will revise the paper to avoid conflating geometric information preservation with universal representational usefulness.

[1] https://arxiv.org/abs/1908.05355

[2] https://arxiv.org/abs/1912.02292

AuthorsOpenReview ↗

Summary of revisions based on Reviewer uh36 review

We have uploaded a revised version of the paper that contains your suggestions:

  1. We added the extension to inputs with arbitrary anisotropic covariance and to a teacher with m outputs is now in Sec. 4.1.1, with details of the results in the Appendix.
  2. We added the results of two controlled experiments: the strength of the nonlinearity (Fig. 10) and explicit layer-wise weight decay (Fig. 11) are now discussed in Sec. 4.2.
  3. We added the training loss curves for CIFAR-10 with the CNN (Fig. 12) and the ViT (Figs. 13–14), and for MNIST (Fig. 16).
  4. We revised the text to avoid conflating geometric information preservation with universal representational usefulness.

Thanks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.