acceptodds
Under review as a conference paper at ICLR 2027

Noise Ceilings for Representational Convergence

15 views

Abstract

The Platonic Representation Hypothesis links increasing model alignment to a shared representation of the world. Yet a rising alignment score can also reflect more reproducible training outcomes. We develop a noise-ceiling analysis that separates training-seed reliability from consensus alignment. For mutual nearest-neighbor overlap and linear centered kernel alignment (CKA), normalized representation objects give an exact factorization of expected alignment, without assuming additive or isotropic noise. Across nine PolyPythia seeds at each of five scales, reliability accounts for 51.3% of the language–vision log-alignment trend; an independent OLMo suite gives 50.7%. Fixed-readout, capability-axis, and replicated-vision controls support a substantial reliability contribution. Fresh extraction of 105 training checkpoints reveals a sharper distinction: Pythia 14M raw CKA alignment rises by 89% while corrected consensus falls by 10%, and the measured training paths differ between Pythia and OLMo. Known-truth experiments expose the limits of correction with few seeds. Raw alignment also predicts held-out retrieval better than corrected alignment, distinguishing consensus from individual-model utility. The resulting protocol measures reproducibility, shared representational structure, and usefulness as separate properties, making convergence claims more precise and testable.

Primary area:
interpretability and explainable AI
Keywords:
Platonic Representation Hypothesis, representational convergence, noise ceilings, seed reliability, representational similarity, training dynamics, multimodal representations, centered kernel alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EMtTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
3(+1)
Read review
Paper summary

The authors apply an interesting idea from neuroscience of noise ceiling estimation to study scale-related trends of representational alignment. They treat independently trained seeds of identical training recipes as repeated measurements. Using this, they show that for mutual kk-NN and linear CKA, the expected alignment between two recipes can be broken down into a consensus and reliability term, akin to how one defines noise ceilings in e.g., fMRI recordings. They test this on PolyPythia and DataDecide/OLMo against fixed vision models and find that approximately half of the increase in language-vision alignment with scale comes from inter-seed consistency in the same model. Last, they examine training checkpoints, run simulations with known ground truth and test whether raw or corrected alignment better predicts held-out retrieval.

Critical strengths

The paper asks an interesting question, in light of increasing interest in the Platonic representation hypothesis, which posits that scale drives convergence. It is great that people are trying to question where this convergence comes from and whether alignment metrics which are commonplace in the community have confounds.

  1. Using simulations is always great, since we know the exact ground-truth structure and allows one to test where a method breaks down. The authors use this to demonstrate that an exact identity does not guarantee good corrected noise ceiling estimates with only a handful of model seeds.
  2. I like that the authors explore an alternative explanation for rising alignment curves -- that seeds of the same recipe become more consistent with one another, rather than models converging to a shared representation through scale alone.
  3. The math for splitting alignment into consensus and reliability terms, and chance-level floors of kk-NN and linear CKA is fairly straightforward and looks correct.
Critical weaknesses
  1. My biggest concern is that a lot of reliability appears even before the beginning of training. In Table 5, Pythia 7070M for example already has a reliability of 0.3700.370, which increases to 0.4030.403 at the end of training. This means ≈92\approx 92\\% of the final reliability is already present before any training takes place. In this vein, reliability does increase over training, but only modestly, and for both Pythia models it peaks mid-training. Another caveat here is that while the authors describe "33 recipes", they differ in much more than scale (e.g., training data, training length). Another thing to note here is that across recipes, at step 00, the larger Pythia model is already more reliable (1414M =0.168= 0.168; 7070M =0.370= 0.370). This would suggest that part of the increase in reliability can be attributed to scale that is intrinsically present in untrained models, rather than to training being more reproducible at scale. There are a few ways I can think of baseline-ing against step 00 (beyond just looking at raw values at the beginning and end of training) to separate learned reliability from reliability present at initialization:

(i) Percentage increase from step 00, although this risks looking exaggerated when starting from lower reliability, as in the smaller Pythia model.

(ii) The share of headroom gained, i.e., how much of the remaining gap to "perfect reliability" is covered by training. This would handle the ceiling at 11 and mirror the paper’s floor calibration a−f1−f\frac{a-f}{1-f}. Doing so would calibrate reliability to an "initialization floor" over and above the chance floor.

(iii) 12log⁡(ρend/ρ0)\frac{1}{2}\log{(\rho_{\text{end}} / \rho_0)} from the log decomposition in Eq. 3, since it would directly give how much of the reliability contribution is learned.

  1. I am not sure that "roughly half" holds across the scale range. The 51.351.3\\% figure comes from fitting a single line to log alignment across all Pythia sizes. However, the authors’ own exploratory split at 7070M (L279–282) suggests two different regimes. Below 7070M, reliability accounts for ≈99\approx 99\\% of the slope, and the corrected alignment scores presented in Table 4 barely change. That is, almost all of the increase in raw alignment at small scales likely comes from seeds being more consistent with each other. Above 7070M, reliability contributes only ≈10\approx 10\\% of the slope while corrected alignment rises clearly (0.0873→0.1184)(0.0873 \rightarrow 0.1184), suggesting that most of the increase at larger scales does indeed come from representational convergence, not from seeds becoming more consistent. There is also a drop in reliability between 160160M and 410410M (0.462→0.432)(0.462 \rightarrow 0.432). If this is the case, the result supports PRH more at the larger end of the model scale spectrum than implied by the authors. Relatedly, since reliability cannot exceed one, its contribution is bounded as models scale. Take for instance the 410410M model, where reliability is 0.430.43. Here, reliability alone can increase raw alignment by at most a factor of ≈1.5\approx 1.5, which makes it hard to see how reliability could explain a trend that is central to PRH. I think the authors should discuss this separately for small and large models, rather than as a single number.

  2. The reliability share also depends strongly on the choice of metric and readout. For example, the OLMo replication is sensitive to the choice of readout. With independently selected layers, the OLMo share closely matches Pythia, but at fixed final layers the two diverge substantially (66.466.4\\% and 50.850.8\\%), even though the Pythia share does not seem to be affected by the choice of readout (Table 1). Do the authors have insights on why this could be the case? Also, the abstract only reports the pairing where the two lineages agree. I would suggest reporting both readouts in the abstract, or at least qualifying that “roughly half” is specific to mutual kk-NN and selected-layer readouts, for a more complete account of the presented results.

  3. The paper unfortunately reads very terse and hard to follow. Most results are followed by disclaimers, some of which are about claims the paper never makes. For example, the main scaling result is followed by "This is evidence that both factors matter in the measured suite, rather than a population significance test" (L271-272). Also, the training-checkpoint section ends with "these five-checkpoint paths establish neither a universal learning phase nor an information bottleneck" (L321-322), and the 7070M split makes the following note: "does not identify a universal transition" (L282). However, no claims of an information bottleneck or universal learning phase or transition are proposed anywhere in the main text. I would suggest stating each finding succinctly, and perhaps adding caveats as an addendum to the existing discussion section, such that a reader can access all such relevant details in a single place.

Questions for authors
  1. At L102, R\mathbf{R} is used as a random variable over training runs, but Lemma 1 only defines Rf\mathbf{R}_f for a fixed model. Is this the same object?

  2. At L106, what does "fresh training randomness" vary exactly? Is it initialization, data order, or both, and is this the same for PolyPythia and DataDecide? Also, are "reproducibility" in the introduction and "reliability" in Section 2 the same quantity?

  3. I could only find "record" and "reference" (L165–169) defined in the appendix. What do they mean here?

  4. How does the decomposition change if layers are selected by corrected rather than raw alignment?

  5. Does corrected alignment predict retrieval for seed-averaged representations? I would expect raw alignment to win for individual models.

Reviewer_i5LAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose an update to the Platonic Representation Hypothesis. The update proposes that at least part of the correlation we see in the PRH is due to a "model reliability" measure, as defined by how similar the representation spaces are for multiple otherwise-identical models trained on different random seeds. They find evidence that ~50% of the PRH is explained by models becoming less random, with the other half being due to other mechanisms.

Critical strengths

This is a good paper with a clear result. Its findings will provoke discussion within the PRH literature and so I recommend acceptance.

The experimental design is careful, which makes the headline results credible.

Critical weaknesses

The paper is hard to follow. I strongly recommend the authors make a pass for clarity.

There are limited scaling tests carried out. It would be good to see if this trend holds or saturates as models increase in size. I would urge the authors to make an attempt to check this with other off-the-shelf models.

Questions for authors

See weaknesses please.

Reviewer_nmkYOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper asks whether the rising cross-model alignment reported in support of the Platonic Representation Hypothesis reflects genuine convergence or simply more reproducible training outcomes. Treating mutual kNN overlap and linear CKA as inner products of unit-norm objects, it factors expected alignment exactly into a consensus term and the reliability of each recipe across training seeds. On the PolyPythia and DataDecide suites, reliability accounts for roughly half of the language-vision alignment trend, and checkpoint analyses show cases where raw alignment rises while corrected consensus falls.

Critical strengths
  1. The question is relevant. A rising alignment score can come from more consistent training rather than closer representations, and the paper makes this distinction measurable.
  2. The decomposition is simple and exact, and the authors check it with fixed final-layer readouts, an independent OLMo lineage and a known-truth simulation.
Critical weaknesses
  1. Limited novelty. The core result is classical attenuation correction applied to normalized similarity objects, as the paper itself acknowledges, and the remaining identities follow directly from bilinearity.
  2. The main conclusion depends heavily on the metric and readout. Reliability accounts for about half of the trend under mutual kNN but only about 15% under CKA, and the OLMo share moves from 50.7% to 66.4% when switching to final layers, so the headline 51% is not a stable quantity.
  3. The evidence is limited for a claim about convergence. Replicated models only reach 410M in Pythia and 1B in DataDecide, far below the scale that motivates PRH, and the paper's own known-truth study shows that the correction can be badly off in this few-seed regime.
  4. The practical value of the corrected score is unclear. On the only downstream task, a small retrieval experiment, corrected alignment predicts performance worse than raw alignment.
  5. The presentation makes the contribution hard to follow. The text is dense with caveats, repeatedly refers to an earlier local protocol and earlier analyses that only appear in the supplement, and states that aggregation and inference were changed after inspecting the results. Figures 2C, 2D and 3A are also hard to interpret.
Questions for authors
  1. What explains the large gap between the reliability shares under mutual kNN and CKA, and which one should be used to answer the PRH question?
  2. Beyond applying attenuation correction to normalized similarity objects, what new theoretical or methodological insight does the framework offer?
  3. How would the conclusions change at the larger model scales that motivate PRH?
  4. Given that raw alignment predicts retrieval better, in which concrete setting should the corrected score be preferred?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.