acceptodds
Under review as a conference paper at ICLR 2027

When Linear Probes Go Blind: The Order Spectrum of Learned Representations

Abstract

Linear probes are widely used to assess properties encoded in neural network representations. However, poor linear recovery cannot distinguish absent information from information accessible through nonlinear readouts, while commonly used MLP heads do not characterize how information is distributed across orders. We formalize this distinction by introducing the order spectrum, a Hermite decomposition of the target regression function that, under Gaussian representations, resolves predictable variance by polynomial degree. Specifically, we show that optimal affine probes measure the first-degree energy of the spectrum, while increments between optimal nested polynomial probes recover successive energies. For the off-Gaussian case, we bound the discrepancy in terms of the Hermite-feature Gram deviation and a target-dependent approximation residual, and validate the bound in controlled settings. Leveraging this framework, we show that quadratic probes recover substantial board-color information in OthelloGPT and pendulum energy in a JEPA trained without energy labels, where linear probes fail. Further, by analyzing grokking in modular addition, we find that partial linear recovery can coexist with substantial quadratic accessibility, while quantitative Hermite-energy interpretations require explicit distributional and approximation conditions. More broadly, the framework provides a degree-resolved account of how target information becomes accessible in learned representations.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, representation analysis, linear probing, nonlinear probing, polynomial probes, Hermite decomposition

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zxxjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an "order spectrum" to analyze the polynomial degree required to extract target information from learned representations. For Gaussian representations, the paper demonstrates that the regression function of a target can be decomposed into Hermite components; specifically, the R2R^2 of an optimal affine probe corresponds to the first-order energy, while the incremental performance gain of an optimal polynomial probe at each degree aligns with the energy of that corresponding order. For non-Gaussian representations, the paper provides bounds on the error of this relationship by utilizing the Hermite-feature Gram deviation and the target-dependent approximation residual.

The authors numerically validate these bounds within controlled settings and in the posterior mixture distributions of an MNIST β-VAE. Furthermore, they examine cases—such as in OthelloGPT, a pendulum-based JEPA, and models trained on modular addition—where quadratic probes outperform affine probes in predicting targets, and they analyze how the accessibility of this information evolves throughout the training process.

Critical strengths
  1. The conditions for the theory’s application are clearly stated. The error that arises when relating the performance gain of an optimal polynomial probe—within a non-Gaussian representation—to a Gaussian-referenced Hermite energy is bounded by the "Gram deviation" and a target-dependent approximation residual. This formalizes the conditions and limitations under which the interpretation holds.

  2. Numerical validation of the proposed bound is provided. The components of the bound are evaluated using the posterior mixture distribution of an MNIST β-VAE alongside the actual digit labels. Notably, the observation that the bound is violated when the residual term is omitted confirms the necessity of that term.

  3. The scope for interpreting the experimental results is carefully constrained. While predictive performance is compared across different orders in various representation learning settings, the scores obtained from fitted probes are not equated with the calibrated energy ratios. This distinction ensures that the experimental results are evaluated without overinterpretation.

Critical weaknesses
  1. The significance of the additional contributions relative to existing research is not sufficiently compelling. It is already well-established that the failure of a linear probe does not necessarily imply an absence of information, and that additional signals can be extracted using polynomial readouts. Furthermore, the energy relationships across different orders in the Gaussian case are rooted in classical orthogonal expansions and projection theory. Consequently, while the primary technical innovation appears to be the derivation of error bounds for non-Gaussian distributions, it remains to be clearly demonstrated what significant insights or analytical capabilities these bounds offer beyond existing polynomial probing methods.

  2. There is a gap between the validation of the non-Gaussian bounds and their practical utility. While the authors successfully validate these bounds using a posterior mixture distribution from an MNIST β\beta-VAE—thereby demonstrating the necessity of the residual term—the precision provided by these bounds in real-world analytical scenarios is less clear. For instance, at β=2\beta=2, the mean discrepancy is 0.00910.0091, whereas the mean bound is 0.25640.2564, indicating a highly conservative estimate. In the case studies involving OthelloGPT, JEPA, and modular addition, the bounds do not serve to guarantee an energy-based interpretation; rather, they primarily function to diagnose the limitations of such interpretations. While this is a meaningful contribution, the significance of the work would be more persuasive if it included examples where these bounds lead to tangible improvements in analytical judgments—judgments that would otherwise be difficult to reach using conventional probing techniques alone.

  3. Greater control is required to disentangle the effects of "accessibility by order" from the influence of the fitting procedure on actual probe performance. The experiments in the paper evaluate probes that are fitted using regularization on finite samples, rather than utilizing the optimal population-level predictor. Therefore, it is crucial to determine to what extent the observed performance differences reflect the underlying order-based structure of the representations, as opposed to being artifacts of the chosen coordinate system, conditioning, or regularization strategy. As an exploratory check, I conducted a small-scale experiment using three GELU-based models—each with a width of 128 and four hidden layers—trained on the 10-bit full parity task. I compared the representation hh immediately following the second activation layer with the output z=Wh+bz = Wh + b of the subsequent affine layer. All three models achieved 100% classification accuracy on 50 held-out test samples. Following the procedure specified in Appendix G.2 of the paper, I standardized the inputs and employed a quadratic kernel, K(u,v)=(1+u⊤v/128)2K(u, v) = (1 + u^\top v/128)^2. The penalty parameter was selected based on internal validation at the final checkpoint, then refitted to the entire probe-fitting dataset and held constant for earlier checkpoints. Using the sum-of-squared-errors criterion, the quadratic test R2R^2 for the earlier representation was −0.064,−0.087,-0.064, -0.087, and 0.5750.575 across the three random seeds, while for the later representation, it was higher: 0.262,0.683,0.262, 0.683, and 0.9620.962. The respective means were approximately 0.140.14 and 0.640.64. However, substituting z=Wh+bz = Wh + b into the quadratic readout q(z)q(z) for the later representation yields a function q(Wh+b)q(Wh + b) that is also quadratic with respect to hh. By transforming the coefficients, I was able to apply this to the earlier representation and perfectly replicate the predictions obtained from the later one. This indicates that a successful quadratic readout for the earlier representation did exist, but the standard fitting procedure failed to discover it. The observed disparity between the layers persisted even when varying the conventions for the penalty term (sum vs. mean of errors) and the handling of the quadratic intercept. These findings demonstrate that the scores obtained from fitted probes can depend significantly not only on the polynomial degree but also on the choice of coordinate system and regularization. Consequently, when interpreting differences between representations based on performance at various polynomial degrees, conducting sensitivity analyses regarding affine transformations and regularization serves as a valuable supplementary control. Since the authors explicitly state that a low fitted score does not imply the failure of the entire function family, this result does not constitute a counterexample to the theorem. The core of the critique is not whether the authors acknowledged these limitations, but whether they sufficiently evaluated the magnitude of their impact in actual experiments. It should be noted, however, that this additional experiment does not represent a full replication of the authors' implementation; it is an exploratory result limited to three initializations and a single data split.

Questions for authors
  1. Can you provide examples where non-Gaussian bounds improve the assessment of learned representations? While I acknowledge that the necessity of bounds—and the residual terms dependent on the target—has been validated for MNIST β-VAE, in some settings, these bounds appear significantly more conservative than the actual discrepancy. Beyond merely confirming numerical validity, under what conditions do these bounds provide a quantitative guarantee for interpreting probe performance at different polynomial degrees, or enable insights that would be difficult to obtain through standard polynomial probing alone?

  2. Is it possible to demonstrate applicability to higher-order interactions using tasks where the interaction order can be controlled, such as the parity problem? Current empirical results focus on low-order polynomials, specifically comparing affine and quadratic models. For tasks like parity, one could distinguish between the order of the input-target relationship and the order of the readout used for the learned representation, thereby testing whether higher-order probes can recover additional signals. Analyzing how sample size and computational cost scale with the order—along with the behavior of ρk\rho_k, τk\tau_k, and the validity of the bounds—would help clarify the scope of the proposed framework.

  3. How can the influences of the coordinate system, conditioning, and regularization be disentangled from the fitted performance at each order? I agree with the authors' point that a low fitted score does not necessarily imply the failure of an entire family of polynomials. Given this, what controls do you recommend for assessing the reliability of observed performance gains when using this framework as a diagnostic tool? Specifically, can you confirm that your primary findings and interpretations remain robust when accounting for factors such as affine transformations of the representations, data whitening, or sensitivity to regularization?

2 comments

Thank you for the careful review, and especially for running your own parity experiment. Section numbers refer to the revised PDF.

W1. The contribution over existing work isn't compelling. A failed linear probe is known not to imply missing information, and the Gaussian energy relations are classical.

We agree with both points, and the revision mentions this. Section 2 now attributes Proposition 1 to classical Gaussian analysis (Brillinger, 1982), and we don't claim that "linear fails, nonlinear succeeds" is new. What we add is a measurement and its conditions. The order spectrum is now defined as the increments Δk\Delta_k between nested population polynomial probes under the representation's own law, so it assumes nothing about the distribution. We then say exactly when these increments equal Gaussian Hermite energies, bound how far they can move off the Gaussian, and give the controls fitted probes need before such increments are read (Section 2.1, Appendix H). The OthelloGPT result under W2 is an example of what this gives beyond a higher score.

W2 / Q1. There's a gap between validating the bound and its practical use. It's conservative (at β=2\beta=2 a discrepancy of 0.0091 against a bound of 0.2564), and in the case studies it mainly diagnoses limits. Where does it change a judgment that standard probing couldn't reach?

We agree with your reading, and the revision now says it directly. On the OthelloGPT and pendulum activations the bound is loose, so we use Proposition 2 as a stability result relative to the Gaussian reference and not as a certificate for fitted scores (Section 3). It still changes one judgment that would otherwise be made. Gaussian-regularized representations such as the SIGReg JEPA are easy to read as near-Gaussian, but their measured Gram deviation is 6.230 to 8.698, far above one (Section 7), so a Hermite-energy reading of their probe increments isn't justified even though the regularizer targets the Gaussian. We also added a case where theory gives a number that probing alone can't. A sign-product proposition for OthelloGPT (Section 6.1, Appendix E.7), which needs no Gaussian assumption, bounds degree-two accessibility of board color from below at 0.74, 0.81 and 0.82 at layers 2 to 4, while the fitted quadratic probe scores 0.56, 0.64 and 0.66. So the fitted probe loses 0.16 to 0.18 to estimation, which no fitted score could show on its own.

W3 / Q3. Accessibility by degree must be separated from the fitting procedure. What controls do you recommend, and do the main findings survive affine maps, whitening and regularization?

Your experiment shows exactly the failure mode the revision now tests for, and transport through a known map, as in your experiment, is step 5 of the protocol for fitted probes in Appendix H. Each main finding is checked against the controls in Table 10, where W means the claim holds but a number moves by 0.05 or more, or the selected penalty sits at a grid edge with validation still improving. Cells without a reported number are marked as such.

  • Grokking (Appendix G.7). Rotations and affine maps with condition numbers 10 and 100 change final scores by at most 0.014, whitening lowers the final pre-MLP quadratic by 0.04 to 0.11, and a widened penalty grid raises the final affine score to at most 0.64. In every weight-decay run and under every variant, the final pre-MLP quadratic stays at least 0.28 above the affine score. Scores before the transition move by up to 0.54 under affine maps, so we read the trajectory only qualitatively.
  • OthelloGPT (Appendix E.8). The gap between affine and quadratic probes holds under every control, but its size depends on the fit. At layer 4 the quadratic scores 0.78, 0.74 and 0.63 under a rotation and affine maps with condition numbers 10 and 100, 0.25 after fitting-set whitening, and it rises from 0.06 to 0.66 with more fitting games, so we report these scores as lower bounds.
  • Pendulum (Appendix F.6). The SIGReg gap survives every control we ran.

We haven't run the transport check on our own models, and the protocol explains it (Table 10 has no transport column).

Q2. Can the framework be shown on higher-order interactions with a task of controlled order, such as parity, including how sample size, cost, ρk\rho_k, τk\tau_k and the bound scale with order?

We didn't add a parity experiment in this revision, but agree it would be a good test of degrees above two and note it as future work. Within the current paper, degrees above two are covered by the controlled checks in Section 4 and Appendix C, where the nested ladder recovers a target with known energies up to degree three and the degree-four increment stays near zero (−0.002-0.002). Appendix H.1 now also reports cost, i.e., the degree-kk feature count (d+kk)\binom{d+k}{k}, the fitting times and memory, and why sample size, not compute, limits the full-space quadratic probe.

Thank you for the response and the added control protocol. While I appreciate the more transparent framing, my core concerns remain unresolved:

  1. Practical utility of Proposition 2: The non-Gaussian bound remains loose in real-world settings and still only serves as a negative diagnostic, while the new OthelloGPT sign-product bound does not use Proposition 2 at all.
  2. Fitting sensitivity: While the final grokking gap and the existence of the OthelloGPT gap persist, the added controls still confirm strong sensitivity of the estimates to coordinates and regularization (e.g., up to 0.54 shifts in pre-transition grokking scores under affine maps, and large magnitude drops under whitening in OthelloGPT), and the transport check was not run on the paper's models.
  3. Higher-order scope: Empirical evaluation on trained networks remains limited to degree-2, with higher-order tasks deferred to future work.

Accordingly, I will maintain my current rating.

Reviewer_hNBQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors develop a framework of nested polynomial regression probes to expose information in hidden layer activations that are invisible to traditional linear probes. A theoretical framework is provided in sections 2 and 3, and several empirical case studies are provided in sections 4 to 8.

Critical strengths
  • The paper is nicely written and organized
  • There is a nice range of experiments and demonstrations of the framework. (However, this does come at the cost of not going very deep into any single application.)
  • Figure 4a-b shows an interesting finding where a quadratic probe essentially saturates the decoding performance of an MLP
Critical weaknesses
  • The mathematical framework is overly formal and adds unnecessary complexity, in my opinion. In section 2, the authors set up the order spectrum (sks_k, definition 1) as the main quantity of interest, but in most of the applications they just look at the increments in R2R^2 on the test set (Δk\Delta_k, defined at the bottom of page 3 in a paragraph). The Δk\Delta_k values are easier to interpret and they don't make any assumptions on the underlying data distribution.

  • The sks_k and Δk\Delta_k values must be estimated from finite data and the curse of dimensionality bites very hard in hidden layer activation spaces. In practice, the authors seem to fall back on regularized probes (e.g. kernel ridge). The optimal degree of regularization will change with the degree of the polynomial, so they are no longer a nested set. The theory developed in sections 2+3 doesn't cover how one should think about regularization and finite data effects.

  • The VAE analysis seems to just be a validation of proposition 2, which is nice but doesn't show any mechanistic insights into the VAE.

  • The other results seem to basically be showing that regularized nonlinear probes (e.g. quadratic) outperform linear probes, which I don't think is necessarily that surprising or novel. The authors already cite Oldfield et al. (2026) as another paper that uses nested polynomial probes. Additionally there is:

Questions for authors

The Othello-GPT results could use some more interpretation, in particular, any additional benefits of the quadratic probe. For example, does the quadratic probe recover the interaction between player-relative board information and whose turn it is, which could provide an argument for interpretability compared to MLPs? Also, why does the quadratic performance drop so dramatically in the final layer (Figure 4a)?

1 comment

Thank you for the review. We have revised the draft based on your suggestions on Δk\Delta_k and OthelloGPT. Section numbers refer to the revised PDF.

W1. The framework is overly formal. The order spectrum is set up as sks_k, but the applications use Δk\Delta_k.

We agree, and we've restructured the paper around this. In the submission, "order spectrum" named the Hermite energies sks_k, while the experiments mostly read off Δk\Delta_k. The order spectrum is now defined directly as the increments Δk=R≤k2−R≤k−12\Delta_k=R^2_{\le k}-R^2_{\le k-1} between nested population polynomial probes under the law μ\mu of the representation, with R2R^2 and R≤k2R^2_{\le k} defined formally (Section 2). We kept sks_k, but only as a Gaussian reference, since its basis is explicit, shared across representations, and recent works have established theoretical connections between representation learning, uniformity, and Gaussianity, Betser et al. (2026), Domb (2026), Wang and Isola (2020) and Δk=sk\Delta_k=s_k when the representation is Gaussian (Proposition 1, now extended to every kk in Appendix A).

W2. Finite data and regularization.

We agree that the theory is about population quantities, and that fitted, regularized probes aren't nested in general. The revision now states this in Section 2. Least squares with many more samples than features, as in our synthetic, VAE and JEPA probes, estimates R≤k2R^2_{\le k}. The full-space OthelloGPT and grokking quadratic probes have more features than samples (131,841 for 41,300, and 8,385 for 2,634). Their regularized held-out scores are at most R≤k2R^2_{\le k} in expectation, so the difference of two such scores is a difference of lower bounds and not an estimate of Δk\Delta_k. Section 2.1 and Appendix H add controls for fitted probes, and Table 10 checks each main finding against them, including penalty grid edges, sample size, affine maps and whitening.

W3. The VAE analysis.

That's right, and it's what the section is for. The VAE is the one learned representation where the residual τk\tau_k can be computed exactly, so it checks Proposition 2 on a learned representation, not how VAEs work. The revision says so and moves the details to Appendix D. It shows that the bound has no violations beyond numerical tolerance across 500 admissible checks, while dropping the residual terms violates it in 205 of them (Section 5).

W4. The other results mostly show that nonlinear probes beat linear ones, which isn't novel.

Thank you, Pimentel et al. (2020) and White et al. (2021) are now cited alongside Oldfield et al. (2026). We agree that "nonlinear beats linear" alone isn't a contribution, and it isn't our claim. What we add is a degree-resolved account, i.e., what the increments between nested polynomial probes measure under the representation's own law, when they equal Gaussian Hermite energies, and how far they can move from them off the Gaussian. Q1 below shows what this adds beyond a higher score.

Q1. Does the OthelloGPT quadratic probe recover the interaction of player-relative board state and whose turn it is, arguing for interpretability over MLPs?

Yes, and this is one of the clearest results in the revision (Section 6.1). Board color is the product of whose turn it is and the player-relative state. A product of one affine next-mover readout, shared by all squares, and one affine player-relative readout per square reaches R2≈0.86R^2\approx0.86 at layer 4 and stays within 0.01 of the MLP at layers 0 to 3 (Appendix E.6). Unlike the MLP, it reads color from two named directions. The fitted quadratic tracks this product, its test predictions correlate with the product's at a median rr of 0.74 to 0.89 per square at layers 0 to 6, and adding it to the product raises test R2R^2 by a median of at most 0.002. It scores below the product, consistent with having more features than fitting samples, and it still improves with more data, from 0.06 with 88 of 700 fitting games to 0.66 with all at layer 4 (Appendix E.8), so we report its scores as lower bounds. A sign-product proposition (Appendix E.7) explains why affine probes see almost nothing and bounds degree-two accessibility from below at 0.74, 0.81 and 0.82 at layers 2 to 4.

Q2. Why does the quadratic performance drop so dramatically in the final layer (Figure 4a)?

It's a fitting failure, not a loss of information, since at layer 7 the product still reaches 0.71 and the MLP 0.85. Every penalty in the quadratic's search gives negative validation R2R^2, and the largest penalty is selected on all three splits (Appendix E.6). Refitting without the first post-move position leaves the quadratic at −0.024-0.024 (Appendix E.4), and fitting-set whitening only raises it from −0.03-0.03 to 0.130.13 (Appendix E.8). The cause remains open, and the paper says so.

Reviewer_C79SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of probing the hidden layers of a neural network from a theoretical point of view, framing the prediction task from a Hermite decomposition perspective. The analysis starts from defining the Gaussian base case, where the distribution of the hidden layer is taken to be Gaussian. Here it defines the order spectrum of order k as the square of the k-th Hermite coefficient of the predictor divided by the variance of the label, and identifies it as the difference between the explained variances using polynomials of order k and k-1. It then gives its main theoretical contribution, which is a bound on the distance between these Hermite coefficients and the differences between the explained variances defined above in the case where the distribution of the activations are non-Gaussian. Finally, it performs experiments on 4 probing tasks, measuring the explained variance of affine, quadratic and MLP probes for each layer. These experiments show that in many cases a quadratic probe performs better than the affine one, and are close to the MLP in terms of explained variance.

Critical strengths

The conceptual framing is clear, and proposition 2 is to the best of my knowledge correct and interesting.

Critical weaknesses

I find the main weakness of the paper to be the lack of connection between the theory and experiments. Indeed proposition 2 is valid under a hypothesis (ρ2≤1\rho_2 \leq 1) which doesn’t hold in 3 of the 4 experiments. While the authors have been honest about this mismatch, this makes me wonder how necessary the whole theoretical framing is, and whether this could be more a paper on the limitations of affine probes and the effectiveness of quadratic ones.

Questions for authors

-) The explained variance R2R^2 is never defined explicitly in the main text, nor are the quantities R≤kR_{\leq k}. While it is a rather standard measure, and the meaning of R≤kR_{\leq k} can be understood from the text, it would help the presentation to have a formal definition.

-) The Hermite polynomial approach to neural network theory is not new, and many important works are not cited. For example: Ghorbani, Behrooz, et al. "Linearized two-layers neural networks in high dimension." (2021): 1029-1054. Arous, Gerard Ben, Reza Gheissari, and Aukosh Jagannath. "Online stochastic gradient descent on non-convex losses from high-dimensional inference." Journal of Machine Learning Research 22.106 (2021): 1-51. Abbe, Emmanuel, Enric Boix Adsera, and Theodor Misiakiewicz. "Sgd learning on neural networks: leap complexity and saddle-to-saddle dynamics." The Thirty Sixth Annual Conference on Learning Theory. PMLR, 2023.

-) In the above works Gaussianity is obtained via CLT-like properties of networks in the large dimensional limit, while this does not seem to hold here. I wonder what the advantage is of building a theory that uses as a reference the Gaussian measure, when the deviation from such a measure is measurably large in many settings. Discussing with Claude Opus 5.5, another possibility could be to define the μ\mu-adapted order spectrum where the projections are defined in the space L2(μ)L^2(\mu). Have the authors considered this approach? What makes the Gaussian base measure a good choice? Clarifying this point in the main text would make a good addition.

1 comment

Thank you for the review and for the concrete suggestions.

W1. Proposition 2 is stated under a hypothesis that fails in most experiments, which weakens the connection between the theory and the experiments.

You note that Proposition 2 was stated under ρ2<1\rho_2<1, which fails in most experiments. The submitted Appendix B already noted that the bound holds whenever the Gram matrix is positive definite. The revised statement makes this explicit, and Proposition 2 needs only an invertible Hermite-feature Gram matrix, so it applies to every experiment. ρk<1\rho_k<1 marks the regime where the bound keeps the relative discrepancy below one (Section 3). We agree that on the OthelloGPT and pendulum activations the bound is loose. The paper now says this directly and uses Proposition 2 as a stability result relative to the Gaussian reference, not as a certificate for fitted scores. The experiments rest on held-out comparisons of fitted probes, and the revision adds the controls that such comparisons need (Section 2.1, Appendix H). In the revision we also added a link between theory and experiment that needs no condition on ρ\rho. A sign-product proposition (Section 6.1, Appendix E.7) predicts the OthelloGPT affine score of about 0.004 (observed −0.002 to 0.004). It also bounds degree-two accessibility from below at 0.74 to 0.82 at layers 2 to 4, above the fitted quadratic scores of 0.56 to 0.66.

Q1. Can the main text give formal definitions of R2R^2 and R≤k2R^2_{\le k}?

Both are now defined formally in Section 2, and the order spectrum is defined directly as the increments Δk\Delta_k of R≤k2R^2_{\le k}.

Q2. The paper omits important prior work using Hermite polynomials in neural network theory.

Thank you. Related work now cites Ghorbani et al. (2021), Ben Arous et al. (2021), Abbe et al. (2023) and Damian et al. (2022). It notes the difference you point out. Those works choose a Gaussian or spherical input law, whereas we measure trained representations whose law is not Gaussian.

Q3. Have the authors considered a μ\mu-adapted order spectrum, and what makes the Gaussian measure a good reference when the observed deviations are large?

The increments Δk\Delta_k are exactly the μ\mu-adapted spectrum you describe. They are the energies of ff in the polynomial basis orthogonalized in L2(μ)L^2(\mu) by degree, and σ2Δk\sigma^2\Delta_k is the variance explained by the degree-kk block after residualizing on lower degrees (Appendix B, the risk identity for Δk\Delta_k). Thus, they require no Gaussian assumption, and the revision makes Δk\Delta_k the primary object.

We retain the Gaussian energies sks_k as a reference for two reasons, now clarified in Section 2, Related Work and the Conclusion. First, the Hermite basis is explicit and shared across representations, whereas a μ\mu-orthogonal basis depends on the representation law. Second, recent theory motivates Gaussian geometry in learned representations. Betser et al. (2026) establish asymptotic Gaussianity of projections of InfoNCE representations under alignment and concentration assumptions; Domb (2026) develops an information-theoretic argument favoring isotropic Gaussian representations; and Wang and Isola (2020) characterize contrastive learning through alignment and hyperspherical uniformity. Gaussian priors and regularization also motivate this reference in VAEs and SIGReg-trained JEPAs.

These results do not imply Gaussianity of arbitrary learned representations. Indeed, we observe substantial deviations, with ρ^2>6\hat\rho_2>6 for both OthelloGPT and the SIGReg-trained pendulum JEPA (Section 3). This motivates Proposition 2, which does not assume ρk<1\rho_k<1 and quantifies the discrepancy between Δk\Delta_k and sks_k.

Comments on the paper

1 comment

We thank all three reviewers for careful and constructive reviews, which changed the paper quite a bit. The revised PDF is uploaded, and the main changes are below. Section and appendix numbers refer to the revised PDF, and each reviewer's points are answered in the replies under their reviews.

1. The measured object comes first

In the submission, "order spectrum" named the Hermite energies sks_k. Following hNBQ, it now names the increments Δk=R≤k2−R≤k−12\Delta_k=R^2_{\le k}-R^2_{\le k-1} between nested population polynomial probes under the law μ\mu of the representation, with R2R^2 and R≤k2R^2_{\le k} defined formally (Section 2). These increments assume nothing about the distribution. The Hermite energies sks_k are kept as a Gaussian reference, and Δk=sk\Delta_k=s_k when the representation is Gaussian (Proposition 1, extended to every kk in Appendix A).

2. Why a Gaussian reference

Following C79S, Section 2, Related Work and the Conclusion now explain the choice. The Hermite basis is explicit and shared across representations, recent theory connects learned representations to Gaussian geometry (Wang & Isola, 2020; Betser et al., 2026; Domb, 2026), and VAEs and SIGReg-trained JEPAs target it directly. The paper treats the Gaussian as a reference, not as an assumption about learned representations.

3. The off-Gaussian bound is stated without ρk<1\rho_k<1

The submitted Appendix B already noted that the bound holds whenever the Hermite-feature Gram matrix is invertible, including ρ≥1\rho\ge1. The revision moves this into the statement of Proposition 2, and ρk<1\rho_k<1 is now described as the regime where the bound keeps the relative discrepancy below one, while for ρ≥1\rho\ge1 it still bounds Δk\Delta_k from above (Section 3, Appendix B).

4. Controls for fitted probes

Following zxxj, a new subsection (Section 2.1) and Appendix H give a ten-step protocol covering the fitting regime, null targets, unseen inputs, penalty grid edges, transport through known maps, affine maps, whitening, rank cutoff, sample size and a comparison with the structured readout a hypothesized mechanism implies, and Appendix H.1 reports compute cost. Table 10 checks each main finding against these controls, and cells without a reported number are marked as such. Regularized probes with more features than samples are now read as lower bounds on R≤k2R^2_{\le k}, not as estimates of Δk\Delta_k.

5. OthelloGPT

Following hNBQ, a product of one affine next-mover readout and one affine player-relative readout recovers board color nearly as well as the MLP. A sign-product proposition (Appendix E.7) explains why affine probes are blind here and bounds degree-two accessibility from below, and the drop at layer 7 is shown to be a fitting failure (Section 6.1, Appendices E.6 to E.8).

6. Grokking and pendulum

All grokking probes were refit on the original checkpoints under each control (Appendices G.6, G.7), and the pendulum section has a controls summary and a 13-target panel (Appendices F.6, F.7).

7. Presentation

Since the VAE section serves as a validation of Proposition 2, it is now shorter, with details moved to Appendix D. Related Work now covers Hermite analysis in learning theory and nonlinear and Pareto probing, and all figures are redrawn at print size.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.