acceptodds
Under review as a conference paper at ICLR 2027

Learning reshapes power-law anisotropy in internal representations

Abstract

Power-law anisotropy in internal representations has been observed across a wide range of biological and artificial neural systems, from state-of-the-art language models to the mouse cerebral cortex. This anisotropy is a key geometric property of high-dimensional information processing and underlies a variety of theoretical analyses. However, the mechanism by which it emerges from input structure and task-driven learning has remained unclear. Here, we characterize this formation process by exactly solving the learning dynamics of a wide two-layer linear neural network in a teacher-student setting with power-law input and teacher structures. We show that, in the feature-learning regime, the local power-law exponent of the internal-representation spectrum evolves nonmonotonically over the course of training and exhibits up to four distinct asymptotic regimes across modes and training times. By contrast, in the lazy regime, the exponent remains essentially unchanged. We further demonstrate numerically that similar exponent dynamics arise in more realistic nonlinear networks. Together, these results suggest a general mechanism by which the dynamic interaction between input statistics and task structure gives rise to power-law internal representations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Power-law anisotropy, Representation dynamics, Deep linear network, Exact solution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VVjbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper theoretically investigates how the power-law exponent of the representation spectrum evolves during training. For two-layer linear networks, the authors show that, the local power-law exponent of the representation spectrum evolves non-monotonically in the feature-learning regime, and find similar behaviors in the training of non-linear networks in the teacher-student setting.

Critical strengths

To the best of my knowledge, this is the first work to analytically characterize the evolution of the weight/representation power-law exponents during learning. I find the problem interesting and the results novel, particularly the identification of different exponent regimes and their transitions over training.

Critical weaknesses
  1. Limited technical novelty. The main theoretical results are built upon the exact learning dynamics of deep linear networks established by Saxe et al. (2013).

  2. Clarification of the phase diagram. Figure 3(a) could be improved by specifying the boundaries between different exponent regimes, at least up to scalings. In particular, it would be helpful to characterize the size of the hatched regions and explain how these regions depend on the model parameters.

  3. Visualization of the power-law exponents. It is difficult to directly identify the predicted changes in the power-law exponents from Figures 2 and 4. The authors could consider adding reference lines corresponding to the theoretical slopes α\alpha, 3α+β3\alpha+\beta, α+β\alpha+\beta, and α+β/2\alpha+\beta/2, where applicable. This would make the comparison between theoretical predictions and numerical results more convincing.

  4. Validation on real datasets. Although the authors conduct experiments with nonlinear networks, these experiments are still restricted to a synthetic teacher–student setting. It would be interesting to investigate whether qualitatively similar spectral dynamics can be observed in more realistic training scenarios. For example, could one estimate the power-law exponents of the input and effective teacher spectra from a real dataset and compare the observed evolution of the representation spectrum with the theoretical predictions?

Questions for authors

See weakness. It is especially important if the authors could compare their theory with existing empirical observation about how the weight spectrum changes over training, at least qualitatively.

6 comments
AuthorsOpenReview ↗

Response to VVjb

We thank the reviewer for carefully reading and evaluating our manuscript. Below, we describe the revisions made in response to the identified weaknesses.

Limited technical novelty. The main theoretical results are built upon the exact learning dynamics of deep linear networks established by Saxe et al. (2013).

The main novelty of our study is the identification and analysis of a previously uncharacterized phenomenon: the nonmonotonic evolution of power-law exponents, revealed by applying the learning dynamics established by Saxe et al. (2013) to data with power-law spectra. Contributions 1 and 2 in the original manuscript could have misleadingly suggested technical novelty in the derivation of the learning dynamics themselves. We have therefore rewritten the “Our contributions” section to make the novelty described above clear.

Clarification of the phase diagram. Figure 3(a) could be improved by specifying the boundaries between different exponent regimes, at least up to scalings. In particular, it would be helpful to characterize the size of the hatched regions and explain how these regions depend on the model parameters.

We have clarified these points in Appendices C.2 and C.4. The regime boundaries are defined implicitly and are difficult to express in a simple form, but their intersections and asymptotic values can be obtained explicitly. Adding these expressions directly to Fig. 3(a) would make the figure cluttered. We therefore explain how the boundary locations and the sizes of the regimes depend on the model parameters in Appendix C.4 (line 957 onward), with a reference to this discussion in the main text (line 362).

Visualization of the power-law exponents. It is difficult to directly identify the predicted changes in the power-law exponents from Figures 2 and 4. The authors could consider adding reference lines corresponding to the theoretical slopes α,3α+β,α+β,α+β/2\alpha, 3\alpha+\beta, \alpha+\beta, \alpha+\beta/2, where applicable. This would make the comparison between theoretical predictions and numerical results more convincing.

We have added reference lines to Fig. 2 for the four asymptotic exponents, which are α\alpha, 3α+2β3\alpha+2\beta, α+2β\alpha+2\beta, and α+β\alpha+\beta in the revised notation. We have also indicated the input exponent α\alpha in Fig. 4; please see our response to the next question. These additions make the agreement between the theoretical exponents and the spectra in Fig. 2 easier to assess visually.

Validation on real datasets. Although the authors conduct experiments with nonlinear networks, these experiments are still restricted to a synthetic teacher–student setting. It would be interesting to investigate whether qualitatively similar spectral dynamics can be observed in more realistic training scenarios. For example, could one estimate the power-law exponents of the input and effective teacher spectra from a real dataset and compare the observed evolution of the representation spectrum with the theoretical predictions?

Result 3.3 in the original manuscript considered the limited setting of a nonlinear student learning from a linear teacher using synthetic data. In the revised manuscript, we have replaced this experiment with a more general setting in which a nonlinear MLP learns to denoise real images from CIFAR-10. This is an intrinsically high-dimensional problem, unlike the solvable model in which the modes evolve independently. Because this task has no explicitly defined teacher exponent β\beta, we cannot quantitatively compare the evolution of the exponents with our theoretical predictions. Nevertheless, the representation exponent reflects the input exponent α\alpha at initialization and evolves nonmonotonically during training, reproducing the qualitative behavior predicted by the theory. This provides empirical support for the phenomenon in a more general setting beyond the assumptions of our theory.

Best regards,

Authors

Thank you for the detailed response and the revisions. I have one remaining question regarding Figure 3(a). Could the authors clarify whether the relative ordering of the three boundaries, (k_{\mathrm{cross}}(t)), (k_{\mathrm{learn}}(t)), and (k_{\mathrm{layer}}(t)), is fixed or can change during training? In particular, do any of these boundaries intersect or merge, and under what parameter conditions?

Thank you for your question. (1) The ordering of kcrossk_{\mathrm{cross}} and klayerk_{\mathrm{layer}} remains fixed during training, whereas (2) klearnk_{\mathrm{learn}} can intersect either of the other two boundaries.

We first restate the definitions of these boundaries. For each mode, the mode strength gkaw(t)g_k^{aw}(t) increases monotonically from zero toward γ0θk\gamma_0\theta_k during training. The boundaries are implicitly defined by

k_{\mathrm{cross}}(t):\quad& g_k^{aw}(t)=\sqrt{\sigma_w^2(2\sigma_w^2+\sigma_a^2)}, \end{aligned}$$ $$\begin{aligned} k_{\mathrm{learn}}(t):\quad& g_k^{aw}(t)=0.9\gamma_0\theta_k, \end{aligned}$$ $$\begin{aligned} k_{\mathrm{layer}}(t):\quad& g_k^{aw}(t)=\sigma_w^2+\sigma_a^2. \end{aligned}$$ **1. Relationship between $k_{\mathrm{cross}}$ and $k_{\mathrm{layer}}$** The thresholds defining these two boundaries are independent of both $k$ and $t$. Moreover, $g_k^{aw}(t)$ decreases monotonically with $k$. Therefore, their ordering cannot reverse during training. Specifically, solving $$\sqrt{\sigma_w^2(2\sigma_w^2+\sigma_a^2)} < \sigma_w^2+\sigma_a^2$$ gives $$\sigma_a^2>\frac{\sqrt5-1}{2}\sigma_w^2.$$ Under this condition, \$ k_{\mathrm{cross}}(t)>k_{\mathrm{layer}}(t) \$ at all times. If the opposite inequality holds, their ordering is reversed. In the equality case, the two thresholds are identical, and the boundaries coincide. In the feature-learning setting primarily considered in the manuscript, $\sigma_a\gg\sigma_w$, so $k_{\mathrm{cross}}$ lies above $k_{\mathrm{layer}}$. **2. Relationship between $k_{\mathrm{learn}}$ and the other two boundaries** The threshold defining $k_{\mathrm{learn}}$ depends on $k$ through $\theta_k=\theta_0k^{-\beta}$, allowing this boundary to intersect the other two. Equating the respective thresholds gives the following intersection ranks: $$k_{\mathrm{cross,learn}}^* =\left( \frac{0.9\gamma_0\theta_0} {\sqrt{\sigma_w^2(2\sigma_w^2+\sigma_a^2)}} \right)^{1/\beta},$$ $$k_{\mathrm{layer,learn}}^* =\left( \frac{0.9\gamma_0\theta_0} {\sigma_w^2+\sigma_a^2} \right)^{1/\beta}.$$ If an intersection rank lies within the analyzed mode range $1<k<D$, the corresponding pair of boundaries intersects at a finite training time.
Reviewer_B8stOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

In this manuscript, the authors investigate how the spectra of the hidden layer of a two-layer linear neural network evolve as training progesses. Based on the initial condition, the learning regime, and the rank, they identify four regimes of learning, each with their own associated power-law decay. In general they show, that for leading eigendirections, the spectral decay splits into three learning regimes, while eigendirections of higher rank remain in the initialization-dominated regime and therefore appear not to adapt to the task.

Critical strengths
  • I found it intriguing that within the same model, one can have strong adaptation in leading eigendirections, and a basically lazy-learning regime in subleading directions. This could perhaps even imply a coexistence of rich and lazy learning within the same model. Though such an event could perhaps be considered to here result from the highly simplified problem setup, it does make me wonder if similar results could be derived in more practical scenarios. It would have been nice to see the consequences of these learning regimes more explicitly discussed (see suggestion below).
Critical weaknesses
  • The setting is highly constrained, limiting the scope of the results. First, the linearity assumption, and second, the assumption that the teacher weights and the input covariance are both diagonal in the same basis turn the dynamics, in conjunction with the infinite-width assumption, into an effectively one-dimensional problem (see line 1001). This makes the setting tractable, and the application to relu networks with synthetic data is encouraging, it would however considerably broaden the scope if it could be show theoretically or empirically that the results hold up in a non-linear regression setting with real data (e.g. image data, that appear to have the correct spectral decay).
  • The proofs and mathematical definitions are not particularly reader-friendly. Some parts of the appendix could be considerably shortened, and some definitions appear to be superfluous. I give suggestions below.
Questions for authors

Suggestions for further evaluations:
What is the influence of the four regimes on the test loss? How must gamma be chosen to reach a certain regime in a certain eigenvalue range? Do the results hold up for real data evaluated on a non-linear regression objective?

Suggestions for reader-friendliness Some equations and derviations repeat verbatim, for example equation (41) is equal to equation (100) and equation (51) is equal to (107). I believe this part could be considerably shortened. The derivations are also very notation-heavy. For example, the definitions in lines (1002-1005) could be made far more user-friendly by reusing the superscripts {w,wa,a} to distinguish between the three variables instead of introducing new notation that has to be memorized.

Lines 1129-1134 furthermore appear to imply that the dynamics of the spectra can be closed without the use of the variable v_k. If so, is there why this simplified representation is not used in the eq. (12)?

Similarily, eq. (15) could be simplified by expressing the function R using g or vice versa.

Additionally, while given enough time and effort, one could perhaps read off the three regimes from eq. (15), there is no explanation given for why these three regimes should emerge from these rather involved formulae.

3 comments
AuthorsOpenReview ↗

Response to B8st

Thank you for reading our manuscript carefully and for the many constructive suggestions. We address them below.

Response to Weaknesses

As you note, the original manuscript relied on strong assumptions that make the modes independent. In the revision, an MLP trained to denoise CIFAR-10 images shows qualitatively similar changes in the power-law exponent in a multidimensional nonlinear regression task (Result 3.3). This extends our findings to a more general setting.

Also, we revised the definitions and explanations to improve readability. Please see our responses below.

Response to Questions

What is the influence of the four regimes on the test loss?

In our setting, the test loss is a sum over modes:

Ltest(t)=12∑kλk(θk−gkaw(t)γ0)2.L_{\mathrm{test}}(t)=\frac12\sum_k\lambda_k\left(\theta_k-\frac{g_k^{aw}(t)}{\gamma_0}\right)^2.

Leading modes learn first, and their contributions to the loss decrease. The boundary between learned and unlearned modes is klearn(t)k_{\mathrm{learn}}(t), defined in the main text. Modes in Region IIIa and IIIb therefore contribute little to the test loss, consistent with prior work. However, the link to the spectral exponent is limited: changing a spectrum's overall scale can change the test loss without changing its exponent.

How must gamma be chosen to reach a certain regime in a certain eigenvalue range?

We added an explanation in Appendix C.2. For example, modes k1k_1 through k2k_2 lie in Regime IIIa after learning if

σw2(σw2+c)θ0k2β≪γ0≪cθ0k1β.\frac{\sqrt{\sigma_w^2(\sigma_w^2+c)}}{\theta_0}k_2^\beta\ll\gamma_0\ll\frac{c}{\theta_0}k_1^\beta.

In general, all four regimes require γ0≫c/θ0\gamma_0\gg c/\theta_0. If γ0≪σw2(σw2+c)/θ0\gamma_0\ll\sqrt{\sigma_w^2(\sigma_w^2+c)}/\theta_0, all modes lie in Regime I.

Do the results hold up for real data evaluated on a non-linear regression objective?

Yes. In the revised Result 3.3, an MLP learns to denoise noisy CIFAR-10 images and shows qualitatively similar changes in the power-law exponent.

Suggestions for reader-friendliness Some equations and derviations repeat verbatim, for example equation (41) is equal to equation (100) and equation (51) is equal to (107). I believe this part could be considerably shortened.

The original Appendices B and C analyzed training only the first layer to explain the derivation in a simpler setting. We agree that this was redundant and have removed it, along with other repeated derivations.

The derivations are also very notation-heavy. For example, the definitions in lines (1002-1005) could be made far more user-friendly by reusing the superscripts {w,wa,a} to distinguish between the three variables instead of introducing new notation that has to be memorized.

Following your suggestion, we now denote the diagonal entries of Gaw,Gw,GaG^{aw},G^w,G^a by gkaw,gkw,gkag_k^{aw},g_k^w,g_k^a instead of ξk,ηk,ζk\xi_k,\eta_k,\zeta_k. We also adopted Saxe et al.'s notation and removed unnecessary variables in Eqs. (7) and (10). Using θ\theta instead of μ\mu changes the teacher exponent to βnew=βold/2\beta_{\mathrm{new}}=\beta_{\mathrm{old}}/2.

Lines 1129-1134 furthermore appear to imply that the dynamics of the spectra can be closed without the use of the variable v_k. If so, is there why this simplified representation is not used in the eq. (12)?

The cited lines expressed ρk\rho_k in terms of uku_k. The original Eq. (12) solved the differential equation for uku_k and used vkv_k to write the complicated solution. We moved this exact solution to Appendix B (line 743) and use the simpler expression in Eq. (7) in the main text.

Similarily, eq. (15) could be simplified by expressing the function R using g or vice versa.

For αeff\alpha_{\mathrm{eff}}, as for ρk\rho_k, we now use a simpler expression in the main text (Eq. (10)) and put the full solution in Appendix C. We removed the temporal functions RR and gg and rewrote the expression using gawg^{aw}.

Additionally, while given enough time and effort, one could perhaps read off the three regimes from eq. (15), there is no explanation given for why these three regimes should emerge from these rather involved formulae.

In the revised manuscript, equation (10) replaces the original Eq. (15). We explain its three factors immediately afterwards to clarify how the exponent changes. We also explain why the different exponents arise when discussing the boundaries (line 315) and each regime (lines 249–251 and 327–344).

Best regards,

Authors

I thank the authors for their revisions. I find the clarity of the paper to be improved. The additional evaluation is limited, but arguably shows that relevant mechanisms carry over. I will raise my score, however, I would like to add that I did not check the new derivations and results in detail, which is reflected in my reduced confidence.

We sincerely thank you for your constructive comments, which have helped us improve the clarity and quality of our manuscript. We also appreciate your consideration of our revisions and your decision to raise your score.

Reviewer_A4HyOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper “Learning shapes power law anisotropy” studies how the spectrum of the kernel of a two-layer linear network evolves through training. In particular, they study networks trained on power-law data, study the evolution of the power-law exponent of the kernel spectrum, and show that there are four distinct regimes in which the power changes. Consistent with existing literature, they find that in lazy learning, the change to the power law exponent is minimal, whereas in rich learning they observe more significant changes. The authors validate their theoretical predictions experimentally, with good agreement. They also argue that this analysis can be extended to nonlinear networks by taking a ReLU student trained on a linear teacher, which they argue exhibits similar qualitative behaviour.

Critical strengths

The observation, if correct, that the local power law exponent is independent of training time within each regime and the transition across regimes is an interesting one, which I would be interested to explore further.

Critical weaknesses

Incremental contribution: The evolution of the kernel of the internal representations has been expressed analytically in a wide class of cases, some of which are cited in this paper- Bordelon & Pehlevan 2022, Bordelon et al. 2025, Saxe et al. 2013, Braun et al. 2022, and Domine et al. 2025. One of the main results, stated in Eq. 12 (contribution 1 of main contributions) is a previously known result, and the power-law exponent regime (contribution 2 of main contributions) is a direct corollary of this known result, under restrictive assumptions. Finally, the authors' claim that “Finite-width nonlinear networks validate the theory” (contribution 3 of main contributions) is not sufficiently convincing, since the authors use a nonlinear network trained on a linear teacher, which in many cases mirrors linear network behaviour.

Poor presentation: Many of the key results of the paper are simply stated, the key steps of the derivation are not mentioned in the main text, and there is largely no motivation as to why these results should hold, thus making the paper difficult to follow, and the soundness difficult to judge. This difficulty is further exacerbated by the overuse of notation. For instance, the authors define θk=μk\theta_k=\sqrt{\mu_k} and then proceed to use θk2\theta_k^2 throughout the whole paper, Ek,vk,rk+E_k, v_k, r_k^+ (and possibly others) are defined and used once. On the other hand, important quantities, such as uku_k, are defined mathematically and then used extensively, while their construction remains entirely unmotivated theoretically. The combination of all of the above obscures an already incremental contribution.

Questions for authors

For Authors*

  1. Could the authors address how these results differ from those in the literature? Especially Saxe et al., 2013?

  2. Can the authors extend their experimental analysis to (a) for networks trained on nonlinear tasks, and (b) cases where α≠β\alpha\neq \beta?

  3. Could the authors clarify what they mean by: “Most existing theories, however, treat the eigenspectrum of the internal representation as a given quantity that remains fixed during learning.” ? Is this a general statement about existing theories for kernel evolution? Or only in the power law context? The former is obviously incorrect, and the statement should be framed accordingly.

  4. Bayesian approaches analyze how the kernel evolves, albeit as a function of data rather than time, and show how the kernel adapts to the training data for both linear and non-linear networks (Li & Sompolinski, 2020, Pacelli et al 2023, Ringel et al 2023 Van Meegen & Sompolinsky 2024, Rubin et al 2025 and others). Since in the setting of this paper it would seem that the network spectrum is largely static within each regime, it would be important to understand how the analyses differ in these papers from the dynamic one.

2 comments
AuthorsOpenReview ↗

Response to A4Hy

We thank the reviewer for carefully reading our manuscript and for the thoughtful questions and comments. Below, we address each point and describe the corresponding revisions.

Response to Weaknesses

  • Incremental contribution: The original wording of the “Our contributions” section could have given a misleading impression of our contributions. In the revised manuscript, we have rewritten contributions 1 and 2 to clarify the novelty of our results. Please also see our response to the first question. Also, we have substantially strengthened the experiments in nonlinear settings by introducing a nonlinear regression task using real image data, as described in our response to the second question.
  • Poor presentation: We have improved the notation and derivation throughout the main text and appendices. Please see the responses to Questions below

Response to Questions

Could the authors address how these results differ from those in the literature? Especially Saxe et al., 2013?

The learning dynamics of the two-layer linear network build on prior work, and we do not intend to claim a new exact solution method as our main contribution. Our contribution is to analytically derive the local power-law exponent of the representation spectrum from these dynamics and to characterize its nonmonotonic evolution, the coexistence of regimes with different exponents, and the conditions under which these regimes arise. Although our analysis builds on Saxe et al., we believe that this characterization of the dynamics of power-law exponents constitutes a distinct contribution, rather than a direct corollary of their results. We have revised the items in the “Our contributions” section to make this novelty clear.

Can the authors extend their experimental analysis to (a) for networks trained on nonlinear tasks, and (b) cases where α≠β\alpha \neq \beta?

(a) Result 3.3 in the original manuscript considered the limited setting of a nonlinear student learning from a linear teacher using synthetic data. In the revised manuscript, we have replaced this experiment with a more general setting in which a nonlinear MLP learns to denoise real images from CIFAR-10. This is an intrinsically high-dimensional problem, unlike the solvable model in which the modes evolve independently. Because this task has no explicitly defined teacher exponent β\beta, we cannot quantitatively compare the evolution of the exponents with our theoretical predictions. Nevertheless, we observe the same qualitative nonmonotonic evolution of the power-law exponent during training. This provides empirical support for the phenomenon in a more general setting beyond the assumptions of our theory.

(b) The analytical solutions in Eqs. (7) and (10) hold for general α\alpha and β\beta. To make this clearer, we have updated the plots to show a case with α≠β\alpha \neq \beta. Figure 2 now uses α=0.6\alpha=0.6 and β=0.7\beta=0.7. Please note that the revised notation defines βnew=βold/2\beta_{\mathrm{new}}=\beta_{\mathrm{old}}/2.

Could the authors clarify what they mean by: “Most existing theories, however, treat the eigenspectrum of the internal representation as a given quantity that remains fixed during learning.” ? Is this a general statement about existing theories for kernel evolution? Or only in the power law context? The former is obviously incorrect, and the statement should be framed accordingly.

Our intention was to refer specifically to the temporal evolution of power-law exponents of internal representation spectra, but the original wording was misleading. We have revised the relevant passage in the Introduction to make our focus on the power-law exponent of the internal representation kernel explicit (lines 47–53).

Bayesian approaches analyze how the kernel evolves, albeit as a function of data rather than time, and show how the kernel adapts to the training data for both linear and non-linear networks (Li & Sompolinski, 2020, Pacelli et al 2023, Ringel et al 2023 Van Meegen & Sompolinsky 2024, Rubin et al 2025 and others). Since in the setting of this paper it would seem that the network spectrum is largely static within each regime, it would be important to understand how the analyses differ in these papers from the dynamic one.

Differences in the input statistics and the scaling of hyperparameters make a direct comparison with the cited studies difficult. Extending our results to Bayesian neural network settings would be an interesting direction for future research. We would also like to clarify that a constant power-law exponent does not imply a static spectrum. In Regime II, for example, ρk(t)∝λk3θk2t2\rho_k(t) \propto \lambda_k^3\theta_k^2t^2. Thus, although the exponent remains constant at 3α+2β3\alpha+2\beta, the kernel evolves quadratically with training time tt. In Regimes IIIa and IIIb, where learning has converged, the kernel also becomes stationary.

Best regards, Authors

Thank you again for your feedback. With the discussion deadline approaching, we would appreciate it if you could review our responses and let us know whether any concerns remain.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Author-Reviewer Discussion

We thank the reviewers and the Area Chair for their time and consideration.

We are pleased that reviewers B8st and VVjb have found our responses satisfactory and expressed support for acceptance.

Although we have not yet received a follow-up response from reviewer A4Hy, we have addressed their concerns through additional experiments and revisions, which we believe adequately respond to the points raised. In particular, to address the concern that our contribution relative to prior work is limited, we have clarified the relationship to Saxe et al. (2013) and highlighted the novelty of our analysis of the power-law exponents. We have also provided evidence for the generality of our findings by showing similar behavior in inherently multidimensional, nonlinear tasks using real-world data.

We therefore respectfully request that, during the upcoming reviewer–AC discussion, the Area Chair and reviewers explicitly discuss whether the assessment that our contributions are limited remains justified in light of these clarifications and additional results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.