acceptodds
Under review as a conference paper at ICLR 2027

Emergence of Dynamic Similarity in Neural Representations

Abstract

Physical systems can look very different in dimensional coordinates while still obeying the same governing dynamics. When neural networks learn such systems, do their internal representations organize around the dimensionless relationships identified by classical physics? We study neural networks trained on publicly available physics datasets from seven dynamical systems spanning transport, mechanics, reaction–diffusion, and waves. We find that training consistently reorganizes hidden representations around the dimensionless physical relationships that govern the observed dynamics. This organization can strengthen in two ways: by emphasizing changes that alter the underlying physics or by becoming less sensitive to equivalent dimensional descriptions. Yet independently trained networks need not share the same neural coordinates, suggesting that representational convergence can occur at the level of physical relationships rather than a common latent basis. These results motivate the broader hypothesis that independently learned representations may converge on the relationships that structure the underlying physical world.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Representation Learning, Neural Representations, Scientific Machine Learning, Dynamic Similarity, Dimensional Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nN5SOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper hypothesizes that neural networks trained on dimensional physical data may learn hidden representations that capture dimensionless relationships, and empirically investigates this hypothesis.

Using the Buckingham-Pi theorem, the authors formulate a linear map ξ=B⊤s\xi = B^\top s from dimensional inputs (ss) to dimensionless variables (ξ\xi) in logarithmic coordinates. They analyze the learned neural representation z(s)z(s) by decomposing its Jacobian JzJ_z using the orthogonal projector PP onto im(B)\mathrm{im} (B). Specifically, the authors measure the energies of the projected components as Ephys=∥JzP∥F2E_\mathrm{phys} = \Vert J_z P \Vert^2_F and Escale=∥Jz(I–P)∥F2E_\mathrm{scale} = \Vert J_z (I – P) \Vert^2_F, and then compute the physical share score as a fraction of the physical component, Sphys=Ephys/(Ephys+Escale)S_\mathrm{phys} = E_\mathrm{phys} / (E_\mathrm{phys} + E_\mathrm{scale}). This decomposition has a natural interpretation from the quotient perspective of the Buckingham-Pi theorem: im(B)\mathrm{im}(B) represents directions in ss that change ξ\xi, capturing nondimensional physical changes; on the other hand, its orthogonal complement, ker⁡(B⊤)\ker (B^\top), represents directions that preserve ξ\xi, corresponding to mere dimensional scale variations of the same physical law. Accordingly, EphysE_\mathrm{phys} and EscaleE_\mathrm{scale} quantify the representation's sensitivity to physical and scale variations, respectively.

Importantly, if the neural representation depends only on the scale-invariant dimensionless coordinates, i.e., z(s)=h(ξ)z(s) = h(\xi), the chain rule gives Jz=JhB⊤J_z = J_h B^\top. Consequently, in this case, $ E_\mathrm{scale}$ vanishes, leaving only EphysE_\mathrm{phys} and accordingly Sphys→1S_\mathrm{phys} \to 1. The authors train FNOs, CNNs, MLPs, and Transformers on raw dimensional data from seven different physical systems and measure changes in physical share before and after training. They find that physical share increases across all seven systems, reaching 0.9-1.0 in many cases. They further validate this finding through various controlled experiments, and also show that this alignment extends beyond the infinitesimal Jacobian level to finite input transformations.

In the subsequent experiments, the authors investigate whether this phenomenon simply reflects different architectures learning similar latent coordinates. They find that linear mappings between hidden representations do not consistently enable successful model transfer. Based on these results, the authors argue that the observed convergence toward dimensionless, scale-invariant representations reflects the underlying physical relationship rather than bias toward a common latent coordinate system.

Critical strengths

This paper proposes an interesting hypothesis in scientific machine learning. Inspired by the Platonic representation hypothesis and the Buckingham-Pi theorem, the authors hypothesize that neural models trained on raw dimensional data may learn internal representations organized around dimensionless coordinates. Through carefully designed experiments based on a quotient-space perspective, the authors provide empirical evidence supporting this hypothesis.

Critical weaknesses

W1. Overall, I find the hypothesis and supporting evidence presented in this paper interesting. However, I believe the paper would benefit from improved readability, particularly in how the experiments are presented.

While the authors relatively clearly explain their motivation, hypothesis, and the mathematical tools developed to test it, the presentation of the experimental designs and results is somewhat vague. For example, Fig. 1 occupies more than half of page 3 but is never discussed in the main text. Moreover, its relevance to the central hypothesis is not immediately apparent.

Fig. 2 presents the main experimental results and receives some explanation in the text, but the purpose of each panel is not easy to follow. For instance, Fig. 2 (c) shows that physical share remains elevated even after removing the clock direction, but the motivation of this control experiment is not clearly explained. I find the control experiments in Fig. 2 (b-d) well designed, which makes the lack of clear explanations unfortunate. Similarly, Fig. 3 (a-c) receives very little explanation, making it difficult to understand exactly how the experiments were conducted without considerable inference.

Given that there is still substantial unused space within the page limit, I suggest that the authors more clearly explain the purpose of each experiment, the control variables, and how the results support the central hypothesis.


W2. As the authors acknowledge in Section 6, this paper presents empirical evidence supporting the proposed hypothesis but does not explain the underlying mechanism. Moreover, the current version does not demonstrate how the observed phenomenon could lead to practical benefits. I do not consider these as particularly serious weaknesses. Nevertheless, while the paper deserves recognition for its novelty in reporting an interesting phenomenon, its immediate practical significance and contribution to a deeper theoretical understanding remain limited.

Questions for authors

Q1. I understand that the purpose of Section 5 is to distinguish physical organization from prediction accuracy. However, I am curious about the (empirical) relationship between these two properties. For example, what would we observe if one plotted SphysS_\mathrm{phys} against prediction error or EΔE_\Delta across different models and tasks? Would there be no clear correlation, or might some systematic relationship emerge? Since many models achieve physical shares close to 1, such a relationship might be difficult to observe. However, could a more detailed analysis of individual runs, different initializations, and training checkpoints reveal meaningful trends?


Q2. It would also be interesting to compare models trained on raw dimensional inputs with those trained on explicitly dimensionless coordinates. The latter naturally enforces dimensional scale invariance. Do the two approaches yield learned representations with similar geometric structures?


Q3. Buckingham-Pi coordinates can be interpreted as quotient coordinates under dimensional scaling symmetry, which is a particular class of Lie group symmetries. If the governing physical laws possess a larger Lie group symmetry beyond dimensional scaling, what properties would the learned representations exhibit? Would they remain invariant only under dimensional scaling, or would they also capture the broader Lie symmetries?


Q4. This may already be explained in the Appendix, but I would appreciate more details on the experimental setup. Specifically, if the training data consists of raw dimensional quantities, the target outputs may also depend on dimensional scales. In that case, a scale-invariant hidden representation would not be sufficient to produce the correct outputs (unless the model has an explicit scale-dependent output head). Could the authors clarify whether the learning objectives and target outputs in each task are scale-invariant or scale-equivariant, and how dimensional scaling is handled in the model?


Q5. Both the orthogonal projection and the energy norm depend on the choice of metric. How robust are the findings to this choice? Have the authors examined whether the results hold under metrics other than the Euclidean metric?


Minor comment: Although Section 6 is titled "WHAT EMERGES ACROSS PHYSICAL SYSTEMS?", the authors acknowledge that "Why this organization emerges remains unresolved." Therefore, I suggest renaming this section to something more appropriate, such as "Limitations."

1 comment

Thank you so much for your positive (and also critical) feedbacks!

Firstly, you are right about the readability of the experiments. Section 2.1 now discusses Figure 1 as motivation and distinguishes its illustrative networks from the new 72-predictor study. Section 2.2 explains the Burgers example before the measurements: joint rescaling preserves the normalized solution, while changing viscosity alters it. Section 2.3 explains why we compare with initialization, match input distance and hold the clock fixed. Holding the clock fixed asks whether selectivity extends beyond elapsed progress. Appendix C.1 specifies the cases, controls and uncertainty. The figure numbers have changed with the new study.

Furthermore, we agree that the submission left its consequences unclear. Section 3 now checks recovered relations against independent physical calculations, using finite Burgers paths and an anomalous exponent within a prescribed family. Section 4.3 and Appendix F.3 connect scale sensitivity to finite generalization. Output sensitivity ranks generalization as closely, so the hidden measure has no demonstrated predictive advantage here. A unique learning mechanism remains unresolved.

A1: We find a strong within-run association and a weaker one across settings. Following your suggestion, Appendix F.4 compares 48 Burgers predictors. Median within-run rank correlations between error and contrast exceed 0.98 at every measured layer. Here a more negative contrast means stronger selectivity: error and contrast decrease together. Across selected checkpoints, the correlations are 0.63 for FNOs and 0.54 for Transformers. In Table 22, width-256 Transformers at learning rates 10⁻³ and 3×10⁻⁴ have median errors of 2.05% and 2.23%, with last-hidden contrasts of −1.13 and −0.76. Thus similar accuracy can accompany different selectivity. These validation-selected checkpoints establish a Burgers comparison, not a cross-task law.

A2: They show related scale selectivity expressed at different depths; we have not established identical complete hidden geometries. We added matched Burgers predictors with raw parameters, logarithms, dimensionless groups plus scale, and groups alone (Section 4.3; Appendix F.2). With logarithms or groups plus scale, dependence on scale is largely removed by the first affine stage. With raw inputs it decreases through nonlinear layers. Groups alone enforce invariance by construction. The encoding also changes the coordinate channel and training normalizers.

A3: Some are, to different degrees. Sections 4.1 and 4.2 (Appendix E) add time re-anchoring, translation and Galilean boosts of periodic Burgers dynamics. In the same twelve evolved-profile predictors, hidden layers attenuate scaling more strongly than time re-anchoring, although both preserve the target. Accurate outputs approximately respect both. Translation and boost responses depend on architecture and layer. The finite tests use a specified feature-action class; they do not establish a full Lie-group representation or composition laws. We exclude the projective transformation because it does not preserve the periodic problem.

A4: Your concern is correct: a predictor must retain any scale needed for its output. Section 2.3 and Appendix A clarify that the main study's selected similarity moves preserve its prediction targets. Appendix F.5 adds a separate amplitude test: the network either predicts the dimensional Burgers field or predicts its normalized field with amplitude restored outside the network. Both use the same physical loss. FNO last-hidden amplitude-response shares are 0.16 to 0.17 and 0.005, respectively. Invariance must therefore be assessed relative to the actual target.

A5: The original feature-rescaling checks did not establish metric independence. Appendix B.4 gives the revised-cohort tests: on Burgers and the pendulum tasks, sampled channel transformations with condition numbers up to 10 retain more than 92% of the training-induced changes in similarity share and weak-direction angle; standardization retains more than 90% (system means). General invertible transformations can change these measurements. But, the claim is limited to these transformations and populations.

Thank you again for recognizing the potential in our research. We are grateful for your review and it has allowed us improve the manuscript. We are happy to answer more questions!

Reviewer_vR6fOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper asks whether neural networks trained to predict physical dynamics organize their hidden representations around dimensionless relationships. Across seven dynamical systems and multiple neural architectures, the authors analyze hidden-feature Jacobians with respect to dimensional inputs. They decompose sensitivity into directions that change governing dimensionless quantities and directions that preserve the normalized physical problem. The main finding is that training consistently increases the representational sensitivity lying in the designated physical subspace. The paper further argues that this can occur either by increasing separation along physically meaningful directions or by contracting responses to equivalent dimensional descriptions. Experimental results suggest that independently trained models can encode similar physical dependencies without sharing a linearly transferable neural coordinate system. Output tests emphasize that representational selectivity and accurate physical prediction are distinct.

Critical strengths
  • The paper asks a conceptually interesting question about whether predictive learning recovers physically meaningful relationships.
  • The authors carefully distinguish representational selectivity, invariance, cross-model coordinate alignment, and prediction accuracy.
  • Empirical work is documented well. Initialized network controls, random subspace comparisons, feature-normalization tests, finite transformations, and output level evaluations are useful.
Critical weaknesses
  1. Unclear novelty and contribution. The quantitative Jacobian/subspace analysis is close to standard sensitivity and active subspace methodology. The principal novelty therefore appears to be the application of familiar diagnostics to several simulation datasets, rather than a new representation learning method, theoretical result, or operator analysis framework. The paper should identify more precisely what conclusion could not already be obtained using existing sensitivity and invariance analyses.

  2. Overstating the physical scope. Although the paper studies seven named systems, the effective physical variation is narrow. Advection, Gray–Scott, acoustic scattering, and Schrödinger evolution primarily test a single normalized-time or progress direction. Only Burgers, pendulum, and compressible flow expose additional non-temporal directions, and those directions still span very low-dimensional predefined subspaces. The output fields may be high-dimensional, but the tested physical manifolds are not. The experiments use canonical equations, restricted geometries and boundary conditions, limited coefficient variation, and deliberately constructed similarity transformations. They do not examine chaotic, turbulent, externally forced, multiscale, irregular-geometry, or otherwise complex physical regimes.

  3. No demonstrated operational importance. The paper reports large response errors for compressible flow, negative long-horizon for the held-out Gray–Scott setting, substantial acoustic interpolation error, and weak Transformer mechanics. Some coverage interventions also worsen prediction. The paper reports these negative findings honestly, but they leave the significance of the proposed representation metric unclear. The paper does not demonstrate that greater physical share predicts improved accuracy, scale generalization, robustness, transfer, sample efficiency, or conservation. It is therefore uncertain why this particular hidden-layer organization should matter beyond being descriptively measurable.

Questions for authors
  1. Are the conclusions consistent across layers, and was the structural middle layer selected before inspecting the representation results?
  2. Is there any downstream task for which higher physical share reliably predicts better generalization or robustness?
1 comment

Thank you for your review! Our original contribution statement did not answer that clearly enough. In the revision, we test what the recovered relations mean, whether they extend beyond a clock, and when they accompany better generalization.

For the novelty and contribution of the paper. You are right that Jacobian and subspace analysis are established tools, as we now state in Appendix B.3 and the related-work discussion preceding Appendix A. The original study mainly measured sensitivity along supplied physical directions; Section 3 now extracts relations from hidden responses and checks them against independent physics. This includes finite Burgers paths and an anomalous exponent within a prescribed scaling family. Changing the target and input encoding tests which relation is learned and where it appears (Section 4.3). Our contribution is empirical; we claim neither a new sensitivity estimator nor a mechanism unique to physics. We have also made clear of our contribution and novelty in the intro and the conclusion in the revised manuscript.

Several original panels exposed only a progress direction, as you noted. The revised central study uses a separate cohort of 72 predictors across six systems and four architectures: Burgers, KdV, an acoustic interface, a harmonic trap and both pendulums (Section 2.3). These remain restricted families; turbulence and forced systems remain future work.

To test dependence beyond progress directly, we added single-constant changes at fixed normalized time (Appendix C.1, Table 10). The last-hidden contrast has a negative interval in 68/72 predictors, including 68/69 meeting the accuracy criterion. The four exceptions are the three acoustic CNNs and Burgers CNN seed 2. These are follow-up evaluations on the existing 96 held-out initial conditions per system, with whole-parent uncertainty conditional on the fitted models.

We fixed the further control procedures before evaluating their responses. Complete-input-distance matching gives negative trained intervals in all 60 predictors on five complete panels. Acoustic density matching gives 12/12 on only 192/768 supported comparisons; that restricted result does not replace Table 10 (Table 11). An impedance-only acoustic move preserves both speeds, normalized time and the pulse, yet gives negative intervals in 9/12, retaining all three CNN exceptions (Table 12). For the pendulums, solver solutions change in angles and normalized momenta while canonical initial states stay fixed, excluding a purely dimensional-momentum explanation (Table 13). Thus the revised evidence tests specific non-clock dependencies, without claiming broad coverage of complex physical regimes.

Lastly, we agree that the original prediction failures prevent physical share from being treated as a certificate of competence. Changing the principal study population does not repair those earlier failures. We now give the recovered relation a physical use: it proposes parameter paths whose solutions are checked independently. For Burgers, the solution changes by about 1% or less in the median along recovered paths, versus 7.9% along an equal-length viscosity-changing path (Section 3.1; Appendix D.1). Section 3.2 separately tests a recovered anomalous coefficient against an eigenvalue reference and retains unsuccessful hidden recoveries. These establish that hidden responses can yield testable physical information. They do not establish that increasing physical share generally improves a predictor, so the generalization evidence has the narrower scope now (which we answer below for your questions).

A1: The original Appendix B.1/Table A1 names a structural-middle hook but does not establish that it was fixed before inspecting representation results. In the revised manuscript, the last-hidden endpoint was fixed before generating the evaluation cases; middle and output results are also reported (Appendix C.1). Selectivity strengthens from middle to last hidden and again to output. Recovery is less uniform: system-median alignment improves when evaluated at the analytical similarity dimension, while blind selection finds the correct hidden rank in 36/72 predictors (Appendix D.3, Table 15).

A2: We now observe an association in the Burgers scale tests, but cannot claim a reliable predictor across tasks. Smaller scale response accompanies better interpolation between trained scales. One decade outside training, raw-input errors reach 7.5 to 125 times their in-range values, while logarithmic and groups-plus-scale encodings approximately retain their accuracy (Section 4.3; Appendix F.3). Output responses rank generalization as closely as hidden responses. We therefore report these comparisons without claiming an advantage for the hidden metric, or demonstrating robustness beyond these tests.

Thank you again for helping us clarify our own understanding. I hope that I have addressed some of the concerns you have, and I would be happy to answer more questions!

Reviewer_AcWPOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper considers the learned representations of neural networks trained on physics problems, and in particular the effect of dimensionless relationships. Fourier neural operators, CNNs, MLPs, and transformers are trained on data from seven different physics problems, and it is discovered that hidden representations become relatively insensitive to pure dimensional input rescalings after training, compared to physically meaningful rescalings. Different networks are shown to learn this structure with different internal coordinate systems. Additionally, some networks are shown to have a strong representational correspondence to the physical system, yet at the same time still show high error, demonstrating that the physical organization is not enough for accurate predictions.

Critical strengths

The paper tests the results in a fairly broad range of settings, i.e. seven tasks and four genuinely different architectures. It is also quite honest about its limitations and mentions negative results. The separation between accuracy and physical organization is also somewhat surprising.

Critical weaknesses

Novelty is not fully clear to me. From what I understand, the main results of the paper center around redundant dimensions coming from dimensional rescaling becoming relatively small during training compared to physics-relevant directions, resulting in a representation more closely matching the underlying physical system. Currently, I fail to understand how this is different from the already documented (e.g. Saxe 2014, Achille 2018, Flesch 2022) phenomenon of irrelevant feature collapse. I'd be more inclined to give the paper a more positive score if e.g. either more is claimed here about the learned representation beyond the redundant dimensions becoming relatively insignificant, or it is argued that this has particularly significant consequences in the physics setting here, that are not applicable to the general setting.

I'm not entirely sure about the influence of the way data was sampled on the learned representation, as this is done in a non-standard way as far as I understand.

The paper is also lacking in clarity, e.g. terminology is not always clearly introduced before using it, making the text take longer to read than necessary.

Questions for authors

Could your results be a consequence of a more general phenomenon unrelated to physics, e.g. the observation that feature directions which do not affect the target predictions will generally become comparatively smaller in a rich learning regime? If you were to train a network on some arbitrary non-physical task with a task-invariant dimension, would any of the results change from what is observed for the physical systems in the paper?

"Our training distributions include similarity-structured coverage, so these experiments do not establish that the same organization would arise under arbitrary dimensional sampling.” What was the motivation for using this, and do the results still hold up under completely independent sampling?

Are there any particularly significant implications of any of the phenomena observed here, specific to the physics setting?

2 comments

Thank you so much for your careful review! Your review helped us see that the initial submission did not explain clearly enough what the physical interpretation adds to established accounts of irrelevant-feature suppression. We have revised that framing and added experiments that directly test the sampling and target assumptions you raised.

A1: Yes, that remains a plausible explanation. Our original experiments did not test a non-physical target, so they could not separate that account from a mechanism specific to physics. We also now explicitly discuss the connection to the works you suggested us in related-work Appendix A.

We also ran the control you suggested: a smooth non-physical target with the same redundant direction as the physical task. The ordering between raw and logarithmic input encodings persists in all 32 architecture/seed/layer comparisons (Appendix F.2, Table 21). This supports your interpretation that the suppression itself can arise from general feature learning. To ask what determines the relation being learned, we then changed the target's redundancy while holding the inputs, initial weights and minibatch order fixed. All 16 paired-target predictors turn their weakest last-hidden direction toward their own target's redundancy; 14 also meet the stable rank-one criterion (Section 4.3; Appendix F.1). Both candidate directions preserve normalized time, so their different responses support dependence beyond that clock in these sixteen predictors. Well, we do not infer a new learning mechanism unique to physics for the current scope of the paper (at least for now).

A2: Our original experimental design generated dimensional realizations of archived normalized problems. This gave us equivalent descriptions whose physical relationship was known, but, as you point out, it also exposed that structure during training. That limitation left us unable to tell whether deliberately grouped training descriptions were necessary. However, we have now addressed that question with a separate central experiment: 72 predictors across six systems and four architectures, trained with dimensional parameters sampled independently conditional on each initial profile or state. Every predictor meets the selectivity and paired training-gain criteria. All 72 also retain selectivity when complete-input displacement is matched (Section 2.3; Appendix C.1). Profiles are still reused, so this establishes the result for the tested sampling families, not for arbitrary data distributions. These are new experiments; the 38 fits in the original seven-system study are not being treated as independent-sampling evidence.

A3: In the current revision, we go beyond measuring relative sensitivity. A relation extracted from hidden features is used to propose finite parameter paths, which an independent physical calculation then checks. In our experiments, the recovered Burgers paths change the solution by about 1% or less in the median, compared with 7.9% along an equal-length viscosity-changing path (Section 3.1; Appendix D.1). We also test this on Barenblatt model to recover an anomalous scaling coefficient. The median local-derivative estimates from eight-block FNO hidden features are within 0.005 of a separate eigenvalue reference at each tested equation coefficient (Section 3.2, Table 1; Appendix D.2). This is the empirical contribution we now emphasize: learned relations can be read from hidden responses and tested as physical hypotheses. There is also a practical consequence in the new encoding comparison. One decade beyond training, the tested raw-input predictors have 7.5 to 125 times their in-range error, whereas logarithmic and groups-plus-scale encodings retain approximately their in-range accuracy (Section 4.3; Appendix F.3). Hidden and output responses both track these outcomes; we have not established that the hidden metric predicts generalization better than the output metric.

Lastly, you are absolutely write with the paper being hard to follow. We restructured the paper around the new experiments and made sure that the story and the flow of the paper is easy to keep track.

Your review led us to test assumptions that the original submission did not state precisely. Thank you so much again.

If you have more questions. I am happy to answer!

Thank you for the changes. I appreciate the improvements to the writing which did help with clarity, and the control does alleviate some of my concerns about the validity of the results. However, I'm still having a hard time grasping what is novel about this paper, so I cannot currently recommend it, although I am unfamiliar with applications of deep learning to physics, and so there may things I am overlooking. But don't currently see any clear contribution to deep learning as a whole.

Comments on the paper

1 comment

Thank you all for your amazing feedbacks!

We thank all the reviewers and the area chair for their time and consideration. Your questions helped us identify where the original submission needed clearer explanations and stronger evidence. We have revised the manuscript to address concerns from reviewers (we have replied those changes individually).

General changes summarized below:

  1. A controlled study of emergence. The revised main comparison uses 72 predictors across six systems, four architectures and three seeds. This is a separate cohort from the original seven-system study. Dimensional parameters are sampled independently conditional on each initial profile or state, without deliberately constructing equivalent-description training pairs. All 72 predictors become more selective through training, and their trained selectivity persists when complete-input distance is matched. Additional fixed-clock comparisons support dependence beyond elapsed progress in 68/72 predictors, with all four exceptions reported (Section 2; Appendix C.1).

  2. Independent physical tests of recovered relationships. We now extract parameter paths from Burgers hidden responses and check their solutions independently. These paths nearly preserve the physical solution. In a separate modified-diffusion experiment, FNO hidden responses yield an anomalous scaling coefficient within a prescribed scaling family, checked against an independent eigenvalue reference (Section 3). We distinguish improved alignment from complete recovery: system-median alignment improves across the six systems, while blind selection returns the correct hidden rank in 36/72 predictors.

  3. Experiments on what shapes the organization. We added comparisons that change the prediction target, input encoding and training settings. The recovered direction follows the target, while the encoding affects where scale dependence is removed and how predictions generalize across scales. A nonphysical target exhibits the same encoding-dependent ordering, consistent with general feature learning. Output responses track generalization as closely as hidden responses, so we do not claim a unique learning mechanism or an independent predictive advantage for the hidden metric (Section 4.3; Appendix F).

  4. Broader symmetry tests and clearer presentation. We added finite tests of time re-anchoring, translation and Galilean boosts. These show that different symmetries can be expressed differently across hidden layers, even when accurate outputs respect them (Sections 4.1–4.2).

  5. Clearer structure and writing flow. We substantially reorganized the manuscript with the reader’s perspective in mind. We wanted each section to make clear why a question matters, how we test it, and what the evidence allows us to conclude. The revised paper moves from observing emergence to testing recovered relationships against physics, then examining what shapes that organization. We also improved transitions and integrated the figures and controls into the discussion, aiming to keep the narrative engaging (hopefully) while connecting each claim to its supporting evidence throughout.

Thank you again for helping us improve the manuscript!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.