acceptodds
Under review as a conference paper at ICLR 2027

Root Multiplicity Sets the Finite-Width Edge of Chaos in MLPs and Message-Passing Networks

Abstract

How far a finite-width network's order/chaos crossing sits from its mean-field location is set by one integer, the multiplicity of the mean-field root: an correction to the Lyapunov exponent moves the crossing by . In message-passing networks at zero bias the root is double. We prove this through an exact identity, , whose constant is universal () across smooth saturating odd activations, so the finite-width crossing lies at distance of order , not , from both the linearized boundary and mean field, which coincide. For MLPs and regular graphs with uniform input the network reduces exactly to a two-state Markov chain, and we prove the critical-window law . Between and the crossing the exponent is at least , sharply; at the chain converges to an explicit diffusion, and for every . Pre-registered measurements on full networks confirm the window law: the displacement decays as at zero bias (fitted exponent ) and as at nonzero bias (), and softsign is displaced more than by the predicted factor . Bipartite graphs follow a predicted second branch without the cost, which a registered field-level test confirms. We prove, and a registered test at held-out widths confirms, a shift for ReLU, whose root is simple.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
edge of chaos, finite-width corrections, Lyapunov exponents, mean-field theory, signal propagation, message-passing neural networks, graph neural networks, random matrix products, Markov chains, pre-registration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MKVzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper establishes a root-multiplicity principle for finite-width criticality: an O(F−1)O(F^{-1}) correction to the quenched Lyapunov exponent displaces a mean-field root of multiplicity rr by O(F−1/r)O(F^{-1/r}). It derives an exact mean-field identity showing that smooth saturating odd activations at zero bias have a double root with a universal coefficient, explaining the F−1/2F^{-1/2} rather than F−1F^{-1} displacement. For MLPs and regular message-passing networks with uniform input, it obtains an exact two-state Markov reduction, the critical-window law, and an explicit boundary-layer diffusion limit. Registered experiments support the predicted zero- and nonzero-bias scaling, activation-dependent prefactor, bipartite branch, and the O(F−1)O(F^{-1}) ReLU shift.

Critical strengths
  1. Theorem 2 gives a concise structural explanation for the double root and derives the constants κ2\kappa_2 and κ1\kappa_1, rather than merely fitting a width-scaling exponent.
  2. The exact (q,ρ)(q,\rho) Markov reduction and the stationary alignment cost FEπ[ρ2qK(q)]→1F\mathbb E_\pi[\rho^2qK(q)]\to1 jointly explain both the F−1/2F^{-1/2} scaling and its leading constant.
  3. The analysis treats ergodicity, the singular corner ∣ρ∣=1|\rho|=1, stationary moments, the critical window, and the x/Fx/F boundary-layer diffusion instead of relying only on a formal Taylor expansion.
  4. The paper distinguishes the linearized, mean-field, annealed, finite-depth empirical, and quenched Lyapunov boundaries, preventing several common conceptual confusions.
  5. The authors report falsified, void, inconclusive, and estimator-dependent results, including conclusions that changed after auditing the first-sign-change estimator.
  6. Linear controls, float64 checks, exact-chain comparisons, an independent reimplementation, fixed seeds, and explicit reproduction commands address important numerical failure modes.
Critical weaknesses
  1. The abstract introduces FF, gg, GG, κp\kappa_p, pp, glin(F)g_{\mathrm{lin}}(F), χ⋆\chi^\star, and xx without sufficient definition. Since these quantities encode different notions of gain, criticality, and finite-width scaling, the abstract is difficult to follow without consulting the main text. The authors should define FF and gg explicitly, describe GG and χ⋆\chi^\star in words, and either define pp and κp\kappa_p or omit those subscripts from the abstract.

  2. The Introduction provides some motivation--locating the finite-width order/chaos boundary before measuring its geometry--and cites foundational signal-propagation work, so the context is not entirely absent. However, the motivation remains narrow and the closest comparisons are compressed into a few sentences: the Introduction does not clearly explain how the proposed quenched Lyapunov crossing differs from existing finite-width criticality criteria, nor does it connect the result sufficiently to current work on deep-GNN signal propagation, initialization, oversmoothing, and random bifurcations. Much of this positioning appears only in the Discussion or Appendix B, after the principal claims have already been presented. The central research gap and closest distinctions should be moved into the Introduction, leaving the Discussion for implications, limitations, and open problems.

  3. Theorem 3 gives a uniform critical-window expansion and locates every sign change that occurs, but the abstract and main text then refer to "the crossing" as an established zero. Turning opposite asymptotic signs into an actual zero requires continuity of the stationary Lyapunov exponent in gg, including parameter-continuity of the invariant law and suitable uniform integrability; local uniqueness needs additional control. The paper should prove or cite these properties, explicitly bracket a zero, and avoid uniqueness language unless uniqueness is established.

  4. The sharp critical-window law requires an MLP or a connected regular graph with node-uniform input. For irregular graphs, the paper states a conjectural quenched bound and presents empirical transfer results that are partly inconclusive or depth-limited. The title and abstract can therefore be read as claiming a general message-passing-network theorem that is not currently proved. The framing should be narrowed to regular message-passing networks with node-uniform states, while irregular-graph findings should be labeled clearly as conjectural or empirical.

  5. Proposition 10 conditions on at least one active unit at every layer. For fixed finite FF, each independent layer dies with probability 2−F2^{-F}, so infinite-horizon survival has probability zero. The proof instead appears to use a product process in which every layer is conditioned individually on survival. The authors should define this as a layerwise-conditioned or "immortal" process, derive it as an appropriate finite-horizon conditional limit, or restrict the claim to finite depth before extinction. This issue is localized to the ReLU extension and does not invalidate the central smooth-activation theorem.

  6. The paper refers to anonymized code, stored fields, and preregistration files, but these materials were not part of the reviewed PDF. This matters because pooled-field reanalysis changed earlier first-sign-change conclusions, while bipartite fields can remain nearly flat over a broad interval. The supplement should include timestamped preregistrations and amendments, raw per-seed fields, exact pooling and interpolation rules, and scripts reproducing every headline interval. For flat fields, uncertainty should describe the full plausible zero region rather than only the bootstrap variability of a selected first sign change.

Questions for authors
  1. Can you prove continuity and existence of a zero of χF⋆(g)\chi_F^\star(g) in the critical window, rather than only locate any sign change that exists? This is needed to connect the uniform expansion to the headline statement that the crossing lies at the predicted location.
  2. Is local uniqueness of the non-bipartite crossing established? Finite-depth empirical fields can change sign multiple times, so it is important to distinguish estimator noise from possible non-monotonicity of the stationary exponent.
  3. How should Proposition 10 be interpreted when infinite-horizon ReLU survival has probability zero? Please clarify whether the theorem concerns an immortal layerwise-conditioned process, a finite-horizon conditional limit, or an ordinary network before extinction.
  4. Which message-passing claims remain theoretically valid for irregular graphs without assuming the empirically observed cubic reduction? The answer determines whether the graph-level contribution is a theorem, conjecture, or empirical generalization.
  5. Will the supplementary material expose raw fields and timestamped preregistrations, including all amendments and estimator-audit decisions? These artifacts are necessary to verify the empirical claims whose reported values changed after pooling the fields.
1 comment
AuthorsOpenReview ↗

Summary of revisions: what changed and where, and the corrections we made

We thank the three reviewers. The reviews found a real gap in the theory, an overclaim in the scope, and a paper that was hard to read. The revised PDF addresses each.

What to read. Pages 1–10 are self-contained. The new results are Lemma 10 and Corollary 2 (pp. 60–63), Theorem U1 on uniqueness (pp. 64–78) and Propositions G1 and G2 (pp. 89–90). The paper grew from 69 to 125 pages, all of it in the appendix: proofs were split into steps with headings, long formulas were set on their own lines, each main-text result is repeated before its proof, and records were added. None of it is needed to follow the main text.

ChangeRaised byWhere
Title narrowed to "…in MLPs and Regular Message-Passing Networks with Uniform Inputs"MKVz W4, bghv W1p. 1
Abstract rewritten: every symbol said in words, limit statedzwde, MKVz W1p. 1
Introduction starts from the phenomenon, states the gap and the closest prior work, and has a "Scope" paragraph; setup moved to a new §2; "In words" after Theorems 2–4zwde, MKVz W2, bghv W1pp. 1–5, 8
A crossing exists: Lemma 10 (continuity in the gain) and Corollary 2. For the MLP chain it is unique for large F: Theorem U1MKVz W3, Q1, Q2pp. 60–62; pp. 64–78; p. 3
What is proved, measured and conjectured on irregular graphs; Limitations; two new statements for irregular graphs (G1, G2)bghv W1, Q1; MKVz W4, Q4pp. 9–10; pp. 89–90
Registered predictions shown as prediction, rule, outcome; later explanations kept separatebghv W2, Q2pp. 7–9; Table 4, p. 104
"A diagnostic, not an initialization rule"bghv W3, Q3pp. 3, 10
New registered test: where nearby inputs stop merging, on four irregular graphsbghv W1, W3, Q3pp. 9, 101–102
ReLU result restated for a named conditioned processMKVz W5, Q3p. 80
Eight longest proofs rewritten as a goal plus labelled steps; appendix roadmapzwdepp. 24–81; p. 18
Index of all registrations with their limits; zero region for flat fields; pooling rules; commandsMKVz W6, Q5pp. 120–123; p. 108

Corrections not prompted by the reviews. We re-checked the paper with AI review tools and re-ran its commands. The location law, its constant and every registered verdict are unchanged. Appendix N (pp. 124–125) lists every change. In brief:

  • Statements. Theorem 2 gains a hypothesis at infinity that every activation in the paper satisfies (p. 4). Theorem 3 names the direction in which the gain varies, and says "large F" for bipartite graphs (p. 5). Assumption (A2) gains a hypothesis that the proof used. Theorem 5(a) holds above a width that depends on x (p. 47). §6 no longer claims the quenched bound beyond regular graphs.
  • Proofs repaired, conclusions unchanged. Lemma 1 (p. 27), the bipartite block of Theorem 3 (pp. 56–59), and steps of the boundary-layer results (pp. 34–55).
  • Numbers. A boundary value printed as 0.9729 is 0.9707. A registered run at F = 64 was completed, so registered predictions go from 18 to 21 (one more confirmed, one falsified, one void). A test registered during this discussion adds a 22nd, confirmed (pp. 101–102); a second one, on the backward signal, is a 23rd, outside Table 4, also supported (p. 102). One sentence on p. 111 is corrected.
  • A further AI audit (14 Nov) of the uniqueness proof, Propositions G1 and G2 and pages 1–10 found errors of statement or prose, now corrected: the definition of the exponent (Equation 2), the explanation of the 1/F cost, a width condition in Lemma U7(b), the uniqueness overview and the bootstrap count. No number, verdict or location law changed (Appendix N, p. 125).
  • An outside AI review of the rendered paper (14 Nov, later) led to further corrections: scope statements (the initial tangent and the existence window in Theorem 3, uniqueness stated in the window, Corollary 2 on the graph-active ray, where the limit in Equation 2 exists), the width condition and the trace step in Lemma 4, the range λ0 in Lemma 6, and wording on the pooled estimator, the tally of predictions, resample counts and the list of proof checks. No number, verdict or location law changed (Appendix N, p. 125). Two one-line edits came last: the range of the fits in the caption of Figure 4(b), and a lemma pointer in the proof of Theorem U1. A reader's comments on clarity then led to three small edits to Sections 1 and 2 (Appendix N). A last AI review of the final PDF (15 Nov) led to six small corrections: "Registered" for "Pre-registered" in the abstract, the note to Table 2 moved into its caption, two pointers and one split sentence in the appendices, and F ≥ 6 stated in Lemmas 4 and 6 (p. 125). An author read all of these line by line (p. 17).
  • Reproducibility. Missing per-seed outputs are added, three commands that failed as printed are fixed, and the commands added in the revision were re-run from the supplement (p. 123).
Reviewer_zwdeOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary
Critical strengths
Critical weaknesses

The readability of the paper is insufficient to write a proper summary, strengths and weaknesses. It is worth noting that AI has been extensively used to write this paper. It is acknowledged by the authors, and it is not a weakness in itself. However, it is clear to me that it is the direct cause of the lack of readability.

To be more specific, the abstract itself is impossible to understand as such:

  • FF is not defined;
  • the "Lyapunov exponent" is used outside any precise context (the dynamical system is not defined), and it is unnecessary jargon for an abstract;
  • GG is not defined;
  • o(1)o(1) is used without specifying the variable and the limit;
  • gg is not defined;
  • etc.

The introduction fails to provide sufficient literature to frame the problem and to motivate the paper, and loses itself into technical details before explaining why this work should be valuable for the community.

The clarity issue is even more concerning about the proofs in the Appendix. Most theorems have "inline" proofs of 10+ lines, sometimes more than 30 (lines 1112--1148), which is not acceptable for human reading.

For these reasons, I cannot provide a summary and critical strengths for this paper.

Questions for authors
1 comment
AuthorsOpenReview ↗

Response to Reviewer zwde: what we changed for readability

Thank you for the comments on the writing, and we are sorry for the effort the submitted version cost you. We have updated the draft to address the highlighted concerns:

  • Abstract (p. 1). Rewritten. It first says what "ordered" and "chaotic" mean; F, g and G are said in words; "Lyapunov exponent" is gone; the limit is stated as F → ∞.
  • Introduction (pp. 1–3). It starts from the phenomenon, compares with the closest prior work and states the scope. The technical setup is moved to a new §2.
  • Statements. Theorems 2, 3 and 4 are each followed by an "In words" paragraph.
  • Proofs. The proof you cite (now Lemma 1, p. 27) and the seven other longest are rewritten as a goal and labelled steps. Appendix A (p. 18) is a roadmap. The appendix is re-set: 265 long formulas that sat inside sentences are on their own lines, steps have bold headings, and each main-text result is repeated before its proof.
  • Proofreading. We read every proof in a theorem environment line by line, and have ensured the correctness and readability of the revised submission

Pages 1–10 are self-contained and we hope you would be able to assess the technical merits of the paper.

Reviewer_bghvOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

paper studies how finite network width shifts the order–chaos boundary away from its mean-field prediction. It shows that the displacement depends on the multiplicity of the mean-field root, giving an O(F^{-1/2} shift for double roots and an O(F^{-1} shift for simple roots.

Critical strengths

The connection between root multiplicity and finite width displacement is interesting and represents a meaningful theoretical contribution. The exact mean field identity in Theorem 2 and the critical window law in Theorem 3 provide more than a qualitative scaling argument. The experiments distinguish zero bias, nonzero bias, and activation dependent behavior, while the preregistered tests and explicit reporting of unsuccessful predictions strengthen the evaluation.

Critical weaknesses

main limitation is the restricted applicability of the strongest theoretical results. Theorem 3 requires regular graphs with node uniform inputs, effectively reducing the forward dynamics to an MLP. For irregular graphs and heterogeneous inputs, which are more representative of practical message passing networks, the corresponding scaling law remains unproved. Several experimental predictions concerning graph specific behavior are also inconclusive or refuted. The practical implications remain unclear. The experiments show that locating the corrected chaos boundary does not improve initialization or training performance, and sometimes produces worse outcomes at large depths. Consequently, the significance is primarily theoretical, while the proposed boundary correction has not demonstrated a clear advantage for learning.

Questions for authors

Can the authors clarify which parts of the F^{-1/2} scaling law are expected to survive on irregular graphs with heterogeneous inputs? How much confidence should be placed in the graph specific conclusions given the inconclusive and falsified registered tests? Is there a concrete setting where the corrected boundary improves learning or information propagation beyond what is achieved using the linearized boundary?

1 comment
AuthorsOpenReview ↗

Response to Reviewer bghv

Thank you for the review. Your summary of the limits is accurate, and the revision now states them up front.

What we claim is a theory result: at finite width the edge of chaos moves by F^(−1/r), where r is the multiplicity of the mean-field root; for tanh-like activations r = 2 and the constant is √3.

W1, Q1 (irregular graphs, heterogeneous inputs). Status of each claim (pp. 9–10):

ClaimStatus on irregular graphs
Double root of the mean-field gain (Theorem 2)Proved; does not depend on the graph
Exact linearized boundary (Theorem 1)Proved for every symmetric operator
Location law √(3/F) (Theorem 3)Proved only for MLPs and regular graphs with node-uniform input
Quenched bound (Theorem 4)Regular graphs only; on irregular graphs it is Conjecture 1, numerically supported on 199 of 200 graph × activation cells at F = 6 (bootstrap interval), not proved. What is proved without the damping term: for any inputs and bias, the exponent is at most the linear one when log(g/s₂²) ≥ 2/(F−2) (Proposition G2, p. 90)
Finite-depth bound in expectation (Proposition 11)Proved for every symmetric operator
F^(−1/2) decayA measurement: fitted exponent −0.56 [−0.66, −0.46] on our 24-graph corpus
Where nearby inputs stop mergingA registered test (new): beyond the linearized boundary on four irregular graphs, at 0.72 to 1.56 of the regular-graph value

For heterogeneous inputs there is no finite-width location law; Proposition G2 does bound the finite-width exponent above for arbitrary inputs under its stated conditions. The title now ends "…Regular Message-Passing Networks with Uniform Inputs".

On "effectively reducing the forward dynamics to an MLP": that is correct, and the paper says so (pp. 5, 7). Under these assumptions the graph enters only through the perturbation, which splits into one block per eigenvalue of the graph operator. On a bipartite graph a second block has the same gain, and Theorem 3 gives it a different law, without the 1/F cost (p. 5). That case is the only place where the regular-graph theorem differs from the MLP one. Its measurements are mixed: a registered field-level test confirms it, and the tests of where the crossing lies are inconclusive or falsified (Table 2, p. 9).

W2, Q2 (confidence in the graph conclusions). Moderate for the scaling, low for anything graph-specific. Of 22 registered predictions, eight hold, eight are inconclusive, two void, three falsified or refuted, and one is neither confirmed nor refuted (Table 4, p. 104 lists each). The three falsified or refuted predictions all concern particular graph classes (bipartite graphs, stars and wheels). An audit of our own estimator made the other bipartite crossing-location tests inconclusive. The failures are reported as predicted, and our later explanations are labelled as such (p. 7).

W3, Q3 (does the corrected boundary help?). For information propagation, yes, in three registered and confirmed settings. At depth 16F, two nearly identical inputs merge below the crossing and separate above it. On a scale where the linearized boundary is 0 and the √(3/F) law is 1, the switch sits at 1.23 (F = 32) and 1.03 (F = 128); the exact crossing is at 1.26 and 1.09 (Table 2, p. 9). Prompted by your question, we registered the same test on graphs before running it: a regular control and the four non-bipartite irregular graphs of our width study, at both widths. In all ten cells the switch lies beyond the linearized boundary, at 0.91–1.29 (F = 32) and 1.05–1.70 (F = 128) on that scale, and the registered rule returns "supported" (pp. 101–102). A third test concerns the signal that training sends backward through the layers (registered after your question; its digest was time-stamped by two independent authorities before the first run). On the same scale, the backward signal of a deep tanh MLP switches from vanishing to exploding at 1.32 [1.14, 1.43] (F = 32) and 1.07 [0.97, 1.19] (F = 128), against the exact crossing at 1.26 and 1.09, and a linear control finds its own threshold (0.000 and −0.006); the registered rule returns "supported" (Appendix I, p. 102). Two limits: in five of the eight irregular cells the interval excludes the regular-graph value, so the scale transfers but the constant does not; and one cell clears the registered threshold narrowly (0.908 against 0.885). For training, no. In a registered grid of 4512 runs, initializing at the crossing was no better than at the linearized boundary, and in deeper post hoc runs (L = 4F) it was worse (§7, p. 10). The null is not a dilution effect: on the 37 cells of that grid that can separate the arms, the difference is −0.0062 [−0.0171, +0.0049] (post hoc; p. 99). The theory is consistent with this: between the two boundaries the exponent is within about 1/(2F) of zero per layer. We present the result as a diagnostic (p. 3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.