- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper asks whether the training endpoint of a normalized two-layer quadratic network on a finite group's multiplication table can be predicted by scoring candidate representations independently at initialization. A sector-local score may read its own complete Fourier blocks and all sectors' per-layer magnitudes, including the trivial sector, but not other sectors' orientations. For exact cross-entropy (CE), the endpoint label maximizes the product of the three per-layer sector energies; the endpoint need not have support strictly confined to one sector. For groups with at least three nontrivial real Fourier sectors, Theorem 3.1 characterizes the width-one leading selector under uniform initialization: exact sector-local ranking is possible almost everywhere if and only if the input-sphere metric weight is ω = 2. Every fixed ω ≠ 2 yields a positive error floor for the entire class. For a fixed group and finite width under ordinary normalization, Theorem 4.2 establishes convergence and leading-label inheritance for almost every initialization below a scale threshold that may depend on that initialization. It covers continuous CE and retracted iterations; the latter threshold may also depend on a fixed upper bound on the physical learning rate. At width one, for continuous CE and groups with a C₂ × C₂ quotient, Theorem 5.1 gives a positive optimal error at every sufficiently small positive scale even at ω = 2, although the error tends to zero as the scale vanishes. Retuning the scalar weight within a fixed compact subset of positive weights also fails to restore exactness. Experiments use XOR, D₄, and Q₈. On XOR, 18.43% is the disagreement between joint and separate normalization, 1.70% is the error of one particular sector-local rule, and approximately 0.0809% is the estimated optimal sector-local error. These are distinct quantities. The CE experiments read labels at a finite horizon and do not directly verify infinite-time endpoint agreement.
Critical strengths
- An exact characterization of the full scoring class. Theorem 3.1 constructs paired initializations with identical information available to the two rival sectors and identical magnitudes in every sector. The pair differs only by a reflection of the third sector's output block, which switches the winner between the rivals; the third sector wins at neither start. Appendix C.4-C.6 extends the construction to open sets of the full joint manifold, yielding a positive lower bound shared by all admissible measurable scoring rules. The main novelty is this class-wide converse and its coverage of real, complex, and quaternionic blocks. Matched rates and blow-up-time selection have precedents, as Section 7 acknowledges.
- A connection from the leading approximation to exact-loss endpoints. Theorem 4.2 combines potential gaps around Morse-Bott maximum manifolds with analytic convergence to turn finite-horizon closeness into infinite-time label inheritance, including control of discrete steps crossing the trapping boundary. Theorem 5.1 shows that positive-scale corrections destroy exact sector-local ranking at ω = 2, even with scalar metric retuning in a fixed compact set. Selected key local coefficients in Appendix E were recomputed with AI assistance and matched the manuscript; see the AI disclosure for the scope of these checks. These results distinguish accurate small-scale prediction from exact independent ranking at positive scale.
Critical weaknesses
- The practical error magnitude remains unclear across groups. Sections 2 and 7 explain the connection to existing selector formulas, but the obstruction concerns this particular information class. Remark F.4 and Proposition H.1 (pp. 50 and 60) imply that, on XOR, the additional exact value of the total objective f allows decoding; no noise-stability guarantee is given. The H.3 estimate of 0.0809%, with a Monte Carlo standard error of about 0.0052 percentage points, applies only to the XOR leading selector. Remark C.2 already bounds the optimal error by q_op, but the gap to the H.2 witness lower bounds remains large. Small witness lower bounds do not establish small optimal errors on D₄ and Q₈. Tighter error estimates are needed to assess the practical predictive impact across groups.
- Evidence at positive scale and the scope of the results remain limited. Figure 2(b), Table 3, and H.1 mainly test finite-horizon label inheritance at ω = 1; they do not directly demonstrate the reflected witnesses or boundary displacement of Theorem 5.1 at ω = 2. That theorem covers width one, continuous flow, and groups with a C₂ × C₂ quotient; larger widths and discrete updates remain open. Separately, for the leading selector, exact sector-local ranking at ω ≠ 2 remains unresolved for groups with exactly two nontrivial real sectors (Remark F.3). The Gronwall bound in D.3 is quantitatively uninformative at the reported scales, as acknowledged. Concrete reflected initializations, numerical convergence checks, and boundary displacement across scales at ω = 2 would better substantiate the positive-scale mechanism. The small-logit, full-batch setting also limits extrapolation. These are limitations of evidence and scope, rather than identified errors in the proofs.
Questions for authors
- Error magnitude across groups (request for additional evidence). Remark C.2 already bounds the optimal error by q_op, whose Table 2 estimates are 14.465% on D₄ and 15.845% on Q₈. Could the authors provide tighter upper bounds, for example by extending the H.3 raw-block clock rule to these groups and reporting its error with uncertainty estimates? Neither the XOR optimum estimate of 0.0809% nor the small H.2 witness lower bounds locates the optimal errors on these groups. This would affect my assessment of the practical magnitude of the results on noncommutative groups.
- The positive-scale mechanism at ω = 2 (request for additional evidence). Could the authors provide a specific reflected pair of initializations for exact CE at a sufficiently small positive scale, and test the distinct endpoint labels using the full system, longer horizons, and refined step sizes or tolerances? If feasible, please also show how the boundary displacement varies with scale. The existing label-inheritance experiments at ω = 1 do not directly address this.