acceptodds
Under review as a conference paper at ICLR 2027

When Are Energy Proxies Reliable? Excitation and Curvature in ReLU Equilibria

Abstract

An energy proxy replaces asymmetric recurrent interactions with a symmetric field, but a smaller field mismatch need not imply a more accurate answer. We construct a three-dimensional ReLU family whose original updates are uniformly contractive, yet the metric that minimizes rotational defect produces unbounded proxy error. This metric is fixed, has condition number below , and performs worse than the bounded-error identity proxy. For fixed input distributions on a positive box, the energy penalty grows as and the mean squared state error grows as , where is the proxy curvature margin. We explain these rates with two-sided population bounds for strongly convex proxies, valid across activation changes and at any rotation strength. Inactive-unit slack can screen rotation even when the response second moment is nonsingular, so the lower bound measures excitation through jointly active responses. Complementarity supplies an energy-budget upper bound, and both bounds recover the growth rates on the separating family. Input-coverage and response-balance conditions specify when the lower bound is positive and quantitatively informative. Experiments on trained ReLU networks and a fixed-anchor recurrent Transformer support the directional mechanism: input-excited rotation predicts proxy errors better than rotation magnitude.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.