Gradient Methods with Chaotic-Inspired Step Sizes

This paper investigates gradient methods with dynamically varying step sizes for convex quadratic optimization problems. We propose a family of step-size rules, named FC and FCA, motivated by the chaotic behavior of gradient iterations. The main theoretical contribution is a rigorous analysis of the two-dimensional convex quadratic setting. For this low-dimensional case, we derive closed-form characterizations of the iteration map and establish the stability properties of its fixed points under different parameter regimes. The emergence of chaotic dynamics is demonstrated numerically via positive Lyapunov exponents, initial sensitivity, and ergodicity, though a rigorous mathematical proof of chaos is not established in this work. The theoretical findings in two dimensions inspire the construction of the FC/FCA step-size schemes for high-dimensional problems. We conduct extensive numerical experiments on large-scale convex quadratic test problems with a wide range of condition numbers, including ill-conditioned cases. Our numerical results demonstrate the empirical performance of FC/FCA compared with classical gradient methods such as the BB and CBB step-size rules. We further state several conjectures regarding the high-dimensional dynamical behavior and iteration complexity of the proposed methods. Rigorous proofs for these high-dimensional claims remain open.