Self-Evolution Should Be Stable, Not Frozen: Lyapunov-Gated Recursive Self-Improvement

Recursive self-improving agents increasingly modify not only their task behavior, but also the mechanisms that generate future modifications. This creates a control problem that current self-evolution objectives do not measure: a change can improve the next agent while making future evolution increasingly sensitive to evaluator noise, misleading memories, reward misspecification, or mutation randomness. We formulate recursive self-improvement as a non-stationary stochastic dynamical system and introduce EVOLYAPUNOV, a stability-constrained framework that allows the improver itself to evolve without freezing the meta-operation. EVOLYAPUNOV estimates an interventional evolution response operator from matched perturbation branches, decomposes evolution into capability and risk subspaces, and admits candidate self-modifications only when useful directions improve while measured risk dynamics satisfy a confidence-aware Lyapunov drift condition and an empirical finite-horizon amplification bound. We define the Recursive Amplification Factor (RAF) and evolution risk half-life, which expose failure modes invisible to one-generation benchmark scores. Under a stochastic drift condition, accepted updates keep expected risk energy bounded despite persistent perturbations; a complementary amplification bound connects the certificate to multi-generation error propagation. Our evaluation protocol spans SWE-bench Verified, Multi-SWE-bench, τ²-Bench, BFCL V4, and ARC-AGI-2 with Qwen2.5-Coder-7B-Instruct and Qwen2.5-7B-Instruct backbones. A stress study reveals a sharp recursive stability boundary: unrestricted self-evolution can continue improving immediately after the estimated risk gain exceeds one, whereas Lyapunov-gated evolution remains on a capability–stability Pareto frontier. The central claim is that recursive self-improvement need not choose between open-ended evolution and stability; it should expand useful directions while contracting the directions along which mistakes compound.