Evie-KF: Risk-Sensitive Kronecker-Factored Preconditioning for Adaptive Optimizers

Adaptive optimizers such as Adam adapt each update to the magnitude of its coordinate, but this diagonal view discards correlations in gradient noise. Those correlations contain information about which directions are more reliable. We propose Evie-KF, an adaptive optimizer derived from a risk-sensitive formulation of stochastic optimization. Evie-KF extends Adam's diagonal normalization with a matrix preconditioner obtained by solving the risk-sensitive Riccati equation in coordinates whitened by Adam. The resulting operator, $(I+\gamma {\Sigma }_z{)}^{-1/2}$, uses the centred covariance of gradient noise so that correlated directions are treated jointly. It recovers AdamW exactly at $\gamma =0$ and adds one tuned parameter. Across vision, language, and finance tasks, Evie-KF improves over AdamW and other common adaptive optimizers on most tasks tested across multiple seeds. On CIFAR-100, it improves top-1 accuracy over AdamW by 5.6%, winning in all five seeds. On WikiText-103 language modeling, it improves over AdamW by 11.5% and over SGD by 40.7%, again in all five seeds. On a 33.7M-parameter OpenWebText model, the largest we train, Evie-KF improves over AdamW by 13.1% on three of three seeds at matched wall time. Ablating the Kronecker structure on its own, against a diagonal gate at matched normalization, it gains 6.75% on a finance panel and 6.67% on WikiText-2, on nine and ten of ten seeds, both surviving Holm correction. [Our code is available](https://github.com/rpextra2026-afk/evie-kf-optim), with Evie-KF released as the [`evie-kf`](https://pypi.org/project/evie-kf/) Python package.