Weakly Convex Optimization on Manifolds with Heavy-Tailed Oracles

Weakly convex optimization on manifolds underlies nonsmooth learning problems with orthogonality constraints. Existing Riemannian stochastic subgradient guarantees rely on bounded second moments; we study unbiased oracles with only a finite $p$-th central moment, $1<p\le 2$. Our main result is high-probability convergence of Riemannian stochastic subgradient descent (RSSG) on compact embedded manifolds with globally defined smooth retractions. Heavy-tailed steps can leave the region where local retraction bounds apply. We overcome this mismatch by controlling the iterates' actual movement on the compact manifold, without clipping or assuming finite oracle variance. Its oracle complexity is $\tilde{O}({\epsilon }^{-2p/(p-1)})$, recovering the classical ${\epsilon }^{-4}$ order at $p=2$. Two complementary variants bound every tangent step. GeoClip-RSSG couples clipping and stepsize selection to attain the same oracle exponent within a prescribed retraction neighborhood. GeoNorm-RSSG normalizes mini-batch subgradients, attaining the same iteration-rate order with parameter choices independent of $p,\sigma$, at an additional sample cost. Experiments support the predicted convergence trends and demonstrate the methods' value for subspace recovery from heavy-tailed data.