SOPA-DoS: Second-Order Passive-Aggressive Difference-of-Squares Learning in Open Feature Spaces

Online learning algorithms are essential for processing high-speed data streams, yet most classical algorithms assume that each sample is represented within a fixed feature space. Open-feature streams violate this assumption: features can emerge, disappear, and reappear while the learner must predict in one pass. Existing open-feature methods primarily rely on linear passive-aggressive updates, feature reconstruction, sparsity, or evolving ensembles, whereas recent nonlinear difference-of-squares (DoS) passive-aggressive learning assumes a fixed representation and does not exploit second-order feature geometry. To the end, we propose a second-order passive-aggressive framework SOPA-DoS for low-rank DoS classification in open feature spaces. The key construction replaces Euclidean parameter proximity by a positive-definite right metric that accumulates streaming feature curvature. A whitening argument converts the resulting metric-constrained nonconvex QCQP exactly into the tractable one-constraint DoS passive-aggressive problem, yielding a unique scalar step and closed-form rank-one updates along the preconditioned direction $H_t^{-1}{z}_t/(z_t^{\top }{H}_t^{-1}{z}_t)$. Dynamic block expansion makes the metric and DoS factors compatible with newly appearing features, while zero embedding handles unobserved features; diagonal and sketched variants trade curvature fidelity for scalability. The construction reduces to the original DoS-PA update when $H_t=I$ and is equivariant to feature reindexing. Experimental results on the WDBC stream under increment-decrement and capricious feature protocols demonstrate that curvature-aware variants can substantially reduce cumulative error when feature scales are heterogeneous.