Strict Online Learning for Wild Streaming Data

Real-world streaming data commonly exhibit characteristics such as concept drift, missing values, and heterogeneous features, and such data are referred to as $\text{wild streaming data}$. Existing approaches often rely on buffers that retain past instances to support model adaptation, while the implications of data retention, including privacy and data governance concerns, have received limited attention. We investigate a new problem, $\text{Strict Online Learning for Wild Streaming Data}$, in which each instance is used once for prediction and update and then discarded. This no-retention requirement makes learning challenging, as the model adapts to evolving patterns one instance at a time without destabilizing previously acquired knowledge. To address this challenge, we propose $\text{OnlineNet}$, a fast-slow online neural framework for regulating adaptation under uncertain single-instance evidence. Its slow pathway contextualizes the current observation using accumulated stream context, while its fast pathway captures instantaneous evidence. Bounded Adaptive Modulation coordinates their interaction by controlling how instantaneous evidence influences the slow representation. To stabilize single-instance learning, we introduce Online Prior Adjustment based on a prequential class prior estimated solely from previously revealed labels. Extensive experiments across 81 streaming conditions derived from 27 public datasets demonstrate that OnlineNet achieves the best aggregate predictive performance across all four cumulative metrics. It also maintains high computational efficiency while using only 0.5% of the parameters required by a competitive baseline. The source code and datasets are submitted as Supplementary Materials for Reproducibility.