Runtime Learning in Open World: Adaptive Blending and Cognitive Sampling

This paper proposes OpenRun, a framework that enables safety-critical autonomous systems to perform runtime learning in open-world environments. Here, open-world environments refer to environments whose states, conditions, and operational scenarios are not fully known prior to deployment and may continuously evolve or emerge during operation. OpenRun comprises a Learning-Student, a Safety-Teacher, and an Action-Blender. The Learning-Student builds on actor-critic reinforcement learning and introduces a novel cognitive batch sampling strategy that prioritizes experiences based on temporal recency, occurrence frequency, and contextual similarity, significantly enhancing learning efficiency. The Safety-Teacher operates in parallel with the Learning-Student, providing verifiable safety protocols that enable safe runtime learning across a broad range of action policies, from purely control to vision-to-action policies. The Action-Blender introduces a novel safety- and exploration-aware blending mechanism that adaptively blends the actions of the Learning-Student and Safety-Teacher to enhance safety assurance while preserving exploration. We further establish theoretical foundations demonstrating how these innovations collectively enable safer and more efficient runtime learning in open-world environments. Extensive experiments with a quadruped robot and an off-road autonomous vehicle in challenging real-world and simulated wild-forest environments validate the effectiveness and distinctive capabilities of OpenRun, as well as its generalizability across functions, platforms, and environments.