Online Conformal Shielding: Safe Adaptation under Coupled Distribution Shifts

Safe online adaptation requires balancing useful exploration with execution-time risk control. We identify two challenges: a safety–exploration imbalance between overly conservative and overly permissive control, and safety staleness, where risk control becomes outdated under coupled exogenous environment shifts and endogenous policy/scorer updates. We propose Online Conformal Shielding (OCS), a pre-execution safety framework for online adaptation. OCS selectively executes proposed actions or intervenes, updating its threshold from partial safety feedback while the task policy and scorer continue to adapt. Our generic reduction to online conformal abstention (OCA) transfers suitable selective-risk guarantees to shielding without explicitly modeling the source of distribution shift. We further develop a prior-aware OCA variant that uses pre-adaptation information to guide early threshold selection, with a well-placed prior improving the resulting safety guarantee. Under non-anticipating interaction, we establish a high-probability adaptive safety guarantee: at a prescribed interaction budget, the realized average safety cost among executed task actions is bounded by the target level plus an explicit finite-sample slack. On VLA manipulation in ManiSkill and continuous control in Safety-Gymnasium, OCS keeps the selective safety risk near the target level under coupled shifts, where policy-level safe RL and fixed shields exceed it, while executing over 96% of the steps on ManiSkill.