acceptodds
Under review as a conference paper at ICLR 2027

Who Detects What? Budgeted Detection of Conditional Mechanism Shifts in Frozen-Policy RL Streams

Abstract

Reinforcement learning policies are increasingly deployed as frozen components, yet their environments can change beneath a stable surface: observation marginals remain approximately unchanged while a conditional reward or transition mechanism shifts. We study deployment-time detection: given a frozen-policy stream with a pre-registered episode-level false-alarm budget (alpha=0.05), which detector signal families actually detect which mechanism changes, and at what minimum injected intensity? In two MiniGrid environments we construct controlled mechanism injections—transition slip (seven intensities, a small-marginal-shift variant of the premise) and reward flip (four probabilities, the stable-marginal condition exactly)—verified by independent probes, and compare five detector families in a frozen 165-run matrix. Three findings emerge: (i) detection is mechanism-specific—marginal- and trajectory-statistics detectors are blind to reward flips at all intensities (entailed by their score construction, confirmed empirically), while the conditional discriminator, joint-most-sensitive on slip, is near-blind to weak flips that the value residual detects perfectly; (ii) an episode-stream circular-shift calibration is required for honest false-alarm control, as window-score-sequence shifting is provably weak; (iii) batch-style AUROC rankings do not substitute for budgeted deployment-level detection rates (rho=0.885; one persisted-score seat). A pre-registered hold-out on two unseen intensity levels passes all frozen criteria, replicating the specificity structure, as does a pre-registered cross-cell confirmation of the flip mechanism on the second environment; we release all streams, manifests, per-window artifacts, and the pre-registered protocol.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.