acceptodds
Under review as a conference paper at ICLR 2027

Unseen Is Not New: Partial Identification of Capability Expansion in Reinforcement Learning with Verifiable Rewards

Abstract

Reinforcement learning with verifiable rewards (RLVR) can expose reasoning behaviors that were unseen in a finite base-model sample. However, this observation does not establish capability expansion: a mode with zero probability and one with small positive probability are arbitrarily difficult to distinguish at any finite rollout budget. To address this ambiguity, we develop a probability-spectrum account of what checkpoint evaluations can identify. First, absolute continuity of response laws transports through semantic auditing, yielding support constraints and sharp divergence-to-visibility envelopes. Next, we show that expected discovery has a Hausdorff moment representation: its infinite curve identifies the positive mode-mass spectrum, whereas a finite prefix determines a sharp set of compatible future curves. Consequently, checkpoint rankings can reverse with the rollout budget, and semantic coarsening can conceal opposing probability flows. We connect these results to simultaneous probability-change regions and an access/execution analysis that separates natural conditioning from assigned interventions. Finally, exact certificates and matched neural checkpoints demonstrate identifiable amplification and a Transformer budget crossover. Under the verified full-softmax sampler, architectural support is equal even when observed sets differ. These results support capability claims about probability reallocation and operational visibility without treating finite nonobservation as evidence of structural novelty.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.