What a Relative Improvement Measures: Reference Values for Latent-Regime Reinforcement Learning

Work on reinforcement learning under non-stationarity reports adaptation as a percentage improvement over a non-adaptive baseline and compares algorithms using that number. This approach merges the adaptation the environment offers, the fraction the agent collects, and how far the baseline falls short. Since only the fraction belongs to the algorithm, the same improvement means different things in different environments. The three enter the reported improvement through an exact identity, so any reported improvement already bounds the adaptation that an algorithm collected. Measuring the fraction instead needs a policy optimal under the agent's own information, which commonly used environments do not carry. In this work, we define a class in which that policy, together with a controller told the regime and one whose belief never moves, follows from the environment's parameters, and release 21 of its configurations. We train seven algorithms on them, and across runs the reported improvement varies by a factor of 17 while the fraction collected varies by a factor of 1.2, so almost everything separating the results is the environment. That term is set before any agent trains, and on regime processes fitted to daily equity returns, selling down a stock position and hedging an option differ tenfold in what adaptation is worth. We therefore report the fraction collected together with the reported improvement. Our configurations give that fraction exactly, and outside the class a published number still bounds it, while one algorithm trained twice, once with the regime revealed and once without, estimates from the gap between the two costs what the regime label itself is worth there.