BIRD: Belief-Encoder Index Restless Diffusion for Partially Observable RMABs

Restless multi-armed bandits (RMABs) provide a principled framework for resource-constrained sequential decision-making, yet classical Whittle-index methods assume fully observable, discrete states and known dynamics. These assumptions break down in partially observable RMABs with continuous latent states and unknown or drifting dynamics, motivating stochastic score models beyond deterministic rankings. We present BIRD(Belief Encoder Index Restless Diffusion), a scalable score-based algorithm that uses a shared BeliefEncoder and diffusion actor to sample stochastic priority scores, together with a global Top-K projection to enforce the hard budget. We show that the Top-K projection maps score-space policy mirror descent (PMD) to a corresponding update over budget-feasible RMAB actions. Across synthetic and MIMIC-derived environments, BIRD outperforms random, greedy, NeurWIN, PPO, and learned-rollout baselines. Actor ablations isolate the benefit of the shared per-arm diffusion actor over deterministic MLP actor, Gaussian actor, and global joint-arm diffusion variants.