ALLOTS: Adaptive Multi-Agent Reinforcement Learning with Variable Observation and Action Spaces

Many real-world applications involve dynamic entities, e.g., heat pumps and electric vehicles with flexible electricity consumption, which, in multi-agent reinforcement learning (MARL), lead to variable observation and action spaces. Existing MARL methods handle variability in observation or action spaces in isolation, but cannot directly generate entity-aligned action sets that adapt to a changing entity set. To address this issue, we propose the $\underline{\text{A}}$daptive Mu$\underline{\text{L}}$ti-Agent Reinforcement $\underline{\text{L}}$earning with Variable $\underline{\text{O}}$bservation and ac$\underline{\text{T}}$ion $\underline{\text{S}}$paces ($\text{ALLOTS}$) method that simultaneously handles variable observation and action spaces. First, we present a permutation-equivariant set-to-set actor that maps variable observations to entity-aligned variable actions, preserving valid control semantics as the entity set evolves. Second, we present a set-structured critic that encodes entity-level state–action pairs as joint tokens and bootstraps value targets from the next-step active set, enabling stable learning and efficient adaptation without retraining. Experiments on energy system, multi-agent particle, and multi-drone environments show that ALLOTS outperforms state-of-the-art methods in dealing with variable observation and action spaces. Code is available at https://anonymous.4open.science/r/ALLOTS-EFB4.