Offline Learning of Flexible Autoscaling Policies from Controller Logs

Autoscaling is the primary mechanism through which cloud platforms adapt compute capacity to changing demand. Yet many production systems continue to rely on hand-designed heuristics. For example, Kubernetes primarily scales resources based on CPU utilization. Reinforcement learning (RL) offers a way to learn autoscaling policies from logs, but online exploration is impractical in these systems. Moreover, deployment objectives can vary substantially across operators and workloads, particularly since different users have different latency-cost trade-offs. Training a separate policy for each objective is computationally expensive and scales poorly with the number of users. We therefore study whether logs generated by autoscaling controllers can enable offline policy training and support different downstream objectives without retraining. Using strong controller baselines and realistic workloads in a distributed-system simulator, we answer both questions affirmatively. Our experiments show that pooling logs from multiple controllers and applying bounded local perturbations to their actions enable offline learning that matches the performance of an objective-aligned expert. Building on this result, we adapt Forward–Backward representations to learn a reward-free model from the same controller logs. After training, when queried with rewards, it comes within $1.3\%$ of policies trained for each objective on latency and rejection and outperforms a preference-conditioned baseline on a new capacity objective ($22.27$ versus $21.36$), while utilization remains harder ($5.59$ versus $6.94$). To our knowledge, this is the first reward-free offline policy extraction for autoscaling trained entirely from offline controller logs.