acceptodds
Withdrawn from ICLR 2027

Autoscaling by Imagination: Graph World Models for Safe Distributed ML Services

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern machine learning services are increasingly deployed as distributed microservice systems, where resource allocation decisions must balance latency, reliability, and cost. In practice, however, autoscaling remains largely reactive: controllers add replicas only after the system is already under pressure, while safer deployments often rely on static over-allocation of resources. Existing autoscalers do not learn a model of how scaling actions propagate through the service graph, and therefore cannot anticipate how a decision will affect future load, latency, cost, and service-level violations before it is executed. To fill this research gap, we present NetDream, a graph world model for safe autoscaling of distributed machine learning services. NetDream learns action-conditioned dynamics over a microservice graph, predicting how load, latency, replicas, and violation risk evolve under candidate scaling actions. At runtime, NetDream uses this learned model for imagination-based planning: it rolls out multiple future scaling trajectories, filters out plans predicted to violate service-level constraints, and executes the first action of the best safe plan. This turns autoscaling from threshold-based reaction into topology-aware planning over a learned model of the system. We evaluate NetDream on a production-grade AWS Kubernetes deployment of Online Boutique, an 11-service, 14-edge microservice graph with a 66-dimensional system state and 48,000 collected transitions, under steady, variable, bursty, and flash-crowd traffic. NetDream is the most cost-efficient safe autoscaler among all evaluated methods. Each marginal replica-step it spends buys safety 2.7 times more costefficiently than the best static over-provisioning strategy. On variable-traffic workloads specifically, it achieves about 5 times fewer violations than reactive autoscaling while using only about half the cost of the strongest over-provisioning baseline. Code, trained models, and infrastructure manifests are released anonymously at https://anonymous.4open.science/r/netdream-FCF9.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
world models, microservices, distributed ML systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tUpyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NetDream, a graph world model for safe autoscaling in distributed machine learning services. It uses a graph attention network to predict future system states, rewards, and service-level violations under scaling actions. A model-based planner simulates candidate action sequences and filters out unsafe plans before execution. Experiments on an AWS Kubernetes deployment of Online Boutique show improved cost–safety trade-offs over reactive and static autoscaling baselines.

Critical strengths

The paper addresses a meaningful resource management problem by incorporating predictive planning into Kubernetes autoscaling.

The experiments include an AWS Kubernetes deployment, multiple traffic conditions, and additional ablation studies to evaluate the proposed framework.

Critical weaknesses

The framework mainly combines existing GNN-based dynamics prediction, model predictive control (MPC), and threshold-based safety filtering. The proposed "imagination-based planning" is essentially a standard model-based rollout procedure, and the key methodological novelty remains unclear.

Although Section 2 discusses GraphPilot and KARMA, neither is directly evaluated. The generic PPO baseline does not adequately represent existing graph-based autoscaling approaches, making it difficult to assess improvements over closely related methods.

Appendix C.2 (Table 15) shows that the MLP achieves lower 5-step prediction MSE than GAT (0.00131 vs. 0.00370), despite the proposed planner using a five-step horizon. Moreover, the ablations only compare prediction accuracy, without evaluating alternative graph architectures under the same end-to-end planning framework.

The real-cluster experiments use only three random seeds. Appendix A.6 (Table 11) reports 19.4 average SLO violations for NetDream-Safe versus 19.6 for HPA-min3, with a 95% confidence interval of [-13.7, +13.3] for the difference. These results support comparable safety at lower cost rather than a statistically reliable safety improvement. Figure 2 also omits uncertainty estimates.

Real-cluster experiments are restricted to a single Online Boutique application with 11 microservices and five-minute episodes. Despite targeting distributed machine learning services, the paper does not evaluate representative ML serving workloads or different real-world service topologies.

Questions for authors

How was the offline training dataset collected?

Why is a separate reward prediction head necessary?

Reviewer_cZuaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a graph world-modeling approach called NetDream, which turns distributed ML autoscaling from reactive threshold control into predictive safe planning and learns how load, latency, replicas, and failures evolve across a microservice graph. NetDream is validated on a production-grade Kubernetes deployment of Online Boutique and achieves a stronger cost–safety frontier than reactive HPA, static over-provisioning, model-free RL, and unsafe planning ablations.

Critical strengths

NetDream is validated on a production-grade Kubernetes deployment of Online Boutique and achieves a stronger cost–safety frontier than reactive HPA, static over-provisioning, model-free RL, and unsafe planning ablations.

Critical weaknesses
  1. Figure 1 should be redrawn; it looks more like an AI-generated figure.

  2. The definition of an SLO default event is inconsistent. The authors define the violation indicator as latency for either service exceeding the SLO, but also say: "The SLO violation indicator is triggered when p95 latency exceeds 500 ms or error rate exceeds 5%".

  3. The related work is not well categorized. The related work is written in one long paragraph. What are the existing problems in previous work, and what issues does this paper try to solve?

  4. In Table 22, as the cost weights λ increase from 0.001 to 0.01, the HPA violations rise from 1.7 to 5.0. Since HPA does not select its actions based on λ used in NetDream, its violation outcomes should not change when this coefficient is altered while all other experimental conditions remain constant.

  5. The baselines selected are very weak, the authors only compare against simple algorithmic paradigms rather than their exact systems. Random is not directly relevant to active control on graph structures; model-free PPO is not a representative graph-based reinforcement learning method.

  6. Many production-grade machine learning (ML) platforms incorporate larger-scale computational graphs, heterogeneous accelerators, multi-layered model-serving architectures, and more complex routing strategies. Current experimental results are limited to constrained Kubernetes test environments.

  7. The mathematical formulas should be numbered.

  8. Key experiments on prediction accuracy are derived from simulators, lack multi-step prediction and action-sequencing validation based on independent EKS trajectories.

Questions for authors
  1. The related work is not well categorized. The related work is written in one long paragraph. What are the existing problems in previous work, and what issues does this paper try to solve?

  2. Current experimental results are limited to constrained Kubernetes test environments, can they guarantee that the same performance gains would persist in much larger-scale production deployments?

Reviewer_V3jbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NetDream, a graph world model-based framework for safe autoscaling of distributed machine learning services, aiming to address the delayed responses and high resource costs of conventional autoscaling approaches. NetDream employs a graph neural network to predict how different scaling decisions affect future system states and the risk of service-level objective (SLO) violations. By integrating model predictive control with an explicit safety filter, the framework selects cost-efficient scaling actions that are predicted to satisfy SLO constraints. Experimental results demonstrate that NetDream can effectively reduce SLO violations while achieving lower resource costs than static over-provisioning strategies.

Critical strengths
  • The paper addresses an important challenge in microservice autoscaling, where conventional reactive approaches struggle to balance service reliability and resource efficiency. NetDream integrates a graph world model, model predictive control, and an explicit safety filtering mechanism to capture inter-service dependencies and evaluate the potential effects of scaling decisions on future system states and SLO violation risks before execution. Compared with traditional reactive resource allocation, this predictive planning approach offers practical value and provides an interesting direction for improving autoscaling decisions.
  • The authors implement and deploy NetDream on a real AWS Kubernetes cluster and evaluate its performance under different workload patterns against conventional HPA, static over-provisioning strategies, and reinforcement learning-based methods. In addition, prediction accuracy evaluations and ablation studies are conducted to examine the effectiveness of graph-based modeling and safety filtering. The experimental results show that NetDream can achieve a relatively low SLO violation rate while substantially reducing resource costs compared with static over-provisioning, demonstrating its potential for practical deployment.
Critical weaknesses
  • NetDream primarily integrates existing techniques, including graph neural networks, world models, model predictive control, and safety filtering. Its fundamental technical novelty over existing predictive autoscaling and model-based reinforcement learning approaches remains insufficiently clear. Furthermore, the paper lacks direct comparisons with representative methods such as GraphPilot, GRAF, and KARMA, making it difficult to fully establish the technical competitiveness and performance advantages of NetDream over recent related work.
  • NetDream's safety filtering mechanism relies on SLO violation probabilities predicted by the learned world model and therefore cannot provide strict guarantees that the selected scaling actions will satisfy SLO constraints in practice, particularly under sudden traffic spikes or workload distribution shifts. Moreover, the evaluation is primarily conducted on a single Online Boutique application, whose scale and workload characteristics may not adequately represent more complex distributed machine learning services. The real-cluster experiments are also limited to three random seeds, and some comparisons against static over-provisioning baselines do not demonstrate statistically significant improvements in SLO violations. These limitations raise concerns about the reliability and generalizability of the proposed approach in larger and more diverse real-world environments.
Questions for authors
  1. NetDream primarily combines existing techniques, including graph neural networks, world models, model predictive control, and safety filtering. Compared with existing GNN-based predictive autoscaling and model-based reinforcement learning approaches, what are the distinctive algorithmic contributions of this work? Could the authors further clarify the core technical innovations beyond the integration and application of established methods?
  2. The paper lacks direct comparisons with representative methods such as GraphPilot, GRAF, and KARMA. Could the authors include more competitive state-of-the-art baselines to better demonstrate NetDream's performance advantages, or justify why such comparisons are infeasible?
  3. NetDream relies on world-model-predicted SLO violation probabilities for safety filtering, with a default threshold of 0.5. How do the authors assess whether these predicted probabilities accurately reflect the actual risk of SLO violations? In particular, how robust is the safety filter under sudden traffic spikes or workload distribution shifts? Furthermore, when all candidate plans are classified as unsafe, could the fallback strategy of maintaining the current replica counts potentially aggravate SLO violations under rapidly increasing workloads?
  4. NetDream relies on randomly sampled scaling sequences and multi-step world-model rollouts, but the action space grows rapidly with the number of microservices and planning horizon. Could the authors further analyze the trade-offs among candidate count, planning horizon, computational overhead, and decision quality, particularly in larger microservice systems?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.