acceptodds
Under review as a conference paper at ICLR 2027

ANSM: Agile Network Substrate for Cloud-based Agent Sandboxes

Abstract

Agent sandboxes let AI agents execute code, invoke tools, and access data in isolated, short-lived environments. At cloud-provider scale, networking must be provisioned for hundreds of sandboxes per node within milliseconds and must give each tenant access to its VPC and the Internet. We present ANSM, an agile network substrate that separates a high-density data plane on sandbox nodes from tenant-scoped network services on gateway nodes. Sandbox nodes pool pre-addressed virtual interfaces and use programmable forwarding, so every sandbox sees an identical network inside the guest while retaining a unique identity across the deployment. Gateway nodes hold public IP addresses and VPC interfaces as network services shared by all of a tenant's sandboxes, allowing these resources to be provisioned, scaled, and updated independently of sandbox creation. As a result, sandbox startup does not wait for slow cloud-resource provisioning, while public addresses and VPC interfaces remain isolated across tenants. In production, ANSM serves a rapidly growing sandbox workload, setting up each sandbox's network within milliseconds and supporting hundreds of sandboxes per node.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Agent Sandboxes, Cloud Networking, Multi-tenant Networking, VPC Connectivity

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_x81zOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

ANSM is an production networking layer for sandboxes for agents in a cloud environment. ANSM operates in two nodes, sandbox and gateway to solve the challenge of fast setup, high density, connectivity, isolation and identity of running agents.

Critical strengths

This is a valuable experience paper with practical measurements and potential real-world impact.

Critical weaknesses
  • My main concern is venue fit. This paper mostly reads like a very valuable experience report, and I think this fits much better and will receive more attention in a networking-oriented conference like NSDI/SIGCOMM or some other industrial tracks.
  • The evaluation setup is narrow. The setup seems limited, and while targeting at agent, there seems to be no end-to-end measurement on any of the agent turnaround time. There is also no objective measurement against other baselines for main results like Figure 2-3. Only a qualitative comparison with existing work is provided.
  • From the writing, it is unclear to me how the design requirements deviates from existing cloud workloads like serverless (which seems to be also demand the traits laid out by the paper).
Questions for authors

Please see weakness

1 comment

We thank the reviewer for the constructive feedback. We would like to clarify two points.

(1) Relevance to ICLR / agent systems.
We agree that ANSM is a systems paper, but its relevance to ICLR comes from the fact that agents are becoming an important component of modern AI systems, and their practical performance increasingly depends on the underlying infrastructure. Networking is a core part of this infrastructure. It directly affects sandbox startup latency, concurrency, connectivity, and therefore the range of agent workloads that can be deployed in practice. From this perspective, sandbox networking is not merely a generic cloud-operations issue, but an important systems problem for large-scale AI agents. We will revise the paper to make this connection more explicit.

(2) Relationship to serverless workloads.
We agree that sandbox networking shares many similarities with serverless networking, and we believe serverless workloads could also benefit from the ANSM architecture. The main differences lie in the workload requirements. First, sandboxes typically require stronger network isolation: serverless code (e.g., Lambda functions) is usually human-written and reviewed, whereas sandboxed agent workloads are often less predictable and may execute more arbitrary or untrusted code. Second, sandbox lifetimes are often shorter than those of conventional serverless pods/functions, which places stricter demands on high-density, highly elastic network provisioning. These differences motivate our focus on an architecture that decouples network-resource lifetime from sandbox lifetime and supports rapid, isolated, large-scale sandbox creation.

Reviewer_5BhpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a networking system framework designed to accelerate MicroVM startup by decoupling network provisioning from sandbox creation, targeting high-performance, short-lived, and highly concurrent agent sandbox workloads. The authors separate the traditional cloud VM networking architecture into sandbox nodes and gateway nodes. On sandbox nodes, TAP interfaces and IP addresses are preallocated and pooled, while a lightweight, programmable networking layer enables efficient network attachment and resource reuse across MicroVMs. On gateway nodes, the authors leverage k8s Deployments to provide elastic and reliable tenant-specific network services, including dedicated public IP pools and VPC connectivity. Experimental results demonstrate that the proposed architecture can handle its target workload efficiently. The authors also report that the system has been deployed in production at a major cloud provider.

Critical strengths

The paper addresses a real and important problem. Traditional cloud virtualization solutions face significant challenges when handling emerging agent sandbox workloads characterized by short lifetimes and high concurrency, including substantial network provisioning overhead during startup, high overall startup latency, and costly resource management. The authors' effort to identify the characteristics of real-world workloads and develop a tailored solution is commendable.

The proposed framework effectively decouples network resource provisioning and allocation from VM creation, allowing network resources to have longer lifecycles than individual VMs and avoiding the overhead of repeated resource creation. The gateway node design supports both Internet and private VPC connectivity while providing tenant isolation. It also leverages k8s to enable elastic scaling and failure recovery. Furthermore, allowing multiple agents belonging to the same tenant to share ENIs and a pool of public IP addresses can substantially reduce cloud networking resource overhead.

Critical weaknesses

Limited relevance to the ICLR community. While ANSM addresses an important infrastructure challenge for AI agents, its technical contributions primarily concern conventional cloud networking. The paper does not establish how these optimizations specifically advance machine learning research or agent capabilities. Consequently, its significance to the broader ICLR community remains unclear.

Trade-off between resource efficiency and intra-tenant isolation. ANSM improves resource efficiency by allowing sandboxes from the same tenant to share gateway endpoints and public IP addresses. However, this aggregation also introduces potential interference among sandboxes. For example, a buggy or misbehaving agent could generate excessive traffic, saturate shared gateway resources, or trigger IP-based rate limits, affecting other sandboxes within the same tenant. While this trade-off may be reasonable, the paper does not sufficiently discuss its implications or evaluate the extent of such interference. A discussion of this trade-off and an evaluation of intra-tenant performance isolation would strengthen the paper.

Limited generalizable insights and insufficient ablation studies. ANSM primarily combines existing networking techniques into a production system, but the paper provides limited insights into the benefits and trade-offs of individual design choices. Since these mechanisms are tightly coupled and the evaluation focuses mainly on end-to-end performance, it is difficult to identify which optimizations contribute to the improvements. For example, port aggregation reduces OVS port management overhead but may introduce additional forwarding costs and shared-ingress contention, yet no ablation is provided to justify this trade-off. Similar concerns apply to namespace-free networking and ARP elimination. As a result, despite its practical value, the paper offers limited generalizable insights that the broader research community can learn from or apply to other systems.

Questions for authors
  1. System-level novelty: While the paper discusses limitations of existing networking solutions, it remains unclear how ANSM fundamentally differs from prior architectures such as AWS Lambda Hyperplane, which already decouple network resource provisioning from short-lived execution environments. Could the authors clarify what novel architectural insights ANSM offers beyond existing approaches, and which challenges cannot be adequately addressed by prior solutions?

  2. Justification of key design choices: The paper introduces several optimizations, but their motivations and trade-offs are not sufficiently analyzed. What evidence supports the benefits of port aggregation over conventional per-TAP OVS ports? Also, more broadly, could the authors clarify which architectural decisions are essential to ANSM's performance gains and what generalizable insights can be drawn from them? The current end-to-end evaluation makes it difficult to distinguish the contributions of individual mechanisms.

2 comments

We thank the reviewer for the constructive feedback. We would like to clarify three points.

(1) Relevance to ICLR / agent systems.
We agree that ANSM is a systems paper, but its relevance to ICLR comes from the fact that agents are becoming an important component of modern AI systems, and their practical performance increasingly depends on the underlying infrastructure. Networking is a core part of this infrastructure. It directly affects sandbox startup latency, concurrency, connectivity, and therefore the range of agent workloads that can be deployed in practice. From this perspective, sandbox networking is not merely a generic cloud-operations issue, but an important systems problem for large-scale AI agents. We will revise the paper to make this connection more explicit.

(2) Isolation vs. resource efficiency is an explicit, user-controlled trade-off in ANSM.
ANSM already provides strict isolation across tenants. For multiple sandboxes within the same tenant, we intentionally expose the isolation/cost trade-off to the user. A tenant that needs stronger internal isolation can provision multiple VPCs and corresponding multiple NetworkService's to separate different workloads. In the extreme case, a tenant can use one NetworkService per sandbox, which degenerates to a per-sandbox/per-IP model. ANSM therefore does not remove intra-tenant isolation; it makes it a configurable choice, allowing users to decide the appropriate balance between cost and isolation. We will clarify this design point and discuss the trade-off more explicitly in the revision.

(3) Novelty beyond Hyperplane / prior decoupled designs.
We agree that AWS’s 2019 Lambda blog describes that Hyperplane addresses ENI density by decoupling network provisioning from short-lived Lambda functions. However, the public information is limited, and AWS does not disclose the architectural details. ANSM makes this architectural direction explicit and generalizable by abstracting gateway clusters and NetworkServices as first-class resources. This makes network resources independently manageable, scalable, isolatable, and orchestratable, with a lifecycle no longer tied to individual sandboxes. We believe this is the key architectural insight of ANSM and the main lesson that can benefit other large-scale sandbox systems. We will revise the paper to articulate this more clearly and better separate the architectural contribution from implementation-level optimizations.

Thank you for the clarifications. I appreciate the authors' responses and their willingness to revise the paper.

Regarding the relevance to ICLR, I acknowledge that efficient sandbox networking addresses a real and important challenge for agent workloads. I therefore find the authors' explanation reasonable and would be satisfied with this point if the connection to AI agent systems were made more explicit in the revised paper.

However, my other concerns remain largely unresolved:

  1. Intra-tenant isolation: While the configurable isolation model is a useful clarification, it does not address the potential performance interference among sandboxes sharing the same network resources, nor does it quantify the associated trade-offs.

  2. Novelty: While the response does not clearly establish the architectural differences between ANSM and existing systems such as AWS Hyperplane, I acknowledge the limited publicly available details of prior systems and find the authors' explanation acceptable, although some concerns regarding novelty remain.

  3. Ablation studies: My concern regarding the lack of ablation studies and quantitative justification for individual design choices, particularly port aggregation, has not been addressed.

While the response addresses my concern about ICLR relevance, the remaining technical concerns would require further evidence or analysis rather than clarification alone. Therefore, I will maintain my current rating.

Reviewer_CovVOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper addresses the challenge of how to provision isolated network resources for thousands of AI agent sandboxes on cloud platforms. The proposed method, ANSM, allows sandbox nodes to reuse preallocated network interfaces and IP addresses to speed up the sandbox startup by avoiding waiting time for cloud-resource provisioning. Experiments compare ANSM with existing solutions such as E2B, showing lower network provisioning latency and better scalability.

Critical strengths
  1. The paper addresses an important practical challenge in large scale AI agent sandbox environments and presents an effective system design with real world deployment.
  2. Comprehensive evaluation in real world deployment.
Critical weaknesses
  1. The paper presents a solid engineering contribution, but the technical novelty appears limited. For example, the concept of network pooling is similar to E2B. The main contribution is integrating existing techniques into a different system.
  2. The comparison with E2B is not explained clearly, and comparison is not sufficiently fair. E2B is still exhausted for 50 concurrent requests after being expanded to 256 pool size while ANSM has a pool size of 4000. The setup latency comparison should be compared when both methods are not exhausted.
  3. Lack of ablation studies. The paper does not provide ablation studies on the key components of the proposed method, making it unclear where the performance improvements come from. For example, comparing the performance with or without interface pooling and address pre-assignment.
Questions for authors

Why does E2B still require on demand network creation with a pool size of 256 under 50 concurrent requests? If you keep increasing the pool size of E2B until pre-created pool won’t be exhausted, will E2B still be slower in setup time?

1 comment

We thank the reviewer for the thoughtful feedback. We would like to clarify two key points.

(1) ANSM’s main contribution is the sandbox networking architecture, not merely an engineering optimization.
The core idea of ANSM is to move the gateway away from the sandbox node and decouple the lifetime of network resources (e.g., NICs/IPs) from the lifetime of sandbox instances. This architectural separation enables both high-density deployment and fast sandbox startup. Network resources can be prepared and reused independently, so sandbox creation no longer waits for per-instance network provisioning on the critical path. ANSM is therefore more than a warm-pool implementation; it is an architectural design for large-scale, highly dynamic sandbox networking.

(2) The E2B results are based on our measured data, including tuned configurations.
We first evaluated E2B under its default configuration and observed the pool bottleneck reported in the paper. We then increased the relevant parameters; however, under stress testing, we still observed a bottleneck in E2B’s network-interface allocation path. In other words, a larger pool did not remove the provisioning bottleneck in our experiments. Our goal is to report the end-to-end behavior observed in practice, rather than determine whether E2B’s remaining bottleneck comes from an implementation issue or a more fundamental design limitation; such root-cause analysis is outside the scope of this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.