acceptodds
Under review as a conference paper at ICLR 2027

Rondo: Segment-Level Continuous Batching for Multi-Tenant Flow-VLA Serving

Abstract

Flow-based vision-language-action (VLA) models generate actions via iterative denoising. In multi-tenant serving, diverse robot tasks ranging from reactive obstacle avoidance to precision manipulation naturally require varying numbers of denoising steps. However, executing complete trajectories as indivisible units restricts batching efficiency across differing step counts, while per-step continuous batching incurs prohibitive kernel launch and scheduling overheads. We propose Rondo, a serving framework that partitions denoising trajectories into schedulable segments through a resumable execution abstraction. It combines completion-boundary segmentation with dynamic remaining-work bucketing to prevent segment fragmentation and preserve multi-step execution efficiency. Evaluated with on 8 Ascend 910C NPUs under theoretical load capacity stress testing, Rondo reduces mean serving latency by up to 75.2% relative to Native. In strict real-time closed-loop robot fleet manipulation across 16-32 concurrent robots, Rondo achieves up to 92.3% task success and delivers up to a improvement over existing serving baselines, sustaining robust physical execution at scale where baseline systems suffer catastrophic buffer starvation.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Vision-Language-Action Models, Multi-Tenant Serving, Continuous Batching, Flow Matching, Inference Scheduling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1YFuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on multi-tenant serving for Flow-based VLA models. Treating complete trajectories as indivisible units limits batching efficiency across requests with differing step counts, while per-step continuous batching incurs prohibitive kernel launch and scheduling overheads. The paper propose RONDO, a framework that serves heterogeneous Flow-VLA requests through resumable execution, segmentation at the earliest request completion, and dynamic remaining-work bucketing. Experiments with π0.5 on eight Ascend 910C NPUs show substantial latency reductions and improved task success in wall-clock LIBERO simulations.

Critical strengths
  • The execution abstraction explicitly preserves each request's integration grid, progress, latent state, and conditioning cache, providing a concrete design for heterogeneous VLA batching.
  • The combination of completion boundaries and load-dependent bucketing addresses a meaningful trade-off between batch occupancy and scheduling overhead.
Critical weaknesses
  • The contribution of completion-boundary scheduling is not isolated. TetriServe (ASPLOS 2026) uses multistep scheduling and compares 1-, 5-, and 10-step granularities. Therefore, a fixed-multistep baseline using the same execution engine is important. Existing bucketing ablations cannot determine whether the gains come from the specific completion rule or simply from less frequent scheduling. A matched overhead breakdown would also help establish how much of Iter-CB's slowdown is intrinsic and how much is implementation-dependent.
  • The overall evaluation is somewhat limited; it would be better to include other flow-based VLA/WAM methods, such as Lingbot-VA, Lingbot-VLA, and Fast-WAM.
Questions for authors

Please refer to the weaknesses.

Reviewer_k663OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces RONDO which serves multiple robots sharing a flow-based VLA model. Requests require different numbers of denoising steps, creating a tradeoff. the trade off is that processing entire trajectories blocks new requests, while rescheduling after every step adds overhead. RONDO keeps each request’s intermediate state on the accelerator and executes batches until the earliest request finishes.

Critical strengths
  • shared inference for robot fleets creates queuing and responsiveness problems that single-request acceleration does not address. Supporting heterogeneous denoising budgets is an interesting and reasonable objective.
  • Clear engineering intuition. Resumable execution, persistent conditioning caches, and multi-step scheduling provide a plausible way to reduce scheduling overhead without intentionally changing the model’s computation...
Critical weaknesses
  • weak contribution. Resumable diffusion inference, dynamic batching, and scheduling based on remaining work already appear in systems such as [1]. RONDO’s most distinctive element is its completion-boundary segment rule combined with load-triggered bucketing. They do not establish that the proposed rule is better than straightforward intermediate choices. the experiments mainly compare two extremes (whole trajectory execution and on-step scheduling). the author should compare against fixed segments of 2,4 and 8 step, a time-budgeted segment scheduler, and an event-driven implementation that avoids unnecessary regrouping.
  • overstating claims. the abstract and introducing repeatedly describe "real-world" fleet manipulation and physical execution but the documented robot experiments use only simulation. Physical deployment claims require corresponding robot experiments.
  • The closed-loop timing and failure protocol need clarification. The paper reports 50-action predictions, ten actions executed per cycle, and 1–2 Hz action execution with asynchronous prefetching. provide a precise timing diagram and report buffer occupancy, deadline misses, and timeout failures separately from manipulation failures.
  • The statistical evidence is weak. Figure 4 explicitly uses one run per point. Closed-loop measurements last 180 seconds, and the paper does not report episode denominators, repeated-seed uncertainty, or confidence intervals.
  • The proposed explanation for improved reliability is not directly demonstrated. The authors attribute success gains at similar mean latency to reduced delivery jitter and fewer buffer-starvation events. The reported ablation does not directly measure those mechanisms.

[1] GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads

Questions for authors
  • refer to weaknesses
Reviewer_YvcWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Rondo schedules flow-VLA inference in resumable segments, ending each segment when at least one request completes. It combines this with remaining-work bucketing to reduce scheduling overhead while handling heterogeneous denoising budgets. Experiments with π₀.₅ on Ascend NPUs show lower serving latency and higher task success in concurrent LIBERO simulations.

Critical strengths
  1. The paper identifies a useful trade-off between monolithic execution and scheduling after every denoising step. Completion-based segments provide a simple way to balance responsiveness and scheduling overhead.

  2. The implementation supports heterogeneous denoising budgets without changing pretrained weights or reducing each request’s assigned steps. Keeping intermediate states and conditioning caches on the device makes the design practical.

  3. The evaluation connects serving performance to closed-loop task success. The bucketing ablation and host-overhead measurements also help explain the observed gains.

Critical weaknesses
  1. The closed-loop evaluation is insufficiently specified. Section 5.3 uses 180-second measurement windows and early stopping below 30% success, but does not report episode counts, task composition or repeated-run variability. This makes the large success-rate differences difficult to assess. These experiments are simulations, despite the broader real-world framing.

  2. The baselines do not isolate the proposed segment-length rule. Comparing against single-step and monolithic scheduling leaves open whether a simple fixed segment length would achieve similar gains. The bucketing ablation does not answer this question.

  3. Robustness to changing load remains unclear. Bucketing stays enabled once triggered, while the synthetic experiments use stationary Poisson arrivals. The evaluation does not establish whether this policy remains effective when traffic falls or changes abruptly.

Questions for authors
  1. How many episodes and which LIBERO tasks contribute to each Table 1 entry? How does early stopping affect the reported success rates and comparisons?
  2. How does Rondo compare with fixed segment lengths such as 2, 4 and 8 under the same implementation?
  3. What happens when traffic changes from high to low load after bucketing is activated?

Comments on the paper

1 comment

Dear Reviewers,

The authors have submitted their rebuttal. Please review it and share your comments. If anything is unclear or requires clarification, please let the authors know as soon as possible. The author–reviewer discussion period ends on 18 November.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.