acceptodds
Under review as a conference paper at ICLR 2027

HexAGenT: Workflow- and Heterogeneity-Aware Scheduling for Agentic LLM Serving

Abstract

Serving agentic LLM applications requires scheduling dependent model calls to meet service-level objectives (SLOs) for complete user workflows. This is difficult when calls and branches emerge during execution, external tools delay progress, and prefill-decode (P-D) disaggregation places each call on separate, heterogeneous workers. Priorities based only on individual calls overlook workflow dependencies, while workflow priorities alone do not account for placement-dependent queueing, transfer, and KV-capacity costs. We present HexAGenT, a workflow- and heterogeneity-aware scheduler that couples workflow progress with joint P-D placement. HexAGenT maintains a critical-path horizon over revealed calls and completed tool phases, then normalizes projected call completion from workflow arrival by this horizon. It uses the resulting priorities to plan queue order and worker pairs, updating projected queues and capacity after each assignment. Local and global KV-prefix reuse refine per-worker prefill costs, and asynchronous plan application allows workers to continue serving during scheduling. In trace-driven simulation with two LLMs, four agentic workloads, and three homogeneous and heterogeneous cluster settings, HexAGenT reduces the SLO scaling factor required for and workflow attainment by up to and , respectively, relative to the best evaluated baseline at each target. Gains also persist on homogeneous workers, and additional experiments characterize sensitivity to service-time estimation error and aggregate planning cost.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
LLM inference, P-D disaggregation, workflow- aware scheduling, agentic LLM workflows.

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_RWy5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HexAGenT schedules dynamic agent workflows on heterogeneous prefill–decode clusters. It jointly determines call priorities and worker assignments using projected completion times normalized by each workflow’s revealed critical-path length. The planner accounts for queueing, KV-cache reuse, transfer costs, and memory constraints.

Critical strengths

The joint planning of workflow priorities and P-D placement is the paper’s strongest contribution. Incorporating placement costs into workflow prioritization provides a useful way to account for dependencies and hardware heterogeneity within the same scheduling decision.

Critical weaknesses
  1. The evaluation does not isolate the benefit of joint planning: the baselines differ in both ordering and placement, and component ablations are missing. Comparisons using the same placement mechanism across ordering policies would make the attribution clearer. An explicitly adapted Autellix priority rule could provide an additional workflow-aware baseline.

  2. The refinement criterion needs stronger justification. Eq. (1) targets a percentile of Cw/SwC_w/S_w, while refinement prioritizes reducing the maximum projected ratio based on Hw(t)H_w(t). These criteria can favor different schedules, but the paper does not examine the implications of this trade-off for Req95/Req99.

Questions for authors
  1. For ready calls within the same workflow, Eq. (7) orders them by Δs(c,t)\Delta_s(c,t). How does this ordering affect workflow completion in a branching example, and why is refinement disabled for fan-out cases?

  2. How are deadlines constructed for SGLang-LLF and Workflow-LLF? Please specify any SLO multiplier and whether the H100 reference SwS_w is used only for evaluation or also to set scheduling deadlines.

6 comments

Weakness 1 .

The evaluation does not isolate the benefit of joint planning: the baselines differ in both ordering and placement, and component ablations are missing. Comparisons using the same placement mechanism across ordering policies would make the attribution clearer. An explicitly adapted Autellix priority rule could provide an additional workflow-aware baseline.

We appreciate this suggestion. Section 6.3 and Figure 5 compare HexAGenT with Workflow-LLF using the same joint worker-pair selector, queue/capacity estimates, and runtime reuse. The experiment uses Llama and the Hetero-2 GPU composition.

Ablation configurations. Refinement is held off throughout this existing four-component comparison. Each configuration changes one mechanism:

  • HexAGenT: retain workflow-relative priority, joint placement, queue awareness, and cache-aware planning.
  • Priority (LLF): substitute Workflow-LLF priority while retaining identical placement logic.
  • Placement (workload-balanced): substitute workload-balanced dispatch across feasible workers.
  • Queue-blind planning: omit queue waiting from placement estimates.
  • Cache-blind planning: omit cache information from planning while retaining runtime KV reuse.

Llama ablation results on Hetero-2. Each cell reports α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better.

TraceHexAGenTPriority (LLF)Placement (workload-balanced)Queue-blind planningCache-blind planning
ShareGPT1.7399 / 1.84992.3556 / 2.60632.4829 / 2.59923.1198 / 3.21791.7521 / 1.8168
BFCL0.7105 / 0.77640.7104 / 0.77640.9260 / 1.02940.7561 / 0.93060.7177 / 0.7839
LATS0.7292 / 0.75360.7512 / 0.77480.9141 / 0.92890.8847 / 0.90720.8174 / 0.8215
Terminal-Bench1.3301 / 1.40331.9483 / 2.31951.6758 / 1.85551.9488 / 2.09111.6174 / 1.7606

Performance change for the same ablation configurations. Each cell gives the percentage change in α0.95\alpha_{0.95} / α0.99\alpha_{0.99} relative to HexAGenT, computed as 100(rablation/rHexAGenT−1)100(r_{\mathrm{ablation}}/r_{\mathrm{HexAGenT}}-1). Positive values indicate degradation.

TracePriority (LLF)Placement (workload-balanced)Queue-blind planningCache-blind planning
ShareGPT+35.38% / +40.89%+42.70% / +40.51%+79.30% / +73.95%+0.70% / -1.79%
BFCL-0.01% / -0.01%+30.34% / +32.58%+6.41% / +19.85%+1.02% / +0.96%
LATS+3.02% / +2.82%+25.36% / +23.27%+21.32% / +20.39%+12.09% / +9.02%
Terminal-Bench+46.48% / +65.29%+25.99% / +32.22%+46.52% / +49.01%+21.60% / +25.46%

What these comparisons isolate. The priority comparison isolates ordering; the remaining configurations isolate placement and its information inputs. All configurations use the same workload and evaluation references.

Additional comparison with Autellix. We have also included ATLAS, the program-aware scheduler in Agentix (formerly Autellix), in all 24 main-evaluation scenarios (Section 6.1; Figures 3–4). Its attained-service-based prioritization provides an additional workflow-aware baseline. This comparison evaluates the complete scheduling policies, while the matched-placement comparison with Workflow-LLF in Section 6.3 isolates the contribution of HexAGenT’s priority mechanism. The following tables report all nine methods across the 24 scenarios. Each cell gives α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better.

Homo results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS6.3694 / 8.32941.7885 / 1.97601.5850 / 1.66765.1824 / 6.2777
Workflow-LLF4.0372 / 5.35941.7885 / 1.97611.5910 / 1.60322.2923 / 2.5853
Po2C+Call-LLF6.4879 / 7.55581.7786 / 1.85701.5569 / 1.55833.0181 / 3.3491
ATLAS3.5888 / 4.50931.5893 / 1.62091.5483 / 1.56412.6440 / 2.6998
AMPD5.4992 / 6.96731.5718 / 1.71371.6209 / 1.67625.0571 / 5.1545
SAGA7.3060 / 12.25641.5593 / 1.58341.5291 / 1.54922.9564 / 3.0246
Parrot6.1484 / 8.36301.5928 / 1.61701.5315 / 1.53472.9028 / 2.9566
Continuum5.8510 / 7.54711.5940 / 1.61701.6478 / 1.68453.9755 / 4.8083
HexAGenT2.9881 / 3.05511.5454 / 1.57801.5140 / 1.52112.2670 / 2.3274

Homo results for Qwen. The table reports all methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS4.9928 / 7.85655.8287 / 9.32301.5303 / 1.58352.0317 / 2.7232
Workflow-LLF3.5562 / 4.10085.7637 / 6.84501.5303 / 1.58351.7095 / 2.0354
Po2C+Call-LLF4.7672 / 5.87345.3650 / 9.09871.5155 / 1.54761.7502 / 1.8729
ATLAS2.8335 / 3.35935.4773 / 5.73721.5089 / 1.51951.7381 / 1.9250
AMPD4.3042 / 6.30955.9795 / 8.22591.8571 / 2.02941.9426 / 1.9759
SAGA4.7207 / 5.31885.9306 / 10.20791.5123 / 1.52631.5479 / 1.5914
Parrot3.8260 / 5.65275.6934 / 8.74271.5260 / 1.53061.5318 / 1.5725
Continuum4.8972 / 7.28035.4239 / 7.26042.0396 / 2.23192.1792 / 2.3852
HexAGenT2.4637 / 2.56613.8878 / 3.93391.5025 / 1.52021.5660 / 1.5773

Hetero-1 results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS2.2156 / 3.57892.0413 / 2.24331.3242 / 1.35081.7430 / 2.0680
Workflow-LLF2.0125 / 2.40091.9796 / 2.11331.3418 / 1.38041.5423 / 1.5873
Po2C+Call-LLF1.6965 / 2.02951.9954 / 2.28601.2903 / 1.33721.4024 / 1.4605
ATLAS1.7340 / 1.97291.7893 / 1.97591.4635 / 1.48051.5057 / 1.5470
AMPD2.0550 / 2.45011.8173 / 2.20891.5130 / 1.51951.8938 / 2.7287
SAGA1.7831 / 2.46071.9357 / 2.25411.5003 / 1.51471.4605 / 1.4772
Parrot1.8800 / 2.27912.1240 / 2.39061.5032 / 1.51981.4209 / 1.4477
Continuum1.8985 / 2.49621.8947 / 2.34251.5286 / 1.54811.4897 / 1.4904
HexAGenT1.6275 / 1.81631.8125 / 1.85471.0004 / 1.00631.1841 / 1.1894

Hetero-1 results for Qwen. The table reports all methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS1.8886 / 2.510420.3224 / 34.28071.4585 / 1.47541.3535 / 1.3907
Workflow-LLF1.6479 / 2.203919.4959 / 30.15161.4585 / 1.47531.3195 / 1.3398
Po2C+Call-LLF1.5064 / 1.595120.2461 / 34.00811.4337 / 1.48791.2827 / 1.3430
ATLAS1.5173 / 1.597914.9956 / 18.88061.4621 / 1.48201.4367 / 1.4450
AMPD1.7598 / 1.805819.1866 / 31.54331.5036 / 1.52191.5086 / 1.5297
SAGA1.5501 / 1.647920.3016 / 33.17901.4904 / 1.51771.3665 / 1.4379
Parrot1.5152 / 1.521619.1765 / 32.12171.5254 / 1.52681.3631 / 1.4011
Continuum1.6186 / 2.264420.2948 / 33.74711.5272 / 1.57971.4356 / 1.4363
HexAGenT1.4139 / 1.517611.6166 / 11.65840.9977 / 1.00511.0314 / 1.0586

Hetero-2 results for Llama. The table reports all methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS26.0393 / 37.30160.9839 / 1.23681.4121 / 1.46585.3190 / 5.9227
Workflow-LLF11.9316 / 18.62610.9839 / 1.23691.4307 / 1.51722.1625 / 2.7362
Po2C+Call-LLF25.9613 / 37.30160.9672 / 1.22351.2656 / 1.31871.5862 / 1.8934
ATLAS15.9441 / 26.39011.0554 / 1.14401.4585 / 1.47921.9331 / 2.1068
AMPD27.6119 / 49.92551.0735 / 1.22611.5144 / 1.53371.7943 / 1.8331
SAGA27.7117 / 50.15330.9961 / 1.10171.4915 / 1.49531.7535 / 1.8099
Parrot26.0738 / 37.34861.1009 / 1.22741.4875 / 1.49591.7232 / 1.7691
Continuum26.5175 / 36.66271.0572 / 1.20181.6064 / 1.64972.9777 / 4.2461
HexAGenT8.6013 / 8.61760.7104 / 0.77640.9852 / 1.00001.2361 / 1.2710

Hetero-2 results for Qwen. The table reports all methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS26.2869 / 38.30890.9314 / 0.95451.4849 / 1.62022.6888 / 3.1597
Workflow-LLF14.7521 / 16.96110.9314 / 0.95451.4723 / 1.48511.4292 / 1.7938
Po2C+Call-LLF26.7000 / 37.97270.8989 / 0.93731.4564 / 1.48471.1504 / 1.3040
ATLAS16.9817 / 18.66861.0089 / 1.01591.4730 / 1.47711.4282 / 1.5051
AMPD28.8821 / 38.52831.0029 / 1.01001.4893 / 1.50541.4775 / 1.6615
SAGA24.7541 / 34.02691.0041 / 1.00821.4889 / 1.49371.2302 / 1.2970
Parrot29.6037 / 40.28411.0101 / 1.01921.4891 / 1.49491.2005 / 1.2295
Continuum26.8737 / 38.30081.0069 / 1.01211.8650 / 2.03401.5684 / 1.9439
HexAGenT7.8969 / 8.15370.6492 / 0.66340.8605 / 0.96671.0098 / 1.0135

Weakness 2 .

The refinement criterion needs stronger justification. Eq. (1) targets a percentile of Cw/SwC_w/S_w, while refinement prioritizes reducing the maximum projected ratio based on Hw(t)H_w(t). These criteria can favor different schedules, but the paper does not examine the implications of this trade-off for Req95/Req99.

We appreciate the reviewer for highlighting this ambiguity. We would like to clarify that these criteria should favor the same schedules, while refinement should follow the attainment target τ\tau in Equation (1). The maximum projected ratio in the original writing corresponds to the special case when τ=1\tau=1, but τ\tau can be other values as well. We apologize for omitting this qualification, which made the criteria appear inconsistent.

Refinement criterion and its relationship to Equation (1). For each candidate schedule, refinement evaluates the τ\tau-quantile of projected workflow ratios over the same active-workflow set. Choosing τ=0.95\tau=0.95 or τ=0.99\tau=0.99 therefore targets the projected 95th or 99th percentile, respectively. These projections use the currently revealed workflow horizon Hw(t)H_w(t), while final evaluation uses completed workflow durations and the fixed standalone reference $S_w$. The projected percentile thus guides refinement toward the same attainment level specified in Equation (1). Section 5.3 and Algorithm 1 now clarify this parameterization.

Reviewer_vVZtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the online scheduling challenge for agentic LLM workflows on heterogeneous prefill-decode (P-D) disaggregated clusters, where existing call-level or workflow-only schedulers fail to jointly account for dynamic workflow dependencies and placement-dependent queueing, transfer, and KV-cache costs. It presents HEXAGENT, a workflow- and heterogeneity-aware scheduler that ranks calls by projected completion normalized against each workflow’s revealed critical-path horizon, and jointly optimizes P-D worker pair selection using KV-aware prefill costs, projected queue states, and downstream decode availability, with asynchronous plan execution to avoid interrupting active serving. Evaluated via trace-driven simulation across two LLMs, four agentic workloads, and three cluster settings, HEXAGENT reduces the SLO scaling factor required for 95% and 99% workflow attainment by up to 49.7% and 63.0% respectively over the strongest baselines, while showing robustness to moderate service-time estimation errors.

Critical strengths
  1. The paper proposes a principled joint scheduling framework that unifies workflow-level critical-path prioritization and heterogeneous prefill-decode placement optimization, filling a key gap between end-to-end workflow SLO objectives and disaggregated execution decisions in agentic LLM serving.
  2. The experimental evaluation is rigorous and practically relevant, covering four distinct agentic workloads, two large language models, and three homogeneous/heterogeneous cluster settings, with substantial and consistent SLO attainment improvements plus supplementary robustness and overhead analyses that strongly validate real-world utility.
  3. The scheduler is designed for deployability with asynchronous planning and state-validated plan application that never interrupts active serving, plus fine-grained KV-prefix reuse cost modeling, making it directly applicable to production LLM serving systems rather than remaining a purely theoretical algorithm.
Critical weaknesses
  1. All performance results are derived from trace-driven simulation rather than end-to-end deployment on a real LLM serving cluster, so the reported SLO attainment gains remain unvalidated against real-world system dynamics like network jitter, KV cache contention, and actual scheduling overhead, materially reducing the practical persuasiveness of the solution.
  2. The experimental baselines are limited to adapted classical scheduling policies (FCFS and LLF at call/workflow levels) and lack direct comparisons with state-of-the-art program-aware agent serving systems such as Parrot and Continuum, preventing a rigorous assessment of HEXAGENT’s relative advantages over existing specialized agentic scheduling work.
  3. The entire scheduling framework relies on empirically designed heuristic rules for both workflow prioritization and joint P-D placement, with no formal approximation ratio or competitive performance guarantee derived, which limits the theoretical depth and rigor of the algorithmic contribution.
Questions for authors
  1. Could you explain why state-of-the-art program-aware agent serving systems such as Parrot and Continuum are absent from the baseline comparison, and how the reported SLO attainment gains would likely change if these specialized agent-scheduling methods were included?
  2. Have you performed any validation of HEXAGENT on a real prefill-decode disaggregated serving cluster instead of only trace-driven simulation, and if so, do real-world measurements align with the magnitude of SLO improvements reported in the paper?
  3. Is there any formal performance guarantee (such as a competitive ratio or approximation bound) for the proposed revealed-horizon prioritization and joint placement heuristic, or is the scheduling strategy justified entirely by empirical performance?
5 comments

Weakness 1 .

All performance results are derived from trace-driven simulation rather than end-to-end deployment on a real LLM serving cluster, so the reported SLO attainment gains remain unvalidated against real-world system dynamics like network jitter, KV cache contention, and actual scheduling overhead, materially reducing the practical persuasiveness of the solution.

Question 2 .

Have you performed any validation of HEXAGENT on a real prefill-decode disaggregated serving cluster instead of only trace-driven simulation, and if so, do real-world measurements align with the magnitude of SLO improvements reported in the paper?

We appreciate the emphasis on validation under real serving conditions. Section 6.2 and Figures 3-4 now report real-system experiments for both models, four traces, and the three cluster settings. All additional experiments reported in this rebuttal were also conducted on real GPU serving clusters. The scheduler uses service-time estimates from a simulator calibrated against GPU profiling to align its estimates with measured GPU behavior.

Magnitude of the measured improvements. The real-system measurements show reductions of up to 46.47% in α0.95\alpha_{0.95} and 53.73% in α0.99\alpha_{0.99} relative to the strongest baseline at each percentile. Representative reductions include 16.74%/32.25% on Homo/Llama/ShareGPT and 22.47%/24.75% on Hetero-1/Llama/LATS. These measurements substantiate the substantial SLO-attainment gains reported in the paper; Section 6.2 and Figures 3–4 present the updated real-system results.

Complete end-to-end results. The following tables report all nine methods across the 24 scenarios. Each cell gives α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better.

Homo results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS6.3694 / 8.32941.7885 / 1.97601.5850 / 1.66765.1824 / 6.2777
Workflow-LLF4.0372 / 5.35941.7885 / 1.97611.5910 / 1.60322.2923 / 2.5853
Po2C+Call-LLF6.4879 / 7.55581.7786 / 1.85701.5569 / 1.55833.0181 / 3.3491
ATLAS3.5888 / 4.50931.5893 / 1.62091.5483 / 1.56412.6440 / 2.6998
AMPD5.4992 / 6.96731.5718 / 1.71371.6209 / 1.67625.0571 / 5.1545
SAGA7.3060 / 12.25641.5593 / 1.58341.5291 / 1.54922.9564 / 3.0246
Parrot6.1484 / 8.36301.5928 / 1.61701.5315 / 1.53472.9028 / 2.9566
Continuum5.8510 / 7.54711.5940 / 1.61701.6478 / 1.68453.9755 / 4.8083
HexAGenT2.9881 / 3.05511.5454 / 1.57801.5140 / 1.52112.2670 / 2.3274

Homo results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS4.9928 / 7.85655.8287 / 9.32301.5303 / 1.58352.0317 / 2.7232
Workflow-LLF3.5562 / 4.10085.7637 / 6.84501.5303 / 1.58351.7095 / 2.0354
Po2C+Call-LLF4.7672 / 5.87345.3650 / 9.09871.5155 / 1.54761.7502 / 1.8729
ATLAS2.8335 / 3.35935.4773 / 5.73721.5089 / 1.51951.7381 / 1.9250
AMPD4.3042 / 6.30955.9795 / 8.22591.8571 / 2.02941.9426 / 1.9759
SAGA4.7207 / 5.31885.9306 / 10.20791.5123 / 1.52631.5479 / 1.5914
Parrot3.8260 / 5.65275.6934 / 8.74271.5260 / 1.53061.5318 / 1.5725
Continuum4.8972 / 7.28035.4239 / 7.26042.0396 / 2.23192.1792 / 2.3852
HexAGenT2.4637 / 2.56613.8878 / 3.93391.5025 / 1.52021.5660 / 1.5773

Hetero-1 results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS2.2156 / 3.57892.0413 / 2.24331.3242 / 1.35081.7430 / 2.0680
Workflow-LLF2.0125 / 2.40091.9796 / 2.11331.3418 / 1.38041.5423 / 1.5873
Po2C+Call-LLF1.6965 / 2.02951.9954 / 2.28601.2903 / 1.33721.4024 / 1.4605
ATLAS1.7340 / 1.97291.7893 / 1.97591.4635 / 1.48051.5057 / 1.5470
AMPD2.0550 / 2.45011.8173 / 2.20891.5130 / 1.51951.8938 / 2.7287
SAGA1.7831 / 2.46071.9357 / 2.25411.5003 / 1.51471.4605 / 1.4772
Parrot1.8800 / 2.27912.1240 / 2.39061.5032 / 1.51981.4209 / 1.4477
Continuum1.8985 / 2.49621.8947 / 2.34251.5286 / 1.54811.4897 / 1.4904
HexAGenT1.6275 / 1.81631.8125 / 1.85471.0004 / 1.00631.1841 / 1.1894

Hetero-1 results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS1.8886 / 2.510420.3224 / 34.28071.4585 / 1.47541.3535 / 1.3907
Workflow-LLF1.6479 / 2.203919.4959 / 30.15161.4585 / 1.47531.3195 / 1.3398
Po2C+Call-LLF1.5064 / 1.595120.2461 / 34.00811.4337 / 1.48791.2827 / 1.3430
ATLAS1.5173 / 1.597914.9956 / 18.88061.4621 / 1.48201.4367 / 1.4450
AMPD1.7598 / 1.805819.1866 / 31.54331.5036 / 1.52191.5086 / 1.5297
SAGA1.5501 / 1.647920.3016 / 33.17901.4904 / 1.51771.3665 / 1.4379
Parrot1.5152 / 1.521619.1765 / 32.12171.5254 / 1.52681.3631 / 1.4011
Continuum1.6186 / 2.264420.2948 / 33.74711.5272 / 1.57971.4356 / 1.4363
HexAGenT1.4139 / 1.517611.6166 / 11.65840.9977 / 1.00511.0314 / 1.0586

Hetero-2 results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS26.0393 / 37.30160.9839 / 1.23681.4121 / 1.46585.3190 / 5.9227
Workflow-LLF11.9316 / 18.62610.9839 / 1.23691.4307 / 1.51722.1625 / 2.7362
Po2C+Call-LLF25.9613 / 37.30160.9672 / 1.22351.2656 / 1.31871.5862 / 1.8934
ATLAS15.9441 / 26.39011.0554 / 1.14401.4585 / 1.47921.9331 / 2.1068
AMPD27.6119 / 49.92551.0735 / 1.22611.5144 / 1.53371.7943 / 1.8331
SAGA27.7117 / 50.15330.9961 / 1.10171.4915 / 1.49531.7535 / 1.8099
Parrot26.0738 / 37.34861.1009 / 1.22741.4875 / 1.49591.7232 / 1.7691
Continuum26.5175 / 36.66271.0572 / 1.20181.6064 / 1.64972.9777 / 4.2461
HexAGenT8.6013 / 8.61760.7104 / 0.77640.9852 / 1.00001.2361 / 1.2710

Hetero-2 results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS26.2869 / 38.30890.9314 / 0.95451.4849 / 1.62022.6888 / 3.1597
Workflow-LLF14.7521 / 16.96110.9314 / 0.95451.4723 / 1.48511.4292 / 1.7938
Po2C+Call-LLF26.7000 / 37.97270.8989 / 0.93731.4564 / 1.48471.1504 / 1.3040
ATLAS16.9817 / 18.66861.0089 / 1.01591.4730 / 1.47711.4282 / 1.5051
AMPD28.8821 / 38.52831.0029 / 1.01001.4893 / 1.50541.4775 / 1.6615
SAGA24.7541 / 34.02691.0041 / 1.00821.4889 / 1.49371.2302 / 1.2970
Parrot29.6037 / 40.28411.0101 / 1.01921.4891 / 1.49491.2005 / 1.2295
Continuum26.8737 / 38.30081.0069 / 1.01211.8650 / 2.03401.5684 / 1.9439
HexAGenT7.8969 / 8.15370.6492 / 0.66340.8605 / 0.96671.0098 / 1.0135

Scheduling overhead during serving. Section 6.4 and Appendix F report scheduling computation time under asynchronous planning, during which admitted calls continue executing.

Weakness 2 .

The experimental baselines are limited to adapted classical scheduling policies (FCFS and LLF at call/workflow levels) and lack direct comparisons with state-of-the-art program-aware agent serving systems such as Parrot and Continuum, preventing a rigorous assessment of HEXAGENT’s relative advantages over existing specialized agentic scheduling work.

Question 1 .

Could you explain why state-of-the-art program-aware agent serving systems such as Parrot and Continuum are absent from the baseline comparison, and how the reported SLO attainment gains would likely change if these specialized agent-scheduling methods were included?

Thank you for highlighting these systems. Parrot and Continuum are now included in all 24 scenarios, as described in Section 6.1. Figures 3-4 and the complete tables in the previous response report both methods.

SLO-attainment gains with the expanded comparison. The gains are measured against the strongest baseline at each percentile, including Parrot and Continuum. For example, on Hetero-1/Llama/LATS, HexAGenT reduces α0.95\alpha_{0.95}/α0.99\alpha_{0.99} by 33.45%/33.79% relative to Parrot and 34.55%/35.00% relative to Continuum. Against the strongest baseline in this scenario, the reductions are 22.47%/24.75%. The complete results appear in previous response.

Reviewer_N4JFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on addressing two problems in the complex system of LLM serving: call prioritization and P-D worker-pair selection, with the objective of minimizing the SLO scaling factor. The authors' claimed contributions mainly include (1) using information from predecessor calls to determine call priorities for partially observable workflows, and (2) employing a completion-time estimation formula to evaluate each available P-D worker pair. The experiments are conducted solely through trace-driven simulation.

Critical strengths
  1. The paper addresses an important and practically relevant problem in LLM serving.
Critical weaknesses
  1. The concepts of "partially observable workflows" and "critical-path-based prioritization" have already been explored in prior work [1][2], while this paper does not sufficiently distinguish its contributions from existing work. The baseline configurations are questionable, and it does not demonstrate any advantages through experimentation.

  2. The proposed algorithmic design is relatively simple, and the paper lacks ablation studies to validate the contributions of the proposed components. Moreover, the experiment is conducted solely on a simulator, not on actual hardware.

Questions for authors
  1. In the context of online scheduling, why do you believe that previously assigned calls influence future decision-making? Could you provide experimental evidence to support this?

  2. I have concerns about the objective function in Equation (1). Why not directly use SLO violations as the metric, as in [3]?

  3. The baselines are overly simplistic. Please compare with the directly related works [1] and [2] by adapting their methods to the same experimental environment.

  4. Pairwise exchanges are performed after call prioritization and P-D pair selection. However, it is unclear whether this step has a significant impact on performance. Please provide an ablation study to demonstrate its contribution.


[1] Autellix: An Efficient Serving Engine for LLM Agents as General Programs

[2] HexGen-Flow: Optimizing LLM Inference Request Scheduling for Agentic Text-to-SQL

[3] Queue Management for SLO-Oriented Large Language Model Serving

13 comments

Weakness 1 .

The concepts of "partially observable workflows" and "critical-path-based prioritization" have already been explored in prior work [1][2], while this paper does not sufficiently distinguish its contributions from existing work. The baseline configurations are questionable, and it does not demonstrate any advantages through experimentation.

Question 3 .

The baselines are overly simplistic. Please compare with the directly related works [1] and [2] by adapting their methods to the same experimental environment.

We appreciate the recommendation. We include Agentix/ATLAS quantitatively, together with AMPD, SAGA, Parrot, Continuum, and the three existing baselines. ATLAS prioritizes attained program service, while HexAGenT couples projected completion relative to revealed work with joint P/D placement.

Why HexGen-Flow falls outside the evaluated baseline methods. HexGen-Flow's deadline budgeting uses a known Text-to-SQL pipeline and estimates of its remaining stages, with configured bounds for dynamic refinement. Its Equation (5) allocates the remaining deadline according to estimated downstream work. Our workflows reveal future calls and branches during execution, leaving those remaining-stage descriptions unavailable to the scheduler. HexGen-Flow also treats SQL execution as negligible and assigns a call to one model instance, whereas our setting includes substantial tool waits and separate P/D workers. These methodological assumptions motivate discussing HexGen-Flow in Section 2 while excluding it from the direct quantitative comparison.

Complete end-to-end results. Sections 6.1-6.2 and Figures 3-4 report all nine methods on the same arrivals, configured worker pools, and evaluation references. Each cell below gives α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better.

Homo results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS6.3694 / 8.32941.7885 / 1.97601.5850 / 1.66765.1824 / 6.2777
Workflow-LLF4.0372 / 5.35941.7885 / 1.97611.5910 / 1.60322.2923 / 2.5853
Po2C+Call-LLF6.4879 / 7.55581.7786 / 1.85701.5569 / 1.55833.0181 / 3.3491
ATLAS3.5888 / 4.50931.5893 / 1.62091.5483 / 1.56412.6440 / 2.6998
AMPD5.4992 / 6.96731.5718 / 1.71371.6209 / 1.67625.0571 / 5.1545
SAGA7.3060 / 12.25641.5593 / 1.58341.5291 / 1.54922.9564 / 3.0246
Parrot6.1484 / 8.36301.5928 / 1.61701.5315 / 1.53472.9028 / 2.9566
Continuum5.8510 / 7.54711.5940 / 1.61701.6478 / 1.68453.9755 / 4.8083
HexAGenT2.9881 / 3.05511.5454 / 1.57801.5140 / 1.52112.2670 / 2.3274

Homo results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS4.9928 / 7.85655.8287 / 9.32301.5303 / 1.58352.0317 / 2.7232
Workflow-LLF3.5562 / 4.10085.7637 / 6.84501.5303 / 1.58351.7095 / 2.0354
Po2C+Call-LLF4.7672 / 5.87345.3650 / 9.09871.5155 / 1.54761.7502 / 1.8729
ATLAS2.8335 / 3.35935.4773 / 5.73721.5089 / 1.51951.7381 / 1.9250
AMPD4.3042 / 6.30955.9795 / 8.22591.8571 / 2.02941.9426 / 1.9759
SAGA4.7207 / 5.31885.9306 / 10.20791.5123 / 1.52631.5479 / 1.5914
Parrot3.8260 / 5.65275.6934 / 8.74271.5260 / 1.53061.5318 / 1.5725
Continuum4.8972 / 7.28035.4239 / 7.26042.0396 / 2.23192.1792 / 2.3852
HexAGenT2.4637 / 2.56613.8878 / 3.93391.5025 / 1.52021.5660 / 1.5773

Hetero-1 results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS2.2156 / 3.57892.0413 / 2.24331.3242 / 1.35081.7430 / 2.0680
Workflow-LLF2.0125 / 2.40091.9796 / 2.11331.3418 / 1.38041.5423 / 1.5873
Po2C+Call-LLF1.6965 / 2.02951.9954 / 2.28601.2903 / 1.33721.4024 / 1.4605
ATLAS1.7340 / 1.97291.7893 / 1.97591.4635 / 1.48051.5057 / 1.5470
AMPD2.0550 / 2.45011.8173 / 2.20891.5130 / 1.51951.8938 / 2.7287
SAGA1.7831 / 2.46071.9357 / 2.25411.5003 / 1.51471.4605 / 1.4772
Parrot1.8800 / 2.27912.1240 / 2.39061.5032 / 1.51981.4209 / 1.4477
Continuum1.8985 / 2.49621.8947 / 2.34251.5286 / 1.54811.4897 / 1.4904
HexAGenT1.6275 / 1.81631.8125 / 1.85471.0004 / 1.00631.1841 / 1.1894

Hetero-1 results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS1.8886 / 2.510420.3224 / 34.28071.4585 / 1.47541.3535 / 1.3907
Workflow-LLF1.6479 / 2.203919.4959 / 30.15161.4585 / 1.47531.3195 / 1.3398
Po2C+Call-LLF1.5064 / 1.595120.2461 / 34.00811.4337 / 1.48791.2827 / 1.3430
ATLAS1.5173 / 1.597914.9956 / 18.88061.4621 / 1.48201.4367 / 1.4450
AMPD1.7598 / 1.805819.1866 / 31.54331.5036 / 1.52191.5086 / 1.5297
SAGA1.5501 / 1.647920.3016 / 33.17901.4904 / 1.51771.3665 / 1.4379
Parrot1.5152 / 1.521619.1765 / 32.12171.5254 / 1.52681.3631 / 1.4011
Continuum1.6186 / 2.264420.2948 / 33.74711.5272 / 1.57971.4356 / 1.4363
HexAGenT1.4139 / 1.517611.6166 / 11.65840.9977 / 1.00511.0314 / 1.0586

Hetero-2 results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS26.0393 / 37.30160.9839 / 1.23681.4121 / 1.46585.3190 / 5.9227
Workflow-LLF11.9316 / 18.62610.9839 / 1.23691.4307 / 1.51722.1625 / 2.7362
Po2C+Call-LLF25.9613 / 37.30160.9672 / 1.22351.2656 / 1.31871.5862 / 1.8934
ATLAS15.9441 / 26.39011.0554 / 1.14401.4585 / 1.47921.9331 / 2.1068
AMPD27.6119 / 49.92551.0735 / 1.22611.5144 / 1.53371.7943 / 1.8331
SAGA27.7117 / 50.15330.9961 / 1.10171.4915 / 1.49531.7535 / 1.8099
Parrot26.0738 / 37.34861.1009 / 1.22741.4875 / 1.49591.7232 / 1.7691
Continuum26.5175 / 36.66271.0572 / 1.20181.6064 / 1.64972.9777 / 4.2461
HexAGenT8.6013 / 8.61760.7104 / 0.77640.9852 / 1.00001.2361 / 1.2710

Hetero-2 results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS26.2869 / 38.30890.9314 / 0.95451.4849 / 1.62022.6888 / 3.1597
Workflow-LLF14.7521 / 16.96110.9314 / 0.95451.4723 / 1.48511.4292 / 1.7938
Po2C+Call-LLF26.7000 / 37.97270.8989 / 0.93731.4564 / 1.48471.1504 / 1.3040
ATLAS16.9817 / 18.66861.0089 / 1.01591.4730 / 1.47711.4282 / 1.5051
AMPD28.8821 / 38.52831.0029 / 1.01001.4893 / 1.50541.4775 / 1.6615
SAGA24.7541 / 34.02691.0041 / 1.00821.4889 / 1.49371.2302 / 1.2970
Parrot29.6037 / 40.28411.0101 / 1.01921.4891 / 1.49491.2005 / 1.2295
Continuum26.8737 / 38.30081.0069 / 1.01211.8650 / 2.03401.5684 / 1.9439
HexAGenT7.8969 / 8.15370.6492 / 0.66340.8605 / 0.96671.0098 / 1.0135

Largest measured reductions. The reductions against the best baseline reach 46.47% for α0.95\alpha_{0.95} and 53.73% for α0.99\alpha_{0.99}.

Weakness 2 .

The proposed algorithmic design is relatively simple, and the paper lacks ablation studies to validate the contributions of the proposed components. Moreover, the experiment is conducted solely on a simulator, not on actual hardware.

Thank you for emphasizing both requirements. Figures 3-4 and the complete tables in the previous response now provide the real-system comparison. All additional experiments reported in this rebuttal were also conducted on real GPU serving clusters. Section 6.3 and Figure 5 separately examine the scheduling components using Llama and the Hetero-2 GPU composition.

Ablation configurations. The same workloads and H100 reference SLOs are used throughout.

  • Full HexAGenT: retain workflow-relative priority, joint placement, queue awareness, and cache-aware planning.
  • Priority (LLF): replace priority with Workflow-LLF under the same placement algorithm.
  • Placement (workload-balanced): use workload-balanced dispatch across feasible workers.
  • Queue-blind planning: exclude queue waiting from planning estimates while retaining actual execution queues.
  • Cache-blind planning: exclude reuse information from planning while retaining runtime reuse.

Llama ablation results on Hetero-2. Each cell reports α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better.

TraceHexAGenTPriority (LLF)Placement (workload-balanced)Queue-blind planningCache-blind planning
ShareGPT1.7399 / 1.84992.3556 / 2.60632.4829 / 2.59923.1198 / 3.21791.7521 / 1.8168
BFCL0.7105 / 0.77640.7104 / 0.77640.9260 / 1.02940.7561 / 0.93060.7177 / 0.7839
LATS0.7292 / 0.75360.7512 / 0.77480.9141 / 0.92890.8847 / 0.90720.8174 / 0.8215
Terminal-Bench1.3301 / 1.40331.9483 / 2.31951.6758 / 1.85551.9488 / 2.09111.6174 / 1.7606

Performance change for the same ablation configurations. Each cell gives the percentage change in α0.95\alpha_{0.95} / α0.99\alpha_{0.99} relative to HexAGenT, computed as 100(rablation/rHexAGenT−1)100(r_{\mathrm{ablation}}/r_{\mathrm{HexAGenT}}-1). Positive values indicate degradation.

TracePriority (LLF)Placement (workload-balanced)Queue-blind planningCache-blind planning
ShareGPT+35.38% / +40.89%+42.70% / +40.51%+79.30% / +73.95%+0.70% / -1.79%
BFCL-0.01% / -0.01%+30.34% / +32.58%+6.41% / +19.85%+1.02% / +0.96%
LATS+3.02% / +2.82%+25.36% / +23.27%+21.32% / +20.39%+12.09% / +9.02%
Terminal-Bench+46.48% / +65.29%+25.99% / +32.22%+46.52% / +49.01%+21.60% / +25.46%
Reviewer_gQKfOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper focuses on how to allocate GPUs efficiently when multiple agent workflows are running simultaneously. Existing systems tackle this known issue in some combination, i.e., deciding which individual LLM requests to prioritize, some optimize progress of entire agent workflows, other optimize which GPUs handle different parts. HexAGenT combines the fragments considering both which agent tasks need attention and which GPUs can complete their LLM requests most efficiently tested on simulated agent workflow traces.

Critical strengths
  1. Focussing on end-to-end agent workflow latency rather than individual calls is a strong scheduling objective.
  2. The proposed scheduler uses estimated call-completion times to determine both which requests should be prioritized and which prefill–decode GPU pairs should execute them. This is the central contribution combining decisions that existing approaches often optimize separately.
  3. The evaluation includes four agentic workload types, two LLMs, and three GPU configurations. It reports improvements of up to 49.7% and 63.0% in the required SLO scaling factors for 95% and 99% workflow completion, respectively. Improvements also persist on homogeneous GPU clusters, indicating that heterogeneous hardware selection alone does not explain the gains.
Critical weaknesses
  1. AMPD (ICML 2026) already coordinates scheduling and resource allocation for multi-round LLM inference under prefill–decode disaggregation, while SAGA (2026) combines agent-workflow scheduling, GPU load balancing, and KV-cache management. Neither is identical to HexAGenT, but the paper does not clearly establish what substantive advantage its particular joint scheduling approach offers beyond these systems. Both works are also missing from the related-work discussion.
  2. HexAGenT combines workflow prioritization, joint prefill–decode placement, queue-aware scheduling, and KV-cache reuse, but does not isolate their individual contributions - an ablation to this effect would be useful.
  3. The evaluation only compares against three adapted FCFS/least-laxity-first baselines. These do not represent the strongest contemporary agent-aware or heterogeneous LLM-serving approaches. Without comparisons against systems such as AMPD and other suitable adaptations of their scheduling mechanisms, the reported improvements do not convincingly establish an advance over the state of the art.
  4. All performance improvements are measured using trace-driven simulations rather than actual GPU execution. Real batching behavior, network contention, KV-cache transfers, and GPU memory constraints may differ from the simulation assumptions. Live-serving experiments while time-consuming could significantly strengthen the paper even if this means a smaller subset of tasks for the rebuttal followed by a more substantial task set for the camera-ready.
Questions for authors
  1. How does HexAGenT substantively differ from AMPD (ICML 2026), SAGA (2026), and Autellix/Agentix (NSDI 2026)? What specific scheduling capability or algorithmic contribution does HexAGenT introduce beyond these approaches? (If there happen to be others that I am missing or that happen to be more closely related please add those as well though the above should suffice)
  2. Why are the evaluations limited to adapted FCFS and least-laxity-first schedulers?
  3. How much of the reported improvement comes specifically from the proposed workflow-priority heuristic versus joint prefill–decode GPU placement? In particular, can the authors compare HexAGenT's priority mechanism against Workflow-LLF using identical GPU placement strategies, and separately evaluate the contribution of KV-cache awareness and queue-aware planning? The current baselines change multiple components simultaneously, making the source of improvement unclear.
  4. How are models such as Qwen3-235B-A22B supported on configurations that appear to allocate only a single A100 or H100 worker to a serving stage?
10 comments

Weakness 1.

AMPD (ICML 2026) already coordinates scheduling and resource allocation for multi-round LLM inference under prefill–decode disaggregation, while SAGA (2026) combines agent-workflow scheduling, GPU load balancing, and KV-cache management. Neither is identical to HexAGenT, but the paper does not clearly establish what substantive advantage its particular joint scheduling approach offers beyond these systems. Both works are also missing from the related-work discussion.

Question 1.

How does HexAGenT substantively differ from AMPD (ICML 2026), SAGA (2026), and Autellix/Agentix (NSDI 2026)? What specific scheduling capability or algorithmic contribution does HexAGenT introduce beyond these approaches? (If there happen to be others that I am missing or that happen to be more closely related please add those as well though the above should suffice)

Thank you for asking us to clarify this distinction. HexAGenT couples workflow-relative priority with joint heterogeneous P/D placement: the same completion estimates inform which call to prioritize and which worker pair to select.

Differences from recent systems. Section 2 explains how these approaches differ in their scheduling objectives and decisions:

  • AMPD targets per-round TTFT/ITL and session affinity. However, delays can accumulate across calls even when individual rounds meet their targets. HexAGenT accounts for elapsed workflow time and projected completion relative to the revealed workflow horizon.
  • SAGA emphasizes session-aware caching, affinity, and work stealing, but a cache-local worker can still incur slow execution or downstream decode waiting. HexAGenT jointly weighs cache savings, GPU-dependent service time, transfers, and queueing at both stages.
  • Agentix/ATLAS prioritizes attained service along observed program critical paths, whereas workflows with similar attained service can face substantially different remaining delays. HexAGenT uses placement-dependent completion estimates to guide both priority and placement.

Weakness 2 .

HexAGenT combines workflow prioritization, joint prefill–decode placement, queue-aware scheduling, and KV-cache reuse, but does not isolate their individual contributions - an ablation to this effect would be useful.

Question 3 .

How much of the reported improvement comes specifically from the proposed workflow-priority heuristic versus joint prefill-decode GPU placement? In particular, can the authors compare HexAGenT's priority mechanism against Workflow-LLF using identical GPU placement strategies, and separately evaluate the contribution of KV-cache awareness and queue-aware planning? The current baselines change multiple components simultaneously, making the source of improvement unclear.

We appreciate this suggestion. Section 6.3 and Figure 5 report matched ablations using Llama and the Hetero-2 GPU composition.

Ablation configurations. Each configuration changes one part of the planner while retaining the same workloads and H100 evaluation references:

  • Full HexAGenT: workflow-relative priority, joint P/D placement, queue-aware estimates, and cache-aware planning.
  • Priority (LLF): replace only the priority rule with Workflow-LLF and retain the same placement algorithm.
  • Placement (workload-balanced): distribute calls across feasible workers using workload balancing while retaining HexAGenT's priority.
  • Queue-blind planning: omit queue waiting from placement estimates while preserving actual queueing during execution.
  • Cache-blind planning: omit reuse information from planning while preserving runtime KV reuse.

Llama ablation results on Hetero-2. Each cell reports α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better.

TraceHexAGenTPriority (LLF)Placement (workload-balanced)Queue-blind planningCache-blind planning
ShareGPT1.7399 / 1.84992.3556 / 2.60632.4829 / 2.59923.1198 / 3.21791.7521 / 1.8168
BFCL0.7105 / 0.77640.7104 / 0.77640.9260 / 1.02940.7561 / 0.93060.7177 / 0.7839
LATS0.7292 / 0.75360.7512 / 0.77480.9141 / 0.92890.8847 / 0.90720.8174 / 0.8215
Terminal-Bench1.3301 / 1.40331.9483 / 2.31951.6758 / 1.85551.9488 / 2.09111.6174 / 1.7606

Performance change for the same ablation configurations. Each cell gives the percentage change in α0.95\alpha_{0.95} / α0.99\alpha_{0.99} relative to HexAGenT, computed as 100(rablation/rHexAGenT−1)100(r_{\mathrm{ablation}}/r_{\mathrm{HexAGenT}}-1). Positive values indicate degradation.

TracePriority (LLF)Placement (workload-balanced)Queue-blind planningCache-blind planning
ShareGPT+35.38% / +40.89%+42.70% / +40.51%+79.30% / +73.95%+0.70% / -1.79%
BFCL-0.01% / -0.01%+30.34% / +32.58%+6.41% / +19.85%+1.02% / +0.96%
LATS+3.02% / +2.82%+25.36% / +23.27%+21.32% / +20.39%+12.09% / +9.02%
Terminal-Bench+46.48% / +65.29%+25.99% / +32.22%+46.52% / +49.01%+21.60% / +25.46%

What these comparisons isolate. The priority comparison measures ordering under identical placement logic. The placement, queue-blind, and cache-blind comparisons measure worker selection and the information used to make that selection. In particular, cache-blind planning isolates the benefit of anticipating KV reuse when choosing a worker.

Weakness 3 .

The evaluation only compares against three adapted FCFS/least-laxity-first baselines. These do not represent the strongest contemporary agent-aware or heterogeneous LLM-serving approaches. Without comparisons against systems such as AMPD and other suitable adaptations of their scheduling mechanisms, the reported improvements do not convincingly establish an advance over the state of the art.

Question 2 .

Why are the evaluations limited to adapted FCFS and least-laxity-first schedulers?

Thank you for recommending a broader comparison. We now evaluate ATLAS, AMPD, SAGA, Parrot, and Continuum, alongside Workflow-FCFS, Workflow-LLF, and Po2C+Call-LLF. The latter is the former SGLang-LLF configuration, renamed to make its dispatch and priority rules explicit (Section 6.1).

Complete end-to-end results. Section 6.2 and Figures 3-4 cover two models, four traces, and three cluster settings. Detailed results regarding all nine methods are presented below. Each cell below reports α0.95\alpha_{0.95} / α0.99\alpha_{0.99}; lower is better. Llama and Qwen denote Llama-3.1-70B and Qwen3-235B-A22B.

Homo results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS6.3694 / 8.32941.7885 / 1.97601.5850 / 1.66765.1824 / 6.2777
Workflow-LLF4.0372 / 5.35941.7885 / 1.97611.5910 / 1.60322.2923 / 2.5853
Po2C+Call-LLF6.4879 / 7.55581.7786 / 1.85701.5569 / 1.55833.0181 / 3.3491
ATLAS3.5888 / 4.50931.5893 / 1.62091.5483 / 1.56412.6440 / 2.6998
AMPD5.4992 / 6.96731.5718 / 1.71371.6209 / 1.67625.0571 / 5.1545
SAGA7.3060 / 12.25641.5593 / 1.58341.5291 / 1.54922.9564 / 3.0246
Parrot6.1484 / 8.36301.5928 / 1.61701.5315 / 1.53472.9028 / 2.9566
Continuum5.8510 / 7.54711.5940 / 1.61701.6478 / 1.68453.9755 / 4.8083
HexAGenT2.9881 / 3.05511.5454 / 1.57801.5140 / 1.52112.2670 / 2.3274

Homo results for Qwen. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS4.9928 / 7.85655.8287 / 9.32301.5303 / 1.58352.0317 / 2.7232
Workflow-LLF3.5562 / 4.10085.7637 / 6.84501.5303 / 1.58351.7095 / 2.0354
Po2C+Call-LLF4.7672 / 5.87345.3650 / 9.09871.5155 / 1.54761.7502 / 1.8729
ATLAS2.8335 / 3.35935.4773 / 5.73721.5089 / 1.51951.7381 / 1.9250
AMPD4.3042 / 6.30955.9795 / 8.22591.8571 / 2.02941.9426 / 1.9759
SAGA4.7207 / 5.31885.9306 / 10.20791.5123 / 1.52631.5479 / 1.5914
Parrot3.8260 / 5.65275.6934 / 8.74271.5260 / 1.53061.5318 / 1.5725
Continuum4.8972 / 7.28035.4239 / 7.26042.0396 / 2.23192.1792 / 2.3852
HexAGenT2.4637 / 2.56613.8878 / 3.93391.5025 / 1.52021.5660 / 1.5773

Hetero-1 results for Llama. The table reports all nine methods across the four traces.

MethodShareGPTBFCLLATSTerminal-Bench
Workflow-FCFS2.2156 / 3.57892.0413 / 2.24331.3242 / 1.35081.7430 / 2.0680
Workflow-LLF2.0125 / 2.40091.9796 / 2.11331.3418 / 1.38041.5423 / 1.5873
Po2C+Call-LLF1.6965 / 2.02951.9954 / 2.28601.2903 / 1.33721.4024 / 1.4605
ATLAS1.7340 / 1.97291.7893 / 1.97591.4635 / 1.48051.5057 / 1.5470
AMPD2.0550 / 2.45011.8173 / 2.20891.5130 / 1.51951.8938 / 2.7287
SAGA1.7831 / 2.46071.9357 / 2.25411.5003 / 1.51471.4605 / 1.4772
Parrot1.8800 / 2.27912.1240 / 2.39061.5032 / 1.51981.4209 / 1.4477
Continuum1.8985 / 2.49621.8947 / 2.34251.5286 / 1.54811.4897 / 1.4904
HexAGenT1.6275 / 1.81631.8125 / 1.85471.0004 / 1.00631.1841 / 1.1894

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank you for your careful reading and constructive feedback. The revised paper expands the baseline comparison, examines the scheduling components’ contributions separately, and clarifies the objective, theoretical guarantee, and baseline deadlines. All reported results are now based on experiments conducted on real GPU serving clusters. All section and figure references below refer to the revised paper.

Summary of the experimental evidence. The evaluation compares HexAGenT with eight baselines across two models, four traces, and three cluster settings. The additional evidence addresses the following questions:

  • Comparison with recent systems: ATLAS, AMPD, SAGA, Parrot, and Continuum join the three existing baselines in the complete 24-scenario comparison (Sections 6.1-6.2; Figures 3-4).
  • Contributions of individual mechanisms: the Llama experiments on the Hetero-2 GPU composition separately evaluate workflow priority, joint placement, queue awareness, and cache-aware planning across four traces (Section 6.3; Figure 5).
  • Real-system validation: the real-system evaluation covers both models, all four traces, and the three cluster settings (Section 6.2; Figures 3-4).

Reading the results. Each relevant response includes complete numerical results and references to the corresponding paper figures. Following Section 6.1, we now denote α0.95\alpha_{0.95} and α0.99\alpha_{0.99} as the SLO scaling factors required for 95% and 99% workflow attainment; lower is better. The measured reductions in the required SLO scaling factors reach 46.47% for α0.95\alpha_{0.95} and 53.73% for α0.99\alpha_{0.99} relative to the best baseline at the corresponding percentile.

Clarifications about the scheduling method. We state the criterion of bounded pairwise refinement using the attainment target τ\tau: the maximum-ratio criterion is the special case τ=1\tau=1. The responses also clarify the rationale for our workflow-level SLO objective and our conditional per-call placement performance guarantee.

Best regards,

The Authors of Submission 6028

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.