acceptodds
Under review as a conference paper at ICLR 2027

Analytical Provisioning for Attention–FFN Disaggregated LLM Serving under Stochastic Workloads

Abstract

Attention–FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache–dominated Attention computation from stateless, compute-intensive FFN computation, connected by per-step communication. While AFD enables independent scaling of memory and compute resources, its performance is highly sensitive to the Attention/FFN provisioning ratio: mis-sizing induces step-level blocking and costly device idle time. We develop an analytical provisioning framework for AFD bundles in an rA-1F topology under stochastic workloads. Two sources of randomness shape the problem: per-slot Attention workload evolves as KV caches grow and completed requests are replenished with random prompt and decode lengths, and synchronized execution across Attention workers introduces a barrier governed by the slowest worker. We address both via a renewal-reward characterization of the per-slot stationary token load, identifying a single workload statistic that governs provisioning under arbitrary prefill-decode distributions and admits a nonparametric estimator from request traces. The analysis yields a closed-form mean-field rule for the optimal A/F ratio decomposing into Attention-, communication-, and FFN-bottleneck regimes, together with a Gaussian barrier-aware refinement that quantifies cross-worker synchronization overhead. A trace-calibrated AFD simulator supports the framework across workloads: the predicted optimal ratio matches the simulation-optimal within 10%. Together, these results provide a compact, calibratable account of how stochastic workload structure determines provisioning in disaggregated LLM serving.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
AF disaggregation, disaggregated LLM serving, LLM serving provisioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rbr2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents a theoretical formulation and analytical framework for attention-FFN disaggregation in LLM inference. The key innovation seems to be the barrier aware attention load modeling and the approximations for cycle times. At the end, the authors validate the theoretical framework with a simulator built for a DeepSeek model on an NPU processor.

Critical strengths
  • A closed-form mean-field provisioning rule
  • A barrier-aware correction that captures synchronization overhead
Critical weaknesses
  • Lack of validation on real serving engines like vLLM or SGLang with real hardware runs
  • Limited to only 1 model and 1 hardware configuration
  • Cross-worker load balancing is mentioned but lack deeper explanation
Questions for authors
  • What's the fidelity of the simulator? Can you also validate the framework with real hardware runs and actual AFD implementation on the serving engine? How would the found best A:F ratio apply to the real engine if there is one?
  • How is imbalanced A:F ratio handled? especially in terms of load balancing and communication?
  • When it comes to GPU stalls due to waiting for data transfer between A and F, how is this modeled in the theoretical framework?
Reviewer_mSdqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses renewal theory to characterize stationary token loads and derives mean-field and barrier-aware provisioning rules for attention–FFN disaggregated serving. A calibrated simulator evaluates how workload statistics and synchronization affect resource allocation and throughput.

Critical strengths

The renewal-based treatment captures the distinction between arrival-level request statistics and loads observed during ongoing decoding. This provides useful insight into how workload variability affects provisioning.

The framework connects workload statistics to interpretable provisioning rules and a synchronization correction, offering a potentially useful alternative to extensive configuration search.

Critical weaknesses

The numerical inputs and accuracy claim require reconciliation. Under independent prompt/decode lengths, Equation (4) and Section 5.2 yield (θ=644\theta=644) and a mean-field ratio of approximately 10.45, whereas Figure 3’s annotated mean load of 150074 reproduces 9.30. The relationship between these inputs is unclear, as is whether the claimed below-10% ratio error concerns continuous predictions or configurations projected onto the evaluated grid. These ambiguities hinder verification of the main quantitative claim.

The cycle-time model is not fully connected to the simulated schedule. Section 3.3 uses the maximum stage time, while Section 5.1 simulates two interleaved batches. Section 5.2 supplies a communication-hiding condition, but does not demonstrate how it yields the stated cycle time. This weakens the attribution of most of the prediction discrepancy to synchronization barriers in Section 5.3.

Questions for authors
  1. How does the mean token load of 150074 in Figure 3 relate to the workload moments in Section 5.2? Were empirical estimates or correlated prompt/decode lengths used to obtain the predicted ratio of 9.3?
  2. How is the below-10% provisioning-ratio error calculated? Please specify the simulation search grid, any projection of the continuous prediction, and the error denominator.
  3. Under the two-batch schedule, what establishes the maximum-stage-time approximation? How are finite-pipeline effects distinguished from synchronization overhead?

I would raise my rating if the authors reconcile the numerical inputs and error metric and substantiate the cycle-time approximation, and lower it if the necessary corrections materially undermine the provisioning conclusions.

Reviewer_gL61OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Disaggregated serving of LLMs divides serving instances into prefill and decode instances, where each instance type is responsible for a request’s execution during that phase. Attention-FFN disaggregation (AFD) approaches further divide the decode instances into attention and FFN instances, with the goal of increasing system throughput. As with general disaggregation, AFD needs to determine the optimal number of attention and FFN instances. The authors perform a theoretical analysis of the disaggregated system and propose a closed-form rule that finds the optimal ratio of attention to FFN instances. In their experiments, they show that the predicted optimal ratio matches the simulated optimal ratio within a 10% error range.

Critical strengths

Provides a detailed analysis of the bottlenecks in the AFD setting.

Critical weaknesses
  • Lacks consideration of practical settings (communication overhead, KV transfer, scheduling in the AF-disaggregated setting, etc.).
  • The experiments are run only on a simulator.
Questions for authors
  1. What is Nexpert/cardN_{expert/card} in appendix B?
  2. In Figure 2, A13A_{13} creates a bubble because it requires more execution time than previous iterations. However, the rate of change at each decoding step is minuscule compared to a prefill request. I wonder how often such cases arise in practice. A motivational study showing how a request’s decoding execution time grows, and where the crossover point lies, would strengthen the paper.
  3. What is the communication overhead of AFD?
  4. Would the closed-form solution still hold if the distributions of prompt length and the number of decode tokens change?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.