- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents a theoretical formulation and analytical framework for attention-FFN disaggregation in LLM inference. The key innovation seems to be the barrier aware attention load modeling and the approximations for cycle times. At the end, the authors validate the theoretical framework with a simulator built for a DeepSeek model on an NPU processor.
Critical strengths
- A closed-form mean-field provisioning rule
- A barrier-aware correction that captures synchronization overhead
Critical weaknesses
- Lack of validation on real serving engines like vLLM or SGLang with real hardware runs
- Limited to only 1 model and 1 hardware configuration
- Cross-worker load balancing is mentioned but lack deeper explanation
Questions for authors
- What's the fidelity of the simulator? Can you also validate the framework with real hardware runs and actual AFD implementation on the serving engine? How would the found best A:F ratio apply to the real engine if there is one?
- How is imbalanced A:F ratio handled? especially in terms of load balancing and communication?
- When it comes to GPU stalls due to waiting for data transfer between A and F, how is this modeled in the theoretical framework?