- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper provides an empirical study on 4 Large Language Models, while varying parallelism layout, batch scheduling policy, workload characteristics, and load. The authors show that parallelism affects inference KPIs more so than the scheduling policy.
Critical strengths
I would first like to thank the authors for their submission. The results section of this paper are experimentally vast and detailed.
Critical weaknesses
Unfortunately, my main issue with this paper is that I cannot find a cohesive story or learning in the narrative. While I appreciate your work, high volume of experimentation, analyses and interpretation does not result in a good empirical study paper. A powerful empirical study often focuses on a key few points, analyzes seldomly studied aspects of a system or problem, or disproves a common assumption about that problem. Some empirical studies are successful because they carry insights from production systems in the industry. Batch scheduling and parallelism are some of the most well-studied topics in LLM inference. The traces, systems and models in this paper are all well-known and open-source. None of the learnings in the paper are at odds with common knowledge in the community.
I believe the authors can substantially improve this paper, if they focus on any of the above. As it stands, the paper is not ready for publication.
Questions for authors
Could the authors describe the key learnings from this work, ideally 1 or 2 insights?