- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to find configurations for attention–FFN disaggregated LLM serving that satisfy latency and throughput SLOs while reducing total GPU energy consumption under a limited budget of physical measurements. The search space includes GPU counts, parallelism degrees, DBO settings, and the frequencies and power caps of the two GPU groups.
ECO constructs an energy prior from measurement-calibrated stage-time and pipeline models, learns prediction residuals with a Gaussian process, and selects the next trial using constrained Bayesian optimization that accounts for SLO feasibility, execution success, and measurement cost. It returns the lowest-energy feasible configuration among those actually measured. Its main contribution is an energy modeling and search method tailored to the configuration hierarchy and pipeline dependencies of AFD.
The evaluation covers 16 scenarios across two models and two GPU platforms, including end-to-end evaluation on independent request sets, search-efficiency comparisons, and ablations of the energy model, joint tuning, and cost-aware strategy.
Critical strengths
-
The paper addresses a practically relevant AFD energy-optimization problem through joint search over deployment structures and operating controls. Its formulation accounts for service constraints, execution failures, and measurement costs.
-
Combining a stage-and-pipeline energy prior with GP residual learning helps guide search under a limited measurement budget. The evaluation includes two models, two hardware platforms, and independent request sets, while the ablations provide support for the roles of the main components.
Critical weaknesses
-
A comparison with the closest AFD optimization approach is missing. Section 6 identifies AFlex as the closest work, but the BO/GA comparisons and component ablations do not establish how much additional energy saving ECO achieves over a specialized AFD optimization method under the same measurement budget.
-
The load pressure underlying the energy results is unclear. Section 5 notes that the selected eight-GPU Qwen configurations have less than 1% headroom above the throughput SLO floor, but this does not establish their proximity to maximum sustainable throughput. The experiments cover 4–16 requests/s without locating these loads relative to that capacity, leaving it unclear whether the reported energy savings persist near the serving capacity limit.
Questions for authors
-
Regarding Weakness 1: With profiling, calibration, and search costs accounted for consistently, could you compare the energy of the final feasible configurations against AFlex or a comparable implementation under the same budget?
-
Regarding Weakness 2: Where do the evaluated loads lie relative to the maximum sustainable throughput under the stated latency SLOs? How do the energy savings change as load approaches that boundary?