- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents a change in how resources are allocated between prefill and decode on a single GPU box for severing systems. The work identifies an additional physical switching cost when switching between prefill/decode profiles that results from already scheduled kernels being on the wrong configuration. They propose a logical lease system that seems to shorten the physical profile transition time and the results show small but consistent gains in the approach when switching is critical.
Critical strengths
-
The paper shows promising results on SG-Lang for workloads in which this switching cost is likely to appear.
-
The authors clearly put significant effort in their implementation with "over 3,000 lines of runtime code" in SG-Lang.
Critical weaknesses
-
The paper was exceptionally difficult to follow. It has the writing style of Opus 5 with short punchy sentences that don't provide great clarity. Example: "LSV does not require a particular predictor. It accepts a target resource request from the policy." I work in this space and still don't fully understand the technique.
-
This is very much a systems paper and would be much better reviewed at a systems venue (e.g., OSDI or SOSP) where there is more room in the paper to properly explain the system design.
Questions for authors
-
Maybe try to re-explain the key-insight that underlies the technique.
-
This is built on the relatively new cuda green-context and I would like to know if that does most of the "heavy lifting".
I thank the authors for the response. While the rebuttal provides useful additional evidence, the proposed mechanisms remain relatively straightforward extensions of existing execution-control techniques, leaving my concern about the significance of the techniques unresolved. I therefore keep my score.