- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces a scheduling framework called BIP, which combines output-length prediction, rolling-horizon integer programming, and online correction of underestimated workloads. It captures job deadlines, GPU-type-dependent speeds, parallel execution, and slot-boundary preemption.
Critical strengths
-
BIP combines instruction-tuned length prediction, rolling-horizon integer programming, and online workload correction.
-
These three modes of expression cover, respectively, the throughput of completed tasks, normalized service levels, and task completion times subject to strict deadlines.
Critical weaknesses
-
The reported performance relies on selecting the best-performing formulation among ILP-1, ILP-2, and ILP-3 after execution, whereas an online scheduler must commit to a policy before the actual completion rate is known.
-
The current evaluation only compares BIP against FCFS and SJF. We recommend adding additional baseline methods to better demonstrate the effectiveness of the proposed framework.
-
The extensibility of BIP is limited, as the predictive performance observed on reference answers may not necessarily apply directly to content generated by other models, nor to results produced using different decoding configurations.
-
The scheduling results are derived from simulations and do not account for throughput, GPU utilization, or end-to-end service costs in terms of actual wall-clock time.
-
A comprehensive related work investigation should be conducted, as several important recent works (2024-2026) are missing.
-
The simulation results reported in this paper do not cover how solver runtime varies with problem scale, nor do they include statistics on feasibility or timeouts for the various modeling approaches.
-
BIP employs an abstract model based on "constant speed" and "total workload," and it assumes that this model is pre-loaded. The authors should provide more detailed and realistic assumptions regarding these pre-loaded models.
-
The evaluation is really limited, as only one figure is given for the deadline-completion rates across normalized-load settings and relative job sizes.
-
There is no reproducibility statement, and the code is not available.
Questions for authors
-
It is unclear how the individual job deadlines are generated in the experimental setup. Please explain the rationale for assigning deadlines.
-
Can the proposed framework maintain its effectiveness when the target inference model changes? Please provide additional experimental results for both length prediction and task scheduling across different target LLMs.
-
The reported performance relies on selecting the best-performing formulation among ILP-1, ILP-2, and ILP-3 after execution, whereas an online scheduler must commit to a policy before the actual completion rate is known. Please explain how a formulation should be selected at runtime.