- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a controlled study of the offline-to-online reinforcement learning (RL) pipeline for cluster scheduling. It examines the utility of production traces for imitation learning, training implementation challenges, the state-information coverage of baselines, and the role of safe fallback mechanisms. Under the specific candidate construction and evaluation objectives considered, direct imitation of production logs performs poorly; after two training flaws are corrected, however, the learned policy outperforms the initial heuristic-based baseline.
The key finding is that adding the number of concurrent tasks on candidate machines to the heuristic rules improves the rule-based frontier by 5.61 percentage points. Across the dual objectives (SLA and hotspots), the number of scenarios in which the learned policy beats the rules drops from 9/20 to 0/20, and hyperparameter tuning yields no further gains. The authors accordingly emphasize the importance of verifying the information coverage of baselines before attributing benefits to learning-based methods. The paper also analyzes how capacity settings affect outcomes and shows that safe gating mechanisms can mitigate catastrophic failures during online learning.
Critical strengths
- The paper addresses a meaningful performance-attribution problem: by separating feature coverage from hyperparameter search density, it provides a more informative experimental analysis than a straightforward comparison between learning-based and rule-based methods.
- The authors are transparent about identifying and fixing training implementation issues and offer a thoughtful analysis of the limitations of log replay and capacity configuration. These efforts strengthen the credibility of the experiments and provide valuable practical insights.
Critical weaknesses
- The representativeness of the core findings, and their contribution to existing knowledge, remain unclear. The number of concurrent tasks is intuitive and readily available state information. Although the paper offers this empirical attribution, it is not fully clear whether the finding reflects a representative evaluation challenge in the field or mainly corrects the authors' own baseline design. It is also unclear whether comparable existing methods already use this variable (or equivalent information).
- The analysis of the two training flaws rules out ineffective training as a confound, which strengthens the validity of the experiments. The paper does not yet articulate, however, what new and transferable insights these diagnostics offer as contributions. The respective contributions of the core attribution findings and the training diagnostics—and the incremental knowledge gained from their combination—therefore require further clarification.
- The impact of the environment simplifications on the core conclusions remains to be verified. The authors commendably discuss the simulator's simplifications and the scope of their experiments in the appendix, but disclosing these limitations delimits the applicability of the findings without demonstrating that the conclusions are robust to the modeled features. For instance, co-location costs affect only new tasks and do not slow down existing ones, which may alter the performance trade-offs between learning strategies and rules. The current analysis does not sufficiently address whether the primary conclusions would still hold if resource contention also affected task completion times and resource release.
Questions for authors
- Which representative existing methods fail to account for the number of concurrent tasks (or equivalent information)? Why was this factor omitted from the paper's original baseline? Is this omission representative enough to support conclusions beyond the specific experimental setup presented here?
- Do the learning-based strategies and the heuristic rules have access to the same information and the same set of candidate machines? Can the full comparison—with and without this variable—be replicated on a different time window or a separate trace?
- When a task is migrated to another machine (or placed on a machine already running other tasks), how does the simulator determine the task's execution duration and the timing of its resource release? Have these predictions been validated?
- Could the same set of tasks be run on a small-scale real cluster to compare the learning-based strategies against the heuristic rules, verifying whether the methods that win in simulation also win in practice? If real-world experiments are infeasible, could the simulation at least model the mutual slowdown caused by co-located tasks to check whether the primary conclusions remain valid?