- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes CGB, a differentiable gradient boosting framework that updates previously added weak learners before each new tree is added.
Critical strengths
The idea of differentiable weak learner is a natural extension of boosting, and the paper is easy to follow.
Critical weaknesses
The experiments are too limited, comparing CGB only against XGBoost. Being a generic framework for differentiable boosting, it should be able to be combined with any existing boosting algorithms (e.g., LightGBM, CatBoost). The paper lacks a systematic view of the experiments. A single baseline is far from enough.
Questions for authors
The experiments limit XGBoost to 30 trees, while giving CGB up to 100 Adam epochs in every iteration. Currently the number of optimization steps is very unbalanced, ~30 for XGBoost vs up to ~30×100 for CGB. How does CGB compare against XGBoost and other non-differentiable boosting algorithms when they have closer optimization budgets (e.g., dropping the 30-tree limit for XGBoost)? If the gap closes, the gains may simply come from more optimization rather than from updating previous trees.