- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes PG-DTS which is a decision tree induction framework using prior guided Monte Carlo Tree Search (MCTS). The paper frames the induction problem as a deterministic Markov Decision Process (MDP) with states as partial trees and actions as target leaf, split feature and threshold. The contributions try to plug the gap between sub optimal greedy induction and brittle global solvers.
Critical strengths
-
The problem is well motivated and positioned. The paper tries to find a middle ground between fast & myopic greedy solvers and the brittle & memory hungry exact solvers.
-
Engineering contributions like GLR, leaf level reward, transposition deduplication plausibly help reduce runtime and memory.
-
Targeted ablations help to isolate the effects of some of the contributions.
-
Authors provide the implementation, well-structured pseudo code and dataset details to make the experiments reproducible.
Critical weaknesses
- Baselines are compared unfairly. CART, DPDT, DL8.5, and GOSDT all get a 300 s time limit, while PG-DTS gets 1200 s. The failures in Table 2 (marked with -) are attributed to OOM or timeout, which makes the results difficult to interpret. Adding the exact reason for the failures and providing similar wall-clock times for all methods would be helpful. Additionally, providing the time to completion (termination under uncapped limit) for exact solvers may also strengthen your argument.
- PG-DTS discretizes the continuous variables into 5 bins. There is no mention of how this number is reached. On a similar note, the authors do not mention hyperparameter choices in their method, such as the exploration constant “c”, smoothing term “ϵ”, softmax temperature “τ” and their potential impact on the algorithm’s performance. Corresponding sensitivity tests should be added.
- The baselines were used out of the box with added restrictions like arbitrarily selected regularization and the small regularization option. No justification is provided for these choices, even though the datasets used for comparison are completely different from the ones mentioned in the baseline papers.
- The choice of datasets is also arbitrary, as the authors never mention how and why these datasets were selected for the experiments.
- The paper makes strong claims with respect to memory usage, but this is never measured.
- Possible closely related work/baseline is missing: “Branches: Efficiently Seeking Optimal Sparse Decision Trees via AO” (Chaouki et al. 2025) from the same authors of TSDT, which is cited in the paper.
- The novelty claims are too strong, as the engineering contributions already exist in most of the related work that is discussed. Deterministic MDP (Kohler et al., 2025; Chaouki et al., 2024); GLR cache is similar to DL8.5.
- The claims “under 20% label noise it is the most accurate method in every synthetic scenario.” and “On 12 OpenML datasets at depth 4, it achieves the highest accuracy on 7” are too strong, and no statistical testing is present in the paper to defend these.
- No variability is mentioned in Table 2, unlike Table 1.
- The ablations do not significantly show any performance improvement except runtime.
Questions for authors
- How would the performance change when all the methods are given the same budgets?
- How much effort was spent to find the optimal hyperparameter configurations for your method?
- Would spending similar effort on tuning the baselines improve their performance?
- How were the features binarized for exact solvers?
- How would a 5 bin ordinal encoding similar to the binning in PG-DTS change the performance of GOSDT or other binarized solvers?
- Can you provide the exact reason for the failure cases in table 2.
- How were the datasets selected?
- Why are these combination of (d,M) = (2, 1000), (3, 3000) and (4, 5000) chosen?
- Can a paired statistical test for PG-DTS and next best methods be added to prove your method’s superiority in different settings?
- Can the authors also add “Branches” method (Chaouki et al. 2025) to their comparison?
- Please add the variability in table 2 and mention what the ± values indicate in both the tables.
- Can you add metrics related to runtime and memory to your analysis, as this is one of the core contributions of the work.
- Can a sensitivity analysis on the number of bins be added?
- Can an ablation on the lazy build be provided as well?
- Why aren’t the exact solvers able to achieve 95% and 80% accuracy for the synthetic data?
- Could the novelty claims be softened a bit and similar prior work be cited appropriately in these scenarios?