- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes sNLL-CT, a differentiable soft classification tree that aims to retain the optimization advantages of soft trees while recovering deterministic single-path routing and conditional computation. The main components are a new scaled negative-log-likelihood (sNLL) activation and a projected Adam procedure for handling leaf-class assignment constraints.
The method is evaluated on 61 real-world datasets against several single-tree baselines, including OCT-H, ORCT, DTSemNet, RADDT, and LatentTree, as well as RF and XGBoost. Synthetic experiments are also used to evaluate recovery of ground-truth tree structures. The results suggest that sNLL-CT is competitive with the strongest baselines considered and that its deterministic prediction closely matches soft aggregation.
Critical strengths
-
Interesting combination of differentiable optimization and deterministic routing.
The proposed activation appears effective at producing concentrated routing, and the experiments show that single-path prediction closely agrees with soft aggregation. This provides meaningful empirical support for the central modeling idea. -
Broad experimental evaluation and useful ablations.
The paper evaluates many datasets, several tree depths, multiple recent baselines, and synthetic ground-truth recovery. The sigmoid and NLL ablations are particularly useful for isolating the role of the proposed activation.
Critical weaknesses
-
The positioning against the strongest multivariate single-tree methods needs stronger justification.
OCT-H is treated as the strongest deterministic multivariate tree baseline, but it is not clear that OCT-H alone adequately represents the strongest contemporary methods in this category. The paper itself cites newer multivariate optimal-tree approaches, such as Blanco et al. (2023) and D'Onofrio et al. (2024), but does not compare against them where applicable. The paper does include recent differentiable methods such as RADDT, which is valuable, but the claim regarding the strongest deterministic multivariate baseline should either be justified more carefully or weakened to "the strongest among the evaluated baselines."In addition, the main tables emphasize average accuracy across a heterogeneous collection of datasets. This is a reasonable descriptive statistic, and the paired Wilcoxon tests partly address this concern, but the mean can still obscure heterogeneous behavior across datasets. Average ranks or win/tie/loss summaries would provide a clearer cross-dataset comparison.
-
The training-efficiency claim is difficult to interpret from the current evaluation.
Table 2 emphasizes substantially lower mean training time for sNLL-CT, but the per-dataset tables show many cases where OCT-H is faster, together with a smaller set of cases with very large OCT-H runtimes. Thus, the arithmetic mean is sensitive to the long-tailed runtime distribution and may not reflect typical per-dataset behavior. Although the paper provides runtime boxplots, reporting median/geometric-mean runtime and the fraction of datasets on which each method is faster would make the efficiency claim substantially clearer.More importantly, the runtime comparison is not hardware-independent: sNLL-CT uses a GPU-enabled PyTorch implementation, whereas OCT-H uses a proprietary CPU implementation. The paper acknowledges this limitation, but some of the headline efficiency claims still appear stronger than warranted. The experimental protocol also differs across methods in tuning and optimization details, which further complicates direct wall-clock comparisons.
Finally, conditional computation is presented as an important practical advantage, but the paper does not report direct inference-time or throughput measurements. Since reduced inference cost is a central motivation for single-path routing, such measurements would strengthen this claim considerably.
Questions for authors
-
Could the authors clarify why OCT-H is considered the strongest deterministic multivariate tree baseline? In particular, could newer multivariate optimal-tree methods be included where applicable, or could the claim be restricted to the strongest baseline among those evaluated?
-
Could the authors report more robust runtime summaries, such as median or geometric-mean training time and win/tie/loss counts? The per-dataset tables suggest substantial runtime heterogeneity, and it would also be helpful to explain the cases where OCT-H becomes extremely slow.
-
What exactly is included in the reported training time for each method (e.g., hyperparameter search, validation-based model selection, pruning selection, and retraining)? Given the different computational backends and tuning protocols, this is important for assessing the fairness of the comparison.
-
Since conditional computation is one of the main motivations of the paper, could the authors provide direct inference-time or throughput comparisons between single-path prediction and soft/full-tree prediction?
3 comments
Thank you for the detailed rebuttal and the substantial additions to the revised manuscript. I appreciate the authors' efforts to address my concerns with additional evidence rather than only textual clarifications.
The rebuttal has resolved several of my original questions and improved the paper. However, the strongest evidence currently supports competitive predictive accuracy and conditional computational advantages, rather than a clear and general improvement over the strongest existing single-tree approaches. Considering the paper's overall novelty, empirical evidence, and remaining limitations, I am not yet sufficiently convinced to change my recommendation to acceptance, and therefore plan to retain my original Weak Reject rating.
Response to Reviewer AUP1; Follow-up
We appreciate the reviewer's acknowledgment of our revisions and additional experiments, which directly address all specific requests raised in the original review.
Our contribution does not claim universal accuracy dominance over existing single-tree methods, but combines competitive accuracy, efficient first-order training without elaborate procedures used by existing differentiable trees (tuned sigmoid scaling, annealing, routing regularization, warm starts, or multiple restarts), and deterministic single-path inference.
This combination is supported by extensive empirical evidence: 61 real-world datasets, 4 depth settings, 5 synthetic experiments, and 8 comparison methods (OCT-H, ORCT, DTSemNet, RADDT, LatentTree, CART, RF, and XGBoost). Our classification benchmark is broader than the original evaluations of all the multivariate single-tree baselines considered. Against the two strongest multivariate single-tree baselines in our main comparison, sNLL-CT achieves accuracy comparable to OCT-H grid with substantially lower observed training time (86.89% vs. 86.73%; 36 vs. 297 seconds in setting i), and outperforms RADDT in average accuracy across all four settings while training faster on all 61 datasets in every setting.
Simplified training and deterministic conditional inference are central methodological contributions, not ancillary benefits. We hope the recommendation can be reconsidered in light of the evidence addressing the reviewer's original concerns and the paper's stated contributions, rather than a broader requirement of general superiority that the paper does not claim.
Response to Reviewer AUP1
We thank the reviewer for the detailed assessment.
Q1 (Strongest deterministic multivariate baseline). We have clarified in the revised main text of the paper that "strongest" refers to the strongest among the evaluated multivariate single-tree baselines. OCT-H is a well-established reference, extensively benchmarked on more than 50 real-world datasets (Bertsimas & Dunn, 2017; 2019), and we use its official Interpretable AI implementation. The suggested alternatives lack comparably broad evidence of accuracy and scalability. Specifically, D'Onofrio et al. (2024) focus on binary classification (10 datasets, all with fewer than 1,000 observations, only depth 2), while Blanco et al. (2023) evaluate 12 datasets at depth 3 and reduce Waveform (5,000 observations) to approximately 100 aggregated training points per fold for tractability. Moreover, both compare against their own Gurobi implementations of OCT-H rather than the official software. Thus, their results do not establish either method as a stronger reference across our 61 binary/multiclass datasets and four depth settings.
Q2 (Robust training times). We have expanded the discussion in Section 4.2 and added detailed runtime statistics (mean/median times and win/tie/loss counts) and a dataset-size analysis in Appendix D (Table 7). sNLL-CT is faster than OCT-H grid on 16/18 datasets with at least 1,000 samples, consistently across all four settings, whereas OCT-H grid tends to be faster on smaller datasets. In setting (i), reported in the main text, median training times on the larger-dataset subset are 45.7 s vs. 120.4 s. Across all 61 datasets, the 90th-percentile times are 79.3 s vs. 381.9 s. Moreover, sNLL-CT is faster than RADDT on all 61 datasets in every setting. Thus, the lower mean reflects a systematic advantage on larger datasets, rather than merely a few extreme OCT-H runtimes.
Q3 (Training-time protocol). Reported times measure the complete training pipeline adopted for each method, not merely the final fitting step, including hyperparameter search, validation-based model selection, pruning selection, and retraining wherever applicable. All methods were evaluated on the same datasets and folds, using the authors' code wherever available and following their documented training protocols. Importantly, we also examined alternative configurations of two key baselines: OCT-H was evaluated both without pruning (OCT-H single) and with validation-selected cost-complexity pruning (OCT-H grid), while RADDT was evaluated with 1, 5, and 10 optimization starts, retaining its CART initialization and annealing procedure. sNLL-CT achieves higher average accuracy and substantially lower training time than RADDT in all three configurations (Appendix E). The remaining methods retain their prescribed training and model-selection procedures, detailed in Appendix C.3.
Q4 (Inference time). We added in Appendix I a direct comparison of single-path versus full-tree soft-aggregated inference on the same trained sNLL-CT models, across all 61 datasets and five folds, at depths 4 and 5. After an untimed warm-up, we measure median inference time over five runs on each test fold, normalized per sample. Single-path inference is faster on all 61 datasets at both depths: 0.00647 vs. 0.35310 ms/sample at depth 4 and 0.00756 vs. 0.84505 ms/sample at depth 5, corresponding to 54.56× and 111.86× geometric-mean speedups, respectively. Crucially, prediction agreement remains above 99% on average, demonstrating the computational benefit of conditional computation while closely preserving predictive accuracy.