acceptodds
Under review as a conference paper at ICLR 2027

Soft Classification Trees with Conditional Computation

Abstract

Decision trees are widely used supervised learning models for tabular data. Multivariate deterministic trees provide interpretable decision rules and conditional computation but typically rely on expensive mixed-integer optimization, whereas soft trees are amenable to continuous optimization at the cost of losing single-path routing and conditional computation. We propose a soft classification tree that restores single-path routing and conditional computation while retaining the favorable properties of continuous optimization. Our approach relies on a new activation function for routing samples at branch nodes, defined as a novel scaled variant of the negative log-likelihood that combines its benefits with those of the sigmoid, is differentiable almost everywhere, and is proven to be classification-calibrated. The resulting model is suitably trained using a projected adaptation of the Adam optimizer to handle class assignment constraints. Unlike prior differentiable trees, our method needs only a single first-order run, with no scaling, annealing, warm start, or multi-start. Experiments on 61 classification datasets show that our single-tree method is better than all the compared multivariate single-tree alternatives, attaining essentially the same predictive performance as OCT-H, the strongest multivariate tree baseline, while requiring much lower average training time. It also achieves predictive performance competitive with Random Forest and within about on average of XGBoost. On synthetic datasets, it substantially outperforms OCT-H in recovering the underlying ground-truth tree structure.

Primary area:
optimization
Keywords:
Soft decision trees, Non-convex optimization, Conditional computation, Tabular data, Classification calibration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_AUP1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes sNLL-CT, a differentiable soft classification tree that aims to retain the optimization advantages of soft trees while recovering deterministic single-path routing and conditional computation. The main components are a new scaled negative-log-likelihood (sNLL) activation and a projected Adam procedure for handling leaf-class assignment constraints.

The method is evaluated on 61 real-world datasets against several single-tree baselines, including OCT-H, ORCT, DTSemNet, RADDT, and LatentTree, as well as RF and XGBoost. Synthetic experiments are also used to evaluate recovery of ground-truth tree structures. The results suggest that sNLL-CT is competitive with the strongest baselines considered and that its deterministic prediction closely matches soft aggregation.

Critical strengths
  1. Interesting combination of differentiable optimization and deterministic routing.
    The proposed activation appears effective at producing concentrated routing, and the experiments show that single-path prediction closely agrees with soft aggregation. This provides meaningful empirical support for the central modeling idea.

  2. Broad experimental evaluation and useful ablations.
    The paper evaluates many datasets, several tree depths, multiple recent baselines, and synthetic ground-truth recovery. The sigmoid and NLL ablations are particularly useful for isolating the role of the proposed activation.

Critical weaknesses
  1. The positioning against the strongest multivariate single-tree methods needs stronger justification.
    OCT-H is treated as the strongest deterministic multivariate tree baseline, but it is not clear that OCT-H alone adequately represents the strongest contemporary methods in this category. The paper itself cites newer multivariate optimal-tree approaches, such as Blanco et al. (2023) and D'Onofrio et al. (2024), but does not compare against them where applicable. The paper does include recent differentiable methods such as RADDT, which is valuable, but the claim regarding the strongest deterministic multivariate baseline should either be justified more carefully or weakened to "the strongest among the evaluated baselines."

    In addition, the main tables emphasize average accuracy across a heterogeneous collection of datasets. This is a reasonable descriptive statistic, and the paired Wilcoxon tests partly address this concern, but the mean can still obscure heterogeneous behavior across datasets. Average ranks or win/tie/loss summaries would provide a clearer cross-dataset comparison.

  2. The training-efficiency claim is difficult to interpret from the current evaluation.
    Table 2 emphasizes substantially lower mean training time for sNLL-CT, but the per-dataset tables show many cases where OCT-H is faster, together with a smaller set of cases with very large OCT-H runtimes. Thus, the arithmetic mean is sensitive to the long-tailed runtime distribution and may not reflect typical per-dataset behavior. Although the paper provides runtime boxplots, reporting median/geometric-mean runtime and the fraction of datasets on which each method is faster would make the efficiency claim substantially clearer.

    More importantly, the runtime comparison is not hardware-independent: sNLL-CT uses a GPU-enabled PyTorch implementation, whereas OCT-H uses a proprietary CPU implementation. The paper acknowledges this limitation, but some of the headline efficiency claims still appear stronger than warranted. The experimental protocol also differs across methods in tuning and optimization details, which further complicates direct wall-clock comparisons.

    Finally, conditional computation is presented as an important practical advantage, but the paper does not report direct inference-time or throughput measurements. Since reduced inference cost is a central motivation for single-path routing, such measurements would strengthen this claim considerably.

Questions for authors
  1. Could the authors clarify why OCT-H is considered the strongest deterministic multivariate tree baseline? In particular, could newer multivariate optimal-tree methods be included where applicable, or could the claim be restricted to the strongest baseline among those evaluated?

  2. Could the authors report more robust runtime summaries, such as median or geometric-mean training time and win/tie/loss counts? The per-dataset tables suggest substantial runtime heterogeneity, and it would also be helpful to explain the cases where OCT-H becomes extremely slow.

  3. What exactly is included in the reported training time for each method (e.g., hyperparameter search, validation-based model selection, pruning selection, and retraining)? Given the different computational backends and tuning protocols, this is important for assessing the fairness of the comparison.

  4. Since conditional computation is one of the main motivations of the paper, could the authors provide direct inference-time or throughput comparisons between single-path prediction and soft/full-tree prediction?

3 comments
AuthorsOpenReview ↗

Response to Reviewer AUP1

We thank the reviewer for the detailed assessment.

Q1 (Strongest deterministic multivariate baseline). We have clarified in the revised main text of the paper that "strongest" refers to the strongest among the evaluated multivariate single-tree baselines. OCT-H is a well-established reference, extensively benchmarked on more than 50 real-world datasets (Bertsimas & Dunn, 2017; 2019), and we use its official Interpretable AI implementation. The suggested alternatives lack comparably broad evidence of accuracy and scalability. Specifically, D'Onofrio et al. (2024) focus on binary classification (10 datasets, all with fewer than 1,000 observations, only depth 2), while Blanco et al. (2023) evaluate 12 datasets at depth 3 and reduce Waveform (5,000 observations) to approximately 100 aggregated training points per fold for tractability. Moreover, both compare against their own Gurobi implementations of OCT-H rather than the official software. Thus, their results do not establish either method as a stronger reference across our 61 binary/multiclass datasets and four depth settings.

Q2 (Robust training times). We have expanded the discussion in Section 4.2 and added detailed runtime statistics (mean/median times and win/tie/loss counts) and a dataset-size analysis in Appendix D (Table 7). sNLL-CT is faster than OCT-H grid on 16/18 datasets with at least 1,000 samples, consistently across all four settings, whereas OCT-H grid tends to be faster on smaller datasets. In setting (i), reported in the main text, median training times on the larger-dataset subset are 45.7 s vs. 120.4 s. Across all 61 datasets, the 90th-percentile times are 79.3 s vs. 381.9 s. Moreover, sNLL-CT is faster than RADDT on all 61 datasets in every setting. Thus, the lower mean reflects a systematic advantage on larger datasets, rather than merely a few extreme OCT-H runtimes.

Q3 (Training-time protocol). Reported times measure the complete training pipeline adopted for each method, not merely the final fitting step, including hyperparameter search, validation-based model selection, pruning selection, and retraining wherever applicable. All methods were evaluated on the same datasets and folds, using the authors' code wherever available and following their documented training protocols. Importantly, we also examined alternative configurations of two key baselines: OCT-H was evaluated both without pruning (OCT-H single) and with validation-selected cost-complexity pruning (OCT-H grid), while RADDT was evaluated with 1, 5, and 10 optimization starts, retaining its CART initialization and annealing procedure. sNLL-CT achieves higher average accuracy and substantially lower training time than RADDT in all three configurations (Appendix E). The remaining methods retain their prescribed training and model-selection procedures, detailed in Appendix C.3.

Q4 (Inference time). We added in Appendix I a direct comparison of single-path versus full-tree soft-aggregated inference on the same trained sNLL-CT models, across all 61 datasets and five folds, at depths 4 and 5. After an untimed warm-up, we measure median inference time over five runs on each test fold, normalized per sample. Single-path inference is faster on all 61 datasets at both depths: 0.00647 vs. 0.35310 ms/sample at depth 4 and 0.00756 vs. 0.84505 ms/sample at depth 5, corresponding to 54.56× and 111.86× geometric-mean speedups, respectively. Crucially, prediction agreement remains above 99% on average, demonstrating the computational benefit of conditional computation while closely preserving predictive accuracy.

Thank you for the detailed rebuttal and the substantial additions to the revised manuscript. I appreciate the authors' efforts to address my concerns with additional evidence rather than only textual clarifications.

The rebuttal has resolved several of my original questions and improved the paper. However, the strongest evidence currently supports competitive predictive accuracy and conditional computational advantages, rather than a clear and general improvement over the strongest existing single-tree approaches. Considering the paper's overall novelty, empirical evidence, and remaining limitations, I am not yet sufficiently convinced to change my recommendation to acceptance, and therefore plan to retain my original Weak Reject rating.

AuthorsOpenReview ↗

Response to Reviewer AUP1; Follow-up

We appreciate the reviewer's acknowledgment of our revisions and additional experiments, which directly address all specific requests raised in the original review.

Our contribution does not claim universal accuracy dominance over existing single-tree methods, but combines competitive accuracy, efficient first-order training without elaborate procedures used by existing differentiable trees (tuned sigmoid scaling, annealing, routing regularization, warm starts, or multiple restarts), and deterministic single-path inference.

This combination is supported by extensive empirical evidence: 61 real-world datasets, 4 depth settings, 5 synthetic experiments, and 8 comparison methods (OCT-H, ORCT, DTSemNet, RADDT, LatentTree, CART, RF, and XGBoost). Our classification benchmark is broader than the original evaluations of all the multivariate single-tree baselines considered. Against the two strongest multivariate single-tree baselines in our main comparison, sNLL-CT achieves accuracy comparable to OCT-H grid with substantially lower observed training time (86.89% vs. 86.73%; 36 vs. 297 seconds in setting i), and outperforms RADDT in average accuracy across all four settings while training faster on all 61 datasets in every setting.

Simplified training and deterministic conditional inference are central methodological contributions, not ancillary benefits. We hope the recommendation can be reconsidered in light of the evidence addressing the reviewer's original concerns and the paper's stated contributions, rather than a broader requirement of general superiority that the paper does not claim.

Reviewer_3EYMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes to learn a variation of soft decision trees where the main novelty is in the use of a different routing function based on the scaled variant of the negative log-likelihood function. The predictive function is by construction defined to use a single route-to-leaf path by following the branch with the highest score. The objective function being minimized is the misclassification cost over the training set with constraints subject to each leaf predicting only one class and the classes must be covered by at least one leaf. The training algorithm is based on Adam optimizer for unconstrained parameters and projection onto the convex set for constrained parameters. Experiments performed on 61 datasets showing the improved accuracy of the learned trees vs other recent oblique decision tree baselines.

Critical strengths

The proposed routing function has more favorable properties than sigmoid. Perhaps, it might be used in training neural networks.

Critical weaknesses
  • Limiter novelty of the paper. While using scaled variant of the negative log-likelihood function as a routing function is interesting, it is just one out-of-many possibilities to use. For example, "Differentiable Decision Tree via "ReLU+Argmin" Reformulation" Q. Mao et al. uses ReLU based formulation.
  • The objective function in eq. 3a does not look differentiable; it is a cost-sensitive 0/1 loss. How is it possible to calculate its gradient?
  • The proposed training algorithm is not guaranteed to converge. In traditional soft decision trees, everything is differentiable and gradient-based optimization methods will converge to some local optima. In this paper, there is an ad-hoc step of projecting onto the convex set and Adam optimizer for the other parameters. There are no theoretical guarantees of convergence.
  • In experiments, many of the datasets are very small, and the accuracy difference between methods corresponds to just a few misclassified points which might also be due to the chance of different training and test splits. Some important baselines such CART and Tree Alternating Optimization (TAO) are missing.
  • The paper claims the interpretability of the learned trees. However, multivariate splits are much harder to interpret than axis-aligned ones. This makes it harder to argue for interpretability in spite of using a single route-to-leaf path.
Questions for authors
  1. What are the convergence guarantees of the proposed training algorithm?
  2. The sNLL routing function is not differentiable at the margin. Can it affect the stability of the training?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 3EYM

W1 (Novelty). The contribution is not merely a new routing activation. sNLL is part of a complete framework combining continuous first-order optimization, constrained leaf-class assignment, deterministic single-path inference, and conditional computation. Unlike previous differentiable approaches relying on tuned sigmoid scaling, annealing, routing regularization, warm starts, or multiple restarts, sNLL-CT achieves these properties in one training run without any of them. ReLU+Argmin cited by the reviewer (RADDT) is already compared across all datasets and settings. sNLL-CT achieves higher average accuracy with substantially lower training time, even against 1-start RADDT (Appendix E). The contribution goes beyond the activation: simpler training with competitive accuracy and conditional computation.

W2/Q2 (Differentiability; zero-margin behavior). Contrary to the reviewer's claim, Eq. (3a) is a continuous, almost-everywhere differentiable training objective, not a discrete 0/1 loss. As stated below Eq. (3e), eyike_{y_i k} are fixed misclassification costs, whereas ckt∈[0,1]c_{kt}\in[0,1] are continuous optimization variables and Mit(w)M_{it}(w) are continuous routing functions. The objective is linear in cktc_{kt}, with gradients with respect to ww existing almost everywhere. The hard 0/1 indicators appear in Eq. (2) for prediction, not in the objective (3a). Thus, characterizing Eq. (3a) as a nondifferentiable 0/1 loss is mathematically incorrect.

As with ReLU, nondifferentiability at zero routing margins does not prevent gradient-based training. sNLL is continuous, locally Lipschitz, and differentiable almost everywhere; at zero margin, PyTorch's autograd selects a valid generalized derivative, as standard for nonsmooth activations. We observed no related training instability across all datasets and depth settings.

W3/Q1 (Convergence guarantees). The reviewer's claim that gradient-based optimization of differentiable soft trees necessarily converges to a local minimum is mathematically incorrect. Even standard gradient descent requires additional assumptions to guarantee convergence to local minima (Lee et al., COLT'16), while convergence results for nonconvex stochastic optimization generally concern first-order stationarity under specific conditions (Bottou et al., SIAM Review'18). Moreover, Adam can fail to converge to an optimum even on convex problems (Reddi et al., ICLR'18). Our projected-Adam uses Euclidean projection to maintain feasible leaf assignments; we claim no guarantee of convergence to a local minimum. The absence of such a guarantee is a limitation of theoretical scope, not a flaw in the formulation or a contradiction of the empirical findings.

W4 (Dataset size and missing baselines). Our 61-dataset benchmark includes the datasets used by Bertsimas & Dunn plus additional multiclass tasks, ensuring comparability with established optimization-based trees. Dataset sizes reflect competing methods' computational limitations, not those of sNLL-CT: OCT-H incurs prohibitive runtimes on datasets with tens of thousands of samples. Our conclusions rely on 5-fold evaluations and paired statistical tests across 61 datasets, not a few misclassified samples from one split.

We added CART comparisons (Appendix L): sNLL-CT outperforms validation-tuned CART in all settings; CART trains faster. To jointly address the reviewer's dataset-size and TAO concerns, we evaluated sNLL-CT on the 3 largest of the 5 datasets reported by TAO's paper, the 3 with at least 17000 samples and 50 features. Since TAO uses validation to select both depth (up to 12) and sparsity, we use for sNLL-CT a simpler protocol, validating depth over 2–7, on the same test sets. sNLL-CT achieves competitive test accuracy: 73.4% vs. 72.19% on Seismic, 92.8% vs. 94.3% on MNIST, and 68.0% vs. 68.3% on Protein (sNLL-CT vs. TAO). Thus, sNLL-CT is competitive on the 3 datasets despite simpler model selection.

W5 (Interpretability). Interpretability depends on the complexity of the entire decision process, not just on the simplicity of single splits. The new figure (Section 4.2; details in Appendix K) illustrates this trade-off: on synthetic data with oblique ground-truth splits, sNLL-CT achieves 98.0% test accuracy with 8 leaves, 29 parameters, and 3 splits per prediction, whereas the best CART tested up to depth 12 achieves 91.9% with 98 leaves and 292 parameters. Thus, simpler univariate splits can require substantially larger trees to approximate the same decision boundaries. Our interpretability claim is procedural: sNLL-CT follows one auditable root-to-leaf path, unlike conventional soft trees aggregating contributions from multiple leaves. Each leaf defines a convex polyhedral region, enabling exact minimum-cost counterfactual explanations (Carreira-Perpiñán & Hada, AAAI'21); soft trees lack this direct polyhedral, path-based representation.

I thank the authors for providing a detailed response.

  • Regarding the novelty, I still remain unconvinced regarding the fundamental novelty of the paper. Continuous first-order optimization, deterministic single-path inference, and conditional computation all have been previously explored within soft decision trees literature.
  • Regarding differentiability, thanks for the clarification. I got confused by line number 181 "minimizing the expected misclassification cost over the training set". Providing in words what is being optimized could help clarify the objective function of eq. 3.
  • Convergence guarantees. Thanks for the response. I was confused in my wording, I meant to say a stationarity point. I agree with the authors that this is a limitation of theoretical scope and is really not important in practice. But the paper does not show any learning curves demonstrating convergence behavior in practice.
  • On dataset size. Choosing the datasets used by Bertsimas & Dunn is not a good idea. They train optimal decision trees which is an NP-hard problem. And so, they necessarily have to select very small scale datasets as otherwise their MIO solvers deal with exponential problem complexity.
  • MNIST results. In my experience, one could easily achieve ~95% test accuracy on MNIST by training soft decision trees and just making it hard during inference.
  • Interpretability. Showing that CART gives larger trees on synthetic data might not be fair. In many real world dataset, axis-aligned decision trees achieve decent accuracy while still remaining small.
AuthorsOpenReview ↗

Response to Reviewer 3EYM; Follow-up

Novelty. We do not claim that first-order optimization, single-path inference, or conditional computation are individually new. The novelty lies in achieving these properties together through a substantially simplified soft-tree formulation and single-run training procedure, enabled by the proposed sNLL activation and its classification-calibrated loss. The resulting method achieves state-of-the-art test accuracy among the evaluated single-tree methods and favorable empirical scalability. Merely listing previously known components does not address this methodological contribution.

Differentiability. Equation (3) minimizes a continuous expected misclassification cost, computed using soft routing probabilities and continuous leaf-class assignment variables. The description at line 181 is mathematically correct; it does not imply optimization of a discrete 0–1 loss. We can clarify the wording, but the original objection concerning nondifferentiability was based on an incorrect interpretation of the objective.

Stationarity and learning curves. We appreciate the clarification, but differentiability alone does not guarantee convergence even to stationary points; such claims require additional algorithm-specific assumptions. Learning curves are a new request raised only in this follow-up and could illustrate empirical optimization behavior, but cannot establish convergence guarantees.

Dataset selection and scalability. The original OCT-H benchmarks provide an established basis for comparing deterministic and differentiable multivariate classification trees. Moreover, our evaluation extends well beyond small datasets, including Seismic, MNIST, Protein, Adult, and Skin Segmentation, with tens to hundreds of thousands of observations. Our runtime analysis shows that sNLL-CT's advantage over OCT-H grid is particularly pronounced on larger datasets; against RADDT, it trains faster on all 61 datasets across all four settings. Scalability is a demonstrated strength of our approach, not a limitation imposed by the benchmark.

MNIST. Published results do not support treating approximately 95% accuracy as automatically attainable: Frosst and Hinton (2017) report 94.45% for a depth-8 soft decision tree trained without distillation, while Panda et al. (2024) report 94.00% for DGT at depth 8. An anecdotal expectation based on a single dataset cannot establish a deficiency in our experimental evaluation.

Interpretability. Our synthetic experiment assesses recovery of a known oblique decision structure; it does not claim that CART always produces large trees. More importantly, our notion of interpretability is operational, not merely a measure of tree size: each prediction follows a deterministic sequence of explicit hyperplane decisions, defining a polyhedral region and enabling counterfactual analysis. The fact that CART may perform well with small trees on other datasets does not challenge either contribution.

We ask that the assessment address the actual methodological contribution and the evidence presented, rather than objections that have been clarified or expectations unsupported by the cited literature.

Reviewer_B1ijOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose sNLL-CT, a method for training oblique classification trees using a scaled softplus activation and projected Adam for the leaf assignments. It trains on a continuous objective over leaf weights and then switches to standard, deterministic root-to-leaf routing at inference time. The paper proves classification calibration for its binary margin loss and tests the approach on synthetic data and 61 benchmark datasets.

Critical strengths
  1. good objective Single tree with simple training procedure is valuable, and the results suggest sNLL-CT brought advantages compares to current, more expensive oblique tree algorithms.

  2. broad empirical evaluations the evaluation set covers 60+ datasets, various depth and recent differentiable-tree baselines. Which provides useful insights on the effectiveness of the method.

  3. simple & elegant solution the proposed activation has an intuitive optimization motivation. The decay behaviour is different from exponential decay of more widely used sigmoid surrogates, which may of use in general ML/DL domains.

Critical weaknesses

limited practical usage As a tree classification algorithm, sNLL is not cheaper-to-train or more accurate than XGBoost. The author focused on these efficiency & acucracy prospects, comparing with previous SOTA oblique tree algos, which makes sense to me. But ultimately, how to convince others to use sNLL boils down to the benefit of 1) Inference Cost (which seems missing from the main paper) 2) Explainability due to it produces oblique trees. Having these two would make the paper in a stronger position.

performance margin is small comparing with RADDT, and baselines, the margin is small. However I understand there is an upper-limit towards how high the performance can be. Some analysis on when to use sNLL maybe helpful/

Questions for authors

I'd like to know more about its inference comparison / comparing with more activations (smooth-step?). Also see the weakness section pls.

3 comments
AuthorsOpenReview ↗

Response to Reviewer B1ij

We thank the reviewer for the constructive assessment of our approach.

W1 (Interpretability). Interpretability depends not only on the simplicity of individual splits, but also on the complexity of the entire decision process. The new Figure in Section 4.2, with further details in Appendix K, illustrates this: on a synthetic dataset with oblique ground-truth boundaries, sNLL-CT achieves 98.0% test accuracy with 8 leaves and 29 parameters, versus 91.9% with 98 leaves and 292 parameters for the best CART tested up to depth 12. Thus, simpler univariate splits can require substantially larger trees. Beyond compactness, sNLL-CT provides auditable single-path predictions, evaluating only D linear splits, with margin-based routing scores. Each leaf defines a convex polyhedral region, enabling exact minimum-cost counterfactual explanations under suitable cost functions (Carreira-Perpiñán & Hada, AAAI'21). Unlike conventional soft trees, which aggregate contributions across multiple leaves, sNLL-CT provides a directly exploitable, path-based representation for actionable explanations, while retaining efficient continuous training.

W1 (Inference time). We added in Appendix I a direct comparison of single-path versus full-tree soft-aggregated inference on the same trained sNLL-CT models, across all 61 datasets and five folds, at depths 4 and 5. After an untimed warm-up, we measure median inference time over five runs on each test fold. Single-path inference is faster on all 61 datasets at both depths, achieving geometric-mean speedups of 54.56× (depth 4) and 111.86× (depth 5), while retaining over 99% prediction agreement on average with full-tree aggregation. These results demonstrate how conditional computation benefits inference as tree size grows: single-path routing evaluates only D branch nodes rather than the full tree.

W2 (Performance margin is small). The accuracy differences are statistically supported. Across all four depth settings, sNLL-CT significantly outperforms ORCT, DTSemNet, and OCT-H single after Holm correction (Tables 4, 11, 13, 15). Direct paired Wilcoxon tests also favor sNLL-CT over RADDT in every setting (unadjusted p = 0.030, 0.005, 0.035, 0.035). Crucially, these gains come with substantially simpler and faster training: in setting (i), sNLL-CT achieves 86.89% versus 85.90% accuracy in 36 versus 399.5 seconds on average (Table 2), without RADDT's warm start, multiple restarts, or annealing. This accuracy-efficiency trade-off is central to our contribution.

W2 (When sNLL-CT is useful). The synthetic experiments identify particularly favorable conditions. Under feature noise (Exp3), sNLL-CT significantly outperforms OCT-H in test accuracy and ground-truth partition recovery (Rand Index) at all four noise levels. As feature dimensionality increases (Exp5), sNLL-CT also achieves significantly higher test accuracy and Rand Index at all three higher-dimensional settings (Table 5, Appendix G). These results demonstrate advantages beyond average accuracy differences on real-world datasets.

Q (Comparison with more activations). We have extended the activation ablation in Table 3 to include cubic smoothstep alongside sigmoid and unscaled NLL, evaluating all four activations across 61 datasets and four depth settings. For smoothstep, we select h∈{1,3}h\in\{1,3\} by validation accuracy under the same training framework. Despite tuning, smoothstep achieves 70.35–74.31% average test accuracy versus 85.95–86.89% for sNLL-CT, a 12.4–16.5 percentage-point gap. Its training time is also higher (61–73 vs. 35–39 seconds), including both candidate runs. Full per-dataset results are reported in the new Appendix M. These results further support the advantage of the proposed tuning-free sNLL activation.

Appreciate the authors response and revisions. The added results and figures further strengthen the paper (Figure 3 is great addition!).

But as Reviewer 3EYM mentioned, the performance & novelty of the proposed method maybe limited. Keeping the current scoring as it stands.

AuthorsOpenReview ↗

Response to Reviewer B1ij; Follow-up

The follow-up acknowledges that our additional results and Figure 3 strengthen the paper, yet retains the original score based on a vague concern that “the performance & novelty of the proposed method maybe limited,” citing Reviewer 3EYM. This is difficult to reconcile with the original assessment, which reported no substantive soundness concerns and explicitly identified a valuable objective, broad evaluation across 60+ datasets, and a simple, elegant activation with potential broader relevance as strengths.

We have directly addressed the reviewer's specific requests with additional activation ablations, statistical analyses, and interpretability evidence. Moreover, sNLL-CT achieves accuracy comparable to OCT-H grid at substantially lower observed training time (86.89% vs. 86.73%; 36 vs. 297 seconds) and outperforms RADDT in average accuracy and training efficiency across all four settings. The novelty lies in combining efficient first-order training without elaborate procedures with deterministic single-path inference, not in claiming universal accuracy dominance.

Referring to another reviewer's general reservations does not identify a concrete unresolved limitation. Given the original assessment and the new evidence, we ask the reviewer to clarify what specific concern still justifies the unchanged recommendation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.