- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SLtree, a multiclass classifier that constructs a balanced binary hierarchy by grouping entire classes through a label-constrained -means objective, then trains an SVM at each internal node to route samples toward class-specific leaves. Its extension, ISLtree, uses an improved particle swarm optimizer to select each node’s SVM hyperparameters independently.
Critical strengths
The label-constrained grouping formulation keeps each class within a single branch while controlling the number of classes assigned to each child. This provides a clear way to construct a shallow hierarchy without post-hoc pruning, which is the method’s main structural advantage.
Critical weaknesses
- The optimization procedure does not substantiate the claimed optimality or scalability. Equations (3.7)–(3.8) define a discrete, nonconvex grouping problem, while Appendix Algorithm 2 enumerates class assignments and alternates updates without establishing global optimality. The number of candidate groupings grows combinatorially, yet evaluation is limited to 3–18 classes. This limits confidence in the central claim of an efficient, generally applicable construction procedure.
- The empirical comparisons do not clearly establish the source or reliability of the gains. ISLtree receives extensive node-specific tuning without a demonstrated matched search budget for competing methods, and its accuracy improvement over SLtree is not statistically significant in the reported test. Moreover, Tables 2 and 3 list inconsistent baseline ranks. These issues make it difficult to determine whether the improvements reflect the proposed method, additional tuning effort, or errors in the statistical analysis.
Questions for authors
- Could you clarify whether Algorithm 2 exhaustively evaluates all groupings at each update, and what optimality guarantee applies to the final solution? Please also specify the rounding convention in Equation (3.8), particularly for even numbers of classes.
- What hyperparameter-search budgets were used for each baseline? Does any existing comparison separate the benefit of node-specific tuning from the choice of the improved PSO optimizer?
- Could you explain the different baseline ranks in Tables 2 and 3 and clarify whether correcting them changes the Holm-adjusted significance conclusions?
4 comments
Thanks for the response. The explanation of the rank differences between Tables 2 and 3 makes sense, and the rounding convention is now clear.
I still have two concerns:
- If all valid assignments are enumerated and the centroids have a closed-form solution, the global optimum can be found directly, so I'm not sure what the alternating step adds. Could you clarify which procedure is actually used? Since the number of groupings grows combinatorially, how does construction time scale beyond 18 classes?
- The node-level PSO in ISLtree is a form of hyperparameter tuning, while the baselines use default settings. Which data is used to evaluate the PSO objective, and does any experiment isolate the effect of node-specific tuning from that of the optimizer itself? The improvement of ISLtree over SLtree also wasn't statistically significant, which needs some discussion.
We thank the reviewer for this incisive observation, and we agree with it. We address the first question below. With a complete enumeration of valid groupings and a closed-form solution for the centroids, the joint problem can indeed be evaluated directly by scoring each with its closed-form , and the global optimum is then obtained in one pass. In that regime the alternating update adds nothing. The procedure actually used in this work is therefore the exact search over groupings, whose global optimality follows from the completeness of the enumeration rather than from the alternation.
That said, we would like to clarify that we also have a variant that genuinely follows the alternating-optimization idea, and it is this variant that we have in mind as the scalable generalization. Once is fixed, the decision unit is the class rather than the individual sample, because the label constraint requires all samples of a class to enter the same child. Each class therefore has only two options, with closed-form costs and for going left or right, and the total reconstruction error is the sum of over the classes sent left and over those sent right. Since the classes are chosen independently, the optimum under the balance constraint is simply to send the classes with the smallest to the left branch. This is exactly optimal and requires only sorting, at cost , or with linear-time selection, rather than enumerating all groupings. Combined with the closed-form centroid update, this gives a block-coordinate descent whose per-node cost is polynomial in the number of classes. The trade-off is a modest loss in accuracy, since it converges to a block-coordinate stationary point rather than the global optimum, and for the small class counts considered here the runtime saving is limited. Because every dataset in this paper has at most 18 classes, the enumerative version is affordable throughout and is the one underlying all reported results. The alternating variant is intended for settings with far larger numbers of classes, where enumeration becomes prohibitive.
We thank the reviewer for these points, and we address the second question below.
The PSO objective at each node is evaluated on the same node-level training data used by SLtree, so the comparison between ISLtree and SLtree is conducted on identical data under the same splitting framework.
We agree that our comparisons do not separate node-specific tuning from the choice of optimizer. ISLtree is not intended to show that PSO is a better optimizer, but to probe the performance ceiling of the nonlinear-SVM splitting strategy when each node is allowed its own , and the relevant comparison for our contribution is ISLtree versus SLtree under the same framework. A fully factorized study comparing default settings, grid search, and PSO under a matched number of SVM fits would be needed to attribute the gain precisely, and we now state this as a limitation.
We also agree that the advantage of ISLtree over SLtree is modest across the full benchmark, and that its statistical significance is marginal rather than strong. SLtree is now positioned as the recommended default given its much lower training cost, while ISLtree is presented as a performance-ceiling probe preferable when accuracy is the priority and the training budget permits.
We thank the reviewer for the constructive comments. Our responses are organized by point.
In Algorithm 2, all possible G matrices are exhaustively enumerated at initialization; during the alternating optimization of the C matrix and the G matrix, the G matrix is likewise exhaustively searched to find the optimal G matrix satisfying the constraints, whereas the C matrix is the closed-form least-squares solution obtained with the G matrix fixed. The rounding convention in Equation~(3.8) is rounding to the nearest integer with .5 rounded up. Assuming an even number of classes, l=2k, then h=k and l−h=k; assuming an odd number of classes, l=2k+1, then h=k and l−h=k+1. That is, if the number of classes is 10, they should be grouped into 5 classes and 5 classes; if the number of classes is 11, they should be grouped into 5 classes and 6 classes.
Each baseline uses the default parameters set by its authors, and no hyperparameter search was performed. The proposed methods SLtree and ISLtree in this paper also did not undergo hyperparameter tuning; in fact, the node-level SVM parameter optimization implemented in ISLtree should be regarded as an integral part of the proposed model, constituting an adaptive parameter optimization scheme.
The baseline rankings in Table 2 and Table 3 differ because the sets of algorithms in the two tables are different. Table 2 uses SLtree as the control algorithm and excludes ISLtree, with the main purpose of comparing the classification performance of the proposed SLtree method with that of other decision tree methods. Table 3 uses ISLtree as the control algorithm, mainly to compare the degree of improvement of ISLtree over SLtree and to compare its performance with that of other methods. Table 3 is in fact the rank table of all experimental methods.