- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper systematically investigates four optimization methods for Hierarchical-Task Numerical Conjunctive Feature Learning (HTNL), analyzing their theoretical properties, feature selection behavior, and computational efficiency, while proposing an improved ADMM solver and identifying limitations of active-set selection for rare-event forecasting.
Critical strengths
With regard to structured sparse feature selection, distinguishing between the effectiveness of the optimization algorithm itself and that resulting from the design of the objective function is a meaningful area of research.
Critical weaknesses
1.The paper has insufficient motivation, since it doesn't justify why revisiting the HTNL optimization is necessary. 2. The 11 real-world datasets share the same characteristic, limiting its ability to demonstrate advantages over existing methods in general.
Questions for authors
- Why is it important to study optimization in HTNL? From the theoretical analysis and experiment, there is lack of evidence to prove the necessity.
- Your title is “Rethinking …”, what is the major difference before and after your findings about the optimization in HTNL?
- Although there are many datasets tested in experiment, why do you not compare with other baselines or benchmarks?
- For contributions 2 and 3, what is the meaning or can other people learn from the exact ADMM proximal update and structure active-set bound?
- In 2.1, why the objective function written in this form? Why can it forecast the rare event? Are the results sensitive to the primitive features P?
7 comments
Table R2b. Paired Changes When the Primitive Count Doubles
| Solver | Metric | Direction | Mean change magnitude | Improved seeds | Tied seeds | Worse seeds |
|---|---|---|---|---|---|---|
| Original | F1 | Decrease | 0.014 | 2 | 15 | 3 |
| Original | Recall | Decrease | 0.017 | 0 | 19 | 1 |
| ADMM-Bisect | F1 | Increase | 0.043 | 8 | 8 | 4 |
| ADMM-Bisect | Recall | Decrease | 0.004 | 1 | 18 | 1 |
The results show that adding noise features causes only small changes in mean recall. Original's mean F1 decreases slightly, while ADMM-Bisect maintains its performance and achieves a higher mean F1. This suggests that both methods remain relatively stable in this setting, although the results are limited to this experiment.
All four solvers and benchmark runners are open source under the MIT license. We will release the code for the new experiments, preprocessing steps, random seeds, and complete results.
As the rebuttal period ends tomorrow, please let us know if you have any remaining concerns. We would be grateful if you could revisit your assessment in light of our rebuttal and additional experiments. Thank you again for your review and consideration!
rebuttal to Reviewer 4ZQu
Thank you for the careful review.
1. Weakness 1 — Motivation for Revisiting HTNL Optimization
HTNL's current fit determines which candidates are considered in the next round (Secs. 1–2, Fig. 1). Thus, optimization affects not only the current solution but also future feature selection. Manuscript Table 5 shows the effect of improving the proximal solver under fixed search settings, while Fig. 2d shows the separate benefit of expanding the search. Both directly affect feature recovery.
2. Weakness 2 — Diversity of the 11 Benchmarks
Manuscript Table 2 includes 10 LatAm civil-unrest datasets and one flu dataset. Default WGL selects no features on all 10 LatAm tasks but retains features on flu (App. H.6), showing different behavior across tasks. We preserve all 11 original splits and provide a separate six-dataset evaluation in manuscript Table 9. We limit our empirical conclusions to these evaluated tasks.
3. Question 1 — Why Does Optimization Matter in HTNL?
Original and VQF select identical supports (manuscript Table 3), consistent with their equivalent updates. More importantly, manuscript Table 5 and Tables R3a–R3b compare ADMM solvers under matched settings, showing that a more accurate proximal solution improves feature recovery without changing the search procedure.
4. Question 2 — What Changed After the Study?
We establish when MM and VQF are equivalent (Prop. 4.1), improve ADMM's proximal solver using bisection (manuscript Table 5, Table R3a), and adjust WGL regularization to avoid empty supports (manuscript Table 4). Fig. 2d further shows the benefit of a larger search budget. These results separate the effects of solver accuracy, regularization, and search settings on HTNL.
5. Question 3 — Why These Baselines and Benchmarks?
Original is the published HTNL optimizer, while ADMM-FISTA allows us to compare proximal solvers under the same settings. The synthetic datasets provide known true supports for evaluating feature recovery. Tables R1, R4, and R5a add tuned predictive baselines evaluated on the same 11 test splits as manuscript Table 2. Tuned L2 LogReg achieves the highest mean ROC-AUC, while HTNL selects explicit conjunctions. Original and VQF achieve identical ROC-AUC on every split.
Table R6. ROC-AUC by dataset for Tables R1, R4, and R5a
6. Question 4 — What Transfers from the Proximal and Admission Analyses?
Prop. 4.3 bounds the error in the proximal solution, while Lemma 4.4 explains how coupled penalties affect zero blocks. These results can guide solvers with similar group penalties. Prop. 4.5 and Cor. 4.6 show why longer conjunctions are less likely to be selected under the stated assumptions. This also highlights a broader limitation of iterative feature selection: improving the solver alone cannot recover features that the search never considers.
7. Question 5 — Why This Objective, and How Sensitive Is It to P?
Products of primitive signals represent joint precursors. The weighted loss addresses class imbalance in rare-event forecasting, while the two-level penalty selects sparse conjunctions across geographic groups (Sec. 2, App. C). Manuscript Table 2 evaluates prediction on held-out data.
To test sensitivity to the number of primitives, we added 10 independent Bernoulli(0.18) noise features to Dataset 2, keeping the labels, true support, solver settings, and search settings unchanged. Tables R2a and R2b report results from 20 paired seeds at τ=0.40. The rerun of Original at P=10 gives an F1 of 0.594, compared with 0.590 in manuscript Table 1.
Table R2a. Sensitivity to Nuisance Primitives on Dataset 2