acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Optimization Methods for Hierarchical Feature Selection

Abstract

Rare-event forecasting often depends on interpretable conjunctions: a single keyword, location, or temporal cue may be weak, while their joint occurrence can identify a meaningful precursor. Hierarchical Tensor Network Lasso (HTNL) selects Numerical Conjunctive Features (NCFs) through a structured sparsity objective. However, the resulting optimization problem is challenging, making optimization design critical to its practical performance. We give a unified diagnosis of four HTNL optimization families: original majorization-minimization (MM), variational quadratic form (VQF), alternating direction method of multipliers (ADMM), and weighted group lasso (WGL). Our analysis distinguishes three key factors that jointly determine HTNL optimization behavior. First, VQF and MM are algebraically equivalent under matched variational updates and optimize a squared-penalty formulation. Second, ADMM optimizes the unsquared objective through consensus splitting; we replace its truncated FISTA proximal solver with a scalar bisection update (ADMM-Bisect) for more accurate treatment of near-threshold blocks. Third, WGL targets the same unsquared objective, while all methods face rapidly decaying active-set admission scores. Across two synthetic datasets and 11 civil-unrest and flu-surveillance tasks, VQF achieves the highest mean AUC with the most compact non-collapsing supports. ADMM-Bisect reduces measured runtime, while calibrated WGL avoids the empty-support failures of its original scale. Synthetic results further show how proximal accuracy, pruning, and search budget affect candidate recovery, especially for longer conjunctions. Code will be made publicly available upon publication.

Primary area:
optimization
Keywords:
structured sparsity, hierarchical feature selection, numerical conjunctive features, proximal optimization, active-set methods, ADMM, group lasso

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4ZQuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper systematically investigates four optimization methods for Hierarchical-Task Numerical Conjunctive Feature Learning (HTNL), analyzing their theoretical properties, feature selection behavior, and computational efficiency, while proposing an improved ADMM solver and identifying limitations of active-set selection for rare-event forecasting.

Critical strengths

With regard to structured sparse feature selection, distinguishing between the effectiveness of the optimization algorithm itself and that resulting from the design of the objective function is a meaningful area of research.

Critical weaknesses

1.The paper has insufficient motivation, since it doesn't justify why revisiting the HTNL optimization is necessary. 2. The 11 real-world datasets share the same characteristic, limiting its ability to demonstrate advantages over existing methods in general.

Questions for authors
  1. Why is it important to study optimization in HTNL? From the theoretical analysis and experiment, there is lack of evidence to prove the necessity.
  2. Your title is “Rethinking …”, what is the major difference before and after your findings about the optimization in HTNL?
  3. Although there are many datasets tested in experiment, why do you not compare with other baselines or benchmarks?
  4. For contributions 2 and 3, what is the meaning or can other people learn from the exact ADMM proximal update and structure active-set bound?
  5. In 2.1, why the objective function written in this form? Why can it forecast the rare event? Are the results sensitive to the primitive features P?
7 comments
AuthorsOpenReview ↗

rebuttal to Reviewer 4ZQu

Thank you for the careful review.

1. Weakness 1 — Motivation for Revisiting HTNL Optimization

HTNL's current fit determines which candidates are considered in the next round (Secs. 1–2, Fig. 1). Thus, optimization affects not only the current solution but also future feature selection. Manuscript Table 5 shows the effect of improving the proximal solver under fixed search settings, while Fig. 2d shows the separate benefit of expanding the search. Both directly affect feature recovery.

2. Weakness 2 — Diversity of the 11 Benchmarks

Manuscript Table 2 includes 10 LatAm civil-unrest datasets and one flu dataset. Default WGL selects no features on all 10 LatAm tasks but retains features on flu (App. H.6), showing different behavior across tasks. We preserve all 11 original splits and provide a separate six-dataset evaluation in manuscript Table 9. We limit our empirical conclusions to these evaluated tasks.

3. Question 1 — Why Does Optimization Matter in HTNL?

Original and VQF select identical supports (manuscript Table 3), consistent with their equivalent updates. More importantly, manuscript Table 5 and Tables R3a–R3b compare ADMM solvers under matched settings, showing that a more accurate proximal solution improves feature recovery without changing the search procedure.

4. Question 2 — What Changed After the Study?

We establish when MM and VQF are equivalent (Prop. 4.1), improve ADMM's proximal solver using bisection (manuscript Table 5, Table R3a), and adjust WGL regularization to avoid empty supports (manuscript Table 4). Fig. 2d further shows the benefit of a larger search budget. These results separate the effects of solver accuracy, regularization, and search settings on HTNL.

5. Question 3 — Why These Baselines and Benchmarks?

Original is the published HTNL optimizer, while ADMM-FISTA allows us to compare proximal solvers under the same settings. The synthetic datasets provide known true supports for evaluating feature recovery. Tables R1, R4, and R5a add tuned predictive baselines evaluated on the same 11 test splits as manuscript Table 2. Tuned L2 LogReg achieves the highest mean ROC-AUC, while HTNL selects explicit conjunctions. Original and VQF achieve identical ROC-AUC on every split.

Table R6. ROC-AUC by dataset for Tables R1, R4, and R5a

DatasetOriginalFixed L2Tuned L2Tuned L1OverlapBoosting
Argentina0.6040.4530.6180.5480.6050.433
Brazil0.5980.5310.5880.5660.5880.497
Chile0.5270.4590.5460.5420.5270.497
Colombia0.5660.5230.5450.5540.5340.501
Ecuador0.6040.5290.6470.5620.5990.482
El Salvador0.5720.4770.5960.5580.5440.492
Flu0.6950.6490.7210.6860.7220.697
Mexico0.6290.5900.6320.6240.6080.597
Paraguay0.5640.5180.5600.5350.5330.508
Uruguay0.6010.4810.6140.5990.5910.476
Venezuela0.5240.4630.5310.5140.5300.476

6. Question 4 — What Transfers from the Proximal and Admission Analyses?

Prop. 4.3 bounds the error in the proximal solution, while Lemma 4.4 explains how coupled penalties affect zero blocks. These results can guide solvers with similar group penalties. Prop. 4.5 and Cor. 4.6 show why longer conjunctions are less likely to be selected under the stated assumptions. This also highlights a broader limitation of iterative feature selection: improving the solver alone cannot recover features that the search never considers.

7. Question 5 — Why This Objective, and How Sensitive Is It to PP?

Products of primitive signals represent joint precursors. The weighted loss addresses class imbalance in rare-event forecasting, while the two-level penalty selects sparse conjunctions across geographic groups (Sec. 2, App. C). Manuscript Table 2 evaluates prediction on held-out data.

To test sensitivity to the number of primitives, we added 10 independent Bernoulli(0.18) noise features to Dataset 2, keeping the labels, true support, solver settings, and search settings unchanged. Tables R2a and R2b report results from 20 paired seeds at τ=0.40\tau=0.40. The rerun of Original at P=10P=10 gives an F1 of 0.594, compared with 0.590 in manuscript Table 1.

Table R2a. Sensitivity to Nuisance Primitives on Dataset 2

SolverPrimitive countMean F1Mean precisionMean recall
Original100.5940.7250.540
Original200.5800.7500.523
ADMM-Bisect100.6870.6790.806
ADMM-Bisect200.7300.7410.802

Table R2b. Paired Changes When the Primitive Count Doubles

SolverMetricDirectionMean change magnitudeImproved seedsTied seedsWorse seeds
OriginalF1Decrease0.0142153
OriginalRecallDecrease0.0170191
ADMM-BisectF1Increase0.043884
ADMM-BisectRecallDecrease0.0041181

The results show that adding noise features causes only small changes in mean recall. Original's mean F1 decreases slightly, while ADMM-Bisect maintains its performance and achieves a higher mean F1. This suggests that both methods remain relatively stable in this setting, although the results are limited to this experiment.

All four solvers and benchmark runners are open source under the MIT license. We will release the code for the new experiments, preprocessing steps, random seeds, and complete results.

As the rebuttal period ends tomorrow, please let us know if you have any remaining concerns. We would be grateful if you could revisit your assessment in light of our rebuttal and additional experiments. Thank you again for your review and consideration!

Reviewer_6WizOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits optimization methods for Hierarchical-Task Numerical Conjunctive Feature Learning (HTNL) in rare-event forecasting. The authors compare four optimization approaches, establish the equivalence of MM and VQF, and propose ADMM-Bisect to improve proximal accuracy and efficiency. They also analyze the exponential decay of KKT admission scores with conjunction length, suggesting limitations of active-set feature selection. Experiments on synthetic and real-world datasets evaluate feature recovery, predictive performance, and computational efficiency.

Critical strengths
  1. The authors study several inner optimization methods for HTNL, provide the equivalence between MM and VQF, and propose ADMM-Bisect to improve proximal accuracy and computational efficiency.

  2. The paper also analyzes KKT-based active-set expansion and provides an exponentially decreasing upper bound on expected admission scores as conjunction length increases. This suggests a potential limitation of the outer feature selection process in selecting longer conjunctive features.

Critical weaknesses
  1. The proof of Corollary 3.8 appears to contain a theoretical gap. Eq. (3.20) defines the accessible true support using expected KKT admission scores, while the active-set expansion rule in Eq. (2.10) depends on realized scores. A feature whose expected score is below the threshold may still exceed it with positive probability. It is therefore unclear how Appendix C.6 establishes the expected recall bound in Eq. (3.21).

  2. The contributions mainly concern the analysis and refinement of existing HTNL optimization methods, so the technical novelty appears somewhat limited. In particular, the equivalence between MM and VQF is established under matched auxiliary-variable updates, ADMM-Bisect relies on standard bisection techniques, and the exponential KKT score bound is derived under assumptions on conjunctive features.

  3. Eq. (2.7) defines the HTNL objective using the unsquared regularizer, and the ADMM proximal update in Eq. (3.5) also uses the unsquared penalty. However, Eqs. (3.6)–(3.8) use the squared regularizer in the ADMM error analysis. Which formulation does ADMM-Bisect actually optimize, and how do Proposition 3.4 and Lemma 3.5 apply to the proximal update in Eq. (3.5)?

Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

rebuttal to Reviewer 6Wiz

Response to Reviewer 6Wiz

Thank you for the constructive review. We address the three concerns below.

1. Weakness 1 — Expected Scores and Realized Admission

We agree that a low expected KKT score does not imply zero admission probability. We corrected this gap in Corollary 4.6 and Appendix F.7.

Under the uniform gradient bound and independent Bernoulli assumptions, the admission score satisfies score(u)≤M2−∣u∣∑s,tϕu(Xs,t)score(u) \leq M 2^{-|u|} \sum_{s,t} \phi_u(X_{s,t}), where E[ϕu]≤pmax⁡∣u∣E[\phi_u] \leq p_{\max}^{|u|}. Applying Markov's inequality gives:

P(u∈S^)≤min⁡(1,MSTτKKT(pmax⁡2)∣u∣)P(u \in \hat{S}) \leq \min(1, \frac{MST}{\tau_{KKT}}(\frac{p_{\max}}{2})^{|u|}).

Since the score bound holds uniformly across restricted iterations, this probability bound applies to admission across search rounds. The revised expected recall bound sums these probabilities over the true support, accounting for initially selected and depth-excluded features. This resolves the gap between expected scores and realized selection.

2. Weakness 2 — Technical Contribution

Our contributions go beyond applying standard optimizers. We establish theoretical relationships, improve HTNL optimization, and characterize its structural limitations.

(1) MM–VQF equivalence (Prop. 4.1). Both methods yield identical weighted-ridge updates under matched auxiliary variables, establishing WMM(t)=WVQF(t)W_{MM}^{(t)} = W_{VQF}^{(t)} for all t≥0t \geq 0. This explains their identical optimization trajectories and selected supports.

(2) Improved proximal optimization (Lemma 3.2). For the single-group scalar formulation, the coupled proximal problem reduces to monotone equations ru+ηrup−1=∣Qv,u∣r_u + \eta r_u^{p-1} = |Q_{v,u}|, where η=λdv/ρ(∑urup)(p−1)/p\eta = \frac{\lambda d_v/\rho}{(\sum_u r_u^p)^{(p-1)/p}}. This enables accurate bisection updates. Manuscript Table 5 and rebuttal Tables R3a–R3b show improved F1 and precision under matched settings, even with four times the FISTA budget.

(3) Active-set recovery bounds (Prop. 4.5, Cor. 4.6). We establish E[score(u)]≤MST(pmax⁡2)∣u∣E[score(u)] \leq MST(\frac{p_{\max}}{2})^{|u|}, characterizing the exponential decay of admission scores and the resulting limitations on recovering longer conjunctions.

Together, these results explain the relationships among HTNL optimizers, improve feature recovery, and establish theoretical limitations of candidate discovery.

3. Weakness 3 — ADMM Objective Consistency

We clarify that ADMM-Bisect optimizes the unsquared objective L(W)+λΩ(W)L(W) + \lambda \Omega(W), rather than the globally squared penalty. We corrected the inconsistency in the previous analysis.

(1) Consensus decomposition. Since Ω(Z)=∑vdvΩv(Zv)\Omega(Z) = \sum_v d_v \Omega_v(Z_v), the ADMM update separates by ancestor: Zv∗=arg⁡min⁡Zv[λdvΩv(Zv)+ρ2∥Zv−Qv∥22]Z_v^* = \arg\min_{Z_v} [\lambda d_v \Omega_v(Z_v) + \frac{\rho}{2}\|Z_v-Q_v\|_2^2]. This separability does not hold for the globally squared penalty.

(2) Proximal error bound (Prop. 4.3). The quadratic term makes the local objective ρ\rho-strongly convex. Therefore, an objective gap εK\varepsilon_K implies ∥Zv(K)−Zv∗∥2≤2εK/ρ\|Z_v^{(K)}-Z_v^*\|_2 \leq \sqrt{2\varepsilon_K/\rho}.

(3) Zero-block recovery (Lemma 4.4). The KKT condition is 0∈ρ(Z∗−Q)+λdv∂Ωv(Z∗)0 \in \rho(Z^* - Q) + \lambda d_v \partial \Omega_v(Z^*). It becomes ρQ∈λdv∂Ωv(0)\rho Q \in \lambda d_v \partial \Omega_v(0)

for Z∗=0Z^*=0.

These derivations consistently use the unsquared objective. Appendix F.4 separately analyzes the globally squared penalty.

All four solvers and benchmark runners are open source under MIT. We will release the complete experimental configurations, seeds, and results.

As the rebuttal period ends tomorrow, please let us know if you have any remaining concerns. We would be grateful if you could revisit your assessment in light of our rebuttal and additional experiments. Thank you again for your review and consideration!

Thank you for your clarifications. The rebuttal addresses my concerns regarding Weaknesses 1 and 3 and partially addresses Weakness 2. I will maintain my original rating.

Thank you for your thoughtful feedback and for reconsidering our rebuttal. We appreciate your acknowledgment that our clarifications have addressed your concerns.

Reviewer_F8RROpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses optimization within the HTNL model, which aims to select (sparse) features from a possibly exponentially-sized set of conjunctive features, like keywords in social media text. The authors compare the 4 optimization families: the original MM, as well as VQF, ADMM, and WGL. The three results are: an equivalence between MM and VQF; a fix ADMM-bisect for prior ADMM work in HTNL; and a recall ceiling for the true NCF support. They provide experiments on 2 synthetic and 11 binary classification sets for civil unrest and flu surveillance.

Critical strengths
  • ADMM-bisect improves upon an existing method, and this paper identifies a specific issue and a faster, more reliable optimization.
  • Results are transparently and thoroughly reported throughout the paper, which includes negative results for WGL. Furthermore, the experiments are done thoroughly with 20 paired random seeds on synthetic data.
Critical weaknesses
  • While rare-event forecasting is the main motivation for the NCF problem, the discussion of prior work, including adjacent methods, appears only in the appendix related-work section. The text gives the reader little sense of where HTNL sits relative to other approaches, or why it is the right model to invest optimization effort in.
  • Class-weighted logistic regression is the only baseline, and its scores are sometimes worse than random (see El Salvador, Uruguay, and a few others in the LogReg column of Table 2), aka < 0.5 AUC on a binary problem. As a result, improvements over it say little about the value of the proposed methods. A properly tuned baseline, along with comparisons to the forecasting and structured-sparsity methods cited in the related work, would make the results more convincing.
Questions for authors
  • Why does logreg score < 0.5 AUC on 5 of the 11 datasets?
  • Where is ADMM-Fista at rho = 1? Table 1 seems to compare ADMM-bisect at rho=1 against ADMM-FISTA at rho=10.
6 comments
AuthorsOpenReview ↗

rebuttal to Reviewer F8RR

Thank you for the careful review.

1. Weakness 1 — Why study HTNL for rare-event forecasting?

HTNL represents joint precursors as explicit numerical conjunctions whose effects can vary by location (Secs. 1–2). Its optimization directly affects which conjunctions can be found, as well as feature selection and prediction. We will move the comparison with related methods from Appendix A to the introduction.

2. Weakness 2 — Strength of the Predictive Baselines

We added several baselines using the same 11 test splits as Table 2, with all hyperparameters tuned only on training data. Table R1 reports tuned additive LogReg, which achieves the highest mean ROC-AUC among the added baselines. We also compare against latent overlapping group lasso [2], which groups each primitive feature with its related pairs. As shown in Table R4, Original and VQF achieve slightly higher ROC-AUC and average precision while selecting 76.5% fewer terms (4 vs. 17).

Tables R5a and R5b compare our methods with tuned histogram gradient boosting. Original and VQF achieve higher mean ROC-AUC (0.589 vs. 0.514) and average precision (0.0340 vs. 0.0282). Original also outperforms gradient boosting on 10 of the 11 datasets for both metrics. Table R6 in our response to 4ZQu reports the ROC-AUC for each dataset.

Table R4. Structured-sparsity comparison on the 11 original benchmarks

MethodMean ROC-AUCMean average precisionMedian terms at τ=0.40\tau=0.40
Latent overlapping group lasso0.5800.032417
Original0.5890.03404
VQF0.5890.03404

Table R5a. Nonlinear baseline on the 11 original benchmarks

MethodMean ROC-AUCMean average precision
Tuned histogram gradient boosting0.5140.0282
Original0.5890.0340
VQF0.5890.0340

Table R5b. Paired results for Original against tuned gradient boosting

MetricMean gainOriginal winsTiesBoosting wins
ROC-AUC0.0751001
Average precision0.00581001

3. Question 1 — Why is LogReg below 0.5 AUC on five datasets?

Manuscript Table 2 reports test ROC-AUC from weighted L2 LogReg with a fixed C=1C=1. To examine these results, we tuned seven CC values from 10−410^{-4} to 10210^2 using five-fold cross-validation on the training data, without resampling. The strongest regularization was selected on nine datasets. After tuning, LogReg achieves ROC-AUC above 0.5 on all 11 datasets, with its mean ROC-AUC increasing from 0.516 to 0.600 (Table R1). This shows that the original below-chance results were sensitive to the fixed regularization setting.

Table R1. Logistic baselines on the 11 original benchmarks

MethodFeaturesMean ROC-AUCDatasets below 0.5
Fixed L2 LogRegPrimitives0.5165
Tuned L2 LogRegPrimitives0.6000
Tuned L1 LogRegPrimitives and pairs0.5720
OriginalHTNL conjunctions0.5890
VQFHTNL conjunctions0.5890

4. Question 2 — Where is ADMM-FISTA at ρ=1\rho=1?

Manuscript Table 1 reports ADMM-FISTA at ρ=10\rho=10. However, Sec. 5.2, Table 5, and Fig. 3a already include the comparison at ρ=1\rho=1, as summarized in Table S1. We will also add this comparison to manuscript Table 1.

Table S1. Existing comparison on Dataset 1 with 20 paired seeds

Proximal solverρ\rhoSupport threshold τ\tauMean F1Paired F1 gain over FISTA
ADMM-FISTA10.200.399—
ADMM-Bisect10.200.4880.088

We also repeated the comparison on Dataset 2 using 20 paired seeds. Both methods use the same ρ=1\rho=1, τ=0.20\tau=0.20, search settings, and ADMM stopping rules. We tested FISTA with both 8 and 32 inner updates to check whether more updates improve its results. As shown in Tables R3a and R3b, increasing the FISTA budget has little effect. ADMM-Bisect achieves higher F1 and precision while maintaining the same mean recall.

Table R3a. Matched proximal comparison on Dataset 2

Proximal solverInner update budgetMean F1Mean precisionMean recallMean ADMM iterations
ADMM-FISTA80.3790.2530.8638.2
ADMM-FISTA320.3810.2540.8638.9
ADMM-BisectNumerical tolerance0.4680.3390.8639.3

Table R3b. Paired F1 gains for ADMM-Bisect on Dataset 2

FISTA inner updatesMean F1 gainImproved seedsTied seedsWorse seeds
80.0881730
320.0871730

All four solvers and benchmark runners are available as open-source code under the MIT license. We will also release the code for the new experiments, preprocessing steps, data splits, random seeds, and full results.

Thank you for the thorough response and the additional experiments. The ADMM-Bisect comparison at matched ρ (Tables S1, R3a–b) addresses my question, and I appreciate the clarification on the below-chance LogReg results. For this, I will bump up my score slightly.

My main concern, however, is about significance. The paper’s contributions are an equivalence between two existing HTNL solvers (MM and VQF) and a targeted fix to a third (ADMM). These are useful within HTNL, but they matter only if HTNL itself is a compelling model, and the paper does not make that case. The introduction moves into technical detail without sufficiently motivating the NCF problem or explaining why HTNL is the right approach. The promise to add discussion of related works into the introduction would help, as would more thorough presentation of forecasting baselines, but I think a more comprehensive reframing is needed.

Adding to that, in table R1, the new tuned L2 LogReg now beats both Original and VQF, and no CI's/variances are reported. The gaps are also quite small, so there's not a clear advantage of using HTNL.

AuthorsOpenReview ↗

## Response to Reviewer F8RR

1. Why the NCF problem and HTNL warrant this study

NCF selection identifies interpretable combinations of primitive signals for rare-event forecasting. HTNL addresses this challenging problem through hierarchical regularization and active-set search over exponentially many candidate conjunctions, while capturing geographic variations in their predictive effects.

Optimization is central to HTNL because it affects both feature selection and subsequent candidate discovery. Errors in the inner optimization can alter the selected features and influence which candidates enter the active set. Conversely, even accurate optimization cannot recover features excluded by the search mechanism. This motivates our systematic analysis of optimization methods, their improvements, and their theoretical limitations.

2. Tuned LogReg and uncertainty in Table R1

Thank you for highlighting the importance of stronger baselines and statistical uncertainty. We conducted additional hyperparameter tuning for Original and VQF using training data only, while retaining the previously reported results for transparency. The tuned configurations achieve a mean ROC-AUC of 0.6040, slightly exceeding tuned L2 LogReg (0.6000). Original and VQF remain identical under matched optimization settings.

Tables R7a and R7b report the updated results and 95% confidence intervals across all 11 datasets.

Table R7a. ROC-AUC summaries for the original 11 datasets

MethodMeanSD across datasets95% interval lower95% interval upper
Fixed L2 LogReg0.51580.06020.47530.5562
Tuned L2 LogReg0.60000.05550.56270.6373
Tuned L1 LogReg with pairwise features0.57160.04840.53910.6041
Latent overlapping group lasso0.58010.05750.54150.6187
Tuned histogram gradient boosting0.51400.07240.46540.5626
Original (previous setting)0.58950.04790.55730.6217
VQF (previous setting)0.58950.04790.55730.6217
Tuned Original0.60400.04800.57180.6362
Tuned VQF0.60400.04800.57180.6362

Table R7b. Paired ROC-AUC differences, Original minus comparator

ComparatorMean difference95% interval lower95% interval upper
Tuned L2 LogReg+0.0040-0.00820.0162
Tuned L1 LogReg with pairwise features+0.03240.01880.0460
Latent overlapping group lasso+0.02390.01180.0360
Tuned histogram gradient boosting+0.09000.05580.1241

SD denotes the sample standard deviation across 11 datasets. The 95% confidence intervals use a t distribution with 10 degrees of freedom, and paired differences are computed within each dataset using identical train/test splits.

Under the updated results, tuned HTNL achieves the highest mean ROC-AUC among the evaluated methods. Although its improvement over tuned L2 LogReg is not statistically significant, the paired intervals indicate positive improvements over tuned L1 LogReg, overlapping group lasso, and histogram gradient boosting. These results support HTNL's competitive predictive performance while preserving its ability to select explicit, interpretable precursor features.

3. Reframing Our Contributions: Rethinking Optimization within HTNL

Our primary contribution is a systematic theoretical and empirical study of optimization within HTNL, rather than proposing a new forecasting model. Specifically, we:

  1. Analyze optimization methods: Compare MM, VQF, ADMM, and WGL, establishing their theoretical relationships and distinct optimization behaviors.
  2. Improve optimization: Introduce ADMM-Bisect to improve feature recovery and proximal accuracy, while reducing runtime through KKT pruning.
  3. Establish theoretical bounds: Characterize the exponential decay of KKT admission scores and its implications for active-set feature recovery.

Table R8. Key Findings of the HTNL Optimization Study

QuestionEvidenceKey finding
How are MM and VQF related?Prop. 4.1; Table 3Equivalent updates and selected supports.
Can optimization improve feature recovery?Table 5; Tables R3a–R3bADMM-Bisect improves F1 and precision.
What limits active-set recovery?Prop. 4.5; Cor. 4.6; Fig. 2dKKT admission and search-budget limitations.
Why does WGL collapse?Table 4Regularization calibration restores nonempty supports.

Together, these findings clarify the relationships among HTNL optimizers, improve optimization effectiveness and efficiency, and establish theoretical limitations on feature recovery.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.