- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
A coupled residual connection, in which a single shared gates both the linear residual and the nonlinear backbone The empirical contribution is a body of experiments on synthetic and real-world datasets against Lasso/LassoNet/STG/DFS, plus a demonstration that the connection can be bolted onto an existing selector.
Critical strengths
Clearly written. The architecture is stated precisely, the §4.4 three-way distinction between input-gating / partial-linear / single-index models is genuinely clarifying, and both proofs are short and correct under their stated assumptions.
The idea seems working. Results partially support the claims
Critical weaknesses
For the experimental parts, the evaluation cannot measure the quality the paper claims to improve, and omits the baseline that can beat their method without sparsity included. The real data experiments never inspect the selected features, justified by "no ground truth".
There is no full-feature baseline. In my own run, using all features gives a better MSE for unregularized CORAL with selected features, so selection does not improve accuracy here. Therefore, the method's value rests on interpretability, but the paper offers no interpretability analysis and cites the lack of ground truth.
My main concern is the theoretical component. Propositions 1 and 2 are presented as theoretical contributions, but their proofs are elementary, neither uses the coupling that is the paper’s central design choice, and neither explains the empirical results they are linked to. I elaborate below.
My main concern is that the two propositions add little substance to the paper. Their proofs are elementary, they do not depend on the proposed coupling, and they do not support the claims they are cited for. As detailed below, they read as decoration rather than as theoretical support for the method.
-
The proofs are elementary and do not reflect the architecture. Proposition 1 relies only on the fact that is unbounded along a coordinate axis whenever , while is bounded. Proposition 2 follows by setting , so that the Hadamard product reduces to the identity, and applying universal approximation to . Neither argument uses depth, architecture, or the Hadamard coupling. In addition, the experiments use ReLU, which falls outside the bounded-activation assumption of Proposition 1, so the stated theory does not cover the reported results. These results are better presented as remarks than as theoretical contributions.
-
Neither proposition supports the coupling. Replacing by , both proofs go through verbatim. The results therefore hold for any "linear plus bounded nonlinearity" model and provide no justification for , which is the paper's actual design choice. In particular, the claim that "the coupling strategy introduces identifiability" is not supported: the identifiability in Proposition 1 comes entirely from the bounded network being unable to absorb an unbounded linear term, and is equally available to the uncoupled model . The coupling currently has no theoretical support in the manuscript.
-
Proposition 1 cannot explain Fig. 2. Proposition 1 is a statement about exact equality over , and its content lies entirely in the behavior as . It provides no stability on bounded domains. On a compact , under the hypothesis of Proposition 2, for every , every with having nonzero entries, and every , there exists such that . To see this, apply universal approximation to on ; this is the proof of B.2 with left free, and it holds for sigmoid as well. (For analytic activations, exact equality on a set with nonempty interior does extend to , but this offers no robustness to approximation error, which is what matters for estimation.) The only quantitative bounded-domain statement available is the following: if contains for all and , then . This bound is vacuous unless is controlled, e.g. by regularization. It does not require bounded activations; boundedness only makes depend on the output layer alone. The concentration of in Fig. 2 therefore cannot be attributed to Proposition 1. The most plausible explanation is norm control through weight decay or the implicit bias of the optimizer, although other factors such as initialization or early stopping cannot be ruled out without ablation.
Questions for authors
What, specifically, does the Hadamard coupling contribute? As argued above, Propositions 1 and 2 do not use the coupling, and the "with vs. without coupling" experiments only compare against , i.e., they test the presence of the residual term rather than the coupling. The coupling's evident role is to gate features so that removes feature from the whole model, but LassoNet's hierarchy constraint achieves the same gating. Could the authors either provide a theoretical result whose proof genuinely depends on the coupling , for example a stability or estimation guarantee that fails for the uncoupled or LassoNet-style model?