- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper derives an explicit penalty threshold below which ridge-regularized quantile regression collapses to the same minimum-norm interpolant at every quantile level, studies that threshold under random designs, and applies the resulting certificate to neural-network output heads. The formula itself is clean and useful. My reservations are that the step beyond prior work is modest, the probabilistic model is a long way from the network experiments, and the theory does not, in the end, explain what the experiments show.
Critical strengths
The main result is clean and exact: a closed-form penalty boundary, computable with a single linear solve, below which the ridge-penalized quantile fit collapses to the minimum-norm interpolant (Theorems 1–2), and the certificate extends to network output heads at full-rank local minima (Corollary 2). The fixed-design argument is sound.
Critical weaknesses
The writing seems heavily rely on AI and the Authors do not carefully polish it. So I understand nothing from the abstract. There are also illusions in the reference, for example,riedlander & Tseng, Exact Regularization of Convex Programs, SIAM J. Optim. 18(4):1326–1350 is a paper from 2007 not 2008 of the paper. In summary, the paper introduces too many newly coined labels, giving the prose an artificial, AI-like style—for example, “extinction” and “loss-blind” (pp. 2–3), “median-type predictor” (p. 6), “common-penalty level certificates” (p. 66), and “face atlases” and “stable tiles” (pp. 70–71), requiring the AI-assisted reading for readers which might be suitable in the future (2-3 years) but not now.
Questions for authors
-
The theory and the network experiments are only loosely connected. The main probabilistic analysis is for independent Gaussian features with a linear response model (pp. 5–6), and the risk guarantees assume the features are fixed before sampling (pp. 6–7). The network experiments, by contrast, use correlated features that are learned from the training responses. Corollary 2 (p. 5) does give a certificate at a local minimum with a full-row-rank feature matrix, where the output heads solve conditional convex problems, so it applies to whatever features the network has learned. But it says nothing about feature learning or training dynamics, and the networks in practice never reach an exact minimizer (p. 8). The authors do acknowledge these gaps and provide supplementary extensions, but the connection stays conditional. I would like to see evidence that the random-design analysis actually explains the observed network behavior, rather than merely that the deterministic certificate applies after training.
-
Novelty is thin. The paper is upfront about the precedents (pp. 1–2): positive exact-regularization intervals [1], kernel quantile-regression solution paths [2,3], and minimum-norm interpolation with coverage near one half [4]. The explicit threshold and the characterization of cross-level coincidence refine this picture, but the central threshold is a direct consequence of the KKT conditions (p. 3; proof on p. 13), and I am not convinced these refinements change our understanding of quantile regression in a substantial way.
-
The boundary does not translate into calibration guidance. Coverage can stay near one half well above the boundary, and removing exact extinction does not remove severe miscalibration (p. 8; Tables S.21–S.22, p. 97). Consolidating the extinct candidates preserves the full-grid validation selection, and it avoids 55% of the solver fits — but only 8.4% of the solver time, since the time is spent at penalties above the boundary (p. 8). The supplementary reinvestment experiments tackle this, but the equal-elapsed-time risk improvements come to about 0.41% and 0.19% (Table S.2, p. 80). As it stands, the practical value of locating the exact boundary is unclear.
-
The abstract needs a rewrite. I found the abstract hard to follow (p. 1): it races through several theoretical regimes, a conditional network result, and a string of coverage numbers, with little intuition for why the question matters. The main insight is buried. The abstract should state the problem and the contribution plainly before getting into qualifications and numerics.
The fixed-design argument itself looks sound. My concerns are the limited advance over prior work, the idealized probabilistic model, and the absence of a convincing explanation of the network experiments.
References
[1] Michael P. Friedlander and Paul Tseng. Exact Regularization of Convex Programs. SIAM Journal on Optimization, 18(4):1326–1350, 2007.
[2] Youjuan Li, Yufeng Liu, and Ji Zhu. Quantile Regression in Reproducing Kernel Hilbert Spaces. Journal of the American Statistical Association, 102(477):255–268, 2007.
[3] Saharon Rosset. Bi-Level Path Following for Cross Validated Solution of Kernel Quantile Regression. Journal of Machine Learning Research, 10(86):2473–2505, 2009.
[4] Yu Bai, Song Mei, Huan Wang, and Caiming Xiong. Understanding the Under-Coverage Bias in Uncertainty Estimation. Advances in Neural Information Processing Systems, 34:18307–18319, 2021.
In my previous comment, I say ’ There are also illusions in the reference, for example,riedlander & Tseng, Exact Regularization of Convex Programs, SIAM J. Optim. 18(4):1326–1350 is a paper from 2007 not 2008 of the paper. ‘
After investigation, I think this error (wrong year alone) could be caused by typo or the mistake of citation tools (such as google citation), not `illusions'