- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The manuscript proposes a method called TabCAM, which learns a separate regularization strength for each parameter of a neural network for tabular prediction. The method avoids computationally expensive hyperparameter search by learning these regularization strengths online during model training. To achieve this, TabCAM divides the data into two parts at selected stages of training. The first part is used to train the neural network, while the second part is used to correct the model’s predictions. Based on these corrected predictions, TabCAM creates pseudo- outcomes that are intended to represent future data. It then compares the model updates suggested by the training data and the pseudo- outcomes, and uses their difference to adjust the regularization strengths. For regression, the method starts from a Gaussian prediction model; for classification, it uses calibrated class probabilities. Experiments evaluate predictive performance, computational cost, learned regularization patterns, and uncertainty estimates.
Critical strengths
- The online approximation avoids repeated bootstrap refitting and can be applied to several neural tabular architectures with modest overhead.
- There is some evidence of portability and interesting learned structure. More specifically, the method works with MLP, TabM, and FT-Transformer, while simulations show stronger penalties for parameters associated with irrelevant features.
Critical weaknesses
-
Limited technical novelty. The decomposition, per-parameter gradient-matching objective, and alternating optimization closely follow Jiang and Liu as the authors have acknowledged in the manuscript. The major change is replacing bootstrap imputation with online calibrated pseudo-samples, while the calibration ingredients are closely related to Kuleshov et al. and Guo et al. This combination may be new, but it appears technically incremental.
-
Unconvincing empirical improvement. If the empirical performance of the proposed method is strong, then the 1st weakness would become relatively minor. However, TabCAM does not outperform traditional methods, not even to mention the recent tabular foundation-model baselines. Even against matched neural backbones, its improvements are generally around or below 1%, with some confidence intervals including no improvement. These results do not clearly demonstrate that the additional machinery provides a consequential practical advantage.
Questions for authors
- How robust is the Gaussian working model?
- PIT calibration addresses marginal calibration but does not guarantee conditional correctness. Can the authors comment on how this would influence the performance of the method?
- Because pseudo-targets are generated from the current fitted model, can the coupled updates converge to a self-confirming but incorrect solution?
- A gradient evaluated on an independent validation set is already an unbiased estimator of the population gradient under standard i.i.d. assumptions. Having a larger sample size usually leads to a more precise approximation of the population level gradient in the V term. Hence, why should the more indirect calibrated pseudo-sample gradient be used than directly using the calibration-set gradient? What are the advantages?