TabCAM: Online Calibrated Regularization For Tabular Deep Learning

Modern tabular deep learning models are mostly built on overparameterized architectures, which require carefully calibrated regularization to ensure out-of-sample predictive accuracy and reliability. Standard regularization approaches rely on cross-validation via grid search to tune a small number of global penalty coefficients, which inevitably over-regularizes some parameters and under-regularizes others. Ideally, each model parameter should be assigned its own regularization strength, enabling the training to selectively suppress dataset-specific noise while preserving signals that capture true population-level structure. However, since the cost of grid search grows exponentially in the number of hyperparameters, per-parameter regularization is computationally infeasible under standard approaches. To address this bottleneck, we propose TabCAM (Tabular Calibrated Auto-Modeling), an add-on module for differentiable backbones that jointly estimates the model parameters and per-parameter regularization strengths inside a single training loop. Through a novel online construction of calibrated future targets, TabCAM adds minimal computational overhead beyond vanilla model training, making it particularly promising for future adoption with even more complex architectures where principled per-parameter regularization becomes increasingly valuable. Moreover, because TabCAM estimates the population conditional distribution during training, covariate-conditional uncertainty quantification is built in. On the 51-dataset TabArena benchmark, TabCAM ranks third among 16 non-foundation default methods and improves substantially under modest tuning while matching the runtime of vanilla model training. In simulations, TabCAM converges to the population minimizer and produces prediction intervals with near-nominal coverage that is more consistent and narrower than split-conformal baselines.