- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TICL³ (Tabular In-Context Local Linear Learning), a query-adaptive predictor guided by a frozen TabICLv2 backbone. A shared ElasticNet is fitted to the backbone’s predictions, while a query-specific ElasticNet models the remaining residuals using neighbors retrieved in the backbone’s representation space. Their combined coefficients determine both the deployed linear class scores and the corresponding feature contributions. These contributions exactly decompose the query-specific scores, but not the normalized probabilities or the effects of representation-based retrieval and coefficient fitting.
Critical strengths
- A coherent integration of prediction and feature contributions.
- Broad predictive evaluation and competitive results against interpretable baselines.
- Promising results on selected explanation metrics.
Critical weaknesses
W1.The methodological contribution appears incremental, and the feature-selection results do not establish a consistently strong advantage.
W2.The emphasis on direct predictive performance is not matched by a sufficiently clear end-to-end value proposition. Table 3 shows that the matched TabICLv2 with one estimator achieves 0.8710 accuracy and 0.745 seconds of batch prediction time, whereas standard TICL³ achieves 0.8557 accuracy and 6.849 seconds. The fast and regional variants also remain less accurate and slower than this matched backbone. Thus, for prediction alone, the reported configurations provide neither an accuracy advantage nor a speed advantage.
W3.The real-data interpretability evaluation is too narrow to support broad conclusions. The encouraging Table 5 results are based on only 10 CC-18 datasets and 50 explained queries per dataset, using a fixed split and query sample. Although the authors state that this subset was selected before score-function tuning, the scope remains much narrower than that of the predictive evaluation. It is unclear whether the explanation-quality gains persist on TabArena-Lite, TALENT, or datasets with different dimensionality, categorical-feature composition, and task characteristics.
W4. The regression subsection is weakly connected to the central interpretability contribution.
Questions for authors
Q1. What deployment advantage should readers infer from the prediction and runtime comparisons? Given the matched one-estimator results in Table 3, could the authors clarify the intended use case in which TICL³ is preferable to TabICLv2? In particular, could they provide an end-to-end comparison against TabICLv2 paired with an efficient post-hoc explainer, jointly reporting predictive accuracy, explanation quality, one-off setup cost, and online prediction-plus-explanation latency? This would more directly justify the proposed trade-off than additional prediction-only comparisons.
Q2.Do the Table 5 conclusions generalize beyond the selected CC-18 subset? Could the authors extend the interpretability evaluation to TabArena-Lite and TALENT, or to representative, pre-specified subsets from these suites?
Q3. How does the regression extension strengthen the interpretability claim?
7 comments
[1] Qu et al. TabICLv2, 2026.
[2] Ribeiro et al. LIME, 2016.
[3] Plumb et al. MAPLE, 2018.
[4] Nori et al. InterpretML (EBM). 2019.
[5] Prokhorenkova et al. CatBoost, 2018.
[6] Jethani et al. FastSHAP, 2022.
[7] Efron et al. Least Angle Regression, 2004.
[8] Avsec et al. Enformer, 2021.
[9] Jiménez-Luna et al. Drug discovery with explainable AI, 2020.
[10] Toms et al. Interpretable neural networks for geosciences, 2020.
Thank you for the detailed response and the additional experimental results. The expanded CC-18 evaluation helps address my earlier concerns. I encourage the authors to incorporate these results into the main text, as I consider this part of the empirical evaluation to be the paper’s main contribution.
However, I do not find the response regarding the method’s weaker classification performance relative to the TabICL baseline sufficiently convincing. In particular, it remains unclear what makes the evaluation in Section 4.2 a significant contribution. Demonstrating stronger performance than tree-based models does not, on its own, establish a compelling contribution, especially when the method underperforms the relevant TabICL baseline. This is not to suggest that a contribution must necessarily achieve state-of-the-art performance, but rather that the distinct scientific insights or practical advantages demonstrated by this section need to be articulated more clearly. I would appreciate a more substantive explanation of its significance beyond the comparison with tree-based models.
Overall, while the additional CC-18 results strengthen the empirical evaluation, my concerns regarding the significance of Section 4.2 remain unresolved. I therefore maintain my current score at this stage.
We thank the reviewer for the careful review.
Compared with TabICLv2 [1] alone, TICL$^3$ trades a small accuracy loss for an exact account of each prediction’s score. We therefore compare complete prediction-and-explanation pipelines, including TabICLv2 paired with an explainer.
W1.1 — Methodological contribution
We design a simple predictor that is interpretable by design: a global linear model plus a local linear correction fitted on the query’s neighbors in a foundation model’s representation. The same coefficients predict and explain. Our main finding is that this simple rule retains near-foundation-model accuracy across diverse datasets.
It beats CatBoost [5] on 266/386 evaluations, trails TabICLv2 by 1.1 points, and beats MAPLE [3] in all four suites. Ablations support both design choices: representation neighbors improve accuracy by 2.30 points over input-space neighbors (95% CI [1.08, 3.49]); at K=30, the global fit raises feature-contribution coverage from 46.9% to 100% and contribution correlation after 25% row removal from 0.36 to 0.66 (matched valid queries, nine datasets).
W1.2 — Feature selection
On real data, TICL$^3$’s selected features have the highest average utility (Table 5). With the predictor held fixed, we test how much useful information each explainer’s selected features retain. Gen-SVM measures how well an SVM using only those features reproduces the predictor’s decisions. Gen-kmeans measures how well k-means clusters using those features match the true classes. TICL$^3$ leads both, ahead of all nine post-hoc explainers. On 40 evaluations, it beats TabICLv2 + FastSHAP [6] on deletion and Gen-SVM (CIs exclude zero). In the leave-query-out comparison, LIME’s rules [2] are 19.5 and 11.7 points less accurate than TICL$^3$. LIME recovers more generating features on some synthetic tasks (Table 4). Prediction plus explanation takes 27 ms with TICL$^3$ versus 14.6 s with LIME (Table 5).
W2 + Q1 — Deployment advantage
FastSHAP estimates TabICLv2’s Shapley values; TICL$^3$ reports its own score’s terms. We compare 40 evaluations.
Including fitting, TICL$^3$ is faster for 50 queries; FastSHAP is faster online. For prediction alone, one-estimator TabICLv2 is faster and more accurate (0.745 s, 87.10% vs. 6.849 s, 85.57%; Table 3).
On eight datasets with 5,000 training rows, fitting and explaining 50, 200, and 1,000 predictions takes 3.56, 5.55, and 16.23 s with TICL$^3$ versus 35.78, 35.93, and 36.80 s with TabICLv2 + FastSHAP. At 1,000 queries, accuracy is 87.33% versus 87.96%. TICL$^3$ suits frequent refits where inspecting the score calculation matters.
W3 + Q2 — Explanation quality beyond CC-18
We added ten evaluations each from TabArena-Lite, TALENT, and BeyondArena, with three seeds and 50 explained queries per run. Each pipeline explains its own predictor. Deletion checks how steadily class probability falls as features are removed.
Across 40 evaluations, paired 95% gain CIs are [0.006, 0.113] for deletion and [0.90, 5.08] points for Gen-SVM (34 valid pairs). On the 30 new evaluations, deletion is 0.662 vs. 0.595 (+0.067, CI [0.006, 0.131]); Gen-SVM is 92.25% vs. 89.00% (+3.25 points, CI [1.19, 5.61]; 24 valid pairs). FastSHAP is more stable; the clustering difference is uncertain.
Three new CC-18 splits favor TICL$^3$ on average, but CIs include zero. Deletion differences by feature count or categorical share are also uncertain. We will report these and add a class-count breakdown.
W4 + Q3 — Regression and interpretability
Regression extends the same interpretable construction to continuous outcomes: diabetes progression [7], gene expression [8], and drug properties [9]. Attribution also helps interpret climate predictions [10]. Each feature adds or subtracts an amount in the outcome’s units. After undoing target standardization, the baseline and signed contributions sum to the prediction.
Class-separating neighbors need not capture outcome magnitude; TabICLv2 conditions regression representations on continuous targets. On 119 TALENT evaluations, TICL$^3$ has 0.515 nRMSE versus 0.537 for CatBoost, 0.571 for MAPLE, 0.601 for EBM [4], and 0.482 for matched one-estimator TabICLv2 (Table 6). On ten regression datasets and three seeds, we compare feature contributions before and after changing 25% of training rows: correlation is 0.982 for removal and 0.980 for replacement. In both cases, contributions are measured relative to the original training-set mean.