Native Local Linear Explanations for Tabular In-Context Learning

Tabular in-context models adapt to unseen tasks without task-specific retraining, yet their cross-sample attention entangles feature representations into opaque predictions. We introduce TabLLE (Tabular Learning with Local Linear Explanations), an intrinsically interpretable in-context model that recasts support-conditioned prediction as dynamic, query-specific affine hyperplanes. Conditioned on query samples and column-level support sets via a Feature-Context Adapter (FCA), an adaptive hypernetwork generates instance-specific weights and biases in a single forward pass. For the parameters generated in the same forward pass, each class logit decomposes into an exact sum of coordinate-indexed local-linear terms and a sample-dependent additive offset, yielding native instance-level attributions without perturbation-based post-hoc evaluation. To prevent the hypernetwork from bypassing feature pathways via bias shortcuts, TabLLE is trained with a three-term objective that explicitly enforces standalone feature discriminability. Across 20 OpenML benchmarks, TabLLE is competitive with tree ensembles and leading in-context baselines, with 1.5–5.15$\times$ faster inference than TabICLv2 at matched retrieval sizes, while attaining the highest AUC and accuracy among intrinsically interpretable baselines. Finally, we formalize parameter drift under feature interventions and show that a behavioral head reliably accounts for cross-pass masking effects.