Adaptive Aggregation for Partitioned Tabular In-Context Learning

Tabular foundation models perform in-context prediction by conditioning directly on labeled examples, but scaling them to large contexts is constrained by computational and memory requirements. This work studies whether full-context predictions of frozen tabular foundation models can be approximated from multiple shards, where the context is either partitioned to satisfy capacity constraints or naturally distributed across sources. We develop a theoretical framework that characterizes when such reconstruction is possible and how reconstruction error depends on partition structure and aggregation weights, motivating a simple training-free, query-adaptive pooling strategy. Experiments on real-world tabular classification and regression tasks examine when shard-level aggregation provides a useful approximation to full-context prediction and how its behavior varies across partitioning regimes. Together, these results provide a general perspective on reconstructing full-context predictions from partitioned contexts in tabular in-context learning.