acceptodds
Under review as a conference paper at ICLR 2027

Compile the Table: Query-Calibrated Operator Compression for Tabular In-Context Learning

Abstract

Tabular in-context learning (ICL) has emerged as a training-free and accurate paradigm for tabular prediction, but current approaches to compressing its in-context examples face an accuracy–throughput tradeoff: fixed subsets can sacrifice accuracy, while query-specific retrieval limits cache reuse and batching across queries, reducing throughput. We propose QCOC (Query-Calibrated Operator Compression), which exploits the exchangeability and repeated use of in-context examples by compiling their full KV cache once into compact memory shared across subsequent queries. Instead of retaining raw examples, QCOC clusters their states into joint-KV prototypes, preserves per-cluster multiplicities and the original example count, and calibrates prototype values against attention query vectors produced by the in-context examples through an anchored closed-form solution. Prototype compression drives the speedup, while value fitting helps preserve accuracy. On 64 held-out OpenML-CC18 datasets, QCOC achieves the highest mean accuracy among the compared compression and retrieval methods at both retained counts. Across 12 configurations on seven long tables, it ranks first among compressed methods in ten and averages 0.23 percentage points below full context. Compressing 8,192 in-context examples to 512 memory slots yields a 10.5× cache compression ratio; in a single-core CPU online-serving comparison over 1,000 queries, QCOC is up to 508× faster than dynamic retrieval baselines and 1.98× faster than full-context inference. These results show that QCOC enables compact-memory reuse and efficient inference across queries while retaining accuracy close to full context.

Primary area:
foundation or frontier models, including LLMs
Keywords:
tabular in-context learning, tabular foundation models, context compression, KV cache compression, efficient inference, attention approximation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_E6CGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a new method for KV compression of tabular foundation models (TFMs). They first run joint K-Means to learn key and value prototypes. As a second step, they use ridge regression to calibrate value prototypes: full attention output is the target and initial (from K-Means) value vectors serve as anchors. Using TabICLv2 on a subset of CC-18 (classification) datasets, the authors show that their method outperforms other KV compression baselines, both in accuracy and compute time / cost.

Critical strengths
  1. Ridge regression correction. The second step of correcting value prototypes via ridge regression is interesting and appears novel. Empirically. the authors show its benefit by comparing to the naive weighted joint-KV clustering.
  2. Low serving cost. The authors show that their method scales better than competitors (CRUMB, MICP, KATE) on a given subset of datasets, both in terms of performance and time/cost.
Critical weaknesses

While the original idea is novel, I find the empirical evaluation to be lacking.

  1. No true long-context datasets. The largest dataset is <10,000 samples. The recent state-of-the-art models, e.g. TabPFNv3.5, Kumo, Causilo, Limix-2, have scaled to 1M rows.
  2. Evaluations are limited to TabICLv2 and classification. To claim generality, it would be great if the authors could additionally evaluate with more backbones. I also don't understand why the authors have chosen to limit the evaluation to classification.
  3. Dataset choice. The authors have evaluated on a subset of 64 out of 72 CC18 datasets. They provide no justification for this choice.
  4. Statistical significance. The authors don't report re-running on the same dataset with different seeds / splits. K-Means can be sensitive to initialization. Hence it's hard to judge the stability and significance of the obtained results.

Minor comments:

  • Figure 3, panel C, the authors only show the scaling curves for KATE, CRUMB, and MICP. It would be great if they can additionally add LUCos, weighted joint-KV, and input K-Means to the plot.
Questions for authors

See above. Overall, my concerns are that the method has been evaluated on small datasets, where KV-compression is not even justified.

Reviewer_oZ6YOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the problem of context compression for tabular in-context models (such as TabPFN and TabICL). Specifically the paper explores a KV cache compression strategy to maintain a compact memory used for subsequent queries. The claimed contributions are a KV compression method for tabular in-context learning (Query-Calibrated Operator Compression); a derivation for an approximation bound for attention-output errors for key and value approximation errors; and experimental results demonstrating that online inference costs are reduced using cache re-use over queries. Experimentally, the paper demonstrates that the proposed strategy leads to performance improvements (accuracy and serving cost) relative to baseline methods.

Critical strengths

The core problem addressed by the paper (context compression for tabular foundation models) is an important one, and a key area of current interest for the tabular ML community. Related literature at the broad problem level is appropriate. However, lack of citation in the method and proposed derivations makes it difficult to determine how the approach relates to the wider literature.

Experimentally the method has been evaluated on a large number of relevant benchmarks and compared to appropriate benchmarks. Results appear to show improved performance relative to baselines.

Critical weaknesses

The critical weakness for the paper is clarity. Much of the writing appears to have been generated by AI and is difficult to follow, containing unique jargon, and unusual phrasing. This makes the importance and motivation for the approach difficult to understand.

The method sections (3.1, 3.2, 3.3, 3.4, 3.5) contain no citations making it difficult to see how this relates to prior literature. This is particularly an issue for the derivation bound (which is derived in appendix B) as this is listed as a core contribution for the paper.

Figures are overall difficult to parse. Figure 1 (outlining the method) is dense and not self contained, with text that is too small to read directly. It would benefit from outlining directly what the components are, how they connect, and the motivation for these components. The relevant method sections present technical details for these components, however these are largely stated rather than motivated for why the component would be beneficial in this architecture.

Figure 2 is very difficult to understand - the caption is uninformative, and the main text does not clearly provide context. "Figure 2 separates that mechanistic claim from the downstream task claim. The paired accuracy distribution in panel A is heterogeneous: gains over weaker row-selection baselines are broad, while comparisons with stronger retrieval and KV baselines are tighter."

Questions for authors

What does Figure 2 show? (This plot is difficult to interpret)

Could the authors clearly describe the motivation for components (Joint KV Prototype Construction, Constructing Calibration Equations, Value Fitting With Fixed Attention Weights, and Design Analysis) of the proposed method and how this relates to prior literature. Describing this simply (ideally without a LLM) would be beneficial as confusion over the motivation and description of this approach has been the most critical issue in reviewing the current manuscript.

Reviewer_uN6zOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how to reduce attention computation and KV cache overhead in tabular in-context learning, particularly in scenarios where a fixed historical reference set is used to answer a large number of subsequent queries.

Critical strengths

1. The problem is practically relevant, and the method is well designed.

For tabular ICL scenarios where a fixed historical reference set is queried repeatedly, the authors propose building reusable KV memory through offline compilation, rather than selecting historical examples separately for each query. This approach has a clear practical motivation. The cluster-count correction, preservation of the original context length, and Value Fitting also form a coherent design.

2. The experimental evaluation is fairly comprehensive and provides reasonable evidence for both the strengths and limitations of the method.

The paper compares QCOC with several categories of methods, including global row selection, dynamic retrieval, and direct KV compression. It also examines the effectiveness of key components, performance under different context sizes, applicability to other model architectures and prediction tasks, as well as prediction fidelity and computational efficiency. Overall, the experiments provide a fairly systematic evaluation of the method's effectiveness and scope of applicability. The authors also report performance degradation on some datasets and offline compilation costs, presenting a balanced account of the results.

Critical weaknesses

1. The technical differences from related KV compression methods could be clarified further.

QCOC combines joint KV clustering, cluster-count correction, and Value Fitting into a fairly complete and effective compression method for tabular ICL. However, as I understand it, the idea of calibrating compressed values by matching attention outputs bears some technical similarity to existing KV compression methods, such as Fast KV Compaction via Attention Matching and GRKV.

Of course, QCOC does differ from these works in how it constructs prototypes, uses cluster counts, and selects calibration queries. Still, I am not entirely sure whether these differences are sufficient to establish the novelty of the method. I would appreciate some further explanation from the authors about the reasoning behind these design choices and what distinguishes them from existing approaches, which would help me better understand the paper's contribution.

2. Some presentation details in Figure 1 could be improved.

Figure 1 is generally helpful for understanding the overall method, but the different roles of the cluster-count vector wℓw^\ell and the original context size NN could be explained more clearly. In addition, some arrows and connections appear slightly misaligned, and the shape of the schematic plot on the right looks a little irregular. Minor adjustments to these details would improve the clarity of the figure.

Questions for authors

I noticed that QCOC shares some technical ideas with works such as Fast KV Compaction via Attention Matching and GRKV, while also having its own design choices. Could the authors elaborate on the main differences between QCOC and these works, and how they view the novelty of their approach?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.