acceptodds
Under review as a conference paper at ICLR 2027

SuffiKV: Learning Sufficient Task States via Layer-wise KV Memories for Scalable Tabular Foundation Models

Abstract

Tabular foundation models (TFMs) enable supervised prediction by conditioning a frozen model on a labeled support table, but this makes the table part of every inference computation. As support sets grow, full-context inference becomes a latency and memory bottleneck, and additional raw rows may be redundant, noisy, or poorly aligned with the frozen attention mechanism. We introduce SuffiKV, a test-time adaptation framework that compiles a support table into a compact layer-wise key-value task state for a frozen TFM. SuffiKV separates the table used as evidence from the read state used at prediction time: a write stage constructs context-derived KV anchors, a bounded trust-region residual refines them, and a read stage predicts future queries using only the cached KV state. The state is optimized by anchored operator distillation, using full-context predictions and attention effects as early functional guidance while held-out pseudo-query labels refine the state toward supervised task risk. This turns tabular in-context learning into task-state inference: more data can improve the written state without increasing read-time context length. On a long-context tabular benchmark, SuffiKV improves the accuracy–latency–memory Pareto frontier, reducing subsequent inference latency by 89.9 and peak memory by 66.2% while improving AUC by 1.5%. Code and models will be released.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Tabular foundation models, in-context learning, tabular compression

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_K2b9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SuffiKV - a per-task adaptation method that makes the inference of TFMs much faster while preserving or improving the prediction precision. SuffiKV builds a set of K and V and decouples the inference of a TFM into 2 steps: step 1: build K and V from the context, step 2: read K and V to make predictions. Unlike other methods that select a subset of the context, or build abstract rows, SuffiKV acts at the attention stage. The benefit is it decouples the routing and the information.

Critical strengths

The idea is interesting, the paper presents many experiments and ablation studies. The "claimed results" look good, especially the speed.

Critical weaknesses

The evidence does not support the claims.

  • SuffiKV uses labels for training. It is unclear whether the gains in the performance come from the training or from the efficient compresion, or a mix of both. The paper's argument is that the memory can merge redundant rows, downweight noisy ones, reserve slots for rare classes and decision boundaries, and adjust calibration. That’s plausible, but it’s asserted rather than shown by ablation studies.
  • It’s unclear the advantage of decoupling K and V.
  • The results do not support the claims:
    • The paper only reports mean performance. Do the gains come from just a few datasets or homogeneously? win/tie/loss counts should also be reported
    • The +1.5 AUC number hides the spead: it’s an average of +2.7 (LimiX) and +0.2 (TabPFN-2.6), on a self-curated benchmark. Gains for TabPFN on regression is zero, and slower at prediction on TabArena-Reg.
    • The results of TuneTables look broken: on TabArena-Reg, TuneTables performs far worse than TabPFN v2.6 and use more memory.
    • Scaling with context size: The “TFMs saturate” story really holds for LimiX only, the performance TabPFN v2.6 keeps growing when the context grows
    • The rank analysis: the rank result is uninformative. A Gaussian noise matrix can still have high rank.
    • Linear probes: a linear classifier trained on the final embeddings scores higher on SuffiKV’s embeddings than on full context’s. That’s expected: SuffiKV’s state was trained on labels, so its final features align with them
Questions for authors
  • Can you add an ablation study to Table 7: Task-rerm + Op-term (no Label refinement). The objective is to test whether the gain in the performance comes from finetuning for from compression. Additionally, if time permits, is it possible to add an additional baseline: full context plus the same label-trained memory, with no compression. If it matches SuffiKV’s accuracy, the gain is from training.
  • Weakness 2: if time permits, is it possible to report an ablation study with a hidden-summary (Z) version at equal parameter count.
  • Can you justify the claim that SuffiKV “can merge redundant examples into shared slots, reduce the influence of noisy or irrelevant rows”. Otherwise you should relax that claim.
  • Can you justify the advantage of decoupling K and V?
  • The paper only reports mean performance. Do the gains come from just a few datasets or homogeneously? win/tie/loss counts should also be reported
  • Can you explain why SuffiKV is slower than TabPFN v2. on TabArena-Reg? is it because there are many small datasets?
  • Can you explain why TuneTables performs that bad on TabArena-Reg? Is it because the pipeline was broken?
  • Can you define a scenario where the speed up during inference worth the cost of the training of SuffiKV?
1 comment
AuthorsOpenReview ↗

Thank you for recognizing our approach's appeal, extensive experiments and ablations and promising speedups. We appreciate your concerns about the evidence and the chance to clarify our rationale. We welcome further questions and will respond promptly.

Q1. Ablation.

We ran the requested Task+Operator arm on TabMacro-Cls with LimiX v1.

Full-contextLabelLabel+TaskTask+OperatorLabel+Task+Operator
LimiX0.7770.7920.7940.8020.804

Label alone gives +0.015, so some gain is from training, as you suspected, but the label-free arm still yields 0.802 and labels add +0.002, suggesting complementary benefits from label supervision and compression distillation. The label-free 0.802 is our strongest evidence against training as the main cause.

Q2. Hidden summary and decoupled K/V.

TuneTables and TACO are related shared-summary baselines. They derive K and V from a shared hidden summary, whereas SuffiKV optimizes them independently, providing greater flexibility in attention matching and information aggregation. Regarding the requested parameter-matched hidden-summary baseline, we evaluated a layer-wise hidden-summary variant with 192 learned summaries, reaching 0.799 AUC on TabMacro-Cls with LimiX v1, against 0.804 for the KV state at matched read-time slot count.

Direct KV-space learning achieves higher AUC and reduces computational overhead by bypassing feature attention and summary-to-K/V mapping. Together with SuffiKV's reported advantage over the other baselines, these results support the method's effectiveness and the benefits of K/V decoupling.

Q3. Compression and denoising.

We accept this criticism. The listed mechanisms have not been sufficiently validated. We inferred that SuffiKV could merge redundant samples into shared slots because performance improved after compressing the full context into far fewer fixed slots. Flipping the highest-confidence 10% of labels (confidence≥0.95) on five TabMacro-Cls datasets with LimiX v1 gives AUC of 0.979/0.980/0.732/0.944 on 100000; 0.909/0.915/0.890/0.916 on chicago; 0.711/0.735/0.641/0.728 on expedia; 0.846/0.890/0.803/0.890 on kickstarter; 0.962/0.964/0.983/0.954 on web, in the order full context, SuffiKV, noisy full context and noisy SuffiKV. This supports noise mitigation of SuffiKV.

Q4. Win, tie and loss counts.

W/T/LLimiX v1TabPFN v2.6
TabArena-Cls21/6/616/3/14
TabZilla-Cls16/2/917/3/7
TabMacro-Cls26/2/114/2/13
TabArena-Reg9/1/35/1/7
TabMacro-Reg7/1/02/2/4
Total79/12/1954/11/45

LimiX v1 wins 79/110 datasets (72%) and TabPFN v2.6 wins 54 (50%). The +1.5% aggregate hides a large spread, as you say. LimiX v1 gains +2.7% on TabMacro-Cls against +0.2% for TabPFN v2.6. The gains concentrate in LimiX v1 and our self-curated large Kaggle suite. However, SuffiKV yields similar gains for TabPFN v2.6 and LimiX v1 on the other classification benchmarks. Although it does not improve TabPFN v2.6’s regression performance, GPU decoding enables 6×/34.3× speedups on TabArena/TabMacro-Reg, with substantially lower GPU memory overhead than TFMs using standard KV caching (see our response to Reviewer icH3 Q1).

Q5. TabArena-Reg.

Yes, small datasets are one cause, since many contexts are shorter than the 384 K/V pairs. The second cause is CPU decoding of full logits in our original pipeline, while TabPFN v2.6 decodes buckets on the GPU. GPU decoding gives 6.0× and 34.3× over all 13 and 8 datasets (M51z Q2), while the 4.5× and 20.6× figures in icH3 Q1 use only the 11 and 2 cache-completable datasets.

Yes, the TuneTables regression baseline is effectively broken. It reaches R² of 0.345 and 0.108 with higher memory (168.0% and 239.0% in Table 2), and eight TabArena-Reg drops drive its aggregate. Its paper and official code cover classification only, so our regression adaptation is an inadequate extension of our own. We will clarify this.

Q6. Applicable scenarios.

SuffiKV pays off for long contexts setting, with N* ≈ 2.5×10²–3.0×10³ prediction passes at T = 500 (M51z Q2). Per our responses to Reviewers M51z/icH3 Q2, optimization with 50K–100K context examples averages 1.2 s/epoch for 12-layer LimiX v1 and 3.1 s/epoch for 24-layer TabPFN v2.6. 500 epochs suffice for reported performance; early stopping is also enabled. At this manageable cost, SuffiKV learns a compact, sufficient KV state to address OOM, and noise in long contexts, while alternative context-compression and kNN/retrieval methods generally degrade performance in our experiments.

Q7. Rank analysis.

We agree that Gaussian noise is also full-rank. Full-rank Gaussian states give AUC ≈ 0.50 on TabZilla-Cls against 0.946 for SuffiKV. The stronger evidence is predictive, since 99%-energy projection barely changes AUC (Fig. 4a) and shared-direction-only projection costs 0.024 AUC for full context and 0.325 for SuffiKV (Fig. 5).

Q8. Context size and probes.

Figure 2 shows continued but diminishing gains for TabPFN v2.6 near 30K context size. We will clarify that the evaluated models exhibit diminishing returns within the tested range.

In the linear probing experiments, 10% of the full context was held out for testing and was not used to train SuffiKV's KV states.

Reviewer_icH3OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposed to reduce the memory and inference time of tabular foundation models (TFMs) by a technique similar to a KV cache. The KV cache is not exact but approximate. It is represented in a layer-wise KV memory. Given a new dataset, this KV memory is learned using 1000 AdamW epochs with a frozen TFM backbone.

The method is applied to LimiX v1 and TabPFN v2.6 on five benchmarks: TabArena (classification + regression), Tabzilla classification (only using datasets up to 50k rows), as well as two new Kaggle suites with larger datasets (classification + regression). Results indicate reduction of memory and inference time compared to uncached full-context TFMs.

While the idea is interesting, I see several issues:

  • The natural baseline are TFMs with KV cache enabled. This approach was already introduced in TabPFN v2, but it was not very memory efficient, and since TabPFN v3 it is practical. This standard KV caching is not at all discussed in the paper, which is highly unfortunate.
  • 1000 AdamW epochs sounds very expensive. With this, TFMs appear to loose their major selling point of their application just being a forward pass. If ChatGPT would have required gradient-based updates for a new query, it would not have nearly had the impact it did.
Critical strengths

None

Critical weaknesses
  • The natural baseline are TFMs with a standard KV cache, and this baseline is omitted.

  • The method's potential impact is limited: even if it improves performance, 1000 AdamW epochs takes time and make TFMs lose their unique "training-free" advantage against classical methods.

Questions for authors
  • Can you please provide a comparison to a TFM with a standard KV cache?
1 comment
AuthorsOpenReview ↗

Thank you for the two points that materially improve the paper. You correctly identify the missing KV-cache baseline and the loss of forward-only deployment.

Q1. Standard KV cache baseline.

You are right that this is the natural baseline and that the submitted manuscript did not discuss it. On one 32 GB GPU, TabPFN v2.6 with standard KV caching runs out of memory on 22 of the 110 datasets. The failures are 2 of 13 on TabArena-Reg, 6 of 8 on TabMacro-Reg, 1 of 33 on TabArena-Cls, 2 of 27 on TabZilla-Cls and 11 of 29 on TabMacro-Cls. The comparison therefore covers the remaining 88 datasets, with the per-subset counts shown in parentheses. Uncached TFMs use outer-batching to avoid OOM. The failures concentrate in exactly the long-context regimes SuffiKV targets, and SuffiKV completes all 110 datasets without OOM (Sec. 4.3). Excluding them does not favour us, because it removes the regime where the baseline is least viable.

TabArena-Cls (32)TabZilla-Cls (25)TabMacro-Cls (18)
AUCACCSpeed↑Mem↓AUCACCSpeed↑Mem↓AUCACCSpeed↑Mem↓
TabPFN v2.60.850.871.0×100.0%0.940.881.0×100.0%0.770.711.0×100.0%
+ KV cache0.850.872.6×126.6%0.940.884.8×127.6%0.770.715.1×161.7%
+ SuffiKV0.860.873.3×73.3%0.950.895.2×70.3%0.780.7334.8×31.3%

Exact caching reproduces the uncached metrics, as expected. SuffiKV's small AUC edge is not a cache-approximation effect. It comes from the held-out pseudo-query label refinement in Eq. (9)–(12), which is also visible in our response to Reviewer K2b9 Q1, where the label-free Task+Operator arm already reaches 0.802 of 0.804.

Most regression datasets where caching completes are small. As shown in our response to Reviewer M51z Q2, CPU decoding in our original pipeline limited speedups, and decoding is now on the GPU. The original row is kept only to make the fix explicit, and it is not our deployment configuration. The full comparison is as follows.

TabArena-Reg (11)TabMacro-Reg (2)
R²RMSESpeed↑Mem↓R²RMSESpeed↑Mem↓
TabPFN v2.60.800.401.0×100.0%0.270.851.0×100.0%
+KV cache0.800.402.3×156.4%0.270.855.1×191.9%
+SuffiKV (CPU decode, original)0.800.400.5×58.5%0.270.851.2×34.9%
+SuffiKV (GPU decode)0.800.404.5×92.0%0.270.8520.6×101.5%

GPU decoding raises memory relative to our original implementation, but peak memory stays well below caching and near the uncached baseline, with metrics matched. These 4.5× and 20.6× figures use the 11 and 2 cache-completable datasets, while the 6.0× and 34.3× figures in our response to Reviewer M51z Q2 use all 13 and 8 datasets.

Q2. Limited impact.

We agree, and we state the scope plainly. SuffiKV does not replace the training-free, single-forward-pass use of TFMs. For small datasets or one-off predictions, the original forward pass remains the right choice. SuffiKV adds an opt-in "compile once, read many" mode for long-context, repeated-query workloads where the full table otherwise OOMs, fades attention, or is repeatedly re-read despite redundant and noisy rows.

The cost is bounded and one-off. The backbone stays frozen; only the small KV-state parameters are updated, and the state is reused for all later queries in this task without further optimization. Appendix B.3 fixes T = 1000 epochs, and the submitted paper uses early stopping. Contexts with 50K–100K rows take 1.2 s per epoch with the 12-layer LimiX v1 and 3.1 s per epoch with the 24-layer TabPFN v2.6. Since 100 epochs already beat the originals and 500 match the reported performance, the full budget is not required. Our response to Reviewer M51z Q2 gives N* ≈ 2.5×10²–3.0×10³ prediction passes at T = 500. Compression baselines are also task-specific and lose accuracy in Table 3, whereas exact KV caching OOMs on 22 of the 110 datasets. We will state this limitation in the introduction and conclusion.

Reviewer_M51zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SuffiKV, a test-time adaptation framework for improving the scalability of frozen TFMs. Instead of repeatedly conditioning predictions on the full labeled support table, SuffiKV constructs compact layer-wise key-value memories that serve as task-specific states during subsequent inference.

The proposed approach initializes KV memories using information from the support context and refines them through bounded residual parameterization and anchored operator distillation. The optimization combines teacher predictions, intermediate attention outputs, and held-out pseudo-query labels, allowing the resulting memory to improve beyond the original full-context model.

Critical strengths

The paper investigates task-specific adaptation directly in the internal KV space of frozen TFMs, rather than selecting or optimizing input-level context examples. This provides a flexible way to represent support-set information while avoiding modifications to the backbone parameters. The separation between support-set processing and subsequent inference is conceptually appealing and potentially useful for large-scale tabular applications.

Critical weaknesses

Section 3.2 describes a context-derived initialization mechanism in which learnable summary slots attend to support representations through a frozen writer block, followed by projection into layer-wise KV anchors. However, Appendix B.3 describes a substantially different procedure: extracting teacher KV statistics and initializing the anchors by sampling correlated Gaussian variables using their means, standard deviations, and correlations. The relationship between these two initialization procedures is not explained. This distinction matters because the context-derived writer is presented as a central component of the proposed framework. It is therefore difficult to determine which mechanism is actually used in the reported experiments

While the reported prediction-stage speedups are impressive, SuffiKV requires an additional optimization process for each dataset. Appendix B.3 specifies 1,000 training epochs involving full-context teacher guidance and chunk-wise processing. The main efficiency results primarily emphasize subsequent inference latency. Although Figure 1 reports fit-and-predict time, the paper describes fitting time as preprocessing overhead, leaving it unclear whether the complete KV adaptation cost is included. Since this adaptation may be computationally expensive, the end-to-end advantage depends on the number of future queries over which the one-time optimization cost can be amortized. Without measured adaptation costs or a break-even analysis, it is difficult to evaluate the practical scalability of the complete framework. Moreover, the efficiency improvements are not universal: Table 2 reports a 0.5× speedup for TabPFN v2.6 with SuffiKV on TabArena-Reg.

Questions for authors

Could the authors clarify the exact KV initialization procedure used in the experiments? Specifically, how does the frozen writer-block formulation in Section 3.2 relate to the Gaussian-statistics-based initialization described in Appendix B.3? Are both mechanisms used, or does the appendix describe the actual implementation? If possible, an ablation comparing these initialization strategies would help establish the contribution of the proposed writer.

What is the complete computational cost of constructing a SuffiKV task state? Could the authors report actual adaptation times, clarify whether the 1,000 epochs include processing all training chunks, and provide end-to-end runtime comparisons that include state construction? A break-even analysis across different numbers of test queries would help establish when SuffiKV provides a practical efficiency advantage over full-context inference.

1 comment
AuthorsOpenReview ↗

Thank you for recognizing the KV-space adaptation and the write–read separation. We clarify initialization and construction cost below.

Q1. KV state initialization.

We apologize for the inconsistency. All reported experiments use the statistics-based initialization of Appendix B.3. Both passages describe the same write stage, namely a compact anchor derived from the frozen TFM's full-context K/V computation and written before any query is read. Section 3.2 was intended to express the general principle of deriving KV anchors from support-set information, but its formulation does not clearly reflect the implementation. We will revise Section 3.2 accordingly and align the notation and descriptions across both sections. The implemented writer extracts layer-wise frozen model KlDK^D_l and VlDV^D_l, estimates their per-dimension means, standard deviations and K–V correlations, and samples correlated Gaussian anchors from these statistics. The rest is unchanged, including the compact anchor (m=192/384≪nm = 192/384 ≪ n), the bounded trust-region residual (Eq. 6) and the memory-only read path for every query.

The ablation tests whether context-derived anchors matter, since all three arms share the same refinement and read path.

TabZilla-ClsAUCACC
Random0.9320.838
Sampled Full-Context KV0.9440.875
Sampled Full-Context KV Statistics (ours)0.9460.886

Context-derived initialization gives +0.014 AUC and +0.048 ACC over random initialization. Estimating full statistics adds a further +0.002 AUC and +0.011 ACC over sampled full-context KVs.

Q2. Cost of constructing a KV state.

In Figure 1, "fit" refers to preprocessing overhead and excludes the full T-epoch KV state construction process. We have separately analyzed the optimization cost of constructing the complete KV state. The submitted paper uses early stopping, so appendix B.3 fixes T = 1000 epochs for all datasets are conservative upper bounds. Each epoch traverses all training chunks and includes teacher and student forward passes, inter-GPU communication and validation. Times are seconds per epoch, measured on eight 32 GB RTX 5090 GPUs and averaged over eight datasets per bin from TabZilla-Cls and TabMacro-Cls.

Moedl<1K[1K,5K)[5K,10K)[10K,50K)[50K,100K)Avg
LimiX v10.620.660.700.921.220.82
TabPFN v2.60.850.900.921.143.101.38

We also evaluated shorter schedules on TabZilla-Cls, where 100 epochs already outperform the originals and 500 match the reported performance.

EpochLimiX-v1TabPFNv2.6
00.9370.937
1000.9440.938
5000.9460.946
10000.9460.946

The requested end-to-end accounting is the break-even point, computed as N*=(T·t_epoch)/(t_full−t_SuffiKV), where the denominator is the time saved by one prediction pass of the same size. At T = 500 in the 50K–100K bin, construction takes ≈10 min for LimiX v1 and ≈26 min for TabPFN v2.6 on the eight-GPU setup. Using the GPU-decode timings below, N* is ≈2.5×10² passes on TabMacro-Reg and ≈3.0×10³ on TabArena-Reg for TabPFN v2.6, and roughly 40% of these for LimiX v1. Doubling T to 1000 doubles both. SuffiKV pays off for queries on long contexts.

The 0.5× slowdown in Table 2 came from two implementation effects. First, 384 K/V slots exceed many short TabArena-Reg contexts. Second, TabPFN v2.6 decodes bucket regression on the GPU, whereas our original pipeline transferred full logits to the CPU for decoding in order to aggregate pipeline outputs and save memory, and that transfer dominated. We profiled both paths on one 32 GB RTX 5090 using previously learned states. Each cell reports full-context and SuffiKV times in seconds, and the decode columns compare the original CPU pipeline with the revised GPU pipeline.

DatasetForwardDecode(CPU)Decode(GPU)
Another0.024/0.0450.001/0.0680.001/0.027
Food1.291/0.1240.016/2.8850.016/0.154
QSAR-TID1.501/0.4300.003/0.4510.003/0.427
QSAR_fish0.024/0.0530.001/0.0460.001/0.025
airfoil0.027/0.0320.001/0.0670.001/0.032
concrete0.025/0.0550.001/0.0160.001/0.026
diamonds1.731/0.1460.019/3.4050.019/0.182
health0.030/0.0290.001/0.1190.001/0.027
houses0.363/0.0520.008/1.3850.008/0.060
miami0.244/0.0460.005/0.9090.005/0.048
physio1.305/0.1320.016/2.9410.016/0.156
super1.387/0.2030.008/1.4200.008/0.213
wine0.070/0.0340.002/0.5160.002/0.032

The forward column confirms that the KV state is faster on large datasets, and CPU decoding explains the 12 slower datasets. With alignment, predictive-mean decoding moved to the GPU and only final predictions sent to the CPU, we compared the revised implementation with uncached TabPFN v2.6 over all benchmarks.

BenchmarkTabPFN v2.6SuffiKVSppeedupMem
TabArena-Reg0.617s0.102s6.0x105%
TabMacro-Reg6.479s0.189s34.3x100%

The 4.5× and 20.6× figures in Reviewer icH3 Q1 use only the 11 and 2 cache-completable datasets, while the 6.0× and 34.3× figures above use all 13 and 8.

Comments on the paper

1 comment

We thank all reviewers. Most numbers below are new rebuttal measurements, while values taken from the submitted paper (Tables 1–2, 7, 8 and Appendix C) are cited as such. All timings use 32 GB RTX 5090 GPUs, and the "24GB" in Sec. 4.3 is a typo. "Mem" denotes peak GPU memory as a percentage of the uncached baseline on the same hardware, so lower is better and values above 100% mean that the method uses more memory. The 0.5× slowdown in Table 2 of the submitted PDF reflects a CPU-decoding bottleneck in our original regression pipeline, and it is superseded by the GPU-decode timings below.

The manuscript will be revised in the same way for all three responses. Section 3.2 and Appendix B.3 will be aligned around the implemented statistics-based writer. We will add the standard-KV-cache baseline and its OOM discussion to Sections 1 and 4.3, update Tables 2 and 7 with the GPU-decode and Task+Operator results, and report per-backbone and per-benchmark deltas beside every aggregate number. We will soften the untested mechanism claims concerning the "TFMs saturate" claim. The hidden-summary parameter counts will also be reported.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.