acceptodds
Under review as a conference paper at ICLR 2027

Tabular Selective Attention

Abstract

Tabular foundation models (TFMs) predict each test row by attending over a context of labelled training rows, so a prediction is only as good as the set of rows attention averages over. While these models perform well in low-data regimes, current state-of-the-art models employ context curation to generalize over larger datasets. Existing curation methods either sharpen attention with a temperature or shrink the context with nearest-neighbour retrieval. We prove that temperature scaling faces an inherent tradeoff: a single scalar must balance suppressing the uninformative tail against preserving the attention mass over the entire informative set. Retrieval avoids this tradeoff by explicitly selecting which samples enter the context in the first place. Yet in doing so, it uses row similarity as a proxy for the attention that the model would assign to each row. We address the limitations of both curation types with a new model-agnostic mechanism: **Tabular Selective Attention (TSA)**. TSA learns a non-negative score for each test/training-row pair that reshapes the attention distribution into one that favours all informative samples rather than just a subset. Across different architectures, such as TabICLv2 and TabDPT, TSA outperforms existing curation methods on the TabArena benchmark. Leveraging these TSA scores to prune uninformative rows from context, we can speed up its inference while largely preserving accuracy.

Primary area:
foundation or frontier models, including LLMs
Keywords:
selective attention, tabular foundation models, attention mechanism

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1muzOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose a new attention for tabular foundation models. The authors state that the key to long-context attention in TFMs is distinguishing between informative and uninformative samples. They then observe that QASSMax attention proposed in TabICLv2, which is based on the idea of per-query temperature scaling, tends to suppress both the uninformative samples and the lower-scoring informative samples together. Instead, they propose to learn the shift (masking) matrix, which for each query shifts the per-key attention scores. This new attention is only used in the ICL module. Empirically, compared to QASSMax and vanilla attention, the proposed attention mechanism is better on the subsample of 38 TabArena classification datasets.

Critical strengths
  1. Clear and novel problem formulation. The authors propose a novel formulation based on the idea of informative and uninformative samples. They define two quantities: MT/MRM_T/M_R - ratio of attention mass between informative and uninformative samples (tail-to-informative ratio); ρR\rho_R - ratio of the attention scores within informative samples (coverage). Intuitively, we want MT/MRM_T / M_R to be low, while the second ratio ρR\rho_R needs to stay within a constant range. In particular, using this formulation, the authors show that QASSMax is suboptimal as it suppresses both ratios at the same time.
  2. Proposed solution is intuitive. Instead of per-query scaling, the authors propose to introduce a per-query and per-key shifting (masking) term. The attention then has a clear mechanism to drive attention scores closer to 0 for uninformative samples, without affecting the spread (of attention scores) within the informative samples.
  3. Supported by empirical evidence. The authors re-train TabICLv2 from scratch and compare their proposed attention mechanism to the default QASSMax, SSMax, and vanilla attention. On a set of 38 classification datasets from TabArena, the inductive bias in the proposed attention proves beneficial and outperforms other attention versions. On a set of synthetic datasets, the authors also clearly demonstrate that the proposed attention achieves the desired goal: coverage stays high, while the tail-to-informative ratio is low.
Critical weaknesses
  1. Informative and uninformative samples. The authors build on the premise that for each test query the context (in each layer and head) can be naturally divided into informative (with min attention logit δ\delta) and uninformative (whose attention scores are ≈Δ\approx \Delta) samples such that Δ>δ>0\Delta > \delta > 0. Is this assumption supported in practice? I would expect that per-query the distribution of per-key attention scores is smooth and does not have any jumps.
  2. QASSMax underperforms vanilla attention on the reported empirical set. In Table 1, QASSMax has 31% winrate against vanilla, which is below SSMax. Figure 11 in the original TabICLv2 paper, to the contrary, suggests that QASSMax is better than vanilla (albeit it reports log-loss during pre-training).
Questions for authors

See the weaknesses above. Also, I have the following questions:

  1. By default, does your algorithm prune rows? For example, in Figure 11, is the reported performance with or without pruning?
  2. In Figure 6, where is you default algorithm? I see the 3 ablated versions, but not the default (with confidence gates, with warm-up skips, and without compounding).
  3. Does Figure 5 suggest that there are no "uninformative" samples per se? If I am reading it correctly, my interpretation is that the first few layers don't suppress any rows (samples), i.e., the model considers all samples informative (and applies vanilla attention over them). Then, the model compresses the data into few informative tokens, and that is where the suppression starts to help. Is this interpretation correct?
3 comments
AuthorsOpenReview ↗

Response to reviewer 1muz

We thank the reviewer for the careful reading and the thoughtful questions.

W1 (Theory assumptions)

Real per-query logit profiles are indeed smooth: across all 38 TabArena datasets (3 splits), sorting each query's logits and looking for an unusually large gap among the top 64 finds one in only 2.7% of (layer, head, query) cases, and the largest gap is typically just ≈0.04 logit standard deviations.

The two-level setup (RR at depth δ\delta, TT at depth Δ\Delta) is a deliberate simplification: it gives closed forms for ρR\rho_R and MT/MRM_T/M_R, and intuition for why a key-independent temperature trades coverage for suppression. We still do not claim that heads work always like this.

Our results do not need a gap: for any mm, let RR be the top-mm rows. Then g=min⁡Rz−max⁡Tz≥0g=\min_R z-\max_T z\ge0 by construction, and the bounds hold with that gg. In practice g≈0g\approx0 (no jump), so the tail is suppressed by the learned FF and not by a logit gap, which is what a scalar temperature cannot do.

On real data, RR is not accessible, so we take either the top-mm rows by logit or the mm nearest neighbours in feature space (with m=32m=32 or m=2m=2% of nn). TT is penalised more than RR in 85–91% of queries (with mean suppression score of 0.46 on TT vs. 0.001–0.09 on RR). "Informative" is therefore graded: a random nearest neighbour outscores a random other row 82% of the time (median AUC), but the two sets always overlap.

W2 (QASSMax baseline)

Our QASSMax baseline is a from-scratch reproduction of TabICLv2. All variants are pre-trained under TabICLv2's public prior with the official code, identical optimisation settings and curriculum, and FA-3 enabled in Stages 2–3 as in TabICLv2 (App. C.3–C.4), which is consistent with what was reported in the paper.

We observe that both SSMax and QASSMax degrade at later stages, where NN grows to 60K. We hypothesise this is due to the log⁡n\log n term, which sharpens the logits beyond the range seen in earlier stages and becomes unstable at longer contexts. This is consistent with their lower win rates against vanilla (31.9% and 37.9%).

Q1 (Pruning not used as default)

No: by default, TSA keeps every training row in context, and all reported results are without row dropping.

Q2 (Fig 6)

Thank you for flagging this: Figure 6 will now include the default schedule (all components) as well.

Q3 (Fig 5)

We mostly agree, with a few caveats:

At initialisation, roughly half of all pairs are suppressed in every layer (WW is zero-mean Gaussian). After training, the early-layer scores mostly fall just below zero, so F=0F=0 and TSA reduces to vanilla attention in layers 0–3. The model therefore learns to switch suppression off early.

(i) We agree there is no globally uninformative row: informativeness is defined per head and test row. However, early layers do not treat all rows alike: on TabArena, their logits already rank a query's 32 nearest feature-space neighbours above the other rows (as mentioned in W1).

(ii) The context is not compressed into fewer tokens: every training row keeps its own token in every layer, and at most about 2% of rows are flagged per layer.

Please let us know if you have any remaining questions.

Thank you! My questions have been mostly answered. I am still a bit concerned about the QASSMax, but I suppose I need to play around with it myself.

On why QASSMax underperforms SSMax. We thank the reviewer for raising this point about QASSMax. Both SSMax and QASSMax multiply the standard attention logits by a factor of log⁡n\log n (with an additional learned scale, which is query-dependent in QASSMax). Because TabICLv2 does not use QK-normalization, the logits q⊤k/dq^\top k / \sqrt{d} are themselves unbounded, so this rescaling acts on values with no fixed range.

This interpretation is consistent with findings in Kumo-Tabular [1] (Section 7), where the authors report that stable training with learned attention scales required both QK-normalization and weight decay on the scale parameters. We will add this discussion to the revised manuscript.

Reference:

[1] Qu, J., Hudovernik, V., Jurkovič, M., Lorenz, C., Holzmüller, D., Lopez, F., Gordeev, D., Nitta, A., Bendias, R., Wong, A., Jain, V., Kothapalli, V., Titericz Junior, G., Leskovec, J., & Fey, M. (2026). NVIDIA Kumo Tabular: Pushing the Accuracy$\unicode{8211}$Efficiency Frontier for Tabular Prediction. arXiv.

Reviewer_NBL5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors study the data curation problem for tabular foundation models (TFMs) when the number of in-context samples is large. To address this issue, they first identify a problem with existing methods in which the temperature-scaling term in attention layers does not depend on the keys. Based on this observation, they propose Tabular Selective Attention (TSA). Experiments on TabArena demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper is generally well-written and easy to follow.
  2. The proposed method is sound.
  3. The experiments are well-designed and extensive.
Critical weaknesses

I have several clarification questions for the authors.

  1. In Section 3 and Appendix A, the authors claim that QASSMax acts as a query-dependent 1/τ1/\tau and that the gate function is approximately proportional to the all-ones vector. While the design of QASSMax could do much better, the authors should provide more evidence to support these claims.
  2. On line 184 in Section 3, I do not understand what tt means here or how it scales to meet the mass requirements.
  3. The theories in Section 4.1 are all based on the assumption that the value of FF for any feature in the informative set is exactly zero, which is a strong assumption in practice. I would expect the authors to provide more insight into scenarios where some informative features may have larger values of FF than the remaining features.
Questions for authors

See the weakness part.

2 comments
AuthorsOpenReview ↗

Response to reviewer NBL5

We thank the reviewer for the thoughtful comments.


W1 (QASSMax as a query-dependent temperature).

The claim does not require the gate to be all-ones. Appendix A.2 states this is an empirical observation, and Proposition 1 only requires the scaling to be key-independent. Both QASSMax factors are: base(n)\mathrm{base}(n) folds into WQW_Q, and gate(q)=1+tanh⁡(MLP(q))\mathrm{gate}(q)=1+\tanh(\mathrm{MLP}(q)) depends only on the query, so it cannot suppress individual rows, which is what TSA adds. Behaviourally, 23 of 24 QASSMax retrieval heads trade coverage for tail suppression (Fig. 4, left), as Proposition 1 predicts for a temperature.

Fig. 2 is limited on its own, so we ran two new experiments on the synthetic task. They show gate(q)≈c(q)s\mathrm{gate}(q)\approx c(q)s: a query-dependent scalar c(q)c(q) times a fixed per-coordinate vector ss, which folds into WQW_Q like base(n)\mathrm{base}(n):

  • Decomposition (App. A.2.1). The query-dependent part of the gate's shape, the only part that can re-point a query, explains a median of only 2% of its log-variance. The rescaled query and its c(q)sc(q)s approximation have cosine similarity 1.0 (median), at least 0.94 (minimum), and substituting c(q)sc(q)s leaves γ\gamma, MT/MRM_T/M_R, ρR\rho_R and AUCRT\mathrm{AUC_{RT}} unchanged (median change 0.0000).
  • Downstream replacement (App.~A.2.2). Swapping the gate for c(q) sc(q)\,\mathbf{s} in all 12 layers reproduces the model's class-1 probabilities up to the fourth decimal place, whereas removing the gate shifts them in the second decimal place.
  • All-ones is only approximate. Median cos⁡(gate(q),1)\cos(\mathrm{gate}(q),\mathbf 1) is 0.999, but 7 of 96 heads fall below 0.98.

Note: The new appendices are not yet in the pdf, we will update them shortly.


W2 (Meaning of tt and how it scales).

t=e−Δ/τ∈(0,1)t=e^{-\Delta/\tau}\in(0,1) is the weight of each tail row relative to the top row, which has weight 1. For t=0.01t=0.01, a tail row gets 1% of the top row's attention. Lowering τ\tau pushes the tail further below the top row, so tt shrinks, and temperature enters the analysis only through tt: ρR=tγ\rho_R=t^{\gamma} with n−mmt ≤ MTMR ≤ (n−m)t.\tfrac{n-m}{m}t\ \le\ \tfrac{M_T}{M_R}\ \le\ (n-m)t.

Scaling: There are n−mn-m tail rows of weight tt, so together they add up to (n−m)t(n-m)t, while the mm informative rows hold at most mm. To keep the tail below a fraction ϵ\epsilon of the informative mass, we therefore need t≤ϵmn−m≈ϵmnt\le \tfrac{\epsilon m}{n-m}\approx\tfrac{\epsilon m}{n}. Since t=e−Δ/τt=e^{-\Delta/\tau}, this requires 1/τ≥1Δlog⁡n−mϵm1/\tau\ge\tfrac{1}{\Delta}\log\tfrac{n-m}{\epsilon m}, which grows as log⁡n\log n (the SSMax rate).


W3 (The assumption Fij=0F_{ij}=0 on the informative set).

Fij=0F_{ij}=0 on RR is indeed a strong assumption (especially when RR is unknown), and the mechanism does not enforce it. Eq. 10 is the special case Fim=0F_{im}=0, however, in general, aijaim=e−(Fij−Fim)e−(zim−zij).\frac{a_{ij}}{a_{im}} = e^{-(F_{ij}-F_{im})}e^{-(z_{im}-z_{ij})}. Therefore, FijF_{ij} alone does not determine aija_{ij} as the logit gap contributes equally. We expand the analysis for F≠0F\neq0 on RR, including when some informative rows receive a larger FF than others:

  • Coverage. Let ϕmax⁡,ϕmin⁡\phi_{\max},\phi_{\min} be the largest and smallest FijF_{ij} over j∈Rj\in R. Coverage shrinks by at most e−(ϕmax⁡−ϕmin⁡)e^{-(\phi_{\max}-\phi_{\min})}, so a penalty that is the same on all informative rows costs nothing.
  • Suppression. MT/MRM_T/M_R is multiplied by (fraction of TT's attention mass kept) / (fraction of RR's mass kept). The tail only needs to lose a larger share of its mass than RR, not F=0F=0 on RR.

We checked this on all 38 TabArena datasets. RR is not available, so we take either the top-mm rows by logit or the mm nearest neighbours in feature space, with m=32m=32 or m=2m=2% of nn. Note that this is merely a proxy given that we never have access to RR or TT.

  • Rows in RR. FF is non-zero on RR in 51–84% of queries, but the mean penalty is 0.001–0.09 on RR vs. 0.46 on TT. When FF is non-constant on RR, it is larger on the lower-logit rows in 79–85% of queries, which reduces the coverage in RR (ρR\rho_R, Eq. 3).
  • Rows in TT. TT loses a larger share of its attention mass than RR in 85–91% of queries, so MT/MRM_T/M_R shrinks.

The theory is only meant to give intuition for why key-dependent suppression avoids the temperature trade-off, but it does not guarantee that the mechanism works as in the simplified setting.


We would be grateful if you would consider updating your score after these clarifications and additions. Please let us know if you have any remaining questions.

I thank the authors for their response. My overall assessment of the paper remains positive, and I will maintain my original score.

Reviewer_uWhiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Tabular Selective Attention (TSA) to address attention fading in tabular foundation models. Unlike temperature scaling, TSA learns query-key-dependent suppression scores to selectively reduce attention to potentially uninformative training rows. The paper also introduces a suppression-guided row pruning strategy to improve inference efficiency. Experiments on TabICLv2 and TabDPT demonstrate promising improvements on TabArena classification benchmarks.

Critical strengths

TSA provides a simple approach to selectively suppressing uninformative rows without uniformly sharpening the attention distribution. The theoretical analysis offers useful insights into the limitations of temperature scaling under the stated assumptions.

The authors evaluate TSA on two different TFM architectures and provide analyses of attention behavior, downstream performance, and pruning efficiency. The results suggest that TSA can improve predictive performance over existing attention variants.

Critical weaknesses

Although TSA outperforms the temperature-scaling variants, its improvement over vanilla attention is relatively modest (14.6% vs. 15.3% improvability), and their Elo confidence intervals overlap. Meanwhile, inference time increases from 0.36 to 0.49 seconds per 1K samples. This raises questions about whether the performance gains justify the additional complexity. Furthermore, an ablation against a parameter-matched learnable attention mechanism would help establish the specific benefit of selective suppression.

The proposed pruning strategy ranks rows according to their suppression scores. However, a larger suppression score does not necessarily imply a smaller final attention weight, since the original attention logits also determine each row's importance. The bound in Eq 10 depends on the original logit gap, which is not explicitly considered in the pruning criterion. Moreover, the theoretical guarantees assume that informative rows receive no suppression, which is not enforced by the proposed mechanism. Therefore, the effectiveness of suppression scores as indicators of row importance requires further validation.

Questions for authors

Can the authors compare TSA with a parameter-matched learnable attention bias or other content-dependent attention mechanisms to isolate the contribution of non-negative selective suppression?

Can the authors compare suppression-guided pruning against pruning based on actual attention weights or random pruning under the same computational budget? This would help establish whether the learned suppression scores provide a reliable measure of row importance.

3 comments
AuthorsOpenReview ↗

Response to reviewer uWhi

We appreciate the reviewer's effort and thoughtful comments.

W1 (Benchmarks)

Aggregate TabArena metrics (e.g. ELO) rank many methods at once and hide the two-way comparison, so we test TSA against vanilla attention directly:

  1. TabArena: TSA beats vanilla on 25/38 datasets, and these cover TabArena's full size range (22 with <10K rows, 14 with 10K–100K, 2 with ≥100K). Since attention dilution grows with the number of training rows, we also look at the 10 datasets with >10K training rows: TSA wins 9/10, with 3% lower error (Holm p=0.012p=0.012; error =1−=1-ROC AUC for binary and log-loss for multiclass)
  2. TALENT (TabICLv2): On all 40 classification datasets with 10K–162K training rows, TSA beats vanilla on all metrics.
Metricvs. vanillaWilcoxon pRel. error change
Accuracy66.3%0.015−0.77%
Balanced accuracy67.5%0.021−0.81%
ROC-AUC75.0%0.0014−1.07%
Log-loss75.0%0.0012−0.44%
  1. Overhead; It is a constant factor (noted as a limitation in the conclusion). The suppression score has the same O(n m d)\mathcal{O}(n\,m\,d) cost as the attention logits and adds only 1.46% parameters. The overhead comes from computing FF which cannot be fused into a single flash-attention pass.

W2 (On Fij=0F_{ij} = 0 in RR)

Eq. 10 as written is the special case Fim=0F_{im}=0. In general, aijaim=e−(Fij−Fim) e−(zim−zij),\frac{a_{ij}}{a_{im}} = e^{-(F_{ij}-F_{im})}\,e^{-(z_{im}-z_{ij})}, so FijF_{ij} alone does not determine aija_{ij} as the logit gap enters equally. When F≠0F\neq0 on RR:

  • Coverage. Let ϕmax⁡,ϕmin⁡\phi_{\max},\phi_{\min} be the largest and smallest FijF_{ij} over j∈Rj\in R. Coverage shrinks by at most e−(ϕmax⁡−ϕmin⁡)e^{-(\phi_{\max}-\phi_{\min})}, so a penalty that is the same on all informative rows costs nothing.
  • Suppression. MT/MRM_T/M_R is multiplied by (fraction of TT's attention mass kept) / (fraction of RR's mass kept). The tail only needs to lose a larger share of its mass than RR, not F=0F=0 on RR.

We checked this on all 38 TabArena datasets (3 splits). RR is not accesible, so we take either the top-mm rows by logit or the mm nearest neighbours in feature space, with m=32m=32 or m=2m=2% of nn.

  • Rows in RR. FF is non-zero on RR in 51–84% of queries, but the mean penalty is 0.001–0.09 on RR vs. 0.46 on TT. When FF varies on RR, it is larger on the lower-logit rows in 79–85% of queries, which reduces the coverage in RR (ρR\rho_R, Eq. 3).
  • Rows in TT. TT loses a larger share of its attention mass than RR in 85–91% of queries, so MT/MRM_T/M_R shrinks.

As the paper claims throughout, the theory is only meant to provide intuition for how TSA works when R and T are available.

Q1 (Parameter-Matched)

The natural parameter-matched alternative is a learnable bilinear bias on the test-to-train logits without the non-negativity constraint: softmax(QK⊤dk−QWK⊤dk)=softmax(Q(I−W)Ktrain⊤dk).\mathrm{softmax}\left(\tfrac{QK^\top}{\sqrt{d_k}}-\tfrac{QWK^\top}{\sqrt{d_k}}\right)=\mathrm{softmax}\left(\tfrac{Q(I-W)K_{\text{train}}^\top}{\sqrt{d_k}}\right). This signed bias is vanilla attention with test queries projected by WQ(I−W)W_Q(I-W) instead of WQW_Q, i.e. a reparameterization of the query projection. It includes uniform logit scaling (SSMax-like) as the special case W=(1−c)IW=(1-c)I, where cc is the scaling factor applied to all logits.

Note that TSA's ReLU instead makes the penalty non-negative and non-linear, so it can suppress individual rows selectively, which a signed linear bias cannot. We are currently training a Stage 1 version of the non-ReLU approach and will provide the results when complete.

Q2 (Post-suppression scores)

The score sjs_j indeed ignores the logit gap: it is a proxy for the post-suppression attention aija_{ij} (the softmax of z−Fz-F). We compared pruning by sjs_j against aija_{ij} and the pre-suppression attention (the softmax of zz), using the same schedule and budget (38 datasets × 5 splits) and global aggregation (Eq. 11).

Change in ROC-AUC (pp) vs. no dropping

Rows droppedSuppression onlyPost-suppression attentionPre-suppression attention
25%−0.12−0.08−0.08
50%−0.38−0.37−0.38
75%−0.68−1.39−1.83
  • Accuracy. Up to 50% dropped, the attention weights rank rows marginally better. At 75%, attention-based ranking collapses on several datasets, and sjs_j is the most robust.
  • Cost. At 75% dropped, ranking by sjs_j gives a 1.09×/1.12× wall-clock speedup on medium/large datasets, while ranking by aija_{ij} needs an extra attention pass and is slower than not pruning at all (0.49–0.78×).
  • F≈0F\approx0 on informative rows (see W2), so Fij−Fim≈FijF_{ij}-F_{im}\approx F_{ij} in Eq. 10. Post-suppression attention beats pre-suppression attention at 75%, so FF carries information beyond the logit gap.

We would be grateful if you would consider updating your score after these clarifications and additions. Please let us know if you have any remaining questions.

Thank you for your response and clarifications. I will adjust my rating accordingly.

Out of curiosity, I have one additional question: could you share how the GPU memory consumption of TSA, its variants, and the vanilla baseline compares across different datasets? This question will not affect my rating; I am simply interested in gaining a deeper understanding of the proposed method. A few months ago, I experimented with a similar implementation but observed relatively high GPU memory consumption and no substantial performance improvements in the later stages of training. However, these were only preliminary explorations without rigorous ablation studies, so I would be interested in hearing about your experience.

Thank you for your interest and for considering adjusting the rating!

We profiled peak GPU memory during inference on the TabArena classification datasets, using the default inference settings (AMP; FlashAttention-3 wherever the attention allows it).

Datasettrain / test rowsfeaturesVanilla/SSMax/QASSMax (GiB)TSA (GiB)Ratio
Bank_Customer_Churn6.7K / 3.3K100.570.871.53
HR_Analytics12.8K / 6.4K121.071.531.43
Amazon_employee_access21.8K / 10.9K91.432.501.74
bank-marketing30.1K / 15.1K132.483.391.37
SDSS1752.0K / 26.0K113.705.581.51
GiveMeSomeCredit100K / 50K106.5210.571.62
APSFailure50.7K / 25.3K17040.3040.301.00
kddcup09_appetency33.3K / 16.7K21232.9432.941.00
  • SSMax and QASSMax use the same memory as vanilla (≤0.01 GiB difference on every dataset), because both only rescale queries and keep the fused FlashAttention path.
  • TSA's suppression term FF cannot be fused into FlashAttention. To bound its memory, we process test queries in chunks of 2,048, so the overhead does not grow with the number of test rows.
  • Most datasets see no difference. The median TSA/vanilla ratio is 1.00 (the maximum is 1.74), and 25 of the 30 datasets not shown are within ±1%. TSA raises the peak only on narrow tables with many test rows. On wide tables, the column embedder, which is identical in all variants, sets the peak.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.