acceptodds
Under review as a conference paper at ICLR 2027

Evidence Is Not Exchangeable: Query-Conditioned Evidence Routing for Tabular Foundation Models

Abstract

Tabular foundation models predict from labeled examples placed in their context. In long-tailed classification, an additional labeled example has no fixed value: the same evidence can repair one query and interfere with another. Evidence selection is therefore a query-conditioned decision, yet existing selection rules often assign candidates a shared value or retrieve them by representation similarity. We formalize evidence value as a query-candidate utility U(q,e), whose query-specific rankings create regret for global selection. QCUR estimates this utility for a frozen backbone from the query representation and the candidate-induced posterior response, then routes one additional example to each query. Across three tabular foundation-model backbones, nine datasets, and three random seeds, QCUR yields a 4.5% relative improvement in mean Accuracy over backbone-only inference, together with a 4.3% relative improvement in Macro-F1 and a 1.1% relative improvement in Macro ROC-AUC. QCUR also exceeds the strongest protocol-matched context-selection baseline by a 3.1% relative improvement in mean Accuracy under the same one-evidence budget. Query-specific ranking inversions, measurable global-selection regret, and repair-dominant prediction changes show that evidence utility varies across queries. Query-conditioned evidence valuation provides a portable principle for improving frozen tabular foundation models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Tabular foundation models, in-context learning, context selection, long-tailed classification, query-conditioned routing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LKaKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents QCUR, a method for selecting query-specific evidence for tabular foundation models (TFMs) in long-tailed classification tasks. At inference time, QCUR uses a learned router to select a single labeled evidence example from a candidate pool to append to the TFM’s base context, aiming to improve the prediction for the current query. The theoretical analysis characterizes the regret of query-independent evidence selection relative to query-wise optimal selection and bounds the routing regret in terms of utility-estimation error; it does not directly prove improved classification performance. Experiments across 9 OpenML datasets, 3 frozen TFM backbones, and 3 random seeds show improvements in predictive metrics over the backbone-only model and four baseline methods adapted to the same single-evidence selection setting.

Critical strengths
  1. Addresses a challenging problem: Tackles query-specific evidence selection for in-context learning with tabular foundation models (TFMs), particularly in long-tailed classification.
  2. Clear presentation: The paper is well organized, with a clearly motivated problem, an understandable method description, and a coherent presentation of the theoretical analysis and experiments.
  3. Combines theoretical analysis with empirical evaluation: Provides regret bounds that formalize the limitations of query-independent evidence selection, and evaluates QCUR across 9 OpenML datasets, 3 TFM backbones, and multiple baselines adapted to the same single-evidence setting.
  4. Practical inference-time adaptation: Improves predictions by selecting evidence examples without updating the underlying TFM par
Critical weaknesses
  1. Limited scope of single-evidence routing and unclear practical motivation.
  • QCUR selects a single evidence example for each query, whereas several existing context-selection methods are designed to select sets of examples. For instance, LUCoS evaluates selection across varying context budgets, VIP-COp optimizes fixed-size contexts, and LoCalPFN uses query-local retrieval with multiple neighbors. Restricting the evaluation to a single evidence example limits the conclusions that can be drawn about QCUR's effectiveness for general in-context learning. The paper should better motivate when a single-example context is practically important and evaluate whether its benefits extend to selecting multiple examples.

  • Furthermore, the evaluation focuses on long-tailed classification, but the paper does not establish whether the proposed approach is particularly beneficial in this setting or whether it generalizes to other classification regimes. The paper should better motivate the single-example setting and evaluate whether the benefits extend to multi-example contexts and other task settings.

  1. Lack of cross-task and cross-backbone generalization evaluation. The router is trained separately for each dataset–backbone–seed combination, so the experiments do not establish whether it generalizes to unseen tasks or transfers across different TFM architectures without task-specific training. Although the evaluation covers nine datasets and three backbones, this demonstrates performance across multiple settings rather than out-of-distribution generalization. It remains unclear whether a router trained on one set of datasets or TFM backbones can effectively select evidence for unseen datasets or a different backbone without retraining.

  2. Limited scope of the theoretical guarantees. Although the paper provides regret bounds for query-dependent evidence selection and utility estimation, the guarantees are defined in terms of probability-based utility rather than downstream classification performance. Consequently, the theoretical results do not guarantee improvements in predictive metrics, even when the estimated utility is accurate.

Questions for authors
  1. How can QCUR be extended to select multiple evidence examples, accounting for interactions between examples rather than evaluating each candidate independently?
  2. Can the learned router generalize to unseen datasets or TFM backbones without requiring task-specific training on labeled proxy queries? Could a single router be trained across multiple datasets and backbones and then transferred to new tasks?
  3. How does the computational cost of QCUR scale with the size of the candidate pool? Could the authors provide a more detailed scalability analysis, including settings with larger candidate pools?
Reviewer_aNbxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the authors study the evidence selection problem in long-tailed classification under the tabular foundation model (TFM) framework. Specifically, building on frozen TFMs, the authors propose constructing several evidence selection representations and then using a learnable evidence selection module to learn the utility function for evidence selection. Experiments on real-world datasets demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The proposed method is reasonable and sound.
  2. The proposed method shows good empirical performance in the experiments.
Critical weaknesses
  1. The paper is poorly written, and many parts are difficult to follow. I give examples below.

    • There should be a clear definition of the problem formulation and the notation used in the paper. This would make the wording and notation in the paper much clearer.
    • Some notation is introduced but not clearly defined. In line 74, neither qq nor yqy_q has been defined previously. In Equation (6), Sproxy\mathcal{S}_{proxy} is also undefined.
    • In Equation (3), the authors propose changing the posterior by utilizing the empirical prior. It is not clear why this correction is needed.
    • In Section 3.2, the authors introduce a set of representations but provide only a very brief description of them. It is not clear exactly how each part is calculated or why the authors choose to use these representations.
    • In Section 5.3, does the ablation study correspond to the representations in Section 3.2? If so, what are the detailed configurations of each variant (e.g., global-only, query-only, etc.)?
  2. From my understanding of the paper, the authors want to add one auxiliary piece of evidence for each query sample. However, in practice, I would expect that adding more than one auxiliary piece of evidence could help the model make better predictions. As a result, I think the problem formulation in this paper is not very practical. I would expect the authors to provide some discussion of this issue.

Questions for authors

See the weakness part.

3 comments

Following your suggestion to examine whether more auxiliary examples improve prediction, we further extended the joint-insertion experiment to 10 additional examples, covering the same four datasets and three seeds (all 12 diagnostic states).

Global top-10 uses training-side candidate scores; random-10 averages ten fixed draws per intervention unit. All ten examples are inserted jointly into the existing context and evaluated through actual backbone inference. We kept the previously trained QCUR routers fixed and re-evaluated QCUR under the same test-time inference configuration as the new ten-example comparison.

PolicyAdditional examplesAccuracy ↑Macro-F1 ↑WCR ↑NLL ↓
Global top-10100.80140.79770.56300.5474
Random-10100.79510.79020.55050.5589
QCUR10.82210.81720.61680.5045

Increasing the insertion budget to ten examples still does not surpass QCUR's single query-conditioned insertion under either evaluated policy. QCUR improves Accuracy over global top-10 by 2.07 percentage points (descriptive dataset-clustered 95% bootstrap interval: [0.41, 3.54]) and over random-10 by 2.70 points (interval: [1.52, 3.88]). It wins in 10/12 states against each policy and also improves mean Macro-F1, WCR, and NLL.

This larger-budget experiment reinforces the practical value of the one-item decision studied in the paper: selecting evidence for the receiving query can be more effective than inserting ten globally selected or random examples. The evidence budget alone does not determine the predictive benefit; which evidence is supplied to each query remains consequential.

I thank the authors for their response. After considering the rebuttal, I will raise my score to 3.

AuthorsOpenReview ↗

Thank You for Your Feedback

Thank you for your thoughtful feedback and for the follow-up questions during the discussion. In response, we have added the main results from our follow-up experiments to Section 5.6 of the revised manuscript (Tables 5–8). These results compare QCUR with other learned selectors under the same supervision and candidate probe budget, explain where its target-class recall gains come from, compare single-example routing with the joint insertion of up to ten examples, and evaluate router transfer between two frozen backbones.

We will include the remaining experimental results and fuller protocol details in the supplementary material. We appreciate the opportunity to clarify when QCUR offers a practical advantage, and we hope these additions address the questions raised during the discussion.

Reviewer_CL4zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to choose additional context examples for frozen tabular foundation models. Its main point is that the same example can help one query but hurt another. A single global score may therefore miss useful differences between queries.

The proposed method, QCUR, learns to predict the change in true-class probability caused by adding a candidate example. It uses the query representation, candidate-induced posterior responses, and a global candidate prior. The router is trained on labeled proxy queries. At test time, it probes every candidate and selects one example for each query.

The evaluation covers three backbones, nine datasets, and three seeds. Mean Accuracy rises from 0.7386 to 0.7719. The paper also provides feature ablations, ranking-reversal analysis, utility-regret measurements, and runtime results.

Critical strengths

1. The performance gains are meaningful and cover several backbones.

QCUR improves mean Accuracy by 3.33 percentage points over the frozen backbone and by 2.35 points over the strongest external adaptation, VIP-COp. The paper reports gains across most backbone–dataset cells. This makes the result more convincing than a gain on one model or dataset.

2. The paper makes a serious effort to explain the gains.

The ranking-reversal and fixed-selection regret analyses support the motivation. In Table 2, Full QCUR reaches 0.8187 Accuracy, compared with 0.8032 for Functional-only and 0.7859 for Globalized QCUR. These results support the value of combining query states with candidate responses.

The same-probe heuristics and mean-posterior ensemble are useful controls too. The paper also includes candidate probes in its runtime accounting. These details strengthen the experimental case.

Critical weaknesses

1. The claimed conceptual difference from earlier work is too strong.

Section 2 says QCUR changes the selection target from semantic relevance to the signed posterior effect. However, Rubin et al. (2022), Learning To Retrieve Prompts for In-Context Learning, already use the model’s probability of the correct output to supervise query-specific candidate selection. That method is not based only on semantic similarity.

For a fixed query and base context, subtracting the base true-class probability does not change the candidate ranking:

arg⁡max⁡epe(q)yq.\arg\max_e p_e(q)_{y_q}.

The signed target may still help regression and the fallback decision. But it does not, by itself, define a new ranking rule.

Please make the distinction from this earlier line of work more precise. The stronger contribution seems to be the use of candidate posterior responses as router inputs in frozen TFMs. A direct comparison with a model-feedback-trained selector would help establish this contribution.

2. The current evidence does not fully separate utility routing from other uses of the same supervision and predictions.

QCUR is trained separately for each dataset–backbone–seed state. It uses hundreds to thousands of labeled proxy queries and probes all candidates at test time. Matching the final one-example budget does not by itself match these resources.

The existing ablations and same-probe controls are helpful. Still, a stronger comparison would use the same proxy labels, query features, and candidate predictions to train a selector or a supervised posterior combiner. This would test whether the proposed utility model adds value beyond general task-specific learning.

Several limits also need clearer evidence:

a. The provided paper does not include the supplement. Exact context sizes, long-tail ratios, tail-class rules, and complete split and overlap rules cannot be checked. b. The main mechanism ablations cover only four datasets on TabPFN-v2. They provide less support for a shared mechanism across all three backbones. c. Response shuffling also breaks relations between posterior features, as the paper notes. It is therefore not a fully clean test of query–response matching. d. Higher true-class probability does not always mean a correct prediction. TabDPT’s worst-class recall falls from 0.4910 to 0.4856, despite higher Accuracy.

Please provide the full setup, strengthen the supervised comparisons, and keep the conclusions within the tested scope.

Questions for authors

Clarification requests

  1. What is the exact difference from model-feedback-trained retrieval, such as Rubin et al. (2022), beyond using TFMs, posterior-response features, and continuous utility regression? Which result best shows the value of that difference?

  2. Could you provide the full per-dataset setup? This should include context sizes, long-tail construction, tail-class selection, proxy and test sizes, and split and overlap rules. The supplement is referenced but is absent from the provided file.

  3. How closely does lower utility regret track better classification across experimental states? Please explain cases where utility improves but Accuracy or worst-class recall gets worse. How often does the global fallback select a harmful candidate?

Evidence that could change my rating

  1. Can QCUR retain its advantage against a learned selector or posterior combiner using the same proxy labels and candidate probes? Please also separate proxy-data generation, router training, and test-time costs. A matched comparison here would directly address my main concern.

For further improvement, I suggest testing smaller proxy budgets and repeating the key feature ablations on the other backbones. A native-posterior versus fixed-prior versus updated-prior comparison would also help separate the model response from the prior correction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.