- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents QCUR, a method for selecting query-specific evidence for tabular foundation models (TFMs) in long-tailed classification tasks. At inference time, QCUR uses a learned router to select a single labeled evidence example from a candidate pool to append to the TFM’s base context, aiming to improve the prediction for the current query. The theoretical analysis characterizes the regret of query-independent evidence selection relative to query-wise optimal selection and bounds the routing regret in terms of utility-estimation error; it does not directly prove improved classification performance. Experiments across 9 OpenML datasets, 3 frozen TFM backbones, and 3 random seeds show improvements in predictive metrics over the backbone-only model and four baseline methods adapted to the same single-evidence selection setting.
Critical strengths
- Addresses a challenging problem: Tackles query-specific evidence selection for in-context learning with tabular foundation models (TFMs), particularly in long-tailed classification.
- Clear presentation: The paper is well organized, with a clearly motivated problem, an understandable method description, and a coherent presentation of the theoretical analysis and experiments.
- Combines theoretical analysis with empirical evaluation: Provides regret bounds that formalize the limitations of query-independent evidence selection, and evaluates QCUR across 9 OpenML datasets, 3 TFM backbones, and multiple baselines adapted to the same single-evidence setting.
- Practical inference-time adaptation: Improves predictions by selecting evidence examples without updating the underlying TFM par
Critical weaknesses
- Limited scope of single-evidence routing and unclear practical motivation.
-
QCUR selects a single evidence example for each query, whereas several existing context-selection methods are designed to select sets of examples. For instance, LUCoS evaluates selection across varying context budgets, VIP-COp optimizes fixed-size contexts, and LoCalPFN uses query-local retrieval with multiple neighbors. Restricting the evaluation to a single evidence example limits the conclusions that can be drawn about QCUR's effectiveness for general in-context learning. The paper should better motivate when a single-example context is practically important and evaluate whether its benefits extend to selecting multiple examples.
-
Furthermore, the evaluation focuses on long-tailed classification, but the paper does not establish whether the proposed approach is particularly beneficial in this setting or whether it generalizes to other classification regimes. The paper should better motivate the single-example setting and evaluate whether the benefits extend to multi-example contexts and other task settings.
-
Lack of cross-task and cross-backbone generalization evaluation. The router is trained separately for each dataset–backbone–seed combination, so the experiments do not establish whether it generalizes to unseen tasks or transfers across different TFM architectures without task-specific training. Although the evaluation covers nine datasets and three backbones, this demonstrates performance across multiple settings rather than out-of-distribution generalization. It remains unclear whether a router trained on one set of datasets or TFM backbones can effectively select evidence for unseen datasets or a different backbone without retraining.
-
Limited scope of the theoretical guarantees. Although the paper provides regret bounds for query-dependent evidence selection and utility estimation, the guarantees are defined in terms of probability-based utility rather than downstream classification performance. Consequently, the theoretical results do not guarantee improvements in predictive metrics, even when the estimated utility is accurate.
Questions for authors
- How can QCUR be extended to select multiple evidence examples, accounting for interactions between examples rather than evaluating each candidate independently?
- Can the learned router generalize to unseen datasets or TFM backbones without requiring task-specific training on labeled proxy queries? Could a single router be trained across multiple datasets and backbones and then transferred to new tasks?
- How does the computational cost of QCUR scale with the size of the candidate pool? Could the authors provide a more detailed scalability analysis, including settings with larger candidate pools?
Following your suggestion to examine whether more auxiliary examples improve prediction, we further extended the joint-insertion experiment to 10 additional examples, covering the same four datasets and three seeds (all 12 diagnostic states).
Global top-10 uses training-side candidate scores; random-10 averages ten fixed draws per intervention unit. All ten examples are inserted jointly into the existing context and evaluated through actual backbone inference. We kept the previously trained QCUR routers fixed and re-evaluated QCUR under the same test-time inference configuration as the new ten-example comparison.
Increasing the insertion budget to ten examples still does not surpass QCUR's single query-conditioned insertion under either evaluated policy. QCUR improves Accuracy over global top-10 by 2.07 percentage points (descriptive dataset-clustered 95% bootstrap interval: [0.41, 3.54]) and over random-10 by 2.70 points (interval: [1.52, 3.88]). It wins in 10/12 states against each policy and also improves mean Macro-F1, WCR, and NLL.
This larger-budget experiment reinforces the practical value of the one-item decision studied in the paper: selecting evidence for the receiving query can be more effective than inserting ten globally selected or random examples. The evidence budget alone does not determine the predictive benefit; which evidence is supplied to each query remains consequential.