acceptodds
Under review as a conference paper at ICLR 2027

Universal In-Context Learning over Foundation Model Embeddings

Abstract

Foundation-model embeddings are widely used across many domains, yet using them for prediction usually requires training a separate classifier for each dataset. This adds engineering work, makes results sensitive to hyperparameter choices, and increases the risk of overfitting when labeled data are scarce. It also means that embedding benchmarks measure classifier optimization as well as representation quality. Meanwhile, tabular foundation models have shown that transformers can classify arbitrary tabular data in context, in a single forward pass and without task-level training. We adapt a tabular foundation model into a single in-context predictor that works across embedding spaces. Instead of training a separate head for each task, we meta-train one shared model across tasks and encoders. At every training step, we apply a freshly sampled random orthogonal projection that changes the coordinate basis of the embeddings while preserving their dimension and geometry, which prevents the model from relying on any fixed coordinate orientation. At inference, the model averages its predictions over multiple random orthogonal projections of the same inputs and requires no parameter updates. We curate a benchmark spanning 7 domains, 29 encoders, 75 tasks, 308 task–encoder pairs, and 169,554 samples, and evaluate it under in-distribution, novel-task, novel-encoder, and joint task–encoder transfer. Averaged over all 308 pairs, our model reaches 0.8886 ROC-AUC, improving on the strongest baseline by 0.0198 (95% CI ) without a single gradient step at inference. We release all code, benchmark data, and trained model checkpoints.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
In-Context Learning, Foundation Models, Cross-Domain Transfer, Tabular Transformers, Multimodal Representations

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_sUQROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a single in-context predictor for classification over foundation-model embeddings: it adapts the tabular foundation model TabICL with shared LoRA adapters trained across many task–encoder pairs. During training, each step applies a fresh random orthogonal rotation to the embeddings, which keeps their geometry but removes any fixed coordinate system, which helps generalization across unseen tasks. At inference, the model classifies new tasks and encoders with no gradient updates by averaging predictions over several random rotations. On a new benchmark of 7 domains, 75 tasks, 29 encoders and 308 task–encoder pairs, it outperforms the evaluated baselines.

Critical strengths

The problem is clearly stated, and the underlying scientific question—Can we build a task predictor that is agnostic to the input modality?—is both relevant and well motivated. The use of tabular foundation models seems to be a promising approach to addressing this question.

The use of random projections is interesting, providing a clear benefit in terms of improving the model's ability to generalize across different input modalities.

Compared to the evaluated baselines, the proposed method appears to achieve noticeable performance improvements across multiple modalities, although these results should be interpreted with some caution (see weaknesses below).

Critical weaknesses

Lack of comparison with meta-learning approaches. The proposed approach is closely related to meta-learning, where a model is trained episodically on support and query sets, with the objective of predicting the query set given only the support set at inference time. However, none of the baselines considered in the experimental evaluation appears to follow a similar episodic training procedure, potentially making the comparison unfair.

Lack of ablation on cross-modal training. To properly assess the benefits of learning from multiple modalities, I believe it would be important to compare the proposed model against versions trained on individual modalities. Such an ablation would help showing whether cross-modal training actually improves performance over modality-specific training, and to what extent the different modalities contribute to the model's generalization capabilities.

Additional comments and suggestions:

  • Although potentially computationally expensive, it would be interesting to investigate how much each modality or domain contributes to performance on the others. For instance, one might expect training on protein-related tasks to benefit downstream other biological tasks more than training on sentiment analysis tasks.
  • I believe the paper The Platonic Representation Hypothesis (2024) could be interesting to discuss, as it investigates the extent to which representations learned from different modalities converge toward a shared representation of reality. This perspective could help motivate why learning task predictors across modalities might be beneficial.
  • Minor presentation issue: The organization of Section 5 seems uneven. Most subsections consist of approximately one paragraph, whereas Section 5.4 spans an entire page. (this is more to help increase the paper's readability, but it is a very minor weakness)
Questions for authors
  1. Did the authors consider including meta-learning approaches as baselines? If so, could they clarify why these methods were excluded from the experimental evaluation?

  2. Do the authors have any experimental results or analyses that directly assess the benefits of training on multiple modalities within their proposed framework? In particular, how does the performance of the proposed model compare to that of a counterpart trained on a single modality?

2 comments

Thank you for identifying these two gaps. We conducted additional experiments addressing both: episodically trained meta-learning baselines and a comparison between joint and single-domain training. We also evaluated transfer to entirely held-out domains.

W1 / Q1. Comparison with meta-learning approaches

We agree that comparison with other episodically trained predictors strengthens the evaluation. We therefore added adapted implementations of GPICL (Kirsch et al., 2022) and CAML (Fifty et al., 2024), two relevant transformer-based meta-learning approaches that learn from labeled context examples without parameter updates at inference. Both are trained on the same meta-training task–encoder pairs, use the same data splits, and follow the same checkpoint-selection criterion as our method.

MethodMean ROC-AUC
GPICL0.8687
CAML0.8679
Our method0.8886

To accommodate their fixed input width, we map embeddings to 512 dimensions using random linear projections. Random projection has precedent as dimensionality-reduction preprocessing for tabular in-context predictors, including TabFlex (Zeng et al., 2025). We configure both output heads to support up to 10 classes, covering all tasks in our benchmark without changing their labels. These adaptations enable evaluation across the same heterogeneous task–encoder pairs; our method directly accepts each encoder’s original dimensionality.

A same-backbone control further addresses episodic training. In Table 4, TabICL with LoRA trained on the same episodes using raw embeddings achieves 0.8822 overall and 0.8690 on unseen tasks and/or encoders, versus 0.8886 and 0.8879 for ours. Both average 16 inference views—feature permutations for the raw-trained model and orthogonal transformations for ours—so this comparison supports the combined training-and-inference procedure.

W2 / Q2. How does training across domains affect performance and transfer?

We compare three training settings: only the evaluation domain, all other domains, and all seven domains. The first tests domain specialization; the second tests transfer to a domain excluded from meta-training; and the third evaluates joint training. These experiments use a single matched task–encoder partition seed, the same training recipe, maximum optimization budget, and early-stopping procedure. (Table 3 averages three runs, so the all-domain reference scores is a bit differ between the two evaluations).

Evaluation domainOnly this domainAll other domainsAll domains
Audio0.90380.91900.9188
Graphs0.78740.79520.7931
Molecules0.84500.85550.8642
Proteins0.89330.89040.8945
Text0.89470.89470.8948
Time series0.96170.96320.9632
Vision0.90510.89850.9045

Joint training provides one shared predictor with competitive performance across domains. Compared with single-domain training, its largest observed gains are in molecules and audio. The graph-domain average also improves, while differences in the remaining four domains are at most 0.0015. Benefits are not uniform: on the held-out graph-task subset, the graph specialist performs better.

Most of this performance transfers even when the evaluation domain is excluded from meta-training. Across all 308 pairs, the leave-one-domain-out models achieve 0.8854, compared with 0.8883 for the matched all-domain model. They outperform frozen TabICL and the tuned MLP by point estimate in every domain.

To investigate which source domains transfer well, we also evaluated each specialist across all seven domains. Transfer is not confined to related domains: on molecules, protein-only, text-only, and time-series-only training achieve 0.8570, 0.8557, and 0.8577, respectively, compared with 0.8450 for molecule-only training. We will include the complete 7×7 transfer matrix in the revision. This matrix characterizes transfer from individual source domains; it does not isolate their marginal contributions within joint training.

Related work and presentation. Thank you for suggesting The Platonic Representation Hypothesis. We will discuss its connection to our motivation while distinguishing representation alignment from our finding that a shared predictor can transfer across embedding spaces. We will also reorganize Section 5.4 into shorter thematic subsections to improve readability.

As the discussion period closes on November 18, we would welcome your feedback on our response. We would be happy to clarify any points or address any remaining questions or concerns.

Thank you for your time and consideration.

Reviewer_hJpCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies in-context classification across heterogeneous foundation-model embedding spaces. Starting from pretrained TabICL, the authors meta-train shared LoRA parameters across multiple tasks and encoders. They apply freshly sampled orthogonal transformations to support and query embeddings during training and average predictions across transformed views at inference. The resulting predictor handles new task without task-specific parameter updates. Evaluation covers seven domains, 29 encoders, and 75 tasks, including settings where tasks, encoders, or both are absent from meta-training.

Critical strengths
  1. The paper is generally easy to follow. The problem formulation, training procedure, and inference strategy are clearly described.

  2. The evaluation spans diverse domains, encoders, and tasks. Separating generalization to unseen tasks, unseen encoders, and both provides a useful assessment of transfer.

  3. The use of orthogonal transformations is an interesting design. It changes the embedding coordinate system during training while preserving dimensionality and pairwise geometry, encouraging the predictor to rely less on encoder-specific coordinates.

Critical weaknesses
  1. The practical need for the proposed method is not sufficiently established. It improves mean ROC-AUC over a lightweight MLP by approximately 2%, while introducing a more elaborate training and inference pipeline. Inference cost is particularly concerning: the strongest 16-view configuration averages 23 seconds per task, raising questions about its efficiency relative to a simple MLP.

  2. The authors report positive average advantages on 54 of 75 tasks, while improvements for audio and time series remain statistically inconclusive. They acknowledge this variation but provide limited analysis of its causes.

  3. Orthogonal basis randomization is a central contribution of the paper, yet Table 4 shows that the Gaussian variant achieves 0.8831 ROC-AUC, compared with 0.8886 for the orthogonal variant, a gain of only 0.0055. Given these similar results, why is explicitly orthogonalizing the sampled matrix necessary instead of using Gaussian transformations directly?

Questions for authors
  1. Could you compare ROC-AUC, runtime, and memory usage against MLP under comparable computational budgets, accounting for both per-task fitting/tuning and prediction?

  2. What distinguishes tasks where the method improves performance from those where it underperforms? In particular, can you explain the inconclusive gains in audio and time series through analyses of support-set size, embedding properties, or baseline performance?

2 comments

Thank you for the careful review. We address the practical use case, computational cost, variation across tasks, and the role of orthogonalization below.

W1 / Q1. Practical value and computational cost

The table below compares predictive performance and elapsed time, including the MLP’s fitting and hyperparameter search. Each time is the mean time to process one task–encoder pair’s entire test set, not one prediction. Our method uses the full training split as context and runs on one NVIDIA A40 GPU; the MLP runs on 16 CPU cores.

MethodMean time per pairROC-AUCPeak memory
Ours, 1 view1.46 s0.87186.69 GB GPU
Ours, 4 views5.9 s0.88346.69 GB GPU
Ours, 16 views23.0 s0.88866.69 GB GPU
MLP, default parameters0.33 s0.84520.79 GB RAM
MLP, 20-trial search7.2 s0.85230.79 GB RAM
MLP, 100-trial search23.0 s0.86890.79 GB RAM

A single view achieves a similar observed mean ROC-AUC to the 100-trial MLP search (0.8718 vs. 0.8689), with elapsed times of 1.46 and 23.0 seconds, respectively. We do not interpret the small AUC difference of 0.0029 as established superiority. Increasing to four or 16 views raises mean ROC-AUC to 0.8834 or 0.8886.

Shared meta-training additionally costs 1.1–1.7 GPU-hours, separately from the per-pair times above.

We also agree that, once fitted, the MLP predicts in milliseconds and is substantially faster for inference alone. Our method’s practical advantage is therefore strongest when obtaining predictions for new task–encoder pairs without repeated fitting and tuning; an already-trained MLP remains attractive for repeated, low-latency prediction on a fixed pair.

W2 / Q2. Which tasks benefit?

The largest observed gains occur on pairs where the MLP has lower ROC-AUC. Grouping all 308 task–encoder pairs by MLP performance gives:

MLP ROC-AUC rangePairsMLPOursGain [95% CI]
0.983–1.000620.99350.9934−0.0001 [−0.001, +0.001]
0.943–0.982610.96590.9700+0.004 [0.001, 0.007]
0.872–0.943620.90640.9231+0.017 [0.011, 0.024]
0.762–0.871610.81020.8305+0.020 [0.011, 0.030]
0.481–0.759620.66890.7267+0.058 [0.033, 0.087]

For time series, 22 of 36 pairs fall in the highest-performing group, where the MLP already achieves near-perfect ROC-AUC. Limited headroom may therefore contribute to the small aggregate improvement.

For audio, ceiling effects do not provide a sufficient explanation. Among audio pairs with MLP ROC-AUC below 0.95, the gain remains inconclusive: +0.005 [−0.008, 0.013]. Audio also contains the fewest tasks and pairs (7 tasks, 28 pairs), limiting the evidence available for identifying systematic gains. We therefore do not claim that the present analysis explains the audio result.

At task level, our method underperforms the MLP on 14 of 75 tasks, with a maximum mean deficit of 0.011; 11 of these tasks have MLP ROC-AUC above 0.95. This comparison consistently uses the MLP, whereas the paper’s 54-of-75 win count compares against the strongest baseline.

We also examined support-set size, embedding dimension, class count, and class balance. After accounting for baseline performance and domain, we did not identify a clear additional association with the gain.

W3. Why orthogonalize rather than use Gaussian transformations?

We agree that resampled Gaussian transformations are a strong alternative: orthogonalization is not necessary to obtain useful transfer. We choose orthogonal transformations because they preserve inner products and Euclidean distances exactly, achieve higher observed ROC-AUC, and add little measured computational overhead.

In Table 4, the orthogonal configuration improves over the Gaussian configuration by 0.0055 ROC-AUC overall (0.8886 vs. 0.8831) and 0.0079 on unseen pairs (0.8879 vs. 0.8800). These comparisons concern complete training-and-inference configurations, rather than an isolated effect of training-time orthogonalization. Our profiling shows that QR decomposition accounts for less than 0.5% of inference time and less than 2% of training time.

The fixed-basis and finite-pool ablations further support the importance of repeatedly changing the coordinate basis: unseen-pair ROC-AUC increases from 0.8685 with a fixed global orthogonal basis to 0.8830 with a pool of 64 bases and 0.8879 with freshly resampled orthogonal transformations. We therefore interpret orthogonality as a geometry-preserving refinement of basis randomization, rather than a prerequisite for transfer.

As the discussion period closes on November 18, we would welcome your feedback on our response. We would be happy to clarify any points or address any remaining questions or concerns.

Thank you for your time and consideration.

Reviewer_Y668OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper finetunes TabICL to make ICL on embeddings possible. Specifically, the paper needs to deal with (1) X dimensions are different from task to task; (2) the meaning of X is different from task to task. Results

Critical strengths
  1. The idea is interesting to apply ICL on embedding, to make the ICL generalizable to any embeddings.

  2. The method of "RANDOMIZED ORTHOGONAL BASIS CHANGES" brings insights.

Critical weaknesses
  1. Table 2 hides the details of each method. I'm expecting to see the results of zero-shot TabICL and few-shot TabICL. Sec 5.1 says zero-shot TabICL reaches 0.8615. Why does zero-shot TabICL performing well on embeddings? If thats' the case, the contribution from the method is limited. I was expecting TabICL is broken because not able to understand embeddings.

  2. The dimension is a dangerous factor. Assuming input dimensions may range from 256 to 1024, which are lots of cases. How is it possible to make it generalizable? I did not find an analysis on this, correct me if I missed it.

  3. The benefit seems to be weak "improving on the strongest baseline by 0.0198" on a semi-saturated benchmark "reaches 0.8886 ROC-AUC". I suggest to highlight more on the generalization on new dataset, new encoder.

Questions for authors
  1. How many ICL samples (k) are used for training and inference? I see Figure 3 has 4096 support size which is crazy to me, while I assume 4096 are the number of ICL samples, which means we feed 4096 samples together to the model.

  2. Is the Figure 1 AI generated? I see unmatched () under "random orthogonal rotation."

2 comments

We thank the reviewer for the constructive feedback.

1. Per-method results and frozen TabICL (W1)

“Zero-shot TabICL” was misleading: the baseline receives the full labeled training split as context but undergoes no parameter updates. We will rename it “frozen pretrained TabICL.”

Below are mean ROC-AUC scores on the same evaluations across three runs.

Evaluation subsetOursMLPFrozen TabICLXGBoostLinear probek-NN
Overall0.88860.86890.86150.86070.85560.8325
In-Distribution0.88910.86840.86290.85940.85490.8326
Novel Task0.88620.86950.86590.86250.85560.8315
Novel Encoder0.88930.86900.85470.86140.85610.8324
Joint Generalization0.88720.87330.85570.87030.86390.8356
Unseen, pooled0.88790.86970.85900.86280.85680.8324

Regimes describe exposure during our meta-training. Classical baselines are fitted separately for each pair; frozen TabICL uses labeled context. “Unseen, pooled” combines the three transfer regimes.

Frozen TabICL can already use embeddings as numerical features and infer classification rules from context. Our contribution is improved prediction and transfer across embedding spaces: +0.0271 overall over frozen TabICL. Ordinary LoRA training on raw embeddings reaches 0.8690 on unseen evaluations, versus 0.8879 for our method, with similar seen performance (0.8900 versus 0.8891). Orthogonal-view ensembling alone reaches only 0.8659 overall with frozen TabICL (Table 4), supporting the combined adaptation-and-ensembling procedure.

2. Generalization across dimensions (W2)

TabICL accepts variable-width inputs and converts them into fixed-dimensional internal representations. Our square orthogonal transformations preserve each encoder’s dimension and geometry; no encoder-specific adapters are required.

We additionally conducted leave-one-dimension-out experiments over 256, 384, 504, 512, 640, 768, 1024 and 1280, excluding all encoders of the held-out dimension from meta-training. Mean ROC-AUC is 0.8871, compared with 0.8886 in the reference setting: a paired difference of −0.0015 [−0.0023, −0.0007]. This small decrease supports transfer to dimensions absent from embedding-specific meta-training within the tested range.

3. Benefit and generalization (W3)

We will emphasize transfer more prominently. Improvements over the strongest baseline are:

RegimeROC-AUC gain [95% CI]
Novel Task+0.0167 [0.0065, 0.0289]
Novel Encoder+0.0203 [0.0121, 0.0294]
Joint Generalization+0.0140 [0.0064, 0.0225]
In-Distribution+0.0207 [0.0138, 0.0286]

All intervals exclude zero. The benchmark is heterogeneous: on pairs where MLP already achieves ≥0.983, headroom is limited and our mean gain is approximately zero; on the hardest fifth, the gain is +0.058 [0.033, 0.087]. On unseen tasks with support caps of 32 and 64 examples, gains over the strongest comparator are +0.032 and +0.027, respectively (§5.5).

4. Context size (Q1)

Yes, 4,096 labeled examples are processed jointly when available. Figure 3 uses maximum support caps: each task receives the smaller of the cap and its training-set size, with identical support examples across methods.

The main experiments use the full training split: 869 examples on average, median 448, range 38–6,254. During meta-training, this supplies labeled support and within-task validation examples supply queries. Inference uses labeled training context without parameter updates.

These sizes are within TabICL’s intended scope: the original paper (Qu et al., ICML 2025) reports pretraining with up to 60,000 samples and handling datasets with 500,000 samples.

5. Figure 1 (Q2)

Yes, Figure 1 was drafted using GPT. We will correct the mismatched parentheses and disclose figure generation in the AI Use Statement.

As the discussion period closes on November 18, we would welcome your feedback on our response. We would be happy to clarify any points or address any remaining questions or concerns.

Thank you for your time and consideration.

Reviewer_amXVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a universal in-context classifier that operates directly on frozen foundation-model embeddings across different encoders and modalities, without requiring encoder-specific classification heads. By meta-training TabICL with shared LoRA adapters and random orthogonal transformations, the method learns representations that are invariant to coordinate choices. At inference time, it aggregates predictions across multiple orthogonal views. Experiments on a benchmark covering seven domains, 75 tasks, 29 encoders, and 308 task--encoder pairs show that the proposed method achieves an average ROC-AUC of 0.8886, outperforming the strongest baseline by approximately 0.0198.

Critical strengths
  1. Technical novel: the paper introduces a unified in-context learning framework for heterogeneous embedding spaces, addressing the need for task-specific prediction heads. The use of random orthogonal transformations during meta-training preserves the geometric structure of the embeddings while removing dependence on fixed coordinate identities. This allows the model to generalize across different encoders without requiring encoder-specific adapters.

  2. Comprehensive empirical evaluation: experiments cover seven domains, 75 tasks, and 29 encoders, with extensive ablations and evaluations of novel-task, novel-encoder, and joint generalization.

Critical weaknesses
  1. Missing comparisons with relevant baselines: although the paper evaluates several conventional classifiers, it does not include comparisons with strong tabular foundation models, particularly recent variants of TabPFN. Without these comparisons, it is difficult to assess the relative performance of the proposed approach against existing foundation models and to fully support the comparative claims.
Questions for authors
  1. Could the authors include comparisons with TabPFN, particularly its more recent variants, and clarify the settings in which the proposed meta-trained approach provides clear advantages over these models? Since TabPFN is discussed in the related work, such comparisons would help establish the relative effectiveness of the proposed approach.

  2. Could the authors provide more details on the within-task data partitioning for the UCR/UEA datasets? The paper states that official dataset splits are retained when available, but it remains unclear how the validation sets are constructed from the original train/test splits. Could the authors report the exact numbers of training, validation, and test examples for each dataset, and preferably provide the corresponding sample indices?

In addition, the sample counts reported in Table 6 appear to follow inconsistent counting conventions. For example, GunPoint is reported as N=50N=50, whereas Plane and TwoLeadECG are reported as N=210N=210 and N=1,162N=1,162, respectively. Under the standard UCR splits, these correspond to 50/150, 105/105, and 23/1,139 training/test examples. Could the authors clarify whether NN represents the training set size or the total number of examples, and explain these discrepancies? This clarification is particularly important for assessing the evaluation protocol and interpreting the near-ceiling ROC-AUC results reported on some time-series tasks.

2 comments

Thank you for recognizing the technical contribution and breadth of the evaluation. We have added the requested TabPFN comparisons, which also provide evidence that our framework’s benefits extend beyond TabICL, and corrected the description of the time-series evaluation protocol.

Q1. Comparisons with recent TabPFN variants

We evaluated TabPFN-3 and TabPFN-3.5 both as frozen predictors and after adaptation with our framework. The table reports mean ROC-AUC across the same 308 task–encoder pairs:

BackboneFrozen+ our frameworkGain
TabICL0.86150.8886+0.0271
TabPFN-3.50.85530.8786+0.0233
TabPFN-30.85030.8677+0.0174

These results address the requested comparison in two ways.

First, our main model outperforms both frozen TabPFN variants on the embedding benchmark, exceeding TabPFN-3.5 by 0.0333 and TabPFN-3 by 0.0383 mean ROC-AUC. It also exceeds the tuned MLP baseline, which achieves 0.8689.

Second, our framework improves all three evaluated backbones. Applying our adaptation and orthogonal-view inference procedure improves TabPFN-3.5 by 0.0233 and TabPFN-3 by 0.0174. This supports the broader applicability of the approach to heterogeneous foundation-model embeddings. TabICL remains the strongest backbone in these experiments.

The demonstrated advantage is in aggregate performance across the evaluated embedding tasks; we do not claim an improvement on every individual task–encoder pair.

Q2. Time-series partitions and sample counts

Thank you for identifying the ambiguity. These UCR experiments use stratified random partitions, rather than retaining the official train/test partitions. We have corrected the manuscript accordingly and clarified both the source files and the resulting partition sizes.

The source files and partitioning procedures are:

  • Combined official training and test files: For Chinatown, FordB, Plane, SonyAIBORobotSurface1, Strawberry, and TwoLeadECG, we pool the two files and construct five stratified folds, using three for training, one for validation, and one for testing—approximately 60/20/20. Pooling and repartitioning UCR data at this ratio also appears in Liu et al. [1].
  • Official training file only: For ECG200, ElectricDevices, FordA, GunPoint, ItalyPowerDemand, and Wafer, we construct stratified training, validation, and test partitions from the official training file, targeting 70/15/15. The official test file is not used. This partition ratio also appears in Mekonnen et al. [2]; our use of the training file alone is specific to our implementation.

Both procedures use a fixed random seed of 42. NN denotes the number of examples in the source files used, before partitioning. Thus, GunPoint’s N=50N=50 refers to its official training file, while Plane’s N=210N=210 and TwoLeadECG’s N=1,162N=1{,}162 refer to their combined official files.

DatasetSource-pool NNTrainValidationTest
Chinatown3632177373
FordB4,4462,667889890
Plane2101264242
SonyAIBORobotSurface1621372124125
Strawberry983589197197
TwoLeadECG1,162696233233
ECG200100721414
ElectricDevices8,9266,2541,3361,336
FordA3,6012,523539539
GunPoint503866
ItalyPowerDemand674999
Wafer1,000702149149

All methods use identical, disjoint partitions, with test examples excluded from meta-training and checkpoint selection. We have expanded Table 6 with these counts and provide the exact sample indices in the accompanying code.

Near-ceiling ROC-AUC on small test sets, such as GunPoint’s six examples, should be interpreted cautiously. We therefore report partition sizes alongside the per-task–encoder results. These comparisons evaluate methods under our specified partitions and are not directly comparable with results obtained using the official archive splits.

We hope the additional comparisons and precise partition documentation address your concerns. In particular, the new experiments demonstrate both the advantage of our main model over the requested TabPFN baselines and the applicability of our framework across three backbones.

References

[1] Liu et al. (2023). “Temporal-Frequency Co-training for Time Series Semi-supervised Learning.” AAAI.

[2] Mekonnen, Longo, and Dondio (2024). “Interpreting Black-Box Time Series Classifiers using Parameterised Event Primitives.” xAI 2024, CEUR-WS, Vol. 3793.

I appreciate the efforts made by the authors, and the detailed explanation addressed most of my concerns, I maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.