acceptodds
Under review as a conference paper at ICLR 2027

Adapting Prior-Data Fitted Networks for Tabular Anomaly Detection

Abstract

While deep features have transformed anomaly detection in images and video, their impact on tabular data has been less substantial, partly due to the limited availability of strong deep representations. Recently, prior-data fitted networks (PFNs) have emerged as a promising source of such representations for tabular data. In this work, we investigate how PFN representations can be adapted and leveraged for anomaly detection. The question is harder than it looks. No anomalies are available before deployment, so model parameters cannot be tuned with supervision, and the reference set that defines normal behavior may itself contain the very anomalies it is supposed to reveal. We begin our study using frozen TabPFN features. Scoring each sample by its distance to its nearest neighbors in feature space already gives strong results. We identify which layers to use and a feature-extraction procedure suited to the task. Next, to further improve performance, we use the reference set to fine-tune the model, so that the resulting features better separate normal samples from anomalies. On the ADBench benchmark, our fine-tuning-free approach (ZEN) reaches a higher mean AUROC than every baseline, and our fine-tuned method (FOCUS) improves on it further. Our approach also generalizes across PFN models.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Prior Fitted Networks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2Wz7OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The present work tackles the problem of anomaly detection on tabular data. This work builds on existing work in image anomaly detection that leverage pretrained representations and propose a method that leverages pretrained model (e.g. TabICL, TabPFN) to identify anomalies. Their method involves two settings carved for uncontaminated and contaminated settings: a frozen encoder setting and a fine-tuned setting. In short, a sample's anomaly score is obtained as follows:

  • extract a sample's embedding by feeding the sample accompanied by a subset of (or the entire dataset) to the TabPFN model.
  • apply the vanilla k-NN based anomaly detection algorithm that identifies samples as being anomalies when the average distance to their k nearest neighbours is too high. The authors evaluate their method on a widely used benchmark, and demonstrate strong performance.
Critical strengths

S1: The paper is well written and easy to follow.

S2: The experiments are extensive and well motivated. Ablations in particular are especially sound and relevant to demonstrate the relevance of the method.

S3: Appendix and additional experiments are large and further support the relevant of the present method.

Critical weaknesses

W1: While the concept of novelty is of course subjective, the present work's novelty is somewhat limited. The present algorithm consists in using pretrained encoders as feature extractors for anomaly detection on tabular data. While the authors correctly mention that this has never been tried (to our knowledge), the idea of using deep features as sample's representation is far from being novel.

W2: Some strong baselines seem to have been left out of the benchmarks, e.g. ICL [1], NPT-AD [2], DRL [3].

[1] Tom Shenkar and Lior Wolf. Anomaly detection for tabular data with internal contrastive learning. In International Conference on Learning Representations (ICLR), 2022

[2] Hugo Thimonier, Fabrice Popineau, Arpad Rimmel, and Bich-Lieˆn Doan. Beyond individual input for deep anomaly detection on tabular data. In International Conference on Machine Learning (ICML), pp. 48097–48123, 2024.

[3] Hangting Ye, He Zhao, Wei Fan, Mingyuan Zhou, Dandan Guo, Yi Chang. DRL: Decomposed Representation Learning for Tabular Anomaly Detection. ICLR 2025.

Questions for authors

Q1: It would be interesting to apply the soft-cleaning and feature subset selection to the vanilla knn approach in the original feature space rather than in the embedding space. This would allow identifying whether the performance originate from the representation space obtained from the pretrained model, or the inclusion. This seems like a simple experiment to implement while also further showcasing the relevant of the embedding representation.

Q2: The authors discuss in the appendix how iterating on different subsets of features seems to increase performance while iterating on embedding while changing the data subset seems to make no difference different. (Appendix E) How was this experimented ? Did authors vary the subset size? On small datasets, we expect this would be especially effective for contaminated sets.

Q3: A thorough discussion on the difference in performance of the method on different datasets’ characteristics would be particularly if interest. For instance, one could expect datasets that have very few features would hardly benefit from the feature subset selection, it could even be detrimental to the performance.

Q4: While AUROC is relevant as a metric as it benefits from being threshold independent, previous work have also relied on the F1 score (see [4] for how to select the threshold for F1 Score computation). Could the authors also include this metric?

Overall the paper is well written, the experiments are thorough (but could improved see W2), but the core of the paper appears to us as incremental for a conference like ICLR. We lean toward reject but we are open to discussion. We believe that some of the small additional experiment (e.g. Q1) might strengthen the paper.

[4] Liron Bergmand and Yedid Hoshen, Classification-Based Anomaly Detection for General Data, ICLR 2020.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2Wz7

Thank you for the careful review and suggestions, and for highlighting the paper’s clarity, ablations, and appendix experiments.

W1

“the present work's novelty is somewhat limited”

We agree that kNN on pretrained features is an already established method. Yet, even ZEN, our simpler method that uses frozen TabPFN embeddings with feature subsetting, augmented contexts, and pooled soft cleaning, raises mean AUROC from 74.1 to 79.7 under contamination, a gain of 5.6 points over plain kNN on frozen embeddings. More importantly, FOCUS is the first method to fine tune a general-purpose PFN for unsupervised anomaly detection using only an unlabeled, potentially contaminated reference set. Its contribution is a trust weighted adaptation procedure that uses the frozen model’s embeddings to estimate which reference samples to trust (Sec. 2.3, App. D.1).

W2

“Some strong baselines seem to have been left out of the benchmarks”

Thank you for suggesting ICL, DRL, and NPT-AD. FOCUS has higher mean AUROC than all three in both regimes. ICL and DRL completed on all 47 datasets.

MethodDatasetsContaminated AUROCClean AUROC
ICL4764.9175.85
DRL4772.9980.59
FOCUS4780.0285.88

NPT-AD completed on 41 datasets within our compute budget. The following table compares both methods on those same 41 datasets.

MethodDatasetsContaminated AUROCClean AUROC
FOCUS4180.0486.37
NPT-AD4173.1981.57

We will add these comparisons and cite DRL alongside ICL and NPT-AD in Sec. 5.

Q1

“apply the soft-cleaning and feature subset selection to the vanilla knn approach in the original feature space”

Thank you for suggesting this test. We applied ZEN’s feature subsetting and pooled soft cleaning to raw kNN, with soft cleaning applied only on the relevant contaminated setting.

MethodContaminated AUROCClean AUROC
Raw kNN (matched control)76.6284.06
Raw kNN + subsets and soft cleaning77.1483.76
ZEN79.6885.21
FOCUS80.0285.88

These results suggest that the advantage of our method comes from the synergy between our novelty and PFN-based approach. The same steps give raw kNN only a small gain under contamination, and feature subsetting slightly hurts it on clean data. We will add this control to the appendix.

Q2

“The authors discuss in the appendix how iterating on different subsets of features seems to increase performance while iterating on embedding while changing the data subset seems to make no difference different. (Appendix E) How was this experimented ? Did authors vary the subset size?.”

Thank you for raising this point. Appendix E documents alternatives we tried for completeness. In this preliminary test, we averaged anomaly scores over random subsets of reference samples and saw no improvement. The subset size was varied but only as an exploration. We do not claim these experiments as a contribution, but we believe that clearly documenting negative results is useful to the community. Therefore we describe them only sporadically.

Q3

“A thorough discussion on the difference in performance of the method on different datasets’ characteristics would be particularly if interest.”

Thank you for asking. We grouped the 47 datasets by feature count into three groups: 9 or fewer, 10–30, and 31 or more. Under contamination, FOCUS’s largest advantage over Isolation Forest, the strongest baseline overall in this regime, was on the 16 datasets with at least 31 features. FOCUS scored 6.16 AUROC points higher on average in this group.

On clean data, feature subsetting slightly reduced FOCUS’s mean AUROC in the group with 9 or fewer features (−0.23 points) and improved it in the two groups with more features (+0.68 and +0.47 points, respectively). Under contamination, subsetting helped even in the group with the fewest features (+1.46 points).

We also examined dataset size. On the 35 datasets with more than 1,000 samples, FOCUS achieved higher mean AUROC than raw kNN in both regimes, scoring 86.02 vs. 84.78 on clean data and 79.85 vs. 71.03 under contamination. We will add these results to the Discussion and appendix.

Q4

“Could the authors also include this metric?”

Following Bergman and Hoshen (2020), we rank test samples by anomaly score. The table reports mean F1 × 100 over 47 datasets (higher is better).

MethodContaminated F1Clean F1
FOCUS37.0475.16
Raw kNN32.3973.00
Isolation Forest35.3159.78
ICL29.1564.76
DRL29.7866.92

Also when evaluating with a threshold, FOCUS substantially outperforms all compared baselines.

We will add F1 in App. B.5 and reference them from Sec. 3.1.

Reviewer_2Wz7OpenReview ↗

Response to Rebuttal

We thank the authors for carefully addressing the different points raised in our review.

In particular, the addition of the vanilla kNN and its soft-cleaned + subset variant is particularly of interest. The other experiments also strengthen the proposed approach. For this reason I am happy to increase my score.

Thank you for acknowledging our rebuttal and for taking the time to review our work!

Reviewer_vLjEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how general-purpose pretrained tabular models can be used for unsupervised anomaly detection, particularly when the reference set contains unknown anomalies. The authors propose ZEN, a frozen-model method that combines representations from multiple feature subsets and network layers with trust-weighted nearest-neighbor scoring. Presentation:

Critical strengths

Adapting general-purpose tabular foundation models to anomaly detection is a valuable research direction, and contamination in the reference set is an important practical issue that can easily be overlooked. The paper is well written and generally clear.

Critical weaknesses

1.An important motivation of this work is to improve anomaly detection using pretrained PFN representations. However, Section 4 reports an AUROC of 74.1 for nearest-neighbor scoring directly on frozen model representations, which is very close to the 73.9 achieved by kNN on raw features. Most of the subsequent performance gains occur after adding random feature subsets and soft cleaning. My question is how much of this improvement depends on PFN representations, and how much comes from feature ensembling and reference-sample weighting techniques that could also be applied to other detectors. What would the performance be if the same random feature subsets, cross-view trust estimation, and soft cleaning were applied in the raw feature space, with the applicable scoring hyperparameters matched? 2.Section 3.2 states that practitioners can choose FOCUS without knowing whether the reference set is contaminated and obtain the corresponding benefits in either setting. However, Section 3.1 also states that the method uses “one fixed configuration per regime.” Choosing separate configurations for the clean and contaminated settings does not mean that a single configuration can achieve the same performance when the setting is unknown. The main text does not clearly present results using a shared configuration across both settings, nor does it explain how practitioners should select a configuration without anomaly labels. I would like the authors to provide an evaluation using a shared configuration or explain a label-free configuration-selection procedure.

Questions for authors

Please see the specific concerns and questions raised in the Weaknesses section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer vLjE

Thank you for the careful review and for highlighting the value of adapting tabular foundation models to anomaly detection and the practical problem of reference-set contamination.

W1

“What would the performance be if the same random feature subsets, cross-view trust estimation, and soft cleaning were applied in the raw feature space, with the applicable scoring hyperparameters matched?”

Thank you for suggesting this test. We applied ZEN’s feature subsetting and pooled soft cleaning to raw kNN, with soft cleaning applied only on the relevant contaminated setting.

MethodContaminated AUROCClean AUROC
Raw kNN (matched control)76.6284.06
Raw kNN + subsets and soft cleaning77.1483.76
ZEN79.6885.21
FOCUS80.0285.88

These results suggest that the advantage of our method comes from the synergy between our novelty and PFN-based approach. The same steps give raw kNN only a small gain under contamination, and feature subsetting slightly hurts it on clean data. We will add this control to the appendix.

W2

“I would like the authors to provide an evaluation using a shared configuration or explain a label-free configuration-selection procedure.”

Thank you for pointing out that Sec. 3.2 drew a deployment conclusion from results obtained with separate configurations. To address this, we evaluated FOCUS and ZEN with the same configurations on all 47 datasets in both regimes. We recommend this configuration (designed for the contaminated setting) for our methods when the regime is unknown. It requires no estimate of the contamination rate. For Raw kNN and Isolation Forest we use their fixed Table 1 settings. The last column gives equal weight to the two regimes.

MethodContaminated AUROCClean AUROCTwo-regime average AUROC
FOCUS80.0284.3982.20
ZEN79.6884.4782.07
Raw kNN73.9285.1079.51
Isolation Forest76.4079.6478.02

FOCUS has the highest average in this comparison. It scores 4.18 AUROC points higher than Isolation Forest (p = 0.0011) and 2.69 points higher than raw kNN (p = 0.21). We will revise Sec. 3.2 to reflect the comparison and report the results in the appendix.

Thank you for the response.After considering it,I’ll keep my original score.

Thank you for considering our rebuttal and for taking the time to review our work!

Reviewer_AVKhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the use of pretrained tabular foundation model representations for unsupervised anomaly detection. It introduces two methods: ZEN, which applies kNN-based anomaly scoring to frozen TabPFN embeddings with feature-subspace ensembling and reference-sample reweighting, and FOCUS, which further adapts the embeddings through trust-weighted compactness fine-tuning.

Experiments on 47 ADBench datasets under clean and contaminated reference settings show that both methods outperform several conventional and deep anomaly detection baselines. The authors also provide component ablations, analyses of contamination robustness, and experiments with alternative tabular foundation models.

Critical strengths

1. Strong empirical performance under reference contamination.

The proposed methods achieve competitive anomaly detection performance, particularly when the reference set contains undetected anomalies. The soft-cleaning mechanism effectively reduces the influence of suspicious reference samples, improving robustness compared with conventional distance-based detectors.

2. Careful experimental analysis.

The paper provides component-level ablations, statistical comparisons, and experiments across multiple pretrained tabular models. The authors also identify and correct a data-duplication issue in the standard ADBench evaluation pipeline, improving the reliability of the reported comparisons.

Critical weaknesses

1. Very limited methodological novelty and informativeness.

The proposed methods primarily combine established techniques: kNN-based anomaly scoring on pretrained embeddings, random feature-subspace ensembling, isolation-based reference reweighting, and DeepSVDD/PANDA-style compactness fine-tuning. While applying these techniques to TabPFN is a reasonable engineering approach, the paper introduces little fundamentally new methodology or insight into anomaly detection.

In particular, the central ideas have already been explored extensively in representation-based anomaly detection. The paper mainly demonstrates their applicability to pretrained tabular models rather than establishing a new learning principle.

2. Limited contribution from pretrained representations and fine-tuning.

The ablation results suggest that much of the performance improvement comes from conventional anomaly detection techniques rather than the pretrained representation or its adaptation. In the contaminated setting, plain TabPFN embedding-based kNN achieves 74.1 AUROC, compared with 73.9 for raw-feature kNN. The substantial improvements arise primarily from feature-subspace ensembling and soft cleaning.

Furthermore, FOCUS improves over ZEN by only 0.3 AUROC points (80.0 vs. 79.7) in the contaminated setting, despite requiring dataset-specific fine-tuning. This raises questions about whether the additional complexity and computational cost of adapting the pretrained model are justified.

3. High computational cost and limited scalability.

ZEN and FOCUS require multiple TabPFN forward passes over different feature subsets, including repeated passes for reference embedding extraction. The experiments also restrict datasets to at most 10,000 samples and limit high-dimensional inputs to 200 features. Consequently, the scalability of the proposed methods to larger or higher-dimensional datasets remains unclear, particularly compared with substantially cheaper conventional anomaly detectors.

Questions for authors

Q1. Contribution of pretrained representations. Given that plain TabPFN embedding-based kNN performs similarly to raw-feature kNN, how much of the final improvement can be attributed specifically to pretrained representations? Could the authors evaluate raw-feature kNN with matched feature-subspace ensembling and soft cleaning?

Q2. Necessity of fine-tuning. FOCUS improves over ZEN by only 0.3 AUROC points in the contaminated setting. What practical benefit justifies dataset-specific fine-tuning and its additional computational cost?

Q3. Scalability. How do the computational cost and predictive performance scale beyond 10,000 samples or 200 features? Can the proposed method remain competitive with classical detectors under comparable inference budgets?

2 comments
AuthorsOpenReview ↗

Response to Reviewer AVKh

Thank you for the careful review and for highlighting the strong results under reference contamination, the component and cross-backbone analyses, and the correction to ADBench’s data duplication issue.

W1

“The proposed methods primarily combine established techniques”

We agree that the compactness objective builds on DeepSVDD and PANDA. To our knowledge, FOCUS is the first method to fine tune a general-purpose PFN for unsupervised anomaly detection using only an unlabeled, potentially contaminated reference set. It uses the frozen PFN’s embeddings to estimate which reference rows to trust, which results in significant gains (Sec. 2.3, App. D.1).

W2a and Q1

“Could the authors evaluate raw-feature kNN with matched feature-subspace ensembling and soft cleaning?”

Thank you for suggesting this test. We applied ZEN’s feature subsetting and pooled soft cleaning to raw kNN, with soft cleaning applied only on the relevant contaminated setting.

MethodContaminated AUROCClean AUROC
Raw kNN (matched control)76.6284.06
Raw kNN + subsets and soft cleaning77.1483.76
ZEN79.6885.21
FOCUS80.0285.88

These results suggest that the advantage of our method comes from the synergy between our novelty and PFN-based approach. The same steps give raw kNN only a small gain under contamination, and feature subsetting slightly hurts it on clean data. We will add this control to the appendix.

W2b and Q2

“What practical benefit justifies dataset-specific fine-tuning and its additional computational cost?”

With the plain kNN readout held fixed, two epochs of fine-tuning raise mean AUROC from 74.1 to 75.9 under contamination and from 84.2 to 85.1 on clean data (Table 10). In addition to the practical importance, this result makes FOCUS a proof of concept for adapting the embeddings using an unlabelled and potentially contaminated reference set.

Fine-tuning adds a one-time cost of 77.3 seconds per dataset on an A100 in the contaminated regime (Table 17). We believe this cost can be justified in offline applications where detection performance matters most, especially as it is only needed once. FOCUS achieves the highest mean AUROC in both regimes, and fine tuning provides the clearest benefit over ZEN on clean data (+0.68 AUROC points, p = 0.019).

W3 and Q3

“How do the computational cost and predictive performance scale beyond 10,000 samples or 200 features?”

We follow ADBench in subsampling larger datasets to 10,000 rows before splitting (Han et al., 2022, App. B.2). This threshold follows the standard evaluation and is not the result of a limitation of our method. Following your question, we ran an additional scaling experiment on Donors(one of the 47 data sets) using one random seed. ZEN processed 20,000 rows (14,000 reference and 6,000 test samples), achieving 88.44 AUROC under contamination and 99.29 on clean data, compared with 89.47 and 98.27 at 10,000 rows. The improvement on clean data suggests that relaxing the benchmark’s row cap may improve ZEN’s predictive performance in some settings.

We applied the 200 feature cap to only three of the 47 datasets: Census, InternetAds, and Speech. For each, we excluded columns that were constant in the reference set, then randomly selected 200 of the remaining columns using a fixed seed. The ranked baselines use all columns (App. A.3). Even with this restriction, FOCUS achieved the highest mean AUROC across the full 47-dataset benchmark in both regimes (Table 1). Better summarizing the features or using future backbones that can digest more features, would therefore only improve the performance of our method.

“Can the proposed method remain competitive with classical detectors under comparable inference budgets?”

We acknowledge that our method improves detection performance at additional computational cost compared to some of the baselines. Under contamination, FOCUS leads Isolation Forest by 3.62 AUROC points. We can make the method faster by running the model on fewer subsets of the input features, with a possible loss in accuracy that still leaves us in better position then other baselines. For example, evaluating on Donors (one of the 47 data sets), FOCUS using only the full set of input features without the five additional subsets, achieved 77.32 AUROC, compared with 75.71 for Isolation Forest. In that case, scoring took approximately 0.0098 seconds per test sample on an RTX A4500 while still being better than isolation forest. We find this per-sample-rate fast enough for most practical applications.

Thank you for the detailed response and additional experiments. The matched raw-feature comparison provides stronger evidence for the contribution of pretrained PFN representations, and I acknowledge that fine-tuning improves representation quality.

However, my main concerns remain:

  1. Methodological novelty: The proposed approach primarily combines established anomaly detection techniques. Its application to PFNs does not sufficiently establish a distinct methodological contribution.
  2. Fine-tuning benefit: While fine-tuning provides measurable improvements, its incremental gain over ZEN remains modest, particularly under contamination (+0.34 AUROC), relative to the additional computational cost.
  3. Scalability: The additional experiment on a single dataset and seed does not sufficiently address scalability or high-dimensional applicability.

Overall, the rebuttal strengthens the empirical evidence but does not sufficiently resolve my central concerns regarding novelty and practical trade-offs. I therefore maintain my original rating.

Make the novelty concern more specificClarify the ZEN comparison basis

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers

We thank all three reviewers for the time and care you put into evaluating our work. We considered each comment in preparing our replies. Your feedback led us to run new comparisons and clarify what our results support.

Our work studies how to adapt general purpose prior data fitted networks (PFNs) for tabular anomaly detection when the unlabeled reference set may contain anomalies. ZEN combines multiple views of frozen TabPFN embeddings and uses them to reduce the influence of suspicious reference rows. FOCUS uses trust estimates from the frozen model to guide fine-tuning. We appreciate that the reviewers highlighted the paper’s clarity (2Wz7, vLjE), the practical importance of reference set contamination (vLjE), and the thorough experimental analysis (2Wz7, AVKh). Both 2Wz7 and AVKh noted the strong benchmark results, with AVKh emphasizing performance under reference contamination and acknowledging our correction to ADBench’s sample duplication issue.

We detail below three main additions incorporated in the revision.

  • Expansion of the baseline comparison. We added ICL, NPT-AD and DRL to our compared methods. FOCUS achieved higher mean AUROC than all three baselines in both regimes.

  • Added a comparison with kNN together with our data curation addition. We report the results with feature subsetting and pooled soft cleaning applied to kNN (our leading baseline). On raw features, these steps improved mean AUROC by 0.52 points under contamination, and feature subsetting reduced it by 0.30 points on clean data. In both cases our method was significantly still ahead. The larger gains from these steps on frozen TabPFN embeddings suggest that the advantage of our method comes from the synergy between our novelty and PFN-based approach.

  • Revise the comparison on unknown setting. We report how FOCUS performs with the same fixed configuration in both regimes and include the full comparison in the appendix. Using its contaminated configuration in both regimes, FOCUS achieves a mean AUROC of 82.20, compared with 79.51 for raw kNN, the strongest baseline in this comparison - a substantial gain of 2.69 points.

Thank you again for the thoughtful discussion. Your suggestions helped us strengthen the evaluation and clarify the practical trade offs in the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.