acceptodds
Under review as a conference paper at ICLR 2027

GradPAO: Gradient-Based Pseudo-Anomaly Optimization for Tabular Anomaly Detection

Abstract

Tabular anomaly detection (TAD) is an important task since many real-world machine learning applications operate on tabular data, and it remains challenging due to the scarcity of labeled samples and the ambiguous definition of anomalies. Despite the rapid progress of tabular foundation models (TFMs) on predictive tasks, building a foundation model for TAD is especially difficult since its anomaly prior must cover diverse data types and define what counts as abnormal before any target dataset is seen. We therefore ask how a pretrained TFM can be turned into an anomaly detector, rather than developing another foundation model for TAD. The obstacle is that in-context learning of TFMs operates on a labeled context set, while only normal samples are available in TAD, so no anomaly class exists for a test query to be assigned to. To fill this gap, we propose **Grad**ient-Based **P**seudo **A**nomaly **O**ptimization for tabular anomaly detection (**GradPAO**), a novel framework that supplies the missing classes by defining pseudo-anomalies as trainable parameters and optimizing them through the foundation model. **GradPAO** first places the pseudo-anomalies using the inter-sample attention of the foundation model, and then optimizes them to be separable from the normal samples and diverse among themselves. Extensive experiments on two benchmarks with 57 and 690 datasets demonstrate that **GradPAO** achieves state-of-the-art performance.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Tabular Anomaly Detection, Pseudo Anomaly Synthesis, Tabular Foundation Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_eDjWOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper asks whether a pretrained tabular foundation model (TFM) can be reused for anomaly detection when the training data contain only normal samples. GradPAO learn pseudo-anomaly examples that supply the missing anomaly classes in the model's context. It uses the TFM's attention to group normal rows and moves selected rows away from their cluster centers to initialize these examples. Their feature values are then optimized through the frozen model using classification and attention overlap losses. Queries are scored by their predicted probability of belonging to any anomaly class. Experiments report strong aggregate performance across 57 ADBench and 690 public OddBench datasets, with TabICLv2 and TabPFN v3, component ablations, sensitivity studies, and runtime analysis. The contribution is an adaptation learned from normal data that changes the context supplied to an existing predictor.

Critical strengths
  1. I really like the idea of using the pseudo-anomaly examples as a way of adapting the predictive model. The approach takes advantage of the TFM's pretrained knowledge while supplying/optimizing the missing anomaly classes and only requiring normal target data to do it. Using synthetic negatives and context optimization do have precedents, but I believe this combination represents an interesting and meaningful contribution. Additionally, I find it encouraging that results are strong across both TabICLv2 and TabPFNv3 as backbone. This supports the general use and application ofthis methodology across various TFMs (Sections 3.1-3.3 and Table 2).

  2. I found the paper quite clear on a first read which was refreshing. Similarly, I feel its ablations support to main claims nicely and answer the natural questions a reader would have about this work. For example, with backbone and initialization fixed, ADBench's rAUC increases from 0.7589 without optimization to 0.7687 with separability and 0.7817 with attention diversity. This supports learning the examples beyond their initialization. Additionally, five seed main results and per-dataset tables add support, including the acknowledged AUROC case where OutFormer has a higher rescaled score (Tables 1 and 4, Appendices A.2 and A.6).

Critical weaknesses
  1. It seems to me the attention heatmap leaves the diversity explanation open. Identical attention distributions could perhaps have small inner products when spread broadly over normal rows so low Gram entries may reflect flatter attention rather than different attended regions. The attention objective improves over input space diversity by 0.0056 rAUC without reported paired uncertainty, but I think the explanation would be more convincing with attention entropy, normalized similarity, and/or variation in the performance differences across seeds (Equation 7, Table 4, and Figure 3).

  2. Additionally, I think the initialization needs a bit more implementation detail to make the method easier to reproduce, especially on large datasets. Appendix A.2 gives the optimization batch sizes, but the sample budget for constructing the initial attention map is less clear. The attention layer and preprocesing used at this stage also need clarification, along with how clusters containing fewer than eight rows or selected rows coinciding with their centroid are handled. These choices affect the initial pseudo-anomalies and the cost of constructing them. Although the default are listed, it would help to state when they were fixed and which data informed their selection (Section 3.2, Algorithm 1, and Appendices A.2 and A.4).

Questions for authors
  1. Following weakness 1 above, it remains a bit unclear whether the diversity term leads to genuinely distinct behaviors, and I think further clarification would be helpful. For example, do the attention patterns remain distinct after accounting for differences in entropy and normalization? How consistent are the optimization and diversity gains across random seeds? Additionally, does incorporating multiple anomaly classes still provide benefits when the total number of pseudo-anomaly rows is held fixed?

  2. Following weakness 2 above, could you explain in more detail how the initialization was implemented? More specifically, could you clarify the sample caps, attention layer, preprocessing, and small/degenerate cluster rules used for initialization, especially on large datasets? When were the defaults fixed, and which development data informed them?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors [1/2]

Dear Reviewer eDjW,

We sincerely appreciate the reviewer’s time and insightful comments. Below, we address each comment in detail and will incorporate the corresponding analyses and clarifications into the revised manuscript.

[W1/Q1-1] Identical attention distributions could perhaps have small inner products when spread broadly over normal rows so low Gram entries may reflect flatter attention rather than different attended regions. Do the attention patterns remain distinct after accounting for differences in entropy and normalization?

[Response] Thank you for your valuable feedback. Following your suggestion, we computed the cosine similarity between the attention distributions of the pseudo-anomalies, but it was hard to see a distinct trend, since the cosine similarity also increases when the attention becomes flatter. We therefore computed the Pearson correlation between the attention weights of each pair of pseudo-anomalies across the normal samples and averaged these correlations over all pairs. Since the Pearson correlation is invariant to shifts and positive rescaling, it is unaffected by uniform flattening of the attention distributions. The effect of the diversity objective becomes clearer with this measure. As shown in the table below, without the diversity objective, the Pearson correlation increases from 0.602 at initialization to 0.682 after optimization, indicating that the attention patterns become more similar during optimization. With the diversity objective, it decreases to 0.592 after optimization, substantially below the value without the diversity objective. These results show that the diversity objective prevents the attention patterns of the pseudo-anomalies from becoming similar during optimization.

Pearson correlation
Initialization0.602
w/o Ldiv\mathcal L_{\text{div}}0.682
w/ Ldiv\mathcal L_{\text{div}} (ours)0.592

[Q1-2] How consistent are the optimization and diversity gains across random seeds, in terms of paired uncertainty?

[Response] We appreciate this suggestion, which helps verify the reliability of our ablation results. Following your suggestion, we provide the rAUC of each variant in the optimization ablation for each of the 5 seeds on ADBench, as shown in the table below. On average, the optimization improves the rAUC from 0.7531 to 0.7654, and the diversity objective further improves it to 0.7788, with gains larger than their standard deviations across seeds. When comparing the variants within each seed, the optimization improves over the initialization-only variant, and the attention-based diversity objective outperforms the separability-only variant and the input-space and embedding-space objectives on all 5 seeds.

sep.div.seed 0seed 1seed 2seed 3seed 4mean
XX (init only)0.75390.75460.75530.74910.75240.7531 ± 0.0025
VX0.76890.76870.76840.76060.76030.7654 ± 0.0045
Vinput0.77700.77280.77310.76950.77250.7730 ± 0.0027
Vembedding0.77380.77160.77060.76230.76100.7679 ± 0.0058
Vattention (ours)0.78270.78080.78280.77430.77340.7788 ± 0.0046

AuthorsOpenReview ↗

Rebuttal by Authors [2/2]

[Q1-3] Does incorporating multiple anomaly classes still provide benefits when the total number of pseudo-anomaly rows is held fixed?

[Response] We thank the reviewer for this insightful question, which helps isolate the effect of using multiple anomaly classes. To answer your question, we provide the results of varying the number of anomaly classes CC while fixing the total number of pseudo-anomalies C×kC \times k at 72. As shown in the table below, the performance increases with the number of anomaly classes. Using 9 classes improves the rAUC by 0.095 over using a single class and by 0.033 over using 3 classes, and the average rank follows the same trend. Since the number of pseudo-anomalies is identical across the three settings, this gain comes from dividing the pseudo-anomalies into multiple classes rather than from the number of pseudo-anomalies. This supports our design of assigning a separate anomaly class to each cluster to represent diverse abnormal patterns, as described in Section 3.1.

(C,k)(C, k)rAUCRank
(9, 8)0.7814.12
(3, 24)0.7484.61
(1, 72)0.6865.16

We note that a setting with more classes such as (C,k)=(18,4)(C, k) = (18, 4) cannot be evaluated since most TFMs natively support at most 10 classes and one of them is reserved for the normal class, which limits CC to 9. Given the increasing trend with the number of anomaly classes, we expect that using more classes may further improve the performance as future TFMs support a large number of classes.


[W2/Q2] Could the authors provide more implementation details of attention-based initialization such as attention layer, preprocessing, clusters containing fewer than eight rows?

[Response] Thank you for pointing this out. To improve the reproducibility of our method, we provide the details of the preprocessing, the attention layer, and the handling of small clusters in the attention-based initialization, and we have incorporated them into Appendix A.2.

For the preprocessing, each feature is standardized using the mean and standard deviation of the train set, following the pipelines of ADBench and OddBench, and the forward pass then applies the built-in preprocessing of TabICLv2. To build the attention map, we use all normal training rows, or a random subset of 5,000 rows when more are available, and select a pool of up to 512 rows by farthest-point sampling. We then group the pool into C=9C = 9 clusters by agglomerative clustering on the attention map and select the k=8k = 8 least-attended rows in each cluster.

For the attention layer, we use the attention among the pooled rows in the first In-context Learning block of TabICLv2, averaged over its heads. As shown in the attention maps of different layers in Appendix A.4, the first layer shows block-diagonal patterns, where the normal samples attend mainly to the samples in their own cluster, whereas the deeper layers show vertical stripes, where a few rows receive more attention from all the other rows and the cluster patterns become less visible.

When a cluster contains fewer than eight rows, we reuse its rows and vary the displacement scale, so that the resulting pseudo-anomalies do not overlap and receive different gradient updates. Specifically, the displacement scale increases by 0.05 for each selected row, starting from α=1.0\alpha = 1.0, so that repeated rows are placed at different distances from the centroid.

Reviewer_eDjWOpenReview ↗

Reviewer eDjW response to author rebuttal

Thank you for the detailed clarifications and additional experiments. I appreciate the evidence supporting the diversity objective and the consistency of the gains across seeds, as well as the additional implementation details. I am satisfied with the responses and will maintain my clear acceptance recommendation.

Dear Reviewer eDjW,

Thank you for your constructive feedback and for maintaining your recommendation. We are glad that the additional experiments and clarifications addressed your concerns, and we will incorporate them into the final version. If you have any further questions or suggestions, we would be happy to discuss them at any time.

Reviewer_SXV9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The present work follows recent work for tabular anomaly detection that leverages the powerful tabular foundation models. To be able to leverage tabular foundation models for anomaly detection, previous work have constructed anomaly distribution priors on which performance heavily relied: poorly defined priors would lead to poor anomaly detection performance. The present work proposes a novel approach to generate anomalies based on the tabular foundation model used. Their approach relies on two stages:

  • Attention-based pseudo-anomaly initialization that provides a proper initialization scheme to generate the anomalies.
  • Pseudo-anomaly optimization that starts from the previous stage’s result and iteratively improves the anomaly samples. In inference the anomaly score is computed as the 1−p1 - p where pp is the probability that a samples is normal against all generated anomalies. The method is tested on two wide benchmarks and displays strong performance in comparison to tested methods.
Critical strengths

S1: Paper is easy to follow and well structured.

S2: Method is properly explained and can be understood quite easily.

S3: Experiments to demonstrate the method's relevance use extensive benchmarks (~700 datasets).

Critical weaknesses

W1: The proposed method appears quite incremental to us as it mostly slightly modifies existing methods.

W2: The way hyperparameters have been selected really looks like overfitting on the test set from ADBench. One isn't supposed to use a val/test set in AD as the core purpose of this branch of methods is that one does not dispose of any anomalies. Ablations suggest that authors have explored hyperparameters to optimize performance on ADBench.

W3: The reported results are obtained from several runs of the same algorithm on a fixed data split.

W4: There are some discrepancies between the code and what the authors discuss in the paper. (see Questions)

Questions for authors

Q1: It would be interesting to have loss curves printed somewhere to have an idea of how their values evolves over the course of training?

Q2: The proposed method seems to be really robust to hyperparameter modifications per Fig. 4. ; multiplying the lr by 30 from 0.01 to 0.3 seems to offer somewhat similar performance on average. Have you investigated if there were big variations of performance on some datasets that were compensated by others, or if each dataset's performance varied very little when changing the lr? Similarly, what is the ratio of loss value between Ldiv\mathcal{L}_{div} and Lsep\mathcal{L}_{sep}? Same question as for the lr, it seems that going from λ=10\lambda=10 to λ=50\lambda=50 has little impact on the performance. Have you investigated why is that? As also mentioned in Q1, it would be interested to see global training loss and per-loss curves to have a better understanding of the training behaviour.

Q3: The authors mention how they use a single forward pass on the tabular foundation model to extract the attention map. Authors do not mention which layer(s) they use in the paper? Looking at the code, it seems that first layer was used. Any rational for this? Have the other investigated using other layers? Several layers and aggregating them using mean/max pooling etc?

Q4: The models used for training and inference are different but this is never explicitly mentioned in the paper. For instance for pseudo-anomaly optimization, no feature-shuffle ensembling is considered, no sink tokens, and no softmax temperature scaling. In contrary, during inference, those parameters are set to n_estimators=16, softmax_temperature=0.9 and sink tokens are included.

Overall, the method is interesting but the experimental setting needs further discussion and clarifications. We recommend rejection but are open to discussion should the authors clarify the few points mentioned in the weakness and question sections.

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors [1/3]

Dear Reviewer SXV9,

We appreciate the reviewer’s time and thoughtful comments. We respond to each concern step by step, and all clarifications and additional results will be reflected in the revised manuscript.

[W1] The proposed method appears quite incremental to us as it mostly slightly modifies existing methods.

[Response] Thank you for raising this concern. We respectfully disagree that GradPAO is a slight modification of existing methods. GradPAO fundamentally differs from existing TFM-based methods in how it enables a foundation model to perform anomaly detection. Specifically, GradPAO uses an existing TFM pretrained for predictive tasks by constructing the anomaly classes missing from normal-only training data, whereas existing methods pretrain a new foundation model for anomaly detection. We believe our work introduces a novel approach to TFM-based anomaly detection.

We would appreciate it if the reviewer could clarify which aspects of GradPAO seem incremental, or point us to any existing work the reviewer considers closely related, so that we can clarify the difference further.


[W2] The way hyperparameters have been selected really looks like overfitting on the test set from ADBench. One isn't supposed to use a val/test set in AD as the core purpose of this branch of methods is that one does not dispose of any anomalies. Ablations suggest that authors have explored hyperparameters to optimize performance on ADBench.

[Response] We thank the reviewer for this question. To clarify our choice of λ\lambda, we provide the ratio of the gradient norms of the two objectives, separability and diversity, with respect to the pseudo-anomalies. As shown in the table below, without weighting the diversity objective, the gradient norm of the separability objective is more than 20 times larger, whereas with λ=20\lambda = 20, the two gradient norms become comparable, which motivated our default choice. For the number of epochs, we referred to the convergence of the loss curves, as discussed in our response to Q1/Q2-1 and shown in Figure 6 of Appendix A.6.

∥∇Lsep∥/∥∇λLdiv∥\lVert \nabla \mathcal{L}_{\text{sep}} \rVert / \lVert \nabla \lambda \mathcal{L}_{\text{div}} \rVertFirst epochLast epoch
λ=1\lambda =146.3423.2
λ=20\lambda =20 (ours)2.321.16

[W3] The reported results are obtained from several runs of the same algorithm on a fixed data split.

[Response] Thank you for pointing this out. We use a fixed split per dataset with different seeds, following the evaluation pipelines that provide the public baseline results, OutFormer for ADBench and MacrOData for OddBench, and we take all baseline numbers directly from these works. Since every baseline result we compare against was obtained on the same split, running GradPAO on different splits would make its results incomparable with the published baseline results. Therefore, keeping the benchmark split is a requirement for a fair comparison rather than a choice that favors our method.


[Q1/Q2-1] It would be interesting to have loss curves printed somewhere to have an idea of how their values evolve over the course of training.

[Response] Thank you for this helpful comment. Following your suggestion, we provide the curves of Lsep\mathcal L_{\text{sep}} and Ldiv\mathcal L_{\text{div}} over 30 epochs in Figure 6 of Appendix A.6. Both losses decrease rapidly in the early epochs and converge by around 10 epochs, with about 90% of their total reduction achieved within the first 10 epochs, which led us to use 10 epochs as the default.

AuthorsOpenReview ↗

Rebuttal by Authors [2/3]

[Q2-2] The proposed method seems to be really robust to hyperparameters … Have you investigated if there were big variations of performance on some datasets that were compensated by others, or if each dataset's performance varied very little when changing the lr?

[Response] We thank the reviewer for this insightful question. To address your concern on robustness on each dataset across varying hyperparameters, we provide the per-dataset variation of GradPAO's rank across hyperparameter values, showing that the robustness holds on each dataset, not only on average. Specifically, we vary λ∈{10,20,30,50}\lambda \in \lbrace10, 20, 30, 50\rbrace and the learning rate ∈{0.01,0.05,0.1,0.2,0.3}\in \lbrace0.01, 0.05, 0.1, 0.2, 0.3\rbrace, keeping the other hyperparameters at their defaults. For each tested value, we rank GradPAO against the 12 ADBench baselines by AUCPR on each dataset, and then take the standard deviation of GradPAO's rank across the tested values within each dataset.

The table below summarizes the distribution of this per-dataset standard deviation over all datasets. For λ\lambda, the median standard deviation is 0, so the rank of GradPAO does not change on at least half of the datasets. For the learning rate, the median is 0.400 and the third quartile is 0.803, so the standard deviation stays below one rank position on at least 75% of the datasets. The larger deviations for the learning rate come mainly from the largest value of 0.3, where the optimization becomes less stable on some datasets. When measured without the learning rate of 0.3, the mean and median decrease to 0.425 and 0.217, respectively. Therefore, the stable average performance reflects stable rankings on individual datasets rather than large gains and losses that cancel each other out, and even the worst configuration among the tested values achieves the best average rank against the baselines.

Rank variation when changing $\lambda$Rank variation when changing learning rate
mean (0~100%)0.3350.568
median (at 50%)0.0000.400
Q1 (at 25%)0.0000.000
Q3 (at 75%)0.4000.803
AuthorsOpenReview ↗

Rebuttal by Authors [3/3]

[Q3] The authors mention how they use a single forward pass on the tabular foundation model to extract the attention map. Authors do not mention which layer(s) they use in the paper? Looking at the code, it seems that the first layer was used. Any rationale for this? Have the authors investigated using other layers? Several layers and aggregating them using mean/max pooling etc??

[Response] We are grateful to the reviewer for carefully going through our code and pointing out this missing detail. In response, we provide the attention maps of different layers in Appendix A.4 and an ablation over the attention layer used for the initialization. We use the first ICL transformer block of TabICLv2 (layer 0) by default, since its attention preserves the cluster structure of the normal samples on which our initialization relies. As shown in Appendix A.4, the first layer shows block-diagonal patterns, where the normal samples attend mainly to the samples in their own cluster, whereas the deeper layers show vertical stripes, where a few rows receive more attention from all the other rows and the cluster patterns become less visible. The table below reports the rAUC when the initialization uses the attention of each layer, followed by the same optimization. The first and middle layers perform comparably on both benchmarks. The last layer, in contrast, performs clearly worse on the 57 ADBench datasets and slightly worse on the 690 OddBench datasets. These results show that GradPAO is robust to the choice between early and middle layers, which preserve the cluster structure, supporting our choice of the first layer. We have also stated the layer used for the initialization in Appendix A.2 of the revision.

ADBenchOddBench
Layer 00.781 ± 0.0050.775 ± 0.002
Layer 5 (Mid)0.779 ± 0.0120.776 ± 0.004
Layer 11 (Last)0.725 ± 0.0100.770 ± 0.002

[Q4] The models used for training and inference are different but this is never explicitly mentioned in the paper. For instance for pseudo-anomaly optimization, no feature-shuffle ensembling is considered, no sink tokens, and no softmax temperature scaling. In contrast, during inference, those parameters are set to n_estimators=16, softmax_temperature=0.9 and sink tokens are included.

[Response] Thank you for carefully reviewing our code, and we apologize for not stating this difference in the paper. We implemented the optimization without feature-shuffle ensembling and softmax temperature scaling, since this stage only aims to find the pseudo-anomalies rather than to produce final predictions, and the optimization converged well without them. At inference, we used 16 ensemble members, which is fewer than the 50 used by OutFormer, and the default softmax temperature of 0.9 in TabICLv2 to obtain the final anomaly scores. The sink tokens are a remnant of earlier experiments and are not used in our experiments. We have clarified the full configurations of both stages in Appendix A.2.

Reviewer_GDeROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GradPAO, a method that directly uses a pretrained tabular foundation model (TFM) for tabular anomaly detection. To address the fact that only normal samples are available, it treats pseudo-anomalies as trainable parameters and optimizes them by gradient descent through the frozen TFM, thereby constructing the missing anomaly classes. The pseudo-anomalies are first initialized by clustering the normal samples based on the TFM’s inter-sample attention and displacing the boundary samples of each cluster outward. They are then optimized with a separability loss and an attention-based diversity loss. At test time, the normal samples and pseudo-anomalies together form the context, and the probability of not belonging to the normal class serves as the anomaly score. On ADBench and OddBench, GradPAO outperforms existing methods overall.

Critical strengths

• The evaluation is large in scale, and the results are fairly convincing. Since its hyperparameters are set on ADBench and reused unchanged on OddBench, the results can suggest good generalization.

• The method is simple and practical. It requires no anomaly-specific pretraining and optimizes only a small number of pseudo-anomaly rows. The paper is also clearly written.

Critical weaknesses
  1. The ablations in Tables 3 and 4 suggest that most of the gain comes from the framework itself, i.e., a TFM classifier with artificial anomaly classes, while the proposed attention-based initialization and diversity constraint add relatively little. However, casting one-class anomaly detection as classification between normal samples and artificial anomalies is a long-standing idea, and optimizing context rows through a frozen TFM has already been explored by In-Context Data Distillation and TuneTables. Could the authors pdiscuss these closely related lines of work more thoroughly?
  2. Section 4.2 compares GradPAO with TabPFN-OD and attributes the performance gap mainly to how anomaly scores are derived. However, GradPAO uses TabICLv2, whereas TabPFN-OD is built on TabPFN, so the two methods rely on different pretrained backbones. Could the authors add comparisons with the same backbone, for example, a reconstruction-based detector built on TabICLv2?
  3. The introduction states that heuristically constructed pseudo-anomalies do not guarantee effective anomaly classes, but the paper does not report results for heuristic pseudo-anomalies without optimization. I would like to see these results, as well as more insight into how the construction of pseudo-anomalies affects inference. Furthermore, Eq. (4) displaces pseudo-anomalies using the cluster centroid, radius, and direction vectors in Euclidean input space, even though the paper argues that input-space distances are unreliable for heterogeneous tabular columns. Could the resulting pseudo-anomalies, together with the subsequent unconstrained optimization, move outside the valid range of the original data (e.g., out-of-range values or invalid categorical encodings)? If so, how does this affect the model’s ability to distinguish realistic anomalies?
Questions for authors

Please see the questions raised in the Weaknesses section.

5 comments
AuthorsOpenReview ↗

Rebuttal by Authors [1/3]

Dear Reviewer GDeR,

We sincerely thank the reviewer for their time and constructive feedback. Below, we address each comment in detail and will incorporate these discussions into the revision.

[W1] The ablations in Tables 3 and 4 suggest that most of the gain comes from the framework itself, i.e., a TFM classifier with artificial anomaly classes, while the proposed attention-based initialization and diversity constraint add relatively little. However, casting one-class anomaly detection as classification between normal samples and artificial anomalies is a long-standing idea, and optimizing context rows through a frozen TFM has already been explored by In-Context Data Distillation and TuneTables. Could the authors discuss these closely related lines of work more thoroughly?

[Response] Thank you for your thoughtful feedback. We clarify how GradPAO differs from existing works below, and we have incorporated this discussion into Section 2 (Related Work) of the revision.

We agree that casting one-class anomaly detection as classification between normal samples and synthetic anomalies is a long-standing idea, and GradPAO also builds on this view. However, its goal differs from that of prior works. Existing methods synthesize anomalies as a means to train a classifier, whereas GradPAO keeps a pretrained classifier frozen and treats the anomaly classes themselves as the target of optimization. For example, SO-GAAL [1] generates anomalies with an adversarially trained generator, and DROCC [2] finds them by adversarial search around the normal samples. In other domains, CutAddPaste [3] cuts a patch from a time series, adds a linear trend, and pastes it back, and VOS [4] synthesizes virtual outliers in the feature space for image data. In all of these methods, the synthetic anomalies serve as training data, and the classifier trained on them for each dataset is the object of optimization. In contrast, GradPAO optimizes the pseudo-anomalies through the frozen classifier, so that they are shaped by the model that consumes them at test time.

GradPAO shares with In-Context Data Distillation [5] and TuneTables [6] the mechanism of optimizing rows in the context set through a frozen TFM, but the purpose of the optimization differs. In-Context Data Distillation [5] distills a large labeled dataset into a small set of context rows, and TuneTables [6] learns a compact context through prompt tuning, so both summarize classes that already exist in the labeled data into fewer rows or tokens. In anomaly detection, however, this approach cannot be directly applied since there are no real anomaly classes to summarize. GradPAO instead constructs anomaly classes that do not exist in the normal-only train set. GradPAO initializes the pseudo-anomalies from the inter-sample attention without labels and optimizes them to be separable from the normal samples and diverse among themselves.

[1] Y. Liu et al., "Generative Adversarial Active Learning for Unsupervised Outlier Detection," IEEE TKDE 2019.

[2] S. Goyal et al., "DROCC: Deep Robust One-Class Classification," ICML 2020.

[3] R. Wang et al., "CutAddPaste: Time Series Anomaly Detection by Exploiting Abnormal Knowledge," KDD 2024.

[4] X. Du et al., "VOS: Learning What You Don't Know by Virtual Outlier Synthesis," ICLR 2022.

[5] J. Ma et al., "In-Context Data Distillation with TabPFN," arXiv:2402.06971, 2024.

[6] B. Feuer et al., "TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks," NeurIPS 2024.

AuthorsOpenReview ↗

Rebuttal by Authors [2/3]

[W2] Section 4.2 compares GradPAO with TabPFN-OD and attributes the performance gap mainly to how anomaly scores are derived. However, GradPAO uses TabICLv2, whereas TabPFN-OD is built on TabPFN, so the two methods rely on different pretrained backbones. Could the authors add comparisons with the same backbone, for example, a reconstruction-based detector built on TabICLv2?

[Response] We thank the reviewer for the valuable feedback and agree that comparing our framework with a reconstruction-based detector on the same backbone is important. To address your concern, we provide the results of TabICLv2-Recon, a reconstruction-based detector built on TabICLv2 in the same way as TabPFN-OD, so that GradPAO and the reconstruction approach are compared on the same backbone. The table below reports the average rank and rAUC in AUCPR on ADBench and OddBench, computed against all baselines including TabICLv2-Recon. TabICLv2-Recon outperforms TabPFN-OD by 0.072 in rAUC on ADBench and by 0.037 on OddBench. With the backbone fixed to TabICLv2, GradPAO still outperforms TabICLv2-Recon by 1.22 in average rank and 0.042 in rAUC on ADBench, and by 2.27 in average rank and 0.175 in rAUC on OddBench. This shows that, on the same foundation model, classifying a test query against optimized pseudo-anomalies is more effective than scoring it by reconstruction error.

To keep the comparison consistent with the current manuscript during the discussion period, we will incorporate these results into Section 4.2 and revise its interpretation accordingly after the discussion period.

ADBenchOddBench
RankrAUCRankrAUC
Ours4.430.7794.540.765
TabICLv2-Recon5.650.7376.810.590
OutFormer5.790.7196.900.578
DTE-NP6.170.6947.400.542
TabPFN-OD6.270.6657.570.553

[W3-1] The introduction states that heuristically constructed pseudo-anomalies do not guarantee effective anomaly classes, but the paper does not report results for heuristic pseudo-anomalies without optimization. I would like to see these results, as well as more insight into how the construction of pseudo-anomalies affects inference.

[Response] Thank you for this helpful suggestion. Following your suggestion, we provide the rAUC on ADBench when each ADBench synthetic anomaly type is used as pseudo-anomalies, without optimization and with the same optimization as GradPAO. As shown in the table below, optimization improves the rAUC of three of the four heuristics, by 0.060 for Local, by 0.070 for Dependency, and by 0.008 for Cluster. The only exception is Global, where the pseudo-anomalies are sampled uniformly from a range wider than that of the normal samples, regardless of how the normal samples are structured. Since these points do not reflect the structure of the normal samples, the optimization does not improve them. This shows that the optimization is effective when the initialization reflects the structure of the normal samples. Our attention-based initialization reflects this structure by grouping the normal samples into clusters, and GradPAO reaches an rAUC of 0.7817, the highest among all initializations.

w/o optimizationw/ optimization
Global0.7650.754
Cluster0.7360.744
Local0.6710.731
Dependency0.6730.743
Ours0.7580.781
AuthorsOpenReview ↗

Rebuttal by Authors [3/3]

[W3-2] Eq. (4) displaces pseudo-anomalies using the cluster centroid, radius, and direction vectors in Euclidean input space, even though the paper argues that input-space distances are unreliable for heterogeneous tabular columns. Could the resulting pseudo-anomalies, together with the subsequent unconstrained optimization, move outside the valid range of the original data (e.g., out-of-range values or invalid categorical encodings)? If so, how does this affect the model’s ability to distinguish realistic anomalies?

[Response] Thank you for the insightful question. To address your concern on out-of-range values, we provide statistics of the optimized pseudo-anomalies and additional experiments in which they are clipped to the min-max range of the normal samples.

First, we measure how many feature values of the optimized pseudo-anomalies lie within the min-max range of the normal samples. Specifically, for each feature, we compute the minimum and maximum over the normal training samples and the range width w=max⁡−min⁡w = \max - \min, and count the fraction of the feature values of all pseudo-anomalies that fall within this range and within the range extended by 0.1w0.1w on each side. This fraction is computed for each dataset and then averaged over all ADBench datasets. As shown in the table below, 50.4% of the values lie inside the normal range, and 69.4% lie within the range extended by 0.1w0.1w.

Since a portion of the values lies outside the normal range, we further investigate whether these values affect the detection performance by clipping the optimized pseudo-anomalies to each of these ranges and comparing the detection performance. As shown in the table below, the clipped variants degrade the detection performance, with the largest drop when the pseudo-anomalies are restricted to the normal range. This suggests that placing some feature values of the pseudo-anomalies outside the normal range may contribute to making them effective anomaly classes for detecting real anomalies.

RangeValues inside the rangerAUC
[min⁡,max⁡][\min, \max]50.4%0.743
[min⁡−0.1w,max⁡+0.1w][\min - 0.1w, \max + 0.1w]69.4%0.771
No clipping (ours)100%0.781

Comments on the paper

3 comments

Dear Reviewers,

This is a friendly reminder to check the authors' responses to your reviews and leave a follow-up comment if appropriate. Your engagement in the discussion is greatly appreciated.

Thank you!

AC

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewers and AC,

We appreciate your valuable feedback, which has greatly improved our work, and would like to kindly remind you of our rebuttal. We have carefully responded to each comment with additional experiments, analyses, and clarifications, and an overview of these updates is available in the general response.

If you have any remaining questions, we would be happy to discuss them further.

Thank you for your time and consideration.

Best regards,
Authors

AuthorsOpenReview ↗

Gentle Reminder (2 Days Left in the Discussion Period)

Dear Reviewers and AC,

We appreciate your time and effort in reviewing our work. As only two days remain in the discussion period, we would like to kindly remind you of our rebuttal. We have responded to each comment with additional experiments, analyses, and clarifications, and an overview of these updates is available in the general response.

We would be grateful to hear whether our responses have addressed your concerns, and even a brief comment would be greatly appreciated. If any questions remain, we would be happy to discuss them within the remaining time.

Thank you for your time and consideration.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.