acceptodds
Under review as a conference paper at ICLR 2027

LENS-AD: Organizing LLM Knowledge for Few-Shot Tabular Anomaly Detection

Abstract

Tabular anomaly detection becomes particularly challenging under extreme label scarcity, where limited observations provide insufficient evidence to reliably characterize abnormal patterns. Such limited evidence motivates the use of Large Language Models (LLMs) to translate task-related domain knowledge into explicit features, revealing latent concepts and feature relationships absent from raw tables. However, this expanded representation also enlarges the learning space: under scarce supervision, detectors may confuse spurious relationships with meaningful anomaly mechanisms. Our key insight is that LLM knowledge should not only expand what the detector can observe, but also constrain how the expanded information is used. Based on this insight, we propose LENS-AD, which organizes original and knowledge-expanded features into overlapping semantic subspaces, each encoding a candidate anomaly mechanism. Because target data are insufficient to reliably determine the relative importance of these subspaces, LENS-AD learns a mapping from subspace descriptors to aggregation weights across source datasets and applies it to combine their predictions without target validation labels. Extensive experiments on 20 real-world tabular anomaly detection benchmarks show that LENS-AD performs competitively under both few-labeled-anomaly and few-sample settings. Further intermediate analyses and ablation studies validate the effectiveness of the proposed framework.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Anomaly Detection, Foundation Models, LLM Knowledge

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ga7aOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
3(+2)
Read review
Paper summary

LENS-AD uses a large language model (LLM) to construct derived features and overlapping prediction views from task descriptions and column metadata. A TabPFN or TabICL predictor is fitted to each view and to the full feature set using the available anomaly training labels. Semantic-Guided Meta-Aggregation (SGMA) learns from labeled source tasks how to predict view weights, then combines target predictions into an anomaly ranking without target validation labels. All splits and seeds of the target dataset are excluded from meta-training. Experiments cover all 20 ReTabAD task, six setings with few labeled anomalies or few training rows, and three seeds. The contribution is the integration of constrained feature construction, semantic prediction views, and transferred aggregation, with more consistent benefits when few anomalies are labeled.

Critical strengths
  1. I appreciate the practical problem the paper addresses and the constraints placed on knowledge generation. Feature and view proposals use metadata without access to sample values or evaluation labels, and a fixed operator library controls how transformations are executed. This makes the resulting features inspectable and avoids choosing them through unreliable target validation feedback. The evaluated dataset is also excluded from meta-training across all splits and seeds. These are useful design choices for making domain knowledge usable when meaningful metadata is available and target supervision is scarce (Sections 3.2-3.4, Appendices C-D and E.1).

  2. The reported results support the practical value of the complete workflow on metadata rich tasks. Table 4 reports mean area under the precision-recall curve (AUPR) of 0.616 across six settings, compared with 0.600 for the raw all-feature predictor and 0.602 for expansion alone. The expanded system also improves over raw semantic views using the same aggregation architecture by 0.018 and 0.019 at five and ten labeled anomalies. The parent feature controls and matched member stacking comparison add evidence that the integration is useful. ReTabAD is an appropriate metadata rich benchmark, and using all 20 tasks with two base predictors provides a reasonable evaluation of the stated setting (Tables 3-5, Appendices A.1, C.5, and E.5).

Critical weaknesses
  1. As currently presented, the novelty appears incremental, and the specific methodological contribution needs a clearer explanation. SemanticMask (Tao et al., 2024) already uses column semantics and an optional task description to construct contrastive views, while existing LLM-based feature-engineering methods generate derived features. LENS-AD adds mechanism-oriented prediction members and transferred aggregation, but I would like a more precise account of what these additions contribute and why semantic organization should help under scarce supervision. A brief comparison of the semantic inputs, feature/view construction, and aggregation, linked to supporting results, would help establish the distinction. Table 4 supports the value of the complete framework beyond feature expansion alone, but leaves the specific contribution of semantic guidance less clearly separated from the broader benefits of subspace ensembles and learned aggregation (Sections 2-3 and Table 4).

  2. I also found the method difficult to follow, particularly how generated features become prediction views and how the member predictions are combined into an anomaly score. Figure 2 conveys the overall stages, but the per-view predictors, label inputs, and distinction between source-side learning and target inference are not sufficiently clear. It aggregation equation should also be consistent with the scoring rule in the text. A worked example following one normal row and one anomalous row through the full pipeline would make these connections easier to understand. The method's reliance on meaningful column semantics and task metadata also needs a clearer discussion of applicability. The minimum metadata requirements, any fallback behavior, and the expected usefulness of semantic guidance when descriptions are missing, redacted, or unreliable should be stated explicitly (Figure 2, Sections 3.2-3.4, and Appendices A.1, D.2, and E.1-E.4).

Questions for authors
  1. In my experience, many datasets do not always have clearly defined or meaningful semantic columns or metadata this is usable. This is especially prevalent in industries like finance and healthcare were datasets are intentionally opaque or redacted. How does this approach fare on cases like this? I do not see this in the paper anywhere, and it appears to be a significant limitation to this method. More specifically, what semantic metadata does the method minimally require, and what happens when column or metadate descriptions are missing, redacted, or unreliable? Please clarify any fallback behavior and the settings in which the semantic guidance is expected to be useful.

  2. Additionally, it is unclear to me which elements or specific contributions of LENS-AD provide a significant methodological advance or novelty over prior work. SemanticMask, as mentioned in the related work section, already uses column semantics, and its task-aware variant incorporates a task description when constructing contrastive views. LLM-based feature engineering also seem quite similar in that they already generate derived features. What specifically does LENS-AD add to these approaches? A brief comparison of the semantic inputs, feature/view construction, and aggregation, linked to supporting results, would help. In particular, why should semantic organization help under scarce supervision, and what does Table 4 establish about its specific contribution relative to the broader benefits of subspace ensembles and learned aggregation?

  3. Finally, as mentioned, found the method difficult to follow. It took several rereads and section cross referencing before I felt like I understood it. Could you walk through one normal row and one anomalous row from the original features to the final anomaly score? Please show how features are generated, assigned to subspaces, used by predictors fitted with the available anomaly labels, and ultimately combined. Figure 2 conveys the overall stages, but I still struggled to reconstruct the prediction procedure. Explicitly showing the per-view predictors and label inputs, separating source-side learning from target inference, and making the scoring equation consistent with the text would improve clarity.

8 comments
AuthorsOpenReview ↗

Response to Reviewer ga7a (1/3)

Q1. "How does the availability and quality of semantic metadata affect the performance of LENS-AD?"..."What semantic metadata does the method minimally require, and what happens when column or metadate descriptions are missing, redacted, or unreliable? clarify fallback behavior and the settings in which the semantic guidance is expected to be useful."

A1. Thank you for raising this important concern. We agree that the minimum semantic requirements and fallback behavior of LENS-AD should be better clarified. We therefore conducted additional experiments with different levels of metadata availability, progressively removing dataset descriptions, feature descriptions, and interpretable feature names across the LKE–SSD–SGMA pipeline. We organize the results into two analyses.

(1) How do dataset descriptions and feature descriptions affect performance?

We first examine the contributions of dataset descriptions and feature descriptions while retaining interpretable feature names.

Table 1.1 Effect of Dataset Descriptions and Feature Descriptions

Metadata settingDataset DescriptionsFeature DescriptionsAUCPR# LKE Features
Full metadata✓✓0.616430.2
w/o Feature descriptions✓—0.610030.2
w/o Dataset descriptions—✓0.607133.5
w/o both descriptions——0.605138.1

The results indicate that dataset descriptions contributes more to performance than detailed feature descriptions.

(2) What happens when feature names are anonymized or all semantic information is removed?

We further examine the model's behavior when interpretable feature names are unavailable or when no semantic information can be provided.

Table 1.2 AUCPR of Dataset Descriptions and Feature Names (w/o Feature Descriptions)

Metadata settingDataset DescriptionFeature NamesAUCPR# LKE Features
Dataset descriptions + feature names✓✓0.610030.2
Feature names only—✓0.605138.1
Dataset description only✓Anonymous0.605925.8
No semantic information—Anonymous0.60540

As shown in Table 1.2, neither dataset descriptions nor feature names alone noticeably improve AUCPR (approximately 0.605), whereas providing both yields 0.610. This highlights a limitation of LENS-AD: effective semantic guidance requires both dataset context and interpretable feature information. When neither is available, LKE generates no additional features.

Overall, these results highlight two findings: (1) Curated feature descriptions are helpful but not essential; (2) LENS-AD relies on both dataset context and interpretable feature information to effectively leverage semantic knowledge. We will incorporate these findings into the revised ablation study and add a dedicated Limitations and Discussion section (Section 5).

AuthorsOpenReview ↗

Response to Reviewer ga7a (2/3)

Q2.1. What are the specific methodological advances of LENS-AD over existing approaches, particularly SemanticMask and LLM-based feature engineering, in terms of semantic inputs, feature/view construction, and aggregation?

A2.1. Methodological novelty and comparison with prior work.

We thank the reviewer for this question. While prior work has explored column semantics and LLM-based feature engineering, LENS-AD addresses how to complement scarce target supervision with LLM knowledge, using it to guide feature expansion, mechanism-oriented prediction, and aggregation. The methodological differences are summarized below.

Table 2.1: Methodological comparison with related approaches.

MethodKey MotivationSemantic InputsLLM Sample AccessFeature / View ConstructionPrediction AggregationMeta-learning
SemanticMaskBetter contrastive viewsColumn & task semanticsNo LLMSemantic-aware maskingNo aggregationNo
CAAFEBetter feature discoveryColumn & task semanticsSample valuesIterative feature generationNo aggregationNo
FeatLLMFew-shot feature learningColumn & task semanticsLabeled examplesRule generation + baggingEnsemble averagingNo
LENS-AD (Ours)Complementing scarce supervision with LLM knowledgeColumn & task semanticsNo samplesFeature expansion + mechanism groupingSelective view weightingYes (view weighting)

Unlike SemanticMask, which uses column semantics and task descriptions to construct masked views for self-supervised contrastive learning, LENS-AD uses LLM knowledge to derive new features and organize both original and generated features into mechanism-oriented prediction views. Each view serves as an independent anomaly predictor trained with scarce anomaly labels, with SGMA further learning how to aggregate their predictions across tasks.

Additional SemanticMask comparison. Although originally designed for one-class AD with clean normal data, SemanticMask was additionally evaluated under our few-shot protocol using the same splits. Disclosed anomalies were excluded from its unlabeled training pool, while undisclosed anomalies remained as contamination.

Table 2.2: Mean AUCPR over 20 datasets and six few-shot settings (three seeds; TabPFN backbone for LENS-AD).

MethodMean AUCPR (Rank)
CAD0.2573 (13)
WSAD-DT0.3468 (12)
SemanticMask0.3777 (11)
SemanticMask (task-aware)0.3843 (10)
PReNet0.4568 (9)
DevNet0.4607 (8)
FeatLLM0.4649 (7)
XGBoost0.4755 (6)
XGBOD0.5078 (5)
CAAFE0.5795 (4)
CatBoost0.5803 (3)
TabPFN0.6001 (2)
LENS-AD (Ours)0.6164 (1)

LENS-AD achieves the highest mean AUCPR (0.6164), outperforming even the task-aware SemanticMask variant (0.3843) by 23.2 percentage points under the evaluated few-shot protocols.


Q2.2. Why semantic organization helps under scarce supervision.

A2.2. Under scarce supervision, anomaly detection faces two challenges:

  1. Difficulty in discovering anomaly mechanisms: With only a few labeled anomalies, it is difficult to reliably identify which feature combinations are informative for anomaly detection.
  2. Difficulty in selecting anomaly mechanisms: Different feature combinations may capture different aspects of anomalous behavior, but limited labels make it difficult to determine which mechanisms are more useful for identifying anomalies.

Importantly, the three components of LENS-AD are designed to work together rather than independently. Table 4 highlights this complementary effect: LKE alone yields only a modest gain (0.600 to 0.602), while applying SSD and SGMA directly to the raw feature space also brings limited improvement (0.600 to 0.605). In contrast, combining LKE with SSD and SGMA achieves 0.616. This suggests that LLM knowledge contributes not merely by expanding the feature space or organizing existing features, but through their combination: it enhances both what anomaly-relevant patterns can be captured and how they are modeled and integrated through complementary prediction views.

Together, these results support our central insight: LLM knowledge can be more effectively utilized under scarce supervision when it guides not only feature expansion, but also the organization and aggregation of prediction experts.

AuthorsOpenReview ↗

Response to Reviewer ga7a (3/3)

Q3. "Finally, as mentioned, found the method difficult to follow. It took several rereads and section cross referencing before I felt like I understood it. Could you walk through one normal row and one anomalous row from the original features to the final anomaly score? Please show how features are generated, assigned to subspaces, used by predictors fitted with the available anomaly labels, and ultimately combined. Figure 2 conveys the overall stages, but I still struggled to reconstruct the prediction procedure. Explicitly showing the per-view predictors and label inputs, separating source-side learning from target inference, and making the scoring equation consistent with the text would improve clarity."

A3. Clarification of the end-to-end prediction process. We thank the reviewer for this suggestion. We have redrawn Figure 2 to clarify the complete LENS-AD workflow and added Figure 3 to distinguish SGMA's source-side meta-training from target-side inference. We have also revised the method descriptions to improve clarity and consistency throughout the paper. Below, we walk through two real Automobile test samples under NLA=5 (seed 42): one normal and one anomalous.

(1) Dataset-level construction (LKE and SSD). Both stages use task descriptions and feature semantics, without accessing sample values or labels.

  • LKE: The LLM uses domain knowledge to derive new concepts and relationships not explicitly represented in the original features. For example, the original body-style column contains five categories (convertible, hardtop, hatchback, sedan, wagon). The LLM groups convertible and hardtop into a new sporty-body concept, creating a binary feature that indicates whether a car belongs to this group. It also derives a power-to-weight ratio from horsepower and curb-weight, since a car's acceleration capability depends on both engine power and vehicle mass. This creates a new, physically meaningful indicator of vehicle performance that may be relevant to insurance risk. These rules are compiled and applied to all rows, expanding 25 original features to 74 features (49 generated).
  • SSD: Using the task description and the semantics of all 74 features, the LLM proposes two candidate anomaly mechanisms: Luxury Make Risk and High-Performance Engine Risk. It assigns 16 and 53 relevant features to these two prediction views, respectively (e.g., sporty-body and power-to-weight belong to the Performance view). The remaining 5 features form an exploratory view, while a full-feature view using all 74 features serves as a fallback, yielding 4 prediction members in total.

(2) Instance-level scoring. Each of the four prediction members has an independent TabPFN predictor, fitted on the same 111 target training rows with only 5 revealed anomaly labels (all other rows are treated as negative). SGMA learns how to weight these predictors from the other 19 source datasets, without using target test labels. The resulting weights are shared by both test samples.

Table 3.1: End-to-end example on Automobile (NLA=5, seed 42).

Normal Sample (row 28)Anomaly Sample (row 131)Aggregation Weight
Original features
Make / body styleHonda / sedanToyota / convertible
Horsepower / weight86 / 2304116 / 2975
Generated features
Sporty-body indicator01
Power-to-weight0.03730.0390
View predictions
Luxury View0.00530.07930.263
Performance View0.00650.36150.297
Exploratory View0.00630.09660.135
Full-feature View0.00230.33580.304
Final anomaly score0.004450.21006
Anomaly rank (out of 48)421

The final score is computed by taking a weighted average of the view predictions in logit space, followed by a sigmoid transformation:

s(x)=σ(∑mwˉmlogit⁡(pm(x(m)))).s(x)=\sigma\left(\sum_m \bar{w}_m \operatorname{logit}(p_m(x_{(m)}))\right).

Here, pm(x)p_m(x) is the prediction of the mm-th view, and wˉm\bar{w}_m is its SGMA weight. Higher scores indicate stronger evidence of an anomaly. In this example, the anomalous car receives a score of 0.21006 and ranks first among the 48 test samples, while the normal car scores 0.00445 and ranks 42nd.

This example illustrates the full process: LLM-generated features → mechanism-based views → independently fitted predictors → source-learned aggregation → final anomaly scores.

Reviewer_Hr8tOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents LENS-AD, a framework for few-shot tabular anomaly detection that uses LLMs to generate semantic features, groups features into mechanism-oriented subspaces, and learns to aggregate subspace predictions from other datasets. Experiments on 20 ReTabAD tasks show competitive performance across few-labeled-anomaly and few-sample settings, with ablations examining the contributions of the main components.

Critical strengths
  1. The paper addresses a relevant problem: using domain semantics when anomaly labels or training samples are scarce.
  2. The framework combines feature generation, semantic subspaces, and cross-task aggregation in a coherent pipeline.
  3. The experiments include component ablations, comparisons with strong tabular baselines, and analyses of different LLMs and base predictors.
Critical weaknesses
  1. The paper focuses on tabular anomaly detection, but it does not compare LENS-AD with several relevant methods, including AnoLLM and Causal-TAD. These comparisons would help clarify the method’s advantages over approaches that also incorporate language models or causal information. [AnoLLM] Tsai, Che-Ping, et al. "AnoLLM: Large language models for tabular anomaly detection." International Conference on Learning Representations. Vol. 2025. 2025. [Causal-TAD] Wang, Ruiqi, et al. "CausalTAD: Injecting causal knowledge into large language models for tabular anomaly detection." arXiv preprint arXiv:2602.07798 (2026).

  2. The description of SGMA’s teacher models is unclear. The paper states that a logistic-regression teacher is trained for each source dataset, but does not explain how this design scales when the source data contains many datasets or groups. It is also unclear how SGMA should be applied to a single large, heterogeneous dataset: should it train one teacher model for the whole dataset, or multiple teachers for different subsets? Clarifying the unit used to define a source dataset, and evaluating the approach under these different data settings, would make the method easier to understand and reproduce.

Questions for authors

See the comments above

4 comments
AuthorsOpenReview ↗

Response to Reviewer Hr8t (1/2)

Q1. The paper focuses on tabular anomaly detection, but it does not compare LENS-AD with several relevant methods, including AnoLLM and Causal-TAD. These comparisons would help clarify the method’s advantages over approaches that also incorporate language models or causal information.

A1. We appreciate the reviewer's suggestion to consider these methods. AnoLLM and CausalTAD fine-tune LLMs directly on serialized data as generative anomaly detectors. Conversely, LENS-AD uses LLMs solely as semantic knowledge providers to extract and organize features, delegating the actual prediction to efficient tabular models. Thus, LENS-AD focuses on enriching and organizing the information available to tabular learners, rather than directly modeling normal data distributions with an LLM.

AnoLLM Comparison: To provide an empirical reference, we also reproduced AnoLLM under our experimental setup, obtaining an average AUC-PR of 0.4904. The results are summarized below.

MethodTraining protocolAUC-PR
LENS-AD (few-shot)10 labeled training instances0.555
AnoLLMNormal-only training0.4904
LENS-AD (unsupervised)Isolation Forest, full training set0.404

AnoLLM achieves intermediate performance between the two LENS-AD settings. While the differing protocols and base detectors preclude a strict comparison, the results suggest that direct LLM-based modeling of normal data can provide a competitive alternative, whereas incorporating even a small number of anomaly labels can further benefit LENS-AD.

CausalTAD: We were unable to locate the original implementation of CausalTAD. Given the limited time and computational resources available during the rebuttal period, conducting a faithful and rigorous empirical comparison would be challenging. We have therefore clarified its methodological differences from LENS-AD in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Hr8t (2/2)

Q2.1. "The description of SGMA’s teacher models is unclear. The paper states that a logistic-regression teacher is trained for each source dataset, but does not explain how this design scales when the source data contains many datasets or groups. ... Clarifying the unit used to define a source dataset."

A2.1. We thank the reviewer for highlighting this ambiguity. We have redrawn Figure 2 and added Figure 3 to clarify SGMA's source-side meta-training and target-side inference. We also clarify the training procedure and provide additional experimental evidence below.

  • Source-task definition: A source dataset is a dataset used to train SGMA's meta-regressor. Each source dataset DiD_i under few-shot setting ss defines a source task (Di,s)(D_i,s).
  • Teacher construction: For each source task, a logistic-regression model estimates the aggregation weights of all its prediction views using their predictions and available labels. We then train a shared meta-regressor across source tasks to predict each view's aggregation weight from its descriptors.

For an unseen target dataset, LENS-AD first fits a predictor for each view using the available anomaly labels. The meta-regressor then predicts the aggregation weights from the resulting view descriptors. Thus, the view predictors are fitted on the target dataset, while the aggregation weights are transferred from source tasks.


Q2.2. It is also unclear how SGMA should be applied to a single large, heterogeneous dataset: should it train one teacher model for the whole dataset, or multiple teachers for different subsets?

A2.2. LENS-AD constructs prediction views by grouping features, not samples. Thus, a large dataset containing different groups still uses one teacher per source task, provided they share the same anomaly detection objective. There is no need to train separate teachers for different sample groups.

Regarding the number of source datasets, Appendix E.6 (Fig. E2) already evaluates 3–19 source datasets, showing that adding more sources mainly improves aggregation stability.

To examine applicability to large, heterogeneous datasets, we additionally summarize results for the five largest ReTabAD datasets containing both numerical and categorical features.

Table 2.1: SGMA versus uniform aggregation on the five largest mixed-type ReTabAD datasets

Dataset# Samples# Features# CategoricalUniformSGMAΔ AUCPR
census50,00041290.25030.2942+0.0439
covertype50,0001220.91450.9420+0.0276
credit30,0002380.31150.3149+0.0035
backdoor29,2234220.35740.3541−0.0033
campaign7,8421650.48310.4845+0.0014
Average———0.46340.4779+0.0146

SGMA improves over uniform aggregation on four of five datasets, with an average AUCPR gain of 0.0146.


Q2.3. Evaluating the approach under these different data settings, would make the method easier to understand and reproduce.

A2.3. Generalization to unseen datasets. We further evaluate LENS-AD on three unseen MacrOData datasets. The SGMA meta-regressor is trained exclusively on the original 20 ReTabAD datasets, with all three new datasets excluded from meta-training. We also compare against uniform aggregation using identical prediction views to isolate the benefit of transferred weights.

Table 2.2: AUCPR on three unseen MacrOData datasets (rank in parentheses).

MethodCorporateBondDefaultCourierShipmentStatusRestaurantInspectionsAverage
CatBoost0.276 (4)0.197 (4)0.252 (4)0.242 (4)
XGBoost0.247 (5)0.194 (6)0.251 (5)0.231 (5)
XGBOD0.239 (6)0.190 (9)0.250 (6)0.226 (6)
WSAD-DT0.100 (8)0.191 (8)0.249 (8)0.180 (8)
DevNet0.119 (7)0.193 (7)0.249 (7)0.187 (7)
PReNet0.094 (9)0.196 (5)0.247 (9)0.179 (9)
CAD0.083 (10)0.188 (10)0.247 (10)0.173 (10)
TabPFN0.339 (2)0.201 (1)0.254 (3)0.265 (3)
LENS-AD (Uniform)0.331 (3)0.199 (3)0.264 (2)0.265 (2)
LENS-AD (ours)0.349 (1)0.199 (2)0.265 (1)0.271 (1)

LENS-AD achieves the best average AUCPR (0.271), compared with 0.265 for both TabPFN and uniform aggregation, and ranks first on two of the three datasets. However, the gains over uniform aggregation vary across datasets, with the largest improvement on CorporateBondDefault and nearly unchanged performance on the other two. These results suggest that SGMA's learned weights can transfer to unseen datasets, although the benefits depend on the target dataset.

Reviewer_Hr8tOpenReview ↗

Thanks the authors for the response.

Thanks the authors for the response. Q1: The results is convincing. I recommend the authors to discuss these related baselines (and compared them in the experiments if possible). Q2: Thanks for the clarification. Look forward to see these explanations incorporated in the final version.

The authors have fully addressed my concerns, and I will raise my score to 4 accordingly.

Thank you for your constructive feedback. We have incorporated the discussion of related baselines and the clarifications on SGMA into the revised manuscript. We appreciate your suggestions, which have helped improve the clarity and completeness of the paper.

Reviewer_92eaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LENS-AD, a framework for few-shot tabular anomaly detection that leverages LLM-derived semantic knowledge. It consists of three components: LLM Knowledge Expansion (LKE), which generates domain-informed features; Semantic Subspace Discovery (SSD), which organizes features into mechanism-oriented prediction views; and Semantic-Guided Meta-Aggregation (SGMA), which predicts aggregation weights using meta-learning across source datasets.

The method is evaluated on 20 ReTabAD datasets under six supervision-scarcity settings. Experiments with TabPFN and TabICL backbones demonstrate competitive AUCPR performance, supported by component ablations and robustness analyses.

Critical strengths

1. Meaningful problem setting. The paper addresses the challenge of exploiting domain knowledge when anomaly labels are scarce. Organizing LLM-generated features into semantic subspaces is a reasonable approach to introducing domain-informed inductive biases.

2. Comprehensive empirical analysis. The authors provide component ablations, comparisons with random subspaces and target-label stacking, and robustness experiments across LLMs, base predictors, and source-task configurations. These analyses provide useful evidence supporting the proposed framework.

Critical weaknesses

1. Limited methodological novelty and insufficient justification of SGMA.

The framework combines established techniques, including LLM-based feature engineering, semantic-aware view construction, and cross-task meta-learning. In particular, SGMA is conceptually related to MetaOD (NeurIPS 2021) and MetaWCE (Information Sciences 2023), which explore transferring model-selection or weighting information across datasets.

More importantly, SGMA assumes that aggregation weights optimized on source datasets can be predicted from descriptors of newly constructed subspaces on unseen datasets. However, these weights depend on the competing predictors, their correlations, and regularization, and may not represent stable or transferable measures of subspace importance. The paper provides limited justification for this assumption, particularly across different domains.

2. Insufficient isolation of semantic contributions and limited applicability.

Although the authors provide several ablations, the improvements attributable specifically to semantic knowledge remain unclear. For example, LKE alone improves mean AUCPR only from 0.600 to 0.602 (Table 4). Comparisons with non-semantic feature grouping and SGMA without semantic descriptors would help determine whether the gains primarily arise from semantic reasoning or conventional ensemble and meta-learning effects.

Furthermore, all main experiments use ReTabAD, which provides curated dataset and column descriptions. Since LKE and SSD rely heavily on such metadata, their effectiveness on datasets with incomplete descriptions or anonymized feature names remains unverified. This limits the demonstrated practical applicability of the framework.

Questions for authors

Q1. Validity of SGMA. What justifies transferring source-optimized aggregation weights to unseen datasets? How stable are the teacher weights under correlated subspaces, and how well does the learned mapping generalize across domains?

Q2. Semantic contribution. Could the authors evaluate SGMA without semantic descriptors and compare SSD against correlation-based or feature-importance-based subspace construction under the same aggregation procedure?

Q3. Metadata dependence. How does LENS-AD perform without curated column descriptions, such as when only column names or anonymized feature identifiers are available?

Q4. Related work and baselines. How does SGMA differ from MetaOD and MetaWCE? Additionally, why is ReTabAD's direct LLM-based anomaly detector not compared under a compatible evaluation protocol?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 92ea (1/3)

Q1.1. What justifies transferring source-optimized aggregation weights to unseen datasets?

A1.1. Thank you for raising this important concern. SGMA does not directly transfer source-specific aggregation weights to unseen datasets. Instead, it learns a mapping from view meta-features to aggregation weights across source tasks.

SGMA assumes that a view's usefulness follows shared patterns across datasets, even when the underlying anomaly mechanisms differ. Specifically, useful views tend to capture core anomaly mechanisms (semantic relevance), provide complementary evidence (predictive complementarity), and remain learnable under scarce supervision (learnability).

We examine Spearman correlations between representative meta-features and teacher weights across 11 datasets. The correlation signs match our expectations in 63.6%–81.8% of datasets, with median Spearman coefficients ranging from 0.21 to 0.70. It suggest that view meta-features capture recurring patterns of view importance across datasets.


Q1.2. How stable are the teacher weights under correlated subspaces?

A1.2. We acknowledge that highly correlated views can lead to unstable or non-identifiable teacher weights. However, weight instability does not necessarily imply prediction instability, particularly when weights are redistributed among views producing similar predictions.

Theoretical explanation. For two views with logits (z_1,z_2) and weights ((w,1-w)), changing the weights to ((w',1-w')) changes the aggregated logit by

\leq \varepsilon |w-w'|,\quad \text{if } |z_1-z_2|\leq\varepsilon.$$ Thus, aggregation is less sensitive to weight redistribution when view predictions are similar. **Empirical verification.** We bootstrap source data to assess weight instability and aggregation logit sensitivity. Table 1.1 shows that highly correlated views exhibit greater weight variability, but lower Relative Logit Sensitivity. **Table 1.1 Weight Variability and Logit Differences Across Correlated Views** |Metric|Low (ρ < 0.3)|Medium (0.3 ≤ ρ < 0.9)|High (ρ ≥ 0.9)| |---|---:|---:|---:| |Teacher Weight SD |0.0357|0.0798|0.1276| |Relative Logit Sensitivity ↓|1.3540|1.1108|0.4597| *Note: Relative Logit Sensitivity is the normalized RMS difference between two views' logits; lower values indicate less aggregation change per unit weight transfer.* We also assess end-to-end bootstrap stability. As shown in Table 1.2, SGMA's predicted weights and anomaly rankings are more stable than source teacher weights, with negligible AUCPR changes. **Table 1.2 Bootstrap Stability of SGMA** |Metric|Average| |---|---:| |**Source Datasets (Teacher)**|| |Teacher Weight TV ↓|0.2064| |Prediction Spearman ↑|0.9688| |**Target Datasets (SGMA)**|| |Meta-Predicted Weight TV ↓|0.0442| |Prediction Spearman ↑|0.9983| |Original SGMA AUCPR ↑|**0.6164**| |Bootstrap SGMA AUCPR ↑|**0.6162 ± 0.0009**| *Note: Results are averaged over 6 few-shot settings. Total Variation (TV) measures weight differences as $\mathrm{TV}(w,w')=\frac{1}{2}\sum_i|w_i-w_i'|$ (lower is more stable).* Overall, these results suggest that **weight redistribution among similar views limits its impact on aggregation**, and that SGMA maintains stable predicted weights and final predictions under bootstrap perturbations. This provides both theoretical and empirical support for the robustness of the final predictions, without requiring teacher weights themselves to be stable. --- **Q1.3. How well does the learned mapping generalize across domains?** **A1.3.** We conduct leave-one-domain-out (LODO) meta-training, excluding all datasets from the target domain from source tasks. We compare it with standard leave-one-dataset-out (LOO) meta-training and uniform aggregation. **Table 1.3 Cross-Domain Generalization under Leave-One-Domain-Out (LODO) Meta-Training** |Held-out Domain|# Datasets|Uniform|SGMA (LOO)|SGMA (LODO)|Δ LODO| |---|---:|---:|---:|---:|---:| |Healthcare|7|0.6330|0.6408|0.6409|0.0079| |Finance / Business|4|0.3763|0.3885|0.3867|0.0104| |Industrial|2|0.7676|0.7738|0.7747|0.0071| |Natural Science|6|0.7119|0.7311|0.7315|0.0197| |Cybersecurity|1|0.3574|0.3541|0.3536|-0.0038| |**Average**|**20**|0.6050|0.6164|0.6163|0.0113| *Note: AUCPR averaged over 6 few-shot settings. Δ LODO = SGMA (LODO) − Uniform.* The results show that **SGMA retains its aggregation benefits under domain-level holdout**. LODO outperforms uniform aggregation in four of five domains, with an overall AUCPR gain of 0.0113. Its performance (0.6163) nearly matches LOO (0.6164), despite excluding all target-domain source datasets. This supports the transferability of SGMA's learned weighting patterns.
AuthorsOpenReview ↗

Response to Reviewer 92ea (2/3)

Q2. Semantic contribution. Could the authors evaluate SGMA without semantic descriptors and compare SSD against correlation-based or feature-importance-based subspace construction under the same aggregation procedure?

A2. Semantic contribution. We thank the reviewer for this suggestion. We evaluate SGMA without semantic descriptors and replace SSD with correlation-based (Corr) or feature-importance-based (Imp) partitioning. All methods share the same expanded features, predictors, and aggregation protocol; SSD, Corr, and Imp use the same 30-dimensional SGMA descriptors.

Table 2.1 AUCPR comparison across 20 datasets

MethodNLA=5n=10Six-setting Avg.
SSD + Full SGMA0.55780.55620.6167
SSD + SGMA w/o Semantic0.54930.55360.6149
Corr + SGMA0.54520.54190.6103
Imp + SGMA0.53970.54980.6076

Three findings emerge. (1) Complementary semantic descriptors: Removing semantic descriptors slightly reduces AUCPR, suggesting that semantic priors provide additional guidance under scarce anomaly supervision. (2) Advantage under scarce supervision: SSD achieves the highest AUCPR, with particularly favorable results under NLA=5 and train size=10, suggesting that semantic guidance is valuable when labeled anomalies are extremely scarce. (3) Distinct semantic organization: SSD and Corr exhibit limited subspace overlap (best-match Jaccard = 0.349), while SSD produces views with lower within-view feature correlation (0.380 vs. 0.531), indicating that semantic organization is not simply equivalent to correlation-based grouping.


Q3. Metadata dependence. How does LENS-AD perform without curated column descriptions, such as when only column names or anonymized feature identifiers are available?

A3. Thank you for raising this important concern. We conduct additional experiments with different levels of metadata availability, progressively removing dataset descriptions, feature descriptions, and interpretable feature names. We organize the results into two analyses to distinguish the role of curated descriptions from that of feature names.

(1) How important are dataset descriptions and feature descriptions?

We first examine the contribution of dataset and feature descriptions while retaining interpretable feature names.

Table 3.1. AUCPR of Dataset and Feature Descriptions (Feature Names Always Available)

Metadata settingDataset DescriptionsFeature DescriptionsAUCPR# LKE Features
Full metadata✓✓0.616430.2
w/o Feature descriptions✓—0.610030.2
w/o Dataset descriptions—✓0.607133.5
w/o both descriptions——0.605138.1

The results show that both dataset and feature descriptions contribute to performance. Nevertheless, removing feature descriptions leads to a relatively small decrease, suggesting that curated descriptions are helpful but not essential when other semantic information remains available.

(2) Can feature names provide useful semantic information?

We further examine whether interpretable feature names provide meaningful guidance when feature descriptions are unavailable, by comparing them with anonymized identifiers and the complete removal of semantic information.

Table 3.2. AUCPR of Dataset Descriptions and Feature Names (w/o Feature Descriptions)

Metadata settingDataset DescriptionFeature NamesAUCPR# LKE Features
Dataset descriptions + feature names✓✓0.610030.2
Feature names only—✓0.605138.1
Dataset description only✓Anonymous0.605925.8
No semantic information—Anonymous0.60540

As shown in Table 3.2, neither dataset descriptions nor feature names alone noticeably improve AUCPR (approximately 0.605), whereas providing both yields 0.610. This highlights a limitation of LENS-AD: effective semantic guidance requires both dataset context and interpretable feature information. When neither is available, LKE generates no additional features.

Overall, these results highlight two findings: (1) Curated feature descriptions are helpful but not essential; (2) LENS-AD relies on both dataset context and interpretable feature information to effectively leverage semantic knowledge. We will incorporate these findings into the revised ablation study and add a dedicated Limitations and Discussion section (Section 5).

AuthorsOpenReview ↗

Response to Reviewer 92ea (3/3)

Q4. Related work and baselines. How does SGMA differ from MetaOD and MetaWCE? Additionally, why is ReTabAD's direct LLM-based anomaly detector not compared under a compatible evaluation protocol?

A4. We thank the reviewer for highlighting these related methods. Their key methodological differences are summarized below.

Table 4.1: Methodological comparison with related approaches.

MethodTaskKnowledge SourcePrediction ModelsCore ApproachMeta-learningLLM Usage
MetaODUnsupervised ADDetector performancePredefined unsupervised detectorsDetector selectionYesNo LLM
MetaWCEClustering (not AD)Clustering informationBase clustering algorithmsWeighted clustering ensembleYesNo LLM
ReTabADZero-shot ADLLM domain knowledgeLLMDirect anomaly scoringNoPer-sample anomaly scoring
LENS-ADFew-shot ADLLM domain knowledge + detector performanceSemantic-view-specific predictorsFeature expansion + semantic views + aggregationYesSemantic knowledge extraction & organization

Unlike MetaOD and MetaWCE, which learn to select or weight existing detectors or clustering results, LENS-AD constructs prediction views around task-specific anomaly mechanisms and learns to weight them using both semantic knowledge and information from other datasets. Our key contribution is to extend LLM knowledge beyond feature generation to guiding how anomaly mechanisms are modeled and how their predictions are combined, reducing reliance on scarce target supervision.

Additional ReTabAD comparison. Our original baselines focused on methods that train downstream predictors, whereas ReTabAD directly uses LLMs for anomaly scoring. Nevertheless, both approaches leverage the same task descriptions and feature semantics. Following the reviewer's suggestion, we evaluated ReTabAD with MiniMax-M3.1-Flash and DeepSeek-V4.1-Flash under matched data splits and information access.

Table 4.2: Mean AUCPR comparison with ReTabAD (across 11 datasets).

SettingReTabAD (MiniMax-M3.1-Flash)ReTabAD (DeepSeek-V4.1-Flash)LENS-AD
NLA = 100.5100.5530.692
Train size = 200.5260.5480.635
Mean0.5180.5500.663

LENS-AD achieves higher mean AUCPR in both settings, outperforming the stronger direct-LLM baseline by 11.3 percentage points on average. Given the substantial cost of per-sample LLM inference, we limited this comparison to 11 datasets and two representative settings. The datasets were selected solely by test-set size, covering all small- and medium-sized datasets: automobile, wine, glass, gallstone, cirrhosis, vertebral, wbc, glioma, yeast, cardiotocography, and seismic. Table 4.3 further illustrates the computational cost difference between direct LLM scoring and our reusable semantic construction.

Table 4.3: LLM token consumption of ReTabAD and LENS-AD across dataset-size groups.

GroupTest rows# DatasetsReTabAD tokens (est.)LENS-AD tokens (measured)
S (evaluated)≤10062.1–2.4M1.34M
M (evaluated)100–800510.5–11.8M1.11M
L1,400–2,400430.3–34.0M0.89M
XL≥8,7005322.4–361.3M1.20M
Total—20365–410M4.54M

Unlike direct LLM scoring, LENS-AD organizes semantic knowledge into reusable prediction structures, avoiding per-sample LLM inference. These results demonstrate the advantages of this design in both detection performance and inference efficiency within the evaluated settings.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful evaluations and constructive discussion. Their feedback has helped us clarify the contribution of LENS-AD, strengthen its empirical support, and better characterize its applicability. Below, we summarize the main points addressed during the discussion.

1. Methodological contribution and semantic attribution (Reviewers 92ea and ga7a).
We clarified how LENS-AD extends LLM knowledge beyond feature generation to mechanism-oriented prediction views and cross-task aggregation. Component ablations support the complementary roles of these stages, while additional controlled comparisons show that semantic view construction improves over correlation-based and feature-importance-based grouping under the same aggregation procedure. We also clarified the distinctions from SemanticMask, existing LLM-based feature engineering, MetaOD, and MetaWCE.

2. Magnitude and applicability of semantic gains (Reviewer ga7a, follow-up). We added paired bootstrap comparisons with No Semantics baseline, showing that semantic information improves AUCPR on a majority of datasets. The modest overall gain reflects smaller improvements under the training-size settings, whereas gains at low anomaly-label budgets are statistically significant. We further examined when semantic guidance helps or hurts, finding that semantic guidance is most useful when credible domain knowledge aligns with discriminative patterns in the target data. Contrasting Covertype and Gallstone cases illustrate how this alignment can enable effective detection and how its absence can lead to performance degradation.

3. SGMA validity, stability, and generalization (Reviewers 92ea and Hr8t).
We clarified SGMA’s source-task definition and its learned mapping from view descriptors to aggregation weights. In response to 92ea, we provided a theoretical explanation of why aggregation can remain stable despite weight changes among views with similar predictions. Bootstrap experiments further show that changes in source teacher weights have little impact on target anomaly rankings and detection performance. Leave-one-domain-out evaluation also preserves average performance. In response to Hr8t, we additionally evaluated large mixed-type datasets and three unseen MacrOData datasets, providing further evidence of applicability and transfer beyond the original benchmark.

4. Metadata requirements and limitations (Reviewers 92ea and ga7a).
We separately evaluated missing and misleading metadata. Metadata ablations show that curated feature descriptions are helpful but not essential when dataset context and interpretable feature names remain available. Without semantic information, LKE generates no additional features, while SSD primarily groups features by logical data type. The framework can still operate in this setting, obtaining modest ensemble gains from the simple SSD-based views. Following ga7a’s further feedback, we also tested corrupted dataset and column descriptions, finding that misleading column descriptions can distort semantic groupings and be more harmful than missing metadata. We explicitly distinguish these limitations in the revised discussion.

5. Additional baselines and clearer presentation (Reviewers 92ea, Hr8t, and ga7a).
We added comparisons with direct LLM scoring through ReTabAD (92ea), AnoLLM (Hr8t), and SemanticMask (ga7a), explicitly stating the evaluation protocols and comparability limits. We also discussed CausalTAD’s methodological differences (Hr8t). To clarify the workflow, we redrew Figure 2, added Figure 3 to distinguish source-side meta-training from target inference (Hr8t and ga7a), and provided an end-to-end example tracing one normal and one anomalous sample (ga7a).

Thank you so much for devoting your time to making LENS-AD better. We hope our rebuttal has resolved your main concerns. We have revised the manuscript to improve clarity and will incorporate the latest follow-up analyses and clarifications into the revised manuscript and appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.