acceptodds
Under review as a conference paper at ICLR 2027

Tabular Anomaly Detection via Causal Matryoshka Trajectories

Abstract

Structural anomalies are samples where individual feature values appear plausible but their combination violates underlying dependencies. Existing anomaly detection (AD) methods often rely on a single fixed latent representation to separate normal and abnormal data, making localized structural violations susceptible to signal dilution in high-dimensional spaces. We propose NestedAno, an unsupervised tabular AD framework that learns a hierarchy of nested representations, inspired by Matryoshka Representation Learning, and progressively organizes them using topologically aligned causal reconstruction objectives. At inference time, it measures local deviation at each nested scale to construct an anomaly trajectory for each sample. The integrated deviation of this trajectory from the normal reference provides the final anomaly score, enabling anomaly evidence to be captured across multiple scales. Extensive experiments on 30 continuous and mixed-type datasets show that NestedAno achieves the highest average AUC-ROC among 17 representative baselines, yielding superior detection accuracy while providing highly interpretable trajectory separation between normal and anomalous data.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Anomaly Detection, Tabular Data, Causal, Matryoshka Embeddings, Matryoshka Representation Learning, Generative Models, GANs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_E8FTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper extends CausalAno by replacing its single latent representation with causally guided Matryoshka representations at multiple nested scales. Each scale is aligned with progressively larger causal variable subsets, and the resulting multi-scale anomaly deviations are combined into a trajectory-based anomaly score.

Critical strengths
  1. The paper improves upon the single fixed latent representation used in CausalAno by introducing multi-scale nested representations, which can capture anomaly evidence at different representation capacities and potentially reduce the dilution of localized structural anomalies.
  2. The paper provides useful empirical analysis on the limitation of covariance-based Mahalanobis scoring. The observed deviations from multivariate normality support the use of local, non-parametric anomaly scoring instead of relying on a single global covariance geometry.
Critical weaknesses
  1. The choice of progressively increasing latent dimensionality is insufficiently justified. The method adopts exponentially increasing representation sizes (e.g., 128/256/512/1024), implicitly assuming that incorporating a later causal stage requires substantially more representation capacity. However, it is unclear why adding one level of causal information should require approximately twice the latent dimensionality. The paper does not compare against fixed-dimensional representations (e.g., 128 dimensions for all stages), linearly increasing dimensions, or other adaptive designs. Prior work has also shown that the standard Matryoshka design is not necessarily optimal across embedding lengths [2]. Therefore, the necessity of the proposed progressive-capacity design remains unclear.
  2. The benefit of the nested representation over independently learned representations is not established. The method assumes that representations for different causal stages should be nested prefixes of a shared latent vector. However, an alternative is to learn an independent representation for each causal stage while keeping the same causal supervision and trajectory-based scoring. Such a comparison is important because previous work on Matryoshka-style training has shown that jointly nested representations do not necessarily outperform representations trained separately for specific configurations [1]. Without this baseline, it is difficult to determine whether the improvement comes specifically from the proposed nested structure or simply from modeling anomaly evidence at multiple stages.

[1] S. Wang, S. Zhuang, B. Koopman, and G. Zuccon. 2D Matryoshka Training for Information Retrieval. arXiv:2411.17299, 2024. arxiv.org [2] T. Wen, Y. Wang, Z. Zeng, et al. Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation. ICML, 2025.

Questions for authors

Please see Critical Weaknesses 1 and 2 above. Addressing these points, particularly through the corresponding ablations and comparisons, would materially affect my assessment.

6 comments

We sincerely thank Reviewer E8FT for the constructive feedback and for recognizing the value of our multi-scale trajectory-based approach and our analysis of the limitations of Mahalanobis scoring. We appreciate the insightful questions regarding the choice of embedding dimensions and the necessity of nested representations. To address these concerns, we conducted additional ablation experiments comparing alternative dimensionality schedules and independently learned representations. We provide our responses below.

W1: Justification of Exponential Dimensionality

We appreciate the suggestion to investigate alternative dimensionality schedules. We opted for exponential scaling for three reasons:

  1. Mathematical Constraint of the Trajectory: Our trajectory formulation integrates anomaly deviations over the representation-capacity spectrum M\mathcal{M}. Specifically, Equation (9) uses the embedding dimensions mkm_{k} as the integration coordinates, requiring strictly increasing dimensions (mk+1>mkm_{k+1}>m_{k}). If all stages use the same embedding dimension (e.g., 128 for all stages), the integration intervals mk+1−mk2\frac{m_{k+1}-m_{k}}{2} become zero, making the trajectory formulation degenerate.

  2. Alignment with MRL Literature: We adopt exponential dimensionality as a practical design choice to provide progressively increasing representational capacity while maintaining a small number of nested scales. This design is consistent with commonly used Matryoshka representation learning configurations [1, 2, 3].

  3. Empirical Comparison against Linear Scaling: To directly answer your question, we conducted a comparison between exponential and linear dimensionality schedules on our Synthetic dataset (Figure 5a), which contains 200 ground-truth anomalies. The exponential configuration, M={4,8,16,32}\mathcal{M}=\lbrace 4,8,16,32 \rbrace, detects an average of 180.67 anomalies (±1.25\pm1.25), achieving a True Positive Rate (TPR) of 0.9034. The linear configuration, M={4,8,12,16}\mathcal{M}=\lbrace 4,8,12,16 \rbrace, detects 172.33 anomalies (±0.94\pm0.94), dropping the TPR to 0.8617. These results provide empirical evidence that the exponential configuration is beneficial in our setting.

Overall, our intention is not to claim that exponential scaling is theoretically optimal, but to demonstrate that a progressively expanding representation provides a useful design for the proposed multi-scale anomaly trajectory.

[1] Matryoshka Representation Learning, NeurIPS 2022

[2] Cross-lingual Matryoshka Representation Learning across Speech and Text, ACL 2026

[3] MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining, ACL 2026

W2: Nested versus Independent Representations

We appreciate this important suggestion. While learning KK independent representations for the causal stages is possible, we chose the nested architecture for two reasons: computational efficiency and representational regularization.

Because the underlying causal graph (DAG) is fixed, an independent architecture requires KK separate encoders, increasing the parameter count by a factor of KK. By contrast, the nested Matryoshka encoder projects the entire causal structure in a single forward pass.

Per your suggestion, we trained a baseline consisting of 4 independent encoders (each learning a representation for its corresponding cumulative causal subset) on our Synthetic dataset (Figure 5a).

  • Efficiency: The independent architecture increased the runtime by 48.1% (0.80 minutes vs. 0.54 minutes for the nested architecture).
  • Performance: The independent embeddings achieved a TPR of 0.8934, compared to 0.9034 for the nested embeddings.

These results indicate that jointly learning nested representations achieves slightly better detection performance while reducing computational cost relative to independently learning representations for each stage. We believe that the shared-prefix architecture also provides a useful inductive bias by encouraging different representation scales to retain common foundational information while progressively incorporating additional capacity for downstream variables.

We thank the reviewer again for these valuable suggestions, which have helped us strengthen the experimental justification of our architecture. If these results resolve your questions, we kindly ask you to reconsider your evaluation.

Thank you for the additional experiments. I appreciate the authors' efforts, but I would like to clarify two points.

W1: Exponential Dimensionality

I apologize for the ambiguity in my original wording. I intended a fixed dimensional increment (e.g., 128/256/384/512), rather than identical dimensions across stages.

The new comparison is not fully controlled, as the exponential configuration {4, 8, 16, 32} has twice the maximum dimension of the linear configuration {4, 8, 12, 16}. A comparison with the same maximum dimension, such as {8, 16, 24, 32}, would be more convincing. I also have concerns about whether conclusions drawn from this Synthetic dataset generalize to more complex real-world datasets.

W2: Nested versus Non-Nested Representations

My concern was simply whether the final embeddings need to be constrained to a nested-prefix structure. A shared encoder could instead use a larger output projection matrix, with different parts of the resulting embedding supervised for different causal stages, without enforcing nesting.

This would still require only a single encoder forward pass, with additional computation mainly limited to the output projection. Therefore, the comparison with four separate encoders does not directly establish the necessity or computational benefit of the nested structure itself.

I appreciate the additional results, but these concerns remain only partially addressed.

AuthorsOpenReview ↗

Response 1/2

We sincerely thank Reviewer E8FT for the continued discussion and for clarifying the intended comparisons. We appreciate the opportunity to provide additional experimental evidence addressing the architectural choices.

1. Additional Ablation Studies

Following the reviewer's clarification, we conducted two additional experiments to investigate the effects of dimensionality allocation and the nested-prefix constraint.

W1: Exponential versus Linear Dimensionality

We implemented a linear dimensionality schedule with the same maximum embedding dimension as our exponential configuration:

  • Exponential: {4,8,16,32}\lbrace4,8,16,32\rbrace.

  • Linear: {8,16,24,32}\lbrace8,16,24,32\rbrace.

On the Synthetic dataset, which contains 200 ground-truth anomalies, the exponential configuration detects an average of 180.67 anomalies (±1.25\pm1.25), corresponding to a TPR of 90.34%. The linear configuration detects 175.00 anomalies (±2.16\pm2.16), corresponding to a TPR of 87.50%. The respective runtimes are 0.54 and 0.63 minutes.

These results provide empirical support for our default exponential configuration in this controlled synthetic setting. However, we retain exponential scaling as a practical design choice consistent with conventional MRL configurations, rather than claiming it to be theoretically optimal.

W2: Nested versus Non-Nested Representations

We further implemented the alternative suggested by the reviewer: a shared encoder with a larger output projection, where different non-overlapping portions of the output embedding are assigned to different representation scales.

Specifically, the encoder produces a 60-dimensional output, which is partitioned into four disjoint blocks with dimensions {4,8,16,32}\lbrace4,8,16,32\rbrace. Each block receives the same corresponding cumulative causal reconstruction supervision as its nested counterpart.

The non-nested configuration achieves a TPR of 89.17% (178.33 detected anomalies, ±2.05\pm2.05) with a runtime of 0.55 minutes, compared with 90.34% (180.67 detected anomalies, ±1.25\pm1.25) for NestedAno with a runtime of 0.54 minutes.

These results show that the shared-encoder non-nested alternative achieves comparable detection performance, while NestedAno obtains a slightly higher mean TPR using a more compact 32-dimensional maximum representation rather than 60 output coordinates allocated to disjoint blocks. This comparison is consistent with the benefit of sharing foundational information across nested prefixes, although we do not interpret the modest performance difference as conclusive evidence of statistical superiority.

2. Evaluation on Real-World Datasets

To examine whether these observations also hold across real-world datasets, we expanded our evaluation of the linear scaling and non-nested alternative to 10 real-world datasets.

As shown below, NestedAno achieves a higher AUC-ROC than both alternatives on 7 of the 10 datasets and ties for the highest AUC-ROC on one additional dataset. Moreover, NestedAno has the lowest runtime on all 10 datasets. While the performance gaps are modest, this confirms that our default nested exponential configuration generalizes well and consistently provides strong empirical performance.

DatasetLinear-AUCLinear-RuntimeNon-nested-AUCNon-nested-RuntimeNestedAno-AUCNestedAno-Runtime
ACD53.12 (0.01)5.8753.93 (0.01)5.8854.03 (0.01)4.34
Bank66.52 (0.00)31.9166.89 (0.01)35.3566.74 (0.01)27.84
CMC59.18 (0.01)1.2159.56 (0.01)1.2859.87 (0.02)1.02
DAMRE92.83 (0.00)0.6392.86 (0.01)0.5793.00 (0.00)0.48
Lymp98.83 (0.01)0.2398.91 (0.00)0.2199.06 (0.00)0.17
NHIS91.25 (0.00)2.7191.19 (0.00)2.6991.19 (0.00)2.15
OS96.50 (0.01)0.6196.79 (0.01)0.5796.94 (0.01)0.47
Seismic71.80 (0.01)4.0571.97 (0.01)4.8873.24 (0.01)3.97
SMD91.73 (0.00)3.6491.46 (0.01)3.3692.29 (0.01)2.76
SPD53.13 (0.00)0.2052.86 (0.02)0.2053.13 (0.02)0.15
Reviewer_QJm1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NestedAno, a causally guided nested representation learning framework for unsupervised tabular anomaly detection. The key idea is to replace CausalAno’s single fixed-capacity latent representation with a Matryoshka-style hierarchy of nested embeddings. A causal graph estimated by the PC algorithm defines a topological ordering of variables, which is partitioned into cumulative subsets. Each nested prefix representation is supervised via an auxiliary causal reconstruction loss to encode its assigned variable subset.

Critical strengths
  1. The paper identifies a real limitation of CausalAno, a single high-dimensional latent space can dilute localized causal violations. Nested multi-scale representations are a reasonable response.
  2. The paper evaluates on 30 datasets, 17 baselines, both mixed-type and continuous data, and includes visualizations. This is a substantial empirical effort.
Critical weaknesses
  1. Several key hyperparameters are not analyzed on real data. The number of scales K = 4 are fixed and the number of neighbors q are not specified.
  2. Computing q-nearest-neighbor distances at every scale for every test sample requires comparing against all normal training embeddings. This could be expensive. Appendix D reports runtime, but the main text does not discuss scalability.
  3. For imbalanced anomaly detection, AUC-PR is often more informative, but this has not been reported in this paper.
  4. The method aligns latent scales with cumulative causal subsets, which suggests interpretability, but the paper only evaluates ranking performance. It does not show whether the scales actually localize anomalies to the intended causal variables or provide useful diagnostic attribution.
  5. Lack of necessary code and implementation details.
Questions for authors
  1. How is the number of neighbors q set? How sensitive are results to K and q?
  2. Can you provide AUC-PR for all main comparisons, and per-dataset paired significance tests?
  3. Why K = 4, and why partition the topological ordering into equal-sized cumulative subsets rather than using causal depth or graph structure? How does this interact with highly unbalanced causal graphs?
  4. Can you provide more details about the implementation?
6 comments
AuthorsOpenReview ↗

Responses to Weaknesses 1-2 and Question 1

We sincerely thank Reviewer QJm1 for the insightful and detailed feedback. We appreciate the opportunity to clarify the implementation choices and provide additional experimental analyses addressing the reviewer's concerns.

W1 & Q1: Hyperparameters qq (Number of Neighbors) and KK (Number of Embedding Scales)

Sensitivity to the number of neighbors qq

We apologize for not explicitly specifying qq in the manuscript. We use q=5q=5, matching the default n\_neighbors=5 setting in scikit-learn's NearestNeighbors implementation.

Following your suggestion, we conducted a sensitivity analysis over q∈{1,…,10}q\in\lbrace1,\ldots,10\rbrace. As shown below, NestedAno is highly stable for q∈{4,5,6,7}q\in\lbrace4,5,6,7\rbrace, achieving AUC-ROC values between 74.49% and 74.53%. Performance decreases slightly for smaller (q≤3q\leq3) and larger (q≥8q\geq8) neighborhoods. These results indicate that the method is robust to the choice of qq within a reasonable range and support our default choice of q=5q=5.

q=1q=1q=2q=2q=3q=3q=4q=4q=5q=5 (Default)q=6q=6q=7q=7q=8q=8q=9q=9q=10q=10
AUC-ROC (%)73.6974.1574.4074.5074.4974.5374.4974.4774.4374.38

Sensitivity to the number of embedding scales KK

Our default configuration uses K=4K=4, with embedding dimensions {128,256,512,1024}\lbrace 128,256,512,1024 \rbrace. This follows a commonly used nested, progressively expanding dimensionality schedule in Matryoshka-style representation learning [1-3]. We choose four scales to provide a sufficiently resolved multi-scale trajectory while keeping the architecture and computational requirements manageable.

Although the trapezoidal rule in Equation (9) can be evaluated with two or more points, K≥3K\geq3 is preferred because using multiple intervals provides a more informative representation of how anomaly evidence evolves across scales.

Following your suggestion, we evaluated configurations with K∈{3,4,5}K\in\lbrace3,4,5\rbrace. For K=3K=3, we removed either the smallest or largest scale from the default range. For K=5K=5, we added either a smaller or a larger scale.

ConfigurationEmbedding dimensionsAUC-ROC (%)
K=3K=3{128,256,512}\lbrace 128,256,512 \rbrace74.13
K=3K=3{256,512,1024}\lbrace 256,512,1024 \rbrace74.33
K=4K=4 (Default){128,256,512,1024}\lbrace 128,256,512,1024 \rbrace74.49
K=5K=5{64,128,256,512,1024}\lbrace 64,128,256,512,1024 \rbrace74.18
K=5K=5{128,256,512,1024,2048}\lbrace 128,256,512,1024,2048 \rbrace74.21

The results show that NestedAno maintains consistent performance across these configurations. As extending the maximum embedding dimension to 2048 does not yield an improvement, we retain K=4K=4 as a practical configuration that provides a good balance between multi-scale representation and model complexity.

[1] Cross-lingual Matryoshka Representation Learning across Speech and Text, ACL 2026

[2] MatFormer: Nested Transformer for Elastic Inference, NeurIPS 2024

[3] MM-CAD: A Multi-Modal CAD Dataset and Benchmark for Cross-Modal Geometric Learning, Symposium on Geometry Processing (SGP) 2026

W2: Scalability and Complexity of qq-NN Distances

We would like to clarify the computational complexity of the qq-NN operation.

Let NN denote the number of normal training samples and dkd_{k} denote the embedding dimension at scale kk. Computing the distances from one test sample to all training samples across the KK scales requires O(N∑k=1Kdk)\mathcal{O}\left(N\sum_{k=1}^{K}d_{k}\right), which is bounded by O(NKdK)\mathcal{O}(NKd_{K}), where dKd_{K} is the maximum embedding dimension. Thus, the complexity grows linearly with the training-set size.

For comparison, a dense covariance-based Mahalanobis approach (as used in CausalAno, ICML 2026) requires approximately O(Nd2+d3)\mathcal{O}(Nd^{2}+d^{3}) operations to estimate and invert a d×dd\times d covariance matrix, followed by O(d2)\mathcal{O}(d^{2}) operations per query for distance computation, where dd is the latent dimension.

As reported in Figure 8, NestedAno requires an average of only 6.24 minutes across all datasets, whereas CausalAno requires 15.84 minutes (2.5×2.5\times slower).

AuthorsOpenReview ↗

Responses to Weaknesses 3-4 and Questions 2-3

W3 & Q2: AUC-PR Metric

We appreciate the suggestion to include AUC-PR, which is particularly informative for imbalanced anomaly detection. Following this suggestion, we computed AUC-PR for both CausalAno and NestedAno across the 12 mixed-type datasets.

As shown below, NestedAno achieves a higher AUC-PR on 9 of the 12 datasets. The average AUC-PR increases from 45.21% for CausalAno to 47.07% for NestedAno, an improvement of ~2%. The one-sided paired Wilcoxon signed-rank test (testing the directional alternative that NestedAno achieves higher AUC-PR) yields p=0.046p=0.046. The corresponding two-sided pp-value is 0.092. We therefore interpret the result as positive directional evidence in favour of NestedAno.

DatasetCausalAnoNestedAno
ACD10.92 (0.99)11.63 (0.59)
Bank38.60 (0.74)38.11 (0.44)
CMC7.68 (0.14)7.73 (0.17)
DAMRE90.13 (0.51)89.23 (0.18)
Fraud15.63 (0.81)19.36 (0.64)
Lymp88.31 (3.85)91.99 (2.02)
NHIS14.90 (1.27)28.11 (1.08)
OS90.86 (0.66)93.04 (2.11)
Seismic24.97 (1.64)26.66 (1.31)
SMD59.11 (4.16)56.60 (3.29)
SPD88.06 (0.15)88.15 (0.82)
Vifd13.34 (0.32)14.24 (0.36)
Average45.2147.07

W4 & Q3: Interpretability and Equal-Sized Mapping of Latent Scales

We agree with the reviewer that causal DAGs can have highly imbalanced structures, with different numbers of variables at different topological depths. However, our method does not partition variables according to causal depth. Instead, it uses a topological ordering to construct cumulative subsets of variables.

Specifically, we divide the topologically ordered variables into KK approximately equal-sized consecutive groups and define each subset Ck\mathcal{C}_{k} as the cumulative union of the first kk groups. This ensures that the number of representation scales remains fixed across datasets, independently of the estimated DAG depth or shape. It also avoids making the trajectory resolution dependent on potentially imperfect causal graph estimates, as examined in Appendix E. For example, if we dynamically mapped KK to the graph depth, a shallow graph containing only root nodes would yield K=1K=1, which violates the K≥2K\ge2 integration constraint of Equation (9).

We acknowledge that when a group contains multiple variables, its corresponding representation scale is associated with a subset of variables rather than an individual causal node. Therefore, NestedAno provides scale-level diagnostic information aligned with the topological ordering, rather than exact variable-level attribution.

Our Synthetic topological-progression experiment (Section 4.3, Figure 6) illustrates this relationship in a controlled setting. In this experiment, the DAG contains 4 variables and we use K=4K=4, so each incremental group contains one variable. An injected anomaly in the root variable (Age) produces a pronounced signal at the earliest scale, whereas an anomaly in the third variable (Salary) becomes more pronounced at the third scale. This demonstrates how anomaly evidence can evolve along the topological ordering when the scales correspond to individual variables. For datasets with larger groups, the interpretation is correspondingly at the group level.

AuthorsOpenReview ↗

Responses to Weakness 5 and Question 4

W5 & Q4: Code and Implementation Details

The architectural and optimization details are provided in Section 4.1 and Appendix B. For clarity, we summarize the main implementation choices below:

  • Generator architecture: Each structural sub-generator is a 4-layer MLP with hidden widths increasing from 64 to 128 units. LeakyReLU activations with a negative slope of 0.2 are used, and Batch Normalization with momentum 0.8 is applied after the second and third layers. Continuous variables use tanh output activations, while categorical variables use Gumbel-Softmax.

  • Discriminator architecture: The discriminator is a 3-layer MLP with two 256-unit hidden layers using LeakyReLU activations, followed by a linear projection to the maximum embedding dimension. It uses a shared linear critic head and scale-specific causal reconstruction predictors.

  • Optimization: Both the generator and discriminator are trained using Adam with a learning rate of 2×10−42\times10^{-4}, β1=0.5\beta_{1}=0.5, and β2=0.9\beta_{2}=0.9.

  • Training and causal discovery: The PC algorithm uses a significance level of α=0.05\alpha=0.05. The model is trained for 300 epochs with a batch size of 64 and a noise dimension of 16.

  • Method hyperparameters: We use q=5q=5, embedding dimensions {128,256,512,1024}\lbrace128,256,512,1024\rbrace, and a causal reconstruction coefficient of λCR=0.1\lambda_{\mathrm{CR}}=0.1.

  • Experimental protocol: Each experiment is repeated using three independent runs, and the average performance is reported.

As stated in the Reproducibility Statement, the source code will be made publicly available upon acceptance.

We thank the reviewer again for these constructive suggestions. The additional sensitivity analyses, AUC-PR evaluation, and implementation clarifications have helped us strengthen the empirical evaluation of NestedAno. If these responses adequately address your concerns, we kindly ask you to reconsider your evaluation.

Reviewer_cWVBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper addresses unsupervised tabular anomaly detection, focusing on structural anomalies where individual feature values appear plausible but their combination violates underlying dependencies. The authors argue that a single fixed latent representation may dilute localized structural violations and that covariance-based Mahalanobis geometry may be inadequate when the latent distribution is heterogeneous, multimodal, or non-elliptical.

  • The authors propose NestedAno, a causally guided nested representation learning framework inspired by Matryoshka Representation Learning. An estimated DAG and causal topological ordering define cumulative variable subsets, while scale-specific causal reconstruction objectives and multi-scale adversarial learning organize nested representations. At inference time, q-NN distances across nested scales form an anomaly trajectory, whose integrated deviation from the normal reference trajectory provides the final anomaly score.

  • Experiments across 30 continuous and mixed-type datasets show that NestedAno achieves the highest average AUC-ROC among 17 representative baselines, improving over CausalAno(Nguyen et al., 2026) on mixed-type datasets while remaining competitive on continuous datasets. Additional analyses examine Matryoshka embedding ranges and anomaly trajectories, while causal reconstruction, graph quality, and variable ordering are investigated on a four dimension Synthetic dataset.

Critical strengths

1. Extensive Benchmark Evaluation and Competitive Detection Performance

  • The paper evaluates NestedAno on 30 continuous and mixed-type datasets against 17 representative baselines under a common evaluation protocol. NestedAno achieves the highest average AUC-ROC on mixed-type datasets (74.49% versus 71.89% for CausalAno), with a statistically significant improvement (p = 0.03), while remaining competitive on continuous datasets. The experiments use a fixed configuration across datasets and three independent runs.

2. Empirical Evidence of Topologically Organized Nested Representations

  • On the four-variable Synthetic dataset, causal reconstruction supervision produces anomaly trajectories that increasingly reflect the topological ordering. At the highest reconstruction coefficient (λCR=1.0\lambda_{\mathrm{CR}} = 1.0), Age anomalies are most pronounced at the earliest embedding scale, while Salary anomalies become clearer around the third scale. Increasing the reconstruction coefficient also improves AUC-ROC from 89.62% to 95.41%. On NHIS and Mammography, NestedAno captures more anomalies than CausalAno under the same inspection budget, providing complementary evidence beyond aggregate AUC-ROC.
Critical weaknesses

1. Unclear Contributions of Trajectory Scoring and Nested Representations

  • Equation (9) reduces to a weighted sum of multi-scale q-NN distances minus a constant, so the normal reference trajectory does not affect anomaly rankings or AUC-ROC. The trapezoidal weights may also emphasize larger scales, potentially reducing the contribution of lower-dimensional bottlenecks (§3.3).

  • NestedAno changes both CausalAno's representation and Mahalanobis scoring. Although even the smallest Matryoshka embedding range outperforms CausalAno (Figure 2b), without controlled comparisons it is difficult to attribute the improvement to nested representations rather than q-NN scoring, or to nesting rather than multi-scale representations alone.

2. Insufficient Justification of Causal Supervision and Theoretical Claims

  • Section 3.3 uses the PC algorithm to obtain an estimated DAG, although standard PC generally identifies a CPDAG. The treatment of unresolved edge orientations and mixed-type conditional independence tests is unspecified. Evidence for causal topological ordering is largely limited to the four-variable Synthetic experiment (Figures 5–6; Appendices E–F).

  • Theorem 2(ii) requires a Lipschitz critic at each latent scale, whereas Equation (5) penalizes the averaged discriminator in input space. Moreover, the bound does not depend on the nested prefix structure and therefore does not establish how nesting mitigates signal dilution.

Questions for authors

Q1. Role of the Normal Reference Trajectory and Scale Weighting

  • Equation (9) subtracts a normal reference trajectory shared by all test samples, which does not affect anomaly rankings or AUC-ROC. Moreover, the trapezoidal weighting may emphasize larger embedding scales, despite the motivation to capture localized violations in lower-dimensional bottlenecks (§3.3). What is the practical contribution of the reference trajectory, and how does the performance change under equal weighting or scale-normalized aggregation?

Q2. Independent Contributions of Nested Representations and q-NN Scoring

  • NestedAno changes both the representation learning and anomaly scoring of CausalAno. Even the smallest Matryoshka embedding range outperforms CausalAno (Figure 2b), but the source of this improvement remains unclear. Could the authors compare CausalAno representations with q-NN scoring and nested versus non-nested multi-scale representations under identical scoring procedures to isolate the contribution of nesting?

Q3. Validity of Causal Topological Supervision

  • Section 3.3 uses the PC algorithm to obtain an estimated DAG and causal topological ordering, although standard PC generally returns a CPDAG. How are unresolved edge orientations handled, and which conditional independence tests are used for mixed-type data? Could the authors also evaluate the effect of variable ordering on representative real-world datasets, beyond the Synthetic experiment?

Q4. Theoretical Justification of Nested Representations

  • The proof of Theorem 2(ii) invokes Wasserstein duality, which requires a 1-Lipschitz critic at each latent scale, whereas Equation (5) penalizes only the averaged discriminator in input space. How is the required per-scale condition ensured? Furthermore, since the bound does not depend on the nested prefix structure, how does it establish that nesting mitigates signal dilution?
4 comments
AuthorsOpenReview ↗

Responses to Questions 1-2

We sincerely thank Reviewer cWVB for the detailed and constructive feedback. We address the four questions below.

Q1: Role of the Normal Reference Trajectory and Scale Weighting

We agree that subtracting a normal reference trajectory shifts all test scores by a constant, which does not affect the relative anomaly ranking or the threshold-independent AUC-ROC.

However, we include the reference trajectory in Equation (9) for practical interpretability. By subtracting the baseline, we center the trajectories (equivalent to centering the anomaly scores) such that a score near 0 indicates a good match with the normal topological structure. While this does not impact AUC-ROC, it provides a consistent baseline that is beneficial for defining decision boundaries in threshold-dependent metrics (such as F1-score) during real-world deployment. As visualized in Figure 3, this centering anchors normal trajectories near the x-axis, clearly separating them from the anomaly trajectories.

Regarding the scale weighting in Equation (9): To address your concern about the trapezoidal weights emphasizing larger scales, we evaluated a scale-normalized aggregation (equal uniform weighting) on the 12 mixed-type datasets. The equal-weighted formulation yielded an average AUC-ROC of 74.50%, compared to our standard trapezoidal formulation's 74.49%. This result indicates that the overall detection performance is very similar under the two aggregation strategies.

While the trapezoidal rule weights larger dimensions more heavily, it does not obscure the localized causal violations captured at early variables (as demonstrated in Figure 6).

Q2: Independent Contributions of Nested Representations and qq-NN Scoring

We respectfully clarify that modifying CausalAno to use qq-NN scoring would not fully isolate the contribution of the scoring mechanism. CausalAno utilizes a different training paradigm (specifically, Reinforcement Learning for DAG matching) whereas NestedAno does not. Therefore, any performance divergence could still be attributed to the training paradigm rather than the scoring function.

To isolate the effect of the representation architecture within our own framework, we refer to the ablations conducted for Reviewer E8FT. We compared our nested embeddings against non-nested embeddings using the same qq-NN scoring pipeline. Both strategies achieved comparable performance on synthetic and real-world datasets.

We emphasize that we simply utilized the established nested Matryoshka architecture (NeurIPS 2022) as a practical building block to achieve these multiple scales simultaneously, rather than claiming statistical superiority for nesting over independent non-nested scales.

AuthorsOpenReview ↗

Responses to Questions 3-4

Q3: PC Algorithm CPDAG & Variable Ordering

To handle mixed-type data during causal discovery, we employ a practical approximation: we apply ordinal encoding to the categorical variables and utilize the Fisher-Z test (α=0.05\alpha=0.05) to estimate the graph.

Because the PC algorithm returns a CPDAG, we follow the preprocessing step in CausalAno: we retain the successfully oriented directed edges while discard the unresolved undirected edges.

We agree that comparing random variable ordering against causal ordering on real-world datasets is a valuable ablation. However, because we lack ground-truth DAGs for the real-world benchmarks, the structural noise in the estimated graphs makes it difficult to isolate the ordering effect outside of our controlled Synthetic experiments. We view refining DAG estimation to validate ordering effects on real-world tabular data as an important direction for future work.

Q4: Theoretical Justification of Nested Representations

We thank the reviewer for the rigorous examination of Theorem 2 and its proof. We agree that the current theoretical discussion requires clarification.

First, regarding the Lipschitz condition, Wasserstein duality requires each scale-specific critic D(mk)D^{(m_{k})} to be 1-Lipschitz with respect to its corresponding latent representation. However, in Equation (5), our gradient penalty is applied to the averaged critic output Dˉ(x~)\bar{D}(\tilde{x}). This regularizes the ensemble critic as a function of the input space but does not guarantee that every individual scale-specific critic is 1-Lipschitz in its latent space. Therefore, we clarify that Theorem 2(ii) is a conditional result under the per-scale Lipschitz assumption, while the gradient penalty is simply a practical regularization mechanism.

Second, we agree that Theorem 2 does not mathematically establish the specific advantage of the Matryoshka prefix-nesting constraint. It provides a conditional relationship between the mutual information I(Z(mk);Y)I(Z^{(m_{k})};Y). Therefore, the theorem characterizes a per-scale property of adversarial representation learning generally, rather than proving that nesting itself mitigates signal dilution. We will revise the theorem discussion and its interpretation accordingly.

We sincerely thank the reviewer again for these valuable comments. If these clarifications fully address your concerns, we kindly invite the reviewer to reconsider their evaluation.

I thank the authors for their clarifications and additional experiments.

While Q1 is largely addressed, my main concerns remain. The independent contribution of nested representations is still unclear (Q2), the effect of causal ordering lacks real-world validation (Q3), and Theorem 2 does not establish the claimed advantage of nesting or guarantee the required Lipschitz condition (Q4).

Therefore, I maintain my original rating.

As the discussion phase concludes shortly, we would like to provide two quick final clarifications:

  1. We adopt the established nested architecture from Matryoshka Representation Learning (NeurIPS 2022) as a practical way to construct multi-scale representations for anomaly trajectories. As shown in our ablation study for Reviewer E8FT, nested and non-nested multi-scale representations achieve comparable performance. Following your suggestion, we will include a discussion of non-nested multi-scale alternatives in the revised paper.

  2. Within the remaining time, we compared random ordering against causal ordering across 9 small-to-medium real-world mixed-type datasets. Causal ordering achieves 79.19% average AUC-ROC compared to 78.00% for random ordering (~1.2% lower). This aligns with our synthetic experiment in Appendix F (Figure 8), where random ordering also resulted in a ~1% drop in AUC-ROC.

We thank Reviewer cWVB for their time and effort in assessing our work. We will carefully incorporate all of your feedback (especially clarifying the claims and assumptions in Theorem 2) into the revised version.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Response and Summary of Additional Evidence

Dear Area Chair and Reviewers,

Thank you for your constructive feedback. We have responded to all concerns, conducted additional experiments, and clarified our methodological and theoretical claims.

NestedAno captures structural anomalies through causally guided nested representations, local non-parametric distances, and integrated anomaly trajectories. Across 30 datasets and 17 baselines, it achieves the highest overall average AUC-ROC, significantly outperforming CausalAno on mixed-type datasets (74.49% vs. 71.89%, p=0.03p=0.03).

Reviewer concernsSummary of responses and additional evidence
Progressive capacity — W1, E8FTExponential scaling improves detection on Synthetic. With the same maximum dimension, {4,8,16,32}\{4,8,16,32\} achieves a TPR of 90.34% vs. 87.50% for {8,16,24,32}\{8,16,24,32\}. This supports a practical design consistent with MRL literature, not theoretical optimality.
Nested vs. non-nested representations — W2, E8FT; Q2, cWVBNesting offers comparable or slightly better detection with a compact hierarchy. On Synthetic, four independent encoders and shared-encoder disjoint embeddings achieve TPRs of 89.34% and 89.17% (0.80 and 0.55 min), respectively, vs. 90.34% (0.54 min) for NestedAno. Nesting requires 32 output coordinates vs. 60 for disjoint embeddings. Across 10 real-world datasets, NestedAno outperforms both linear and shared-encoder non-nested alternatives on 7 datasets, ties for first on one, and has the lowest runtime on all 10.
Trajectory scoring and component contributions — Q1–Q2, cWVBEqual weighting and trapezoidal integration perform comparably (74.50% vs. 74.49% AUC-ROC on 12 mixed-type datasets). The normal reference centers trajectories for interpretability without changing rankings. CausalAno uses reinforcement learning for DAG matching, complicating cross-framework attribution to scoring alone. Nested/non-nested ablations use identical scoring.
Hyperparameter sensitivity — W1/Q1, Q3 and follow-up, QJm1Performance is stable across qq and KK. With default q=5q=5, AUC-ROC ranges from 74.49% to 74.53% for q=4q=4–77. Tested configurations with K=3,4,5K=3,4,5 achieve 74.13%–74.49%, with K=4K=4 performing best. Fixed KK avoids per-dataset tuning, balances trajectory resolution and capacity, and decouples resolution from shallow or noisy DAGs. K=4K=4 is a practical choice, not a requirement.
Causal supervision and interpretability — Q3, cWVB; W4/Q3, QJm1For mixed-type discovery, we use ordinal encoding and Fisher-Z (α=0.05\alpha=0.05) as a practical approximation, retaining directed CPDAG edges and discarding undirected edges. Approximately equal-sized consecutive groups in topological order yield cumulative subsets independent of DAG depth. Synthetic Age and Salary anomalies become pronounced at the first and third scales, supporting group-level diagnosis rather than exact variable attribution. Real-world ordering validation remains open.
Class imbalance — W3/Q2, QJm1AUC-PR improves from 45.21% to 47.07% (+1.86 pp) over CausalAno, winning on 9/12 mixed-type datasets. Paired Wilcoxon tests yield one-sided p=0.046p=0.046 and two-sided p=0.092p=0.092, providing positive directional evidence.
Scalability, fairness and reproducibility — W2, W5/Q4 and follow-up, QJm1; W2 and follow-up, E8FTAverage runtime: 6.24 min for NestedAno vs. 15.84 min for CausalAno. Linear and shared-encoder non-nested comparisons modify only dimensions and output projections, retaining other settings. We provide implementation details, plan dimensionality-based scalability experiments, and commit to code release upon acceptance.
Theoretical scope — Q4, cWVBTheorem 2(ii) is conditional on the per-scale Lipschitz assumption. The averaged input-space gradient penalty does not guarantee per-scale latent-space 1-Lipschitz critics. The theorem concerns per-scale adversarial learning, not a nesting-specific advantage. We will clarify its interpretation.

Our contribution is a dedicated TAD framework, not a new general-purpose representation-learning architecture. We integrate established MRL and causal generative modeling with local trajectory scoring to address limitations of AnoLLM (repeated permutations), DRL (representation overlap), LLM-DAS (synthetic anomaly quality), and CausalAno (global covariance-based geometry).

We appreciate the reviewers' feedback. Detailed evidence is provided in our individual responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.