- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper extends CausalAno by replacing its single latent representation with causally guided Matryoshka representations at multiple nested scales. Each scale is aligned with progressively larger causal variable subsets, and the resulting multi-scale anomaly deviations are combined into a trajectory-based anomaly score.
Critical strengths
- The paper improves upon the single fixed latent representation used in CausalAno by introducing multi-scale nested representations, which can capture anomaly evidence at different representation capacities and potentially reduce the dilution of localized structural anomalies.
- The paper provides useful empirical analysis on the limitation of covariance-based Mahalanobis scoring. The observed deviations from multivariate normality support the use of local, non-parametric anomaly scoring instead of relying on a single global covariance geometry.
Critical weaknesses
- The choice of progressively increasing latent dimensionality is insufficiently justified. The method adopts exponentially increasing representation sizes (e.g., 128/256/512/1024), implicitly assuming that incorporating a later causal stage requires substantially more representation capacity. However, it is unclear why adding one level of causal information should require approximately twice the latent dimensionality. The paper does not compare against fixed-dimensional representations (e.g., 128 dimensions for all stages), linearly increasing dimensions, or other adaptive designs. Prior work has also shown that the standard Matryoshka design is not necessarily optimal across embedding lengths [2]. Therefore, the necessity of the proposed progressive-capacity design remains unclear.
- The benefit of the nested representation over independently learned representations is not established. The method assumes that representations for different causal stages should be nested prefixes of a shared latent vector. However, an alternative is to learn an independent representation for each causal stage while keeping the same causal supervision and trajectory-based scoring. Such a comparison is important because previous work on Matryoshka-style training has shown that jointly nested representations do not necessarily outperform representations trained separately for specific configurations [1]. Without this baseline, it is difficult to determine whether the improvement comes specifically from the proposed nested structure or simply from modeling anomaly evidence at multiple stages.
[1] S. Wang, S. Zhuang, B. Koopman, and G. Zuccon. 2D Matryoshka Training for Information Retrieval. arXiv:2411.17299, 2024. arxiv.org [2] T. Wen, Y. Wang, Z. Zeng, et al. Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation. ICML, 2025.
Questions for authors
Please see Critical Weaknesses 1 and 2 above. Addressing these points, particularly through the corresponding ablations and comparisons, would materially affect my assessment.
6 comments
Thank you for the additional experiments. I appreciate the authors' efforts, but I would like to clarify two points.
W1: Exponential Dimensionality
I apologize for the ambiguity in my original wording. I intended a fixed dimensional increment (e.g., 128/256/384/512), rather than identical dimensions across stages.
The new comparison is not fully controlled, as the exponential configuration {4, 8, 16, 32} has twice the maximum dimension of the linear configuration {4, 8, 12, 16}. A comparison with the same maximum dimension, such as {8, 16, 24, 32}, would be more convincing. I also have concerns about whether conclusions drawn from this Synthetic dataset generalize to more complex real-world datasets.
W2: Nested versus Non-Nested Representations
My concern was simply whether the final embeddings need to be constrained to a nested-prefix structure. A shared encoder could instead use a larger output projection matrix, with different parts of the resulting embedding supervised for different causal stages, without enforcing nesting.
This would still require only a single encoder forward pass, with additional computation mainly limited to the output projection. Therefore, the comparison with four separate encoders does not directly establish the necessity or computational benefit of the nested structure itself.
I appreciate the additional results, but these concerns remain only partially addressed.
Response 1/2
We sincerely thank Reviewer E8FT for the continued discussion and for clarifying the intended comparisons. We appreciate the opportunity to provide additional experimental evidence addressing the architectural choices.
1. Additional Ablation Studies
Following the reviewer's clarification, we conducted two additional experiments to investigate the effects of dimensionality allocation and the nested-prefix constraint.
W1: Exponential versus Linear Dimensionality
We implemented a linear dimensionality schedule with the same maximum embedding dimension as our exponential configuration:
-
Exponential: .
-
Linear: .
On the Synthetic dataset, which contains 200 ground-truth anomalies, the exponential configuration detects an average of 180.67 anomalies (), corresponding to a TPR of 90.34%. The linear configuration detects 175.00 anomalies (), corresponding to a TPR of 87.50%. The respective runtimes are 0.54 and 0.63 minutes.
These results provide empirical support for our default exponential configuration in this controlled synthetic setting. However, we retain exponential scaling as a practical design choice consistent with conventional MRL configurations, rather than claiming it to be theoretically optimal.
W2: Nested versus Non-Nested Representations
We further implemented the alternative suggested by the reviewer: a shared encoder with a larger output projection, where different non-overlapping portions of the output embedding are assigned to different representation scales.
Specifically, the encoder produces a 60-dimensional output, which is partitioned into four disjoint blocks with dimensions . Each block receives the same corresponding cumulative causal reconstruction supervision as its nested counterpart.
The non-nested configuration achieves a TPR of 89.17% (178.33 detected anomalies, ) with a runtime of 0.55 minutes, compared with 90.34% (180.67 detected anomalies, ) for NestedAno with a runtime of 0.54 minutes.
These results show that the shared-encoder non-nested alternative achieves comparable detection performance, while NestedAno obtains a slightly higher mean TPR using a more compact 32-dimensional maximum representation rather than 60 output coordinates allocated to disjoint blocks. This comparison is consistent with the benefit of sharing foundational information across nested prefixes, although we do not interpret the modest performance difference as conclusive evidence of statistical superiority.
2. Evaluation on Real-World Datasets
To examine whether these observations also hold across real-world datasets, we expanded our evaluation of the linear scaling and non-nested alternative to 10 real-world datasets.
As shown below, NestedAno achieves a higher AUC-ROC than both alternatives on 7 of the 10 datasets and ties for the highest AUC-ROC on one additional dataset. Moreover, NestedAno has the lowest runtime on all 10 datasets. While the performance gaps are modest, this confirms that our default nested exponential configuration generalizes well and consistently provides strong empirical performance.
| Dataset | Linear-AUC | Linear-Runtime | Non-nested-AUC | Non-nested-Runtime | NestedAno-AUC | NestedAno-Runtime |
|---|---|---|---|---|---|---|
| ACD | 53.12 (0.01) | 5.87 | 53.93 (0.01) | 5.88 | 54.03 (0.01) | 4.34 |
| Bank | 66.52 (0.00) | 31.91 | 66.89 (0.01) | 35.35 | 66.74 (0.01) | 27.84 |
| CMC | 59.18 (0.01) | 1.21 | 59.56 (0.01) | 1.28 | 59.87 (0.02) | 1.02 |
| DAMRE | 92.83 (0.00) | 0.63 | 92.86 (0.01) | 0.57 | 93.00 (0.00) | 0.48 |
| Lymp | 98.83 (0.01) | 0.23 | 98.91 (0.00) | 0.21 | 99.06 (0.00) | 0.17 |
| NHIS | 91.25 (0.00) | 2.71 | 91.19 (0.00) | 2.69 | 91.19 (0.00) | 2.15 |
| OS | 96.50 (0.01) | 0.61 | 96.79 (0.01) | 0.57 | 96.94 (0.01) | 0.47 |
| Seismic | 71.80 (0.01) | 4.05 | 71.97 (0.01) | 4.88 | 73.24 (0.01) | 3.97 |
| SMD | 91.73 (0.00) | 3.64 | 91.46 (0.01) | 3.36 | 92.29 (0.01) | 2.76 |
| SPD | 53.13 (0.00) | 0.20 | 52.86 (0.02) | 0.20 | 53.13 (0.02) | 0.15 |
We sincerely thank Reviewer E8FT for the constructive feedback and for recognizing the value of our multi-scale trajectory-based approach and our analysis of the limitations of Mahalanobis scoring. We appreciate the insightful questions regarding the choice of embedding dimensions and the necessity of nested representations. To address these concerns, we conducted additional ablation experiments comparing alternative dimensionality schedules and independently learned representations. We provide our responses below.
W1: Justification of Exponential Dimensionality
We appreciate the suggestion to investigate alternative dimensionality schedules. We opted for exponential scaling for three reasons:
Mathematical Constraint of the Trajectory: Our trajectory formulation integrates anomaly deviations over the representation-capacity spectrum M. Specifically, Equation (9) uses the embedding dimensions mk as the integration coordinates, requiring strictly increasing dimensions (mk+1>mk). If all stages use the same embedding dimension (e.g., 128 for all stages), the integration intervals 2mk+1−mk become zero, making the trajectory formulation degenerate.
Alignment with MRL Literature: We adopt exponential dimensionality as a practical design choice to provide progressively increasing representational capacity while maintaining a small number of nested scales. This design is consistent with commonly used Matryoshka representation learning configurations [1, 2, 3].
Empirical Comparison against Linear Scaling: To directly answer your question, we conducted a comparison between exponential and linear dimensionality schedules on our Synthetic dataset (Figure 5a), which contains 200 ground-truth anomalies. The exponential configuration, M={4,8,16,32}, detects an average of 180.67 anomalies (±1.25), achieving a True Positive Rate (TPR) of 0.9034. The linear configuration, M={4,8,12,16}, detects 172.33 anomalies (±0.94), dropping the TPR to 0.8617. These results provide empirical evidence that the exponential configuration is beneficial in our setting.
Overall, our intention is not to claim that exponential scaling is theoretically optimal, but to demonstrate that a progressively expanding representation provides a useful design for the proposed multi-scale anomaly trajectory.
[1] Matryoshka Representation Learning, NeurIPS 2022
[2] Cross-lingual Matryoshka Representation Learning across Speech and Text, ACL 2026
[3] MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining, ACL 2026
W2: Nested versus Independent Representations
We appreciate this important suggestion. While learning K independent representations for the causal stages is possible, we chose the nested architecture for two reasons: computational efficiency and representational regularization.
Because the underlying causal graph (DAG) is fixed, an independent architecture requires K separate encoders, increasing the parameter count by a factor of K. By contrast, the nested Matryoshka encoder projects the entire causal structure in a single forward pass.
Per your suggestion, we trained a baseline consisting of 4 independent encoders (each learning a representation for its corresponding cumulative causal subset) on our Synthetic dataset (Figure 5a).
These results indicate that jointly learning nested representations achieves slightly better detection performance while reducing computational cost relative to independently learning representations for each stage. We believe that the shared-prefix architecture also provides a useful inductive bias by encouraging different representation scales to retain common foundational information while progressively incorporating additional capacity for downstream variables.
We thank the reviewer again for these valuable suggestions, which have helped us strengthen the experimental justification of our architecture. If these results resolve your questions, we kindly ask you to reconsider your evaluation.