acceptodds
Withdrawn from ICLR 2027

Learning Feature–State Dependence with Sparse Variational Mixtures

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

High-dimensional anomaly detection is often jointly affected by latent multimodality, variable dependence, and heavy-tailed contamination, making a single global distribution inadequate for characterizing the complex probabilistic structure of normal data. Mixture models can represent latent heterogeneous states through multiple local components, but existing methods typically overlook that variables may depend on the latent mixture state to different degrees, leading to parameter redundancy and unstable estimation in high-dimensional, finite-sample settings. To address this issue, we propose the Sparse Variational Student- Mixture Factor Model (SVStMFM). The method uses Bernoulli structural variables to adaptively learn component-dependent and component-invariant parameters, while a low-rank Student- factor structure captures both high-dimensional correlations and heavy-tailed behavior. We further develop structured variational Bayesian inference to jointly learn latent states, factor representations, local scales, and variable structures, and construct a locally adaptive anomaly score through a marginal–conditional decomposition of the Student- distribution. Experiments on the Tennessee Eastman process and real industrial data show that SVStMFM consistently improves anomaly-detection performance while reducing model parameters by 83.1%. These results demonstrate that learning feature–state dependence provides a compact and robust solution for high-dimensional, multimodal, and heavy-tailed data.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Variational Inference, Mixture Models, Sparse Structure Learning, Anomaly Detection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KZXXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents SVStMFM for learning feature-state dependence in high-dimensional anomaly detection. The method uses Bernoulli structural variables to adaptively learn component-dependent and component-invariant parameters without removing observed variables. A low-rank Student-t factor structure captures high-dimensional correlations and heavy-tailed behavior. Structured variational Bayesian inference jointly learns latent states and variable structures. A marginal-conditional decomposition of the Student-t distribution yields a locally adaptive anomaly score. Experiments on the Tennessee Eastman process and real industrial data report improved anomaly-detection performance over the comparison methods while reducing model parameters by 83.1%.

Critical strengths

The robust Student-t mixture factor model is a clear strength. Its low-rank latent representation reduces the complexity of modeling high-dimensional correlations. Local scale variables yield a robust Student-t representation within a single probabilistic framework. Appendix B.1 shows the model size depends approximately linearly on the observed dimension. Figure 6 reports 1213 parameters against 7159 for VBStMM.

Critical weaknesses

W1. Consistency of the main results on the Tennessee Eastman process. Section 4.1 and Appendix D.1.1 use a 72 h sequence for testing with faults injected at 60 h and a sampling interval of 3 min. This setup gives 1200 normal samples and 240 faulty samples in each test sequence. The reported results instead correspond to 160 normal samples and 800 faulty samples per fault. Every per-fault FAR of the three comparison methods in Table 2 is a multiple of 1/1601/160 up to rounding. The Accuracy values of all seven comparison methods in Table 1 also match this split. The FAR and DR of GMM would give an Accuracy of 0.75000.7500 under the stated setup instead of the reported 0.86050.8605.

Averaging the per-fault results of VBStMM in Table 2 over the 21 faults reproduces its Table 1 values of FAR 0.10150.1015, Accuracy 0.84720.8472, and F1 0.86570.8657 up to rounding in the last digit. The same averaging does not reproduce the SVStMFM row. The per-fault results of SVStMFM give FAR 0.06350.0635, Accuracy 0.93850.9385, and F1 0.95770.9577 against the reported 0.06460.0646, 0.93010.9301, and 0.95290.9529. The reported Accuracy of SVStMFM also falls below both its proportion of correctly identified normal samples of 0.93540.9354 and its DR of 0.93870.9387. A single confusion matrix cannot produce these three values for any split of normal samples and faulty samples.

The paper does not define Accuracy. Eq. (85) defines only FAR, DR, Precision, and F1. The main results on the Tennessee Eastman process are therefore difficult to verify.

W2. Insufficient isolation of feature-state dependence learning. Section 4.3 states that Table 2 isolates the effects of heavy-tailed modeling and variable-level structure learning. Appendix D.1.3 attributes the improvement of the mean DR from 0.8370.837 to 0.9390.939 over VBStMM to variable partitioning.

Yet SVStMFM also differs from VBStMM in the low-rank factor representation and in the anomaly score with responsibility weighting and EMA accumulation. Figures 5 and 10 compare the VBStMM NLL with the SVStMFM weighted NLL + EMA. Appendix D.1.3 explains the more stable monitoring curves of SVStMFM by responsibility weighting and EMA accumulation. The comparison therefore leaves open how much of the improvement comes from learning which variables share parameters across mixture components. VBStMM-NS appears only in Table 2 and is never defined. Its role as the ablation of variable-level structure learning remains unclear.

The reduction of model parameters by 83.1% likewise combines the low-rank factor representation with cross-component parameter sharing. Removing cross-component parameter sharing would give each of the 23 component-invariant variables KK parameter sets instead of one. Eq. (47) with the 29 component-dependent variables reported in Section 4.3 bounds the resulting increase by a factor of 52/2952/29 for any KK at fixed rr. The model without this sharing would still have fewer than 2175 parameters and remain about 70% below the 7159 parameters of VBStMM in Figure 6. At most about 13 of the 83.1 percentage points can therefore come from cross-component parameter sharing. The current evidence does not isolate the benefit of the variable-level component-dependence structure-learning mechanism in contribution (1).

Questions for authors

Q1. Experimental setup and metric definitions on the TEP dataset. The per-fault FAR values in Table 2 indicate 160 normal samples per fault. A 72 h test sequence with faults injected at 60 h contains 1200 normal samples. Counting the 60 h from the start of the 48 h training data would leave 240 normal samples in the test sequence. Which test data produce the 160 normal samples per fault? How do the six operating conditions enter Tables 1 and 2?

Please report the numbers of normal samples and faulty samples and the TP, FP, TN, and FN counts for each fault and operating condition. Please also give the definitions of Accuracy and AUC and state how each metric in Table 1 is averaged over faults and operating conditions. How do these counts reconcile the SVStMFM row of Table 1 with its per-fault results in Table 2 and with its own FAR and DR?

Q2. Ablation of variable-level structure learning. Could the authors define VBStMM-NS? Its DRs of 0.0990.099 on d04 and 0.0790.079 on d14 fall far below those of the other three models in Table 2. These three models reach 0.9000.900 or higher on both faults. Is VBStMM-NS the SVStMFM model retrained with γj=1\gamma_j=1 for every variable under the same number of mixture components, latent-factor dimension, prior hyperparameters, anomaly score, EMA smoothing coefficient, and control limit?

Please add this ablation if VBStMM-NS is a different model and report its DR, FAR, AUC, and parameter count next to SVStMFM. Please also state the latent-factor dimension rr and the final number of mixture components used for SVStMFM. How much of the gain in anomaly-detection performance and of the reduction of model parameters remains attributable to learning the variable structure when these settings are held fixed?

Q3. Per-fault comparison across decision thresholds. The FAR of SVStMFM under the 95% control limit ranges from 00 on 14 faults to 0.3740.374 on d21. Its DR of 0.8830.883 on d03 comes with an FAR of 0.2630.263. Do the comparison methods in Tables 1 and 3 use the same EMA accumulation? Could the authors report the per-fault AUC of all methods or their DR at the same FAR?

Q4. Hyperparameter settings. The hyperparameters β0\beta_0, u0u_0, and W0W_0 in Appendix D.1.2 do not appear in the Normal-Gamma priors of Eqs. (11) to (14). The hyperparameters a0a_0, b0b_0, and R0R_0 of these priors are not reported. Could the authors give the complete hyperparameter settings? How is u0c=Dc+2u_0^c = D_c + 2 set before the variable structure is learned? Table 1 reports DAGMM instead of the DPGMM listed among the comparison methods in Appendix D.1.2.

Reviewer_FKzKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes the Sparse Variational Student-t Mixture Factor Model for anomaly detection in multimodal, correlated, and heavy-tailed data. Bernoulli structural indicators determine whether each feature’s parameters are shared across mixture components or estimated separately. A low-rank factor structure models correlations, and structured variational inference jointly estimates model parameters and latent variables. Detection uses a responsibility-weighted score derived through a Student-t marginal–conditional decomposition, followed by EMA smoothing.

Critical strengths

The paper addresses a meaningful modeling problem: estimating separate parameters for every feature in every component can waste data and increase estimation variance. Adaptive sharing offers a practically relevant way to reduce complexity while retaining the full observation space.

Critical weaknesses
  1. The reported evaluation is not currently convincible. Inconsistent accuracy values and sequence descriptions, incomplete implementation settings, and unspecified SRCC splits undermine confidence in the central performance and efficiency claims. More specifically, in Table 1, SVStMFM has a detection rate of 0.9387 and a specificity of 0.9354, but its reported accuracy is 0.9301. For metrics computed from the same pooled binary predictions, accuracy must lie between detection rate and specificity. Furthermore, the stated TEP protocol (i.e., 72 hours of testing, fault injection at 60 hours, and three-minute sampling) implies approximately 1,440 samples with fault onset around sample 1,200, whereas Figure 5 appears to show approximately 960 samples with fault onset around sample 160. These discrepancies directly affect confidence in the evaluation.

  2. The novelty discussion should engage more precisely with closely related mixture models. Robust Student-t factor mixtures with common factor loadings already provide heavy-tailed modeling and cross-component parameter sharing. Additionally, Sun et al.’s feature-saliency model retains density terms for nonselected features, so characterizing previous approaches broadly as “retain–remove” methods is insufficiently precise. The potentially distinctive contribution here is adaptive feature-wise sharing while preserving cross-variable dependence. The manuscript should explain this distinction at the model level and compare against directly relevant alternatives.

Questions for authors
  1. The implementation details do not sufficiently specify the proposed model. Appendix D.1.2 lists prior hyperparameters that do not clearly correspond to the Normal–Gamma priors and prior precision matrices introduced in Section 3.2. The numerical latent-factor dimension, its selection procedure, and several required prior settings are also missing. Although the initial component count is provided, the manuscript does not clearly describe component pruning or report the final effective component count.

  2. The ablations do not convincingly isolate the contributions of the proposed components. Comparisons with VBStMM simultaneously change covariance parameterization, parameter sharing, and potentially the anomaly score and temporal smoothing. Figure 5 explicitly compares the proposed score with EMA against a VBStMM negative log-likelihood without the same smoothing. Consequently, the improvements cannot be attributed specifically to feature–state dependence learning.

  3. The anomaly score requires stronger justification. The marginal–conditional decomposition in Section 3.4 is mathematically natural, but the resulting score is equivalent to a responsibility-weighted negative log-likelihood over the full components. The decomposition itself therefore does not establish an additional detection advantage. Moreover, this score differs from ordinary mixture negative log-likelihood by subtracting the divergence between posterior component responsibilities and prior mixture weights. This changes how confidently assigned and low-weight components are treated, but does not guarantee improved anomaly discrimination.

  4. The correctness of the learned feature structure is not established. Figure 7 shows that posterior probabilities concentrate near zero or one, but posterior confidence does not demonstrate that the inferred partition is correct or stable. Controlled experiments with known component-dependent and component-invariant variables would allow evaluation of structural recovery, especially when invariant variables remain correlated with state-dependent variables.

  5. The efficiency claims need more transparent accounting. The reported 1,213 parameters cannot be readily verified without the final component count and latent-factor dimension. Furthermore, the variational family maintains both component-dependent and shared parameter blocks during training, whereas the reported compression concerns the structure retained after thresholding. Training memory, posterior storage, and final inference-model size should therefore be distinguished. Runtime reporting also needs reconciliation: Figure 6 gives approximately 0.361 seconds for VBStMM, whereas Table 1 reports 0.7165 seconds.

3 comments

Response to Critical Weakness 1

Thank you very much for pointing out these important issues. We re-examined the experimental code, original prediction results, and data-splitting procedures. We have now unified the experimental settings and recomputed the reported results in the revised manuscript.

First, regarding the metric inconsistency in Table 1, we reran the experiments on the TEP dataset and consistently recomputed FAR, DR, Accuracy, Precision, F1, and AUC from the same binary predictions using a unified aggregation protocol. After correction, SVStMFM achieves an average FAR of 0.0467, DR of 0.9310, Accuracy of 0.9347, and F1 score of 0.9515. The related results in the tables and main text have been updated accordingly.

Second, we confirm that the original manuscript incorrectly described the duration and fault-injection time of the TEP test sequences. In the actual experiments, each test sequence contains 960 samples with a sampling interval of 3 min, corresponding to 48 h in total. The first 160 samples (8 h) represent normal operation, and the fault is introduced from sample 161 onward, leaving 800 faulty samples. Therefore, the fault onset around sample 160 in Fig. 5 is consistent with the actual experimental protocol. The original description of “72 h testing with fault injection at 60 h” has been corrected to “48 h testing with fault injection after the first 8 h,” and the corresponding descriptions in the main text, figure captions, and appendix have been revised consistently.

Third, we have clarified the data split for the SRCC dataset. The SRCC dataset contains a total of 46,291 samples, including vibration- and temperature-related variables. Among these samples, 37,033 are used for training and 9,258 are used for testing.

Thank you again for identifying these inconsistencies. These revisions ensure consistency among the experimental protocol, evaluation metrics, dataset splits, and reported figures, thereby improving the transparency and reproducibility of the experimental evaluation.

Updated per-fault results on the TEP dataset

FaultFAR ↓DR ↑Accuracy ↑F1 ↑
IDV 010.00000.99250.99380.9962
IDV 020.00000.97750.98120.9886
IDV 030.02500.94870.95310.9712
IDV 040.00000.99000.99170.9950
IDV 050.00000.99500.99580.9975
IDV 060.00000.99620.99690.9981
IDV 070.00000.99750.99790.9987
IDV 080.00000.98250.98540.9912
IDV 090.30000.89880.86560.9177
IDV 100.00000.96630.97190.9828
IDV 110.00000.98380.98650.9918
IDV 120.00000.99620.99690.9981
IDV 130.00000.94370.95310.9711
IDV 140.00000.99750.99790.9987
IDV 150.00000.32120.43440.4863
IDV 160.35001.00000.94170.9662
IDV 170.00000.96630.97190.9828
IDV 180.00000.94870.95730.9737
IDV 190.00000.98380.98650.9918
IDV 200.00000.90750.92290.9515
IDV 210.30630.75620.74580.8322
Average0.04670.93100.93470.9515

Response to Critical Weakness 2

Thank you very much for this important comment. We agree that the novelty of the original manuscript was not positioned with sufficient precision. To address this issue, we have reorganized Section 2.2 of the revised manuscript and further clarified the contribution of SVStMFM at the model level.

First, Student-tt factor mixture models already provide heavy-tailed robustness and low-rank correlation modeling, while models with common factor loadings can also achieve a certain degree of cross-component parameter sharing. Therefore, the contribution of this work does not lie in introducing Student-tt modeling, factor analysis, or parameter sharing in isolation. The key distinction of SVStMFM is adaptive variable-level sharing structure learning. Specifically, Bernoulli structural variables determine, for each observed variable, whether its mean, factor loading, and residual scale are parameterized in a component-specific or cross-component shared manner, without requiring a globally predefined sharing pattern.

Second, we appreciate the reviewer’s clarification regarding Sun et al. In the revised manuscript, we no longer broadly characterize related approaches as “retain–remove” methods, but instead compare them according to how the relationship between observed variables and latent mixture states is parameterized. Sun et al. mainly use feature saliency to characterize the contribution of each variable to mixture-component discrimination. In contrast, the structural variables in SVStMFM directly govern the component-specific/shared parameterization within the Student-tt factor mixture model, determining whether the mean, factor loading, and residual scale of each variable vary with the latent mixture state.

More importantly, component-invariant variables remain within the unified low-rank factor model and retain statistical dependence with component-dependent variables through the latent factors. Therefore, the core contribution of SVStMFM can be summarized as adaptive variable-level learning of component-dependent/component-invariant parameter sharing within a robust Student-tt factor mixture model, while preserving joint cross-variable dependence.

Thank you for the rebuttal and your efforts to address my concerns. The corrections to the evaluation metrics, experimental protocol, and dataset splits improve the clarity and consistency of the reported results. The revised discussion also provides a clearer positioning of the proposed method relative to existing mixture models. However, several major concerns of mine remain insufficiently addressed, particularly the lack of controlled ablations and direct comparisons to closely related approaches to support the claimed novelty and effectiveness of the proposed approach. I therefore maintain my original score.

Reviewer_TXKJOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes SVStMFM, a variational Bayesian Student-t mixture of factor analyzers. SVStMFM uses

  • a Bernoulli inidicator for each variable to decide whether the variable parameters are component specific or shared
  • anomaly score is based on a Student-t marginal-conditional dist with a EMA smooting
  • experiments are on 2 data sets (TEP, SRCC) and based on this the claim is that the method beats every comparison emthod by over 80% The model and the derivations look reasonable and the CDV/CIV idea appears interesting

Novelty is overstated. The paper presents CDV/CIV as distinct from variable selection and yet this is known in the literature e.g.. classic feature-saliency mixture of Law et al. (2004) and the SRUW model of Maugis et al. (2009) . The methodological contribution is weak and evaluation is limited to two process-monitoring datasets against mostly classical baselines

Critical strengths
  • Separating relevant variables from variables' parameters depend on latent state is interesting. Keeping CIVs in the joint model through shared factor loadings preserves cross-variable correlation (§3.2, Eq. 7–10).
  • the derivations in the Appendix appear complete. The Normal-Gamma conjugate blocks, the structured VB updates, the posterior-odds form of φ_j (Eq. 24), and the Beta-Bernoulli update of ρ are provicded in Appendix C).
  • The pseudocode (Algorithm 1) is detailed.
  • The scoring rule uses the closed-form Student-t marginal and conditional (Eq. 26–28) and p(x_I) is shared across components, so it correctly cancels from the responsibilities (Eq. 27). this is well designed scoring rule.
  • Experiments are done with real-plant data. The SRCC table appears internally consistent:
Critical weaknesses
  • §2.2 characterizes prior variable selection literature as "retain-remove" whilw SVStMFM asks a different question. This mischaracterizes the literature.

    • Law, Figueiredo & Jain (2004, TPAMI) addresses the problem of unsupervised feature selection. In supervised learning non-salient features are not removed. They are modelled by a common density that is independent of the component, which is precisely a "component-invariant variable".
    • Sun et al. (2018): cited by the paper, this extends the same feature-saliency construction to Student-t mixtures.
    • Maugis, Celeux & Martin-Magniette (2009, Biometrics) and Raftery & Dean (2006): irrelevant variables are retained and linked to relevant ones by regression. This preserves the joint correlation the paper claims as new.
  • The base model is a (Bayesian) mixture of t factor analyzers. Examples such as McLachlan et al. (Comp Stats and Data Analysis 2007) and Ghahramani & Beal's (NIPS 1999) VB mixtures of factor analyzers are not cited cited nor discussed

  • The actual novelty appears to be a combination of factor-analytic t-mixtures, per-variable component-invariance (feature saliency), and coupling CIVs through shared loadings. That delta must be stated precisely and positioned against these works.

    • The framing is general "high-dimensional anomaly detection", and §2.1 cites MCM, NPT-AD/Thimonier, DRL, AnoLLM and ConjNorm. None is evaluated. The comparators are PCA, CVA, MPPCA, GMM, VBGMM, DAGMM and VBStMM.
  • Even within process monitoring, standard strong TEP baselines (dynamic PCA/KPCA, LSTM/TCN autoencoders, Isolation Forest, kNN) are absent in the analysis.

  • there is an exploding literature for time series foundation models some of which provide strong benchmarks for anomaly detection such as TSPulse (ICLR 2026),

  • With only two datasets, both from industrial process monitoring, the "general" claim is unsupported. Adding ADBench-style tabular datasets would test it. TSB-AD benchmark (Liu & Paparrizos, 2024) (recent comprehensive leaderboard for AD), which comprises 40 eval datasets, covering both univariate (TSB-AD-U) and multivariate (TSB-AD-M) anomaly detection - need c

Questions for authors
  • provide comprehensive benchmarking against TSB-AD benchmark (Liu & Paparrizos, 2024)
  • a more systematic discussion of the novelty wrt the large body of work to clarify the novelty
4 comments

Thank you for your time and careful evaluation. Below is a consolidated summary of the key clarifications and new results we provided during the discussion.

R1: provide comprehensive benchmarking against TSB-AD benchmark (Liu & Paparrizos, 2024)

We thank the reviewer for highlighting the limitations of the original experimental evaluation. Since SVStMFM is designed for multivariate data modeling, we extended our evaluation to the TSB-AD-M anomaly detection benchmark, comparing SVStMFM with 24 representative methods under a unified VUS-PR protocol. The baselines cover statistical, clustering-based, density-based, tree-based, and deep temporal methods, as well as recent approaches including FITS, OFA, TimesNet, and the reviewer-suggested TSPulse.

As shown in Table 1, SVStMFM achieves the highest VUS-PR on five datasets (CATSv2, Daphnet, Exathlon, LTDB, and MITDB) and ranks second on four others (CreditCard, OPPORTUNITY, SMAP, and SVDB), placing among the top two methods on 9 of the TSB-AD-M datasets.. These results demonstrate SVStMFM's competitiveness beyond industrial process monitoring.

Table 1. VUS-PR comparison on the multivariate TSB-AD datasets. Best results are in bold and second-best results are underlined.

MethodCATSv2CreditCardDaphnetExathlonGECCOGHLGenesisLTDBMITDBMSLOPPORTUNITYPSMSMAPSMDSVDBSWaT
SVStMFM (Ours)0.23910.0664‾\underline{0.0664}0.51060.98460.05350.03540.03840.45080.15550.23040.2798‾\underline{0.2798}0.24540.3745‾\underline{0.3745}0.20290.3592‾\underline{0.3592}0.4884
TSPulse (ZS)0.050.000.35‾\underline{0.35}0.890.17‾\underline{0.17}0.010.010.360.070.200.070.140.300.35‾\underline{0.35}0.380.13
CNN0.080.020.210.680.030.020.100.330.14‾\underline{0.14}0.35‾\underline{0.35}0.160.220.190.35‾\underline{0.35}0.190.41
OmniAnomaly0.040.020.340.840.020.070.000.44‾\underline{0.44}0.110.220.180.160.120.170.350.15
PCA0.120.100.130.95‾\underline{0.95}0.200.010.020.240.070.150.300.160.090.360.110.45
LSTMAD0.040.020.310.820.020.06‾\underline{0.06}0.040.300.090.220.170.240.160.330.150.16
USAD0.040.020.340.840.020.06‾\underline{0.06}0.000.410.120.230.180.190.110.160.320.15
AutoEncoder0.060.030.130.910.050.050.010.210.040.220.140.280.130.300.060.58
KMeansAD0.120.020.300.370.060.030.890.410.060.440.060.210.380.360.200.16
CBLOF0.060.030.100.860.030.020.020.200.040.210.140.190.140.220.070.29
MCD0.13‾\underline{0.13}0.060.140.800.030.010.060.210.040.230.170.26‾\underline{0.26}0.100.260.070.54‾\underline{0.54}
OCSVM0.080.020.060.830.040.040.080.200.040.220.120.190.120.280.060.44
Donut0.070.020.170.660.030.050.180.260.120.300.150.200.180.190.110.44
RobustPCA0.040.020.060.770.020.030.000.230.040.220.130.120.070.100.080.12
FITS0.13‾\underline{0.13}0.020.330.630.030.010.100.230.050.170.050.130.080.170.100.15
OFA0.13‾\underline{0.13}0.020.310.580.040.010.22‾\underline{0.22}0.290.060.140.050.170.080.170.120.12
EIF0.060.020.150.410.040.020.060.190.040.180.100.180.130.320.070.32
COPOD0.050.050.110.400.040.030.080.210.040.210.170.200.100.190.070.31
IForest0.050.030.130.350.040.050.080.210.040.210.180.190.090.260.070.39
HBOS0.050.040.150.320.040.040.080.210.040.230.170.170.090.250.070.30
TimesNet0.070.020.270.420.030.010.020.270.070.170.060.140.090.140.110.14
KNN0.070.020.250.330.110.010.040.190.040.180.060.120.120.300.060.11
TranAD0.040.020.310.100.020.06‾\underline{0.06}0.040.260.070.240.160.230.090.300.120.15
LOF0.050.020.110.160.130.010.080.190.040.140.100.150.090.160.060.15
AnomalyTransformer0.030.020.070.100.020.030.010.210.050.120.070.210.060.070.080.18

The benchmarks provided here address one of my concerns of testing this against the TSB-AD becnhmark. The results look promising and should be submitted to TSB-AD.

The novelty of this work for an AI audience and wrt to the broader literature in statistics is still unclear

R2:a more systematic discussion of the novelty wrt the large body of work to clarify the novelty

Thank you for the reviewer’s suggestion. We agree that the original manuscript did not systematically review the related work. In particular, characterizing existing variable-selection methods as “retain–remove” did not accurately reflect how prior work models non-salient variables, making the novelty of our work insufficiently clear. We will reorganize the related work and clarify the differences from four aspects: variable structure, correlation modeling, robust mixture modeling, and variational inference.

First, our novelty does not lie in first retaining non-salient variables or introducing component-invariant variables. Law, Figueiredo & Jain (2004, TPAMI) model non-salient variables using a component-independent common density, while Sun et al. (2018, TNNLS) introduce feature saliency into Student-tt mixtures. We will therefore remove the inaccurate “retain–remove” characterization.

The key difference is that SVStMFM learns not whether a feature is important, but whether its distributional parameters should vary with the latent state. For each variable, SVStMFM determines whether its mean, factor loading, and residual scale are component-specific or shared across components. Law, Figueiredo & Jain (2004, TPAMI) assume conditional independence among variables given the component. Sun et al. (2018, TNNLS) use a product of dimension-wise densities and also assume conditional independence among observed variables; moreover, the parameters of their non-salient branch remain component-indexed, unlike the cross-component shared parameters of CIVs in our model.

Second, we agree that retaining non-clustering variables and their correlations is not new by itself. Raftery & Dean (2006, JASA) retain non-clustering variables and model their dependence on clustering variables through regression. Maugis, Celeux & Martin-Magniette (2009, Biometrics) further allow them to depend on only a subset of clustering variables. These methods assume that non-clustering variables are conditionally independent of the component label given the clustering variables.

In contrast, SVStMFM shares the marginal means, factor loadings, and residual scales of CIVs across components, while preserving their dependence on CDVs through common low-dimensional latent factors. The CIV–CDV cross-covariance is represented by BC,kBI⊤B_{C,k}B_I^\top. Thus, the marginal distribution of CIVs remains component-invariant, while their correlation with CDVs can still vary across states. Our contribution is joint modeling under a marginal-invariance constraint, rather than simply preserving correlations.

Third, we will acknowledge that Student-tt factor mixtures and variational Bayesian mixtures of factor analyzers are established foundations. McLachlan, Bean & Ben-Tovim Jones (2007, CSDA) proposed mixtures of tt-factor analyzers, while Ghahramani & Beal (1999, NeurIPS) studied variational inference for Bayesian mixtures of factor analyzers. Conventional tt-factor mixtures estimate component-specific parameters for all variables, which can introduce substantial redundancy when only some variables vary across states. SVStMFM introduces variable-level partial parameter sharing: CDVs retain component-specific parameters, whereas the complete parameter blocks of CIVs are shared across components.

Finally, SVStMFM directly uses the learned CDV–CIV structure for anomaly detection. It decomposes the anomaly statistic into the common marginal deviation of CIVs and the local state-dependent deviation of CDVs conditioned on CIVs. This score detects both shared-structure and state-dependent anomalies while preserving the dependence between CIVs and CDVs. The contribution of structured variational inference is to jointly infer component assignments, latent factors, local heavy-tailed scales, and variable-level sharing structures for the proposed model, rather than to introduce VB itself.

Therefore, we summarize the novelty as follows:

SVStMFM introduces variable-level state-dependence learning into a Student-tt factor mixture and adaptively determines whether variable parameters should vary across components. By sharing factor loadings, it reduces CIV parameters while preserving CIV–CDV cross-dependencies. We further derive variational Bayesian inference to jointly learn latent states, factor representations, heavy-tailed scales, and variable structures, and construct a marginal–conditional anomaly score consistent with the learned structure.

We thank the reviewer again for this valuable comment, which has helped us better clarify the novelty of our work. We will add the above references, revise the related-work and contribution sections, and emphasize variable-level cross-component parameter sharing, factor coupling, and joint variational inference.

Thanks for making explicit the contributions of this work compared to the literature.

I will wait to read the responses to the other reviewers who have pointed out discrepancies in the reported results in Table 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.