- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies instance-level outlier interpretation and questions whether axis-aligned feature subsets adequately express the contrast between an anomalous query and normal reference observations. It first measures subspace-separability failures across 47 ADBench datasets, using exhaustive enumeration for datasets with at most 20 features and restricted search budgets at higher dimensionalities. It then proposes TMQ, which learns a separate linear projection for each query using quadruplets consisting of the query, another outlier, a near inlier, and a sampled non-near inlier. One global and four leave-one-position-out attention branches estimate latent weights, while a three-hinge objective imposes a hierarchy among inlier–inlier, query–outlier, and query–inlier distances. The learned projection is subsequently mapped to an input-feature ranking.
Critical strengths
-
A substantial comparative record with promising reported results. The paper provides dataset–detector-level results, component ablations, parameter sensitivity, and resource measurements rather than relying only on a pooled average.
-
An explicit query-specific contrast with a clearly characterized relational objective. TMQ makes its reference structure explicit: the query is compared with both normal observations and another detected anomaly. The global and omission-context branches provide an understandable way to condition the learned projection on different reference configurations.
Critical weaknesses
-
The most immediate issue concerns the relationship between precision and F1. Appendix G.1 states that a weight-based method is converted to a predicted feature set by selecting the top features, where is the reference set. It explicitly notes that precision equals recall because .
Under that protocol, for each query with a nonempty reference set, precision=recall=F1. Thus, these quantities must also coincide when calculated from the same query-level predictions using the same aggregation. However, Table 3 reports TMQ precision of 0.616, whereas Table 4 reports F1 of 0.571, despite describing a matching pooled dataset–detector aggregation. -
The specific contribution of TMQ—and its claimed move beyond feature subsets—is not adequately isolated. The manuscript itself notes that ATON already uses query-conditioned embedding learning with feature-weight readout, while WANDER already uses quadruplets and attention. The main incremental claim therefore concerns TMQ’s particular global/local attention parameterization. However, the WANDER comparison changes complete pipelines rather than holding the quadruplets, objective, optimizer, projection dimension, and readout fixed. The ablations provide partial evidence: the parameter-matched MLP replacement is useful, and its performance drop is substantial. Nevertheless, removing the global or local branches changes AUPR by only 0.004 or 0.002, respectively. The largest drop comes from replacing the quadruplet objective with a triplet objective, but this does not isolate the distinction from WANDER, which also uses quadruplets.
Questions for authors
See Weaknesses.
7 comments
I thank the authors for their detailed responses. The authors have resloved most of my concerns. My remaining concerns are as follows:
- Validity of the reference explanations. The reference subset is selected by maximizing the query’s min–max-normalized anomaly score under separately fitted subspace detectors. However, this criterion does not account for how many normal observations receive scores above the query. Consequently, a subset may be selected even when it poorly distinguishes the query from normal behavior. This issue can occur with a single anomalous query and therefore cannot be explained solely by heterogeneous anomaly subtypes. Please validate the selected reference subsets using query-versus-normal ranks or held-out false-positive rates, accounting for subset selection. More fundamentally, agreement with these binary labels measures agreement with a particular annotation procedure, rather than independently validated explanation accuracy. The distinction is especially relevant when evaluating SHAP/LIME, whose native objective concerns the behavior of a specified predictive model.
- The objective function is too similar to WANDER, which leads to limited novelty.
Since feature shapley and outlier interpretation is not my direct research field, i would also like to see other reviewers' feedback.
Beyond Feature Subspaces: Query-Conditioned Projection Learning for Outlier Interpretation
Response to Reviewer nFNU: Q1
We evaluated the HBOS reference generator used by \texttt{eval/eva_main.py} across all twelve datasets. The original exhaustive search reproduces all 3,863 current local reference subsets; Appendix K.5 reports the validation protocol and results.
The query-versus-normal rank check confirms your concern: 1,092/3,863 queries (28.27%) have more than 5% of normal observations scoring above them in their selected subsets. We report both strict and tie-inclusive normal ranks. Maximizing the normalized score therefore does not guarantee strong separation.
We then tested the selection rule with five splits per dataset, each separating fitting, calibration and test observations. At nominal 5%, the dataset-average held-out false-positive rate is 20.75% when thresholding the selected subset alone and 3.88% when calibrating the maximum over the entire search. Anomaly detection also falls from 55.46% to 23.71%. Thus accounting for subset selection materially changes the results.
This validation covers HBOS; the other reference generators remain unvalidated by these tests. Appendix K.5 also separates the archived and current PageBlocks labels, which differ for 508/510 queries. Tables 3 and 4 retain their interpretation as agreement with detector-derived proxies, without establishing independent explanation accuracy or superior SHAP/LIME fidelity.
Thank you for helping us clarify reference validity and model fidelity. If these clarifications and the supplementary evidence address your concerns, we would be grateful if you would reconsider your assessment and revise your score upward. We sincerely appreciate your time and consideration.
Response to Reviewer nFNU: Q2
Thank you for raising this point. TMQ differs from WANDER in the constraints imposed by its objective, the explanation it seeks, and the computation used to obtain that explanation.
First, the objective functions impose different distance constraints. In common squared-distance notation, the per-quadruplet terms of \href{https://pure.ulster.ac.uk/ws/portalfiles/portal/218009379/InformationFusion_2025PreProof.pdf}{WANDER's Eq. (8)} are
Here distances are measured in each method's learned embedding, is the distance between the two inliers, are WANDER's digraph-derived weights, compares the query with the second inlier, and compares the first inlier with the other outlier. Writing both margins as for comparison, TMQ instead uses
TMQ therefore explicitly bounds the query--other-outlier distance: zero loss requires . WANDER's objective does not impose this lower bound. For example, with one-dimensional embedded positions and , all three WANDER hinges vanish, whereas TMQ's first hinge equals 2. Thus the losses have different zero-loss sets. Proposition 1 and Appendix K.2 also identify and test when this additional constraint conflicts with same-subtype explanations.
Second, the explanation targets differ. WANDER optimizes a kernelized embedding to extract an outlying subset of the original features. TMQ learns and returns the query-conditioned operator , with an explicit signed contrast relative to a normal reference . The target is a weighted relation among features that characterizes the query's departure from normal behavior. Retaining the operator exposes reinforcing and compensating contributions that a feature subset does not specify. Appendix K.1 provides a complete learned operator and a worked contrast; feature rankings are an evaluation readout of this explanation.
Third, TMQ avoids WANDER's weighted-digraph construction and multi-kernel feature computation. It operates directly on an projection and sampled quadruplets, removing those kernel computations when processing higher-dimensional inputs. Projection work is for quadruplets; attention adds under the formulation in Appendix F, so the full cost also depends on the latent width and sampling/batching budget. The reported benchmark provides a concrete scalability result: TMQ completes fraud (, ), while WANDER exceeds the two-week budget (Table 4). This supports a practical difference in the tested setting; a dimension-specific advantage requires a matched high-dimensional runtime comparison.
These distinctions concern the mathematical constraints, the information returned to the analyst, and the computational formulation. We welcome the other reviewers' assessments of these points. We would be grateful for your kind consideration of our revised work.
Beyond Feature Subspaces: Query-Conditioned Projection Learning for Outlier Interpretation
Response to Reviewer nFNU: Q1
The precision in Table 3 and the F1 score in Table 4 are reported for feature-weight evaluation and subspace evaluation, respectively. As defined in the manuscript's Evaluation Metrics paragraph (Appendix G.1), G denotes the ground-truth subspace and P the predicted subspace. We follow the distinction in Xu et al., \emph{Beyond Outlier Detection: Outlier Interpretation by Attention-Guided Triplet Deviation Network} (WWW 2021, Secs. 5.1.2 and 5.1.5).
In the feature-weight setting, each method produces feature importance weights. We obtain P by selecting the top-k features with k=∣G∣, and report precision =∣G∩P∣/∣P∣ and Jaccard, together with AUPR and AUROC for the full ranking. Recall equals precision because ∣P∣=∣G∣=k, so recall is redundant in this comparison; F1 is not a reported metric in Table 3.
In the subspace setting, P is the predicted explanation subspace, and Table 4 reports F1=2∣G∩P∣/(∣G∣+∣P∣) without separately reporting precision and recall. Xu et al. report all three metrics for subspace evaluation methods and distinguish ATON's feature-weight output from its thresholded subspace version ATON$'$. The identity precision = recall =F1 holds for the same pair of sets when ∣P∣=∣G∣. Applying it across Tables 3 and 4 additionally requires identical predictions, queries and aggregation. The original query-level predictions and exact selection configuration for Table 4 are unavailable, so the Type-I/Type-II distinction does not by itself verify the reported 0.616/0.571 difference. The supplementary TMQ-S evaluation checks the identity on saved predictions but does not reconstruct those historical entries. The revisions in Secs. 3.4, 4.1 and 4.2, the table captions, and Appendices G and K.1 now distinguish these protocols.
Response to Reviewer nFNU: Q2
TMQ and WANDER differ in representation learning and optimization. WANDER (Guan et al., Information Fusion 2025) builds a weighted digraph to encode each quadruplet's geometry, learns a multi-kernelized embedding, and optimizes a weighted-digraph-based quadruplet loss. TMQ learns an explicit linear projection W∈Rs×d with latent-coordinate weights a∈Rs. Its three-hinge loss L(q) directly constrains Dii, Dqo and Dqi. Following Proposition 1, L(q)=0 requires Dqi≥Dii+2m and Dqo∈[Dii+m,Dqi−m], where m>0 is the margin. Thus TMQ specifies the desired distance ordering in the learned projection, while WANDER uses weighted-digraph guidance in a kernelized representation (Secs. 3.2--3.3).
The attention design differs in how the representation is weighted. WANDER integrates customized self-attention with its multi-kernelized embedding. TMQ estimates a through an explicit global/local decomposition of q=(x,xo,xi,xi′): one global multi-head branch receives all four embedded positions, and four local multi-head branches each omit a different position. Omitting xo exposes the query--normal contrasts; omitting x isolates relations among the reference samples; omitting either inlier changes the normal context. The branch scores are normalized and averaged across the sampled quadruplets to obtain a, so the latent-coordinate weights reflect the complete quadruplet and the separate reference contexts (Sec. 3.2).
The explanation output is also different. WANDER extracts an outlying subspace, namely a subset of the original features. TMQ returns the query-conditioned operator Bx=diag(ax∗)Wx∗, with ϕx(z)=Bxz (Sec. 3.4). Each row of Bx describes a signed linear combination of input features. For a normal reference b, the contrast ϕx(x)−ϕx(b)=Bx(x−b) shows how feature deviations reinforce or cancel one another in the learned projection. A coordinate subset identifies selected features but does not retain these coefficients or their signed combinations. Appendix K.1 provides the complete learned operator and a worked query--reference contrast to make this output distinction concrete.
The matched TMQ-S attention controls in Sec. 4.5 do not show a combined-branch advantage; replacing TMQ's attention with WANDER's in the same backbone remains untested. The architectural differences therefore do not establish an isolated performance advantage.
Thank you for your careful review and detailed feedback, which helped us clarify the evaluation protocols and distinguish architectural differences from evidence of their contribution. If these clarifications and the matched controls address your concerns, we would be grateful if you would reconsider your assessment and revise your score upward. We sincerely appreciate your time and consideration.