- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes a runtime monitor for out-of-distribution (OOD) detection that is shared across all networks belonging to the same structural architecture family (e.g., ResNet-style networks, Vision Transformers), instead of requiring a separate monitor for each individual network. Per-layer activation statistics are turned into a discretized, depth-normalized sequence of severity levels, over which a single Hidden Markov Model is fitted per family. The paper argues, and empirically supports, that this family-level monitor generalizes to unseen networks of the same family while remaining competitive with existing per-network monitoring baselines.
Critical strengths
S1. The central idea — that a single monitor can be shared across an entire family of structurally related networks instead of requiring individual per-network calibration — is conceptually interesting and, to my knowledge, not explored in this form in prior runtime- monitoring work. The within- vs. cross-family alignment analysis (Table 1) and the leave-one-network-out evaluation (Table 3) provide reasonably convincing evidence that such sharing is possible without a substantial loss in detection performance, which is the paper's most substantive contribution.
Critical weaknesses
W1. The paper does not sufficiently develop or exploit its own central idea. The hypothesis that activation-based severity patterns align within an architecture family is supported only empirically, never explained conceptually. The paper leaves the reader with more open questions than insights, and the reviewer did not gain a deeper understanding of the underlying problem.
W2. The evaluation compares only against other activation-based monitors (Mahalanobis, Outside-the-Box), both of which, like the proposed method, require network-specific calibration. The paper's own related-work discussion identifies a distinct class of output-based OOD detectors (e.g., Hendrycks & Gimpel, 2017) that operate purely on output statistics and may already generalize across architectures without any family-specific or per-network calibration at all. No such baseline is included, so it remains unclear whether the added complexity of the proposed pipeline (per-layer scoring, relative-depth staging with arity correction, family-level HMM fitting) provides any practical advantage over a much simpler alternative. This directly undermines the paper's central motivation of scalable, easy-to-deploy monitoring.
W3. The paper is not self-contained and is written in a vague, unevenly detailed way. The abstract and introduction are generic, and central terminology (what "family" refers to) only becomes concrete well into the paper. The contributions are stated too abstractly to be parsed on first reading. Key concepts are used without explanation: the BUILD/INFER flag parameters, the undefined "combined-layer Mahalanobis" baseline in Table 2, and the Related Work paragraph that lists prior monitors without contrasting them with the proposed approach. Meanwhile, the narrative sections are verbose. A substantial revision of the writing would be needed before the contribution can be assessed fairly by a broad readership.
Questions for authors
Q1. Under what conditions can a monitor be expected to generalize across different network architectures, and why? This is the fundamental question the paper raises but leaves unresolved: the central hypothesis is validated empirically on the specific families tested, but no account is given of the conditions under which such generalization should or should not be expected to hold.
Q2. Is the proposed approach competitive with the state of the art in OOD detection at large? The paper only compares against two activation-based monitoring methods, while the OOD detection literature contains a much broader and more mature body of work. Without such a comparison, it is unclear whether the proposed approach is competitive at all.