- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes “agentification,” an approach that decomposes a structured sensing domain into computational agents with responsibility for local learning and inference. Specifically, a 1,024-channel gamma-ray spectrum is assigned to 1,024 agents, each of which uses a Gaussian process to model the unusualness of background counts in its channel. Local responses are standardized and transformed into anomaly scores, which are aggregated using an equally weighted combination of the mean over all channels and the mean over the top 10%. The evaluation uses 100 measured background spectra and Poisson source injections based on Cs-137 and Co-60 detector response functions, with ten repeated 70/30 holdouts. The method achieves higher mean ROC AUC than PCA, a global residual detector, and an autoencoder in the weak-anomaly regime, while its advantage disappears for stronger anomalies. The paper also evaluates channel ranking using local evidence, robustness to partial channel loss, and the effects of simple neighboring-agent communication. Distinctive aspects include the absence of source response templates from the inference procedure and the treatment of communication as a design choice separate from agent formation.
Critical strengths
- Detection and inspection of local supporting evidence are integrated into the same computational structure. The overall score is constructed from channel-specific anomaly scores, allowing the evidence used by the detector to be inspected directly in the spectral domain. Table 2 reports localization AUCs of 0.7176 and 0.7885 for Cs-137 and Co-60, respectively, even at the weakest source level. These results support the claim that the local scores contain ranking information about the locations of injected source counts. This does not guarantee accurate region boundaries or physical source localization, but it has practical value for making detection results inspectable. The authors also appropriately acknowledge the limitations of boundary estimation for weak signals.
- The paper reports conditions in which its advantage disappears, along with negative results for communication. The methods share the same background splits and source realizations, which is an appropriate basis for comparison. Table 1 limits the advantage of GP-Agent to weak anomalies and reports that the global residual detector performs better at, for example, . The paper also reports performance degradation from neighboring-agent communication. The reversal under contiguous channel loss in Table 3 is another interesting observation for understanding the conditions under which the method may be useful. These results provide useful evidence about conditional design advantages rather than suggesting universal superiority.
Critical weaknesses
- The central contribution of “agentification” is not sufficiently distinguished from the baselines or from the effects of the individual components. The formulation in Equations (3)–(5), consisting of domain partitioning, local models, and aggregation, is broad enough to include the global residual baseline itself. Specifically, that baseline can be written as
It can therefore also be interpreted as having a channel-specific background model and local evidence. Retaining the individual values would allow channel-level evidence to be displayed for this baseline as well. Thus, even accepting the authors’ definition in terms of localized responsibility and inference, the operational criterion that makes the baseline “monolithic” and the proposed method “agentified” remains insufficiently specified. My concern is not the use of the term “agent,” but whether this distinction corresponds to a testable algorithmic difference or prediction. Furthermore, the proposed method simultaneously introduces GP modeling, empirical negative log likelihood, median/MAD standardization, a saturating transformation, and aggregation emphasizing the top 10% of scores. The experiments do not establish whether the improvements in Table 1 arise from the local model or the particular aggregation scheme. At minimum, comparisons against a local residual detector using the same aggregation and a GP detector using only the global mean would be important. Localization should likewise be compared against local residual scores using the same metric. The application contribution would also be clearer with a discussion of domain-specific background-modeling approaches, such as the NMF methods in Bandstra et al. (2023). This concern directly affects both the central claim of a new learning paradigm and the interpretation of the reported performance improvements. 2. The evidence is insufficient to assess the reliability and practical significance of the improvements. At the three weakest source levels, GP-Agent improves mean AUC over the strongest baseline by approximately 0.015–0.020. These differences are promising, but the means and standard deviations in Table 1 do not characterize the uncertainty of the paired performance differences. Overlapping standard deviations do not establish the absence of a difference, just as higher means alone do not establish a reliable advantage. The ten holdouts reuse the same 100 background spectra and should not be treated as ten independent experiments. An uncertainty analysis should account for the dependence induced by reusing the original background spectra and the injected examples derived from them. Moreover, the AUC at the weakest source level is 0.5569. A relative improvement does not by itself establish useful detection performance. Detection rates at a specified false-alarm rate are not reported, making it difficult to assess the practical significance of the results for the stated nuclear-security motivation. The restriction to 100 backgrounds and synthetic injections from two source response functions is a legitimate limitation that the authors acknowledge. I am not requiring a large new measurement campaign during the rebuttal period. However, uncertainty analyses using the existing data and sufficient implementation details for reproduction are needed. In particular, the empirical probability estimator, the precise definition of the GP response, handling of zero MAD, and treatment of previously unobserved count values could materially affect the results.
Questions for authors
The following questions concern issues whose answers could materially affect my assessment.
- Are there comparisons that distinguish agentification from the choice of aggregation? Please specify a testable criterion that distinguishes the global residual baseline from the formulation in Equations (3)–(5). Could you also compare a local residual detector using the same top-10% aggregation with a GP detector using only the global mean? Comparing localization AUC against local residual scores would also help establish the method-specific benefit. Please clarify which data and criteria were used to select the top-10% fraction and the mixing weight of 0.5.
- How stable are the performance differences in the weak-source regime? Please provide the paired AUC differences for each split and an uncertainty analysis that accounts for reuse of the original background data. It is important not to assume that the ten splits are independent observations. Could you also assess sensitivity to the random source-injection realizations? In addition, could you report detection rates at fixed false-alarm rates that can reasonably be estimated with the available sample size, together with the threshold-selection procedure? I am not requesting precise estimates at extremely low false-alarm rates from this dataset alone.
- Can you provide the implementation details needed to reproduce the results? Please specify the estimator for , , the GP mean, noise and kernel parameter settings, the exact definition of the response , and the handling of zero MAD and previously unobserved counts. Are implementation files or supplementary materials available during review that also specify the split and injection random seeds, autoencoder training settings, and aggregation procedure under channel loss? Validation on independent source-present measurements is an important direction for future work. The questions above primarily concern interpreting and verifying the currently reported results.
The reviewer is correct that the basic partition/local-score/aggregation equations also describe established feature-wise anomaly detectors. Section 3.1 now states this limitation explicitly. What changes in the revised learned-communication configuration is the inference dependency: the conditional score for channel i depends not just on x_i, but on the simultaneously observed selected nonlocal channels x_{N_i} (Sections 4.7–4.8). Table 1 compares learned conditional fusion to independent GP pooling, self-only fusion, and a degree-matched random nonlocal graph while holding local evidence and pooling fixed. The effect is modest and weaker in reverse chronology; it is not a claim that agents are mathematically necessary.
The top-10% fraction and equal mean/top-tail weighting are fixed heuristic choices; the revision does not claim they are optimal or that they were established by an independent hyperparameter search. A mean-only versus mixed-pooling audit returned identical background test-score rankings (Spearman ρ = 1.0), so we do not attribute the observed AUC improvements to top-tail pooling (Section 6.2). The supplement retains comparisons to PCA, standardized residuals, and an autoencoder (Appendix A.1), plus a separate HBOS/empirical-tail analysis (Appendix A.5). The main paper adds a matched global shrinkage-covariance Mahalanobis baseline (Section 6.1), which outperforms learned fusion at selected weak strengths. The empirical-tail method is COPOD-inspired, not a verified implementation of published COPOD. A matched official COPOD evaluation and a full GP-versus-residual top-tail factorial ablation remain useful further work.
I agree that the original α = 0.005 AUC of 0.5569 versus 0.5421 is near chance and not, alone, evidence of practical utility. Supplementary Table A1 and its accompanying text now give paired descriptive standard deviations for the original small AUC gains (+0.0148, +0.0197, +0.0196 at α = 0.005, 0.010, 0.020), while explicitly acknowledging repeated use of the same 100 backgrounds. In the follow-up chronological study, Table 2 reports 30-seed paired injection percentile intervals and ΔTPR at a retrospectively selected 10% FPR. At α = 0.20, reverse ΔTPR is −0.0136 versus self-only and −0.0282 versus random-graph fusion despite favorable ΔAUC. The calibrated 550/150/300 protocol in Section 6.1 yields learned-fusion realized source-free FPRs of 5.73% forward and 28.19% reverse at a nominal 10% target (Mahalanobis: 8.81% and 17.62%). Table 4 separately reports 30-seed detection-fraction gains of +3.82, +2.16, +3.28, and +1.99 percentage points over independent GP, residual, self-only, and random-graph controls, respectively; these are spectrum-level synthetic-injection comparisons, not independent field encounters. These findings sharply limit fixed-threshold claims.
Section 4.3 now specifies the chronological smoothed count-frequency estimator: p_hat_i(q) = [n_i(q) + 0.5] / [N + 0.5 |G_i|] on an integer grid of at most 80 values; GP training targets −log(max(p_hat_i(q), realmin)) with a constant mean basis and squared-exponential covariance in MATLAB fitrgp. The local output r_i is the predicted negative-log-frequency surrogate, standardized by the median and a robust scale max(1.4826 MAD, 0.05), then transformed to bounded evidence (Sections 4.3–4.4). Counts outside the grid use GP prediction/extrapolation, with a standardized squared-residual fallback if fitting/prediction fails. Sections 4.7–4.8 detail partner selection (up to three nonadjacent background-correlated channels with |C_ij| ≥ 0.15), ridge fit (λ = 5), and evidence fusion (γ = 0.35). Section 5.1 specifies chronology, synthetic Poisson additions, and the 30-versus-10-versus-30 seed protocols by table. The learned GP hyperparameters are optimized by the library; no environment-independent exact length-scale specification is asserted. Although implementation and output files exist in the research record, the revised paper does not claim that anonymized code, every seed, or a complete executable reproduction package has already been publicly posted.
Supplementary Appendix A.2 reports the ability of GP evidence to rank channels receiving synthetic injected counts, but a matched comparison against residual-score and autoencoder reconstruction-error localization maps has not been verified. Accordingly, the revision does not claim superior localization. The radiation-background discussion cites Bandstra et al. (2023) on gamma-ray background/anomaly learning; no new NMF head-to-head experiment is implied. These are limitations to the breadth of the current evidence, not findings we seek to obscure.