acceptodds
Under review as a conference paper at ICLR 2027

Nonparametric Sequential Goodness-of-Fit Testing under Heterogeneity

Abstract

We consider sequential goodness-of-fit testing when the data generating distribution is an unknown mixture of several reference distributions, with heterogeneity arising through variation in the mixing proportions. In many contemporary applications, this reflects latent or structurally distinct subpopulations in the data that occur at different rates while their underlying distributions remain unchanged. Such changes in composition can substantially alter the overall data distribution and cause standard sequential procedures to spuriously reject. We develop a nonparametric sequential testing framework for detecting distributional departures in the presence of such heterogeneity. Our framework is based on a composite null that allows the data generating distribution to arise from the same collection of reference distributions but with possibly different mixing weights. Motivated by the principle of testing by betting, we propose SeqGoF, which uses kernel mean embeddings to quantify departures from this composite null and constructs a sequential betting strategy against it, without imposing a parametric model for the alternative. We show that SeqGoF has anytime-valid level- control under arbitrary stopping and almost-sure power one against every fixed alternative outside the null, and we study its first-order stopping-time asymptotics. Numerical experiments and an application to hard-drive SMART telemetry demonstrate the strong finite-sample performance of SeqGoF.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Anytime-valid inference, Sequential hypothesis testing, E-processes, Nonparametric goodness-of-fit testing, Composite null hypotheses, Kernel mean embeddings

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xwLrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SeqGoF, a nonparametric, anytime-valid sequential goodness-of-fit test for a composite null where the data distribution is an (unknown) convex mixture of K reference distributions. The method constructs a kernel-based witness by projecting the empirical kernel mean embedding onto the convex hull of the reference embeddings in an RKHS, centers the resulting score by its worst-case expectation over the null, and then performs predictable betting to produce an e-process. The authors prove supermartingale validity (level-α control under arbitrary stopping), almost-sure power one with a positive rate against any fixed alternative outside the convex hull (for characteristic kernels), and first-order stopping-time asymptotics; they also present synthetic experiments and a real-world application to hard-drive SMART telemetry.

Critical strengths
  1. The SeqGoF testing procedure introduced in this paper provides a principled and elegant approach to handling mixture heterogeneity in sequential goodness-of-fit. By leveraging the convex hull of kernel mean embeddings, it reduces composite-null calibration to a tractable finite-dimensional operation.

  2. The construction of a predictable, data-adaptive witness via projection onto the convex hull combined with worst-case component-wise centering is novel in the sequential testing-by-betting literature.

  3. The approach extends kernel-based sequential GoF beyond a single reference distribution to mixture composite nulls in a way that remains computationally manageable.

Critical weaknesses
  1. In the introduction, the authors suggest that the paper is motivated by changing of proportions of mixture of distributions under the null hypothesis. However, in Section 2, the null assumes i.i.d. sampling from a single (but unknown) mixture within the convex hull. Thus, time-varying mixture weights under the null are not covered (despite motivating language about composition shifts), and it is not clear whether the procedure could reject the null hypothesis with only composition drift.

  2. Experiments section is not sufficient. In particular, no baseline is considered. The authors should consider compare with other methods to demonstrate the effectiveness of theirs proposed method in the heterogeneity settings they considered in their paper. Also, there is no sensitivity about kernel choice and bandwidth.

  3. There are multiple hyperparameters in this paper. However, analysis (either theoretically or practically) and guidance on choosing ε, λmax, K, and reference components (especially in the real-data example) is limited.

Questions for authors
  1. Could you clarify the intended null with respect to time variation? The theory assumes i.i.d. from a fixed mixture in the hull; do you consider slowly drifting mixture weights to be a change (to be detected) or part of the null? If the latter, can the framework be adapted to handle time-varying weights while preserving e-validity?

  2. Could you compare other baselines in experiments section?

  3. How sensitive is performance to the kernel and its bandwidth? Would aggregating multiple kernels improve robustness/power, and can the betting framework accommodate such aggregation without breaking validity?

  4. In the real-data study, how were K and the component definitions chosen (tertiles of age)? Did you try alternative partitions or learned components (e.g., clustering) and, if so, how did this affect both level and delay?

3 comments
AuthorsOpenReview ↗

Response to reviewer xwLr

Thank you for you thoughtful questions, below are our responses.

Q1 & W1. Thank you for raising this important point. In our framework, changes in the mixing proportions are considered part of the null as long as the underlying reference components remain unchanged. Motivated by this comment, we have extended our theoretical results to accommodate time-varying mixing proportions (Section 4 and Appendix B). Specifically, we establish that SeqGoF retains its anytime-valid type-I error control even when the mixing weights vary arbitrarily over time and depend on past observations (Theorem 4). Thus, the extended framework accommodates not only slowly drifting but also abrupt changes in the mixing proportions under the null, without requiring any modification to the original betting procedure. Furthermore, we show that the asymptotic wealth growth, power-one guarantee, and first-order stopping-time asymptotics continue to hold when the process follows a time-varying mixture null up to an unknown finite change point and subsequently generates i.i.d. observations from a fixed alternative outside the composite null (Appendix B.2).

Q2 & W2. Thank you for this comment. In the revised manuscript, we include the sequential goodness-of-fit test of Zhou and Liu (2024) as an empirical baseline in both simulation experiments (Figures 1 and 2). Since their method is designed for a simple null, we implement it by instantiating our composite null at specific choices of mixing weights (see Appendix C.3 for details). Our numerical results show that Zhou and Liu (2024) controls the type-I error when the specified mixture coincides with the data-generating distribution but exhibits substantial error inflation when the mixing proportions differ (left panels of Figures 1 and 2). In contrast, SeqGoF maintains empirical type-I error at or near the nominal level across all the scenarios considered. Moreover, SeqGoF demonstrates greater sample efficiency in the Gaussian experiment (right panel of Figure 1). Although Zhou and Liu (2024) achieves higher sample efficiency in the non-Gaussian experiment (Figure 2), this apparent advantage should be interpreted with caution given its lack of type-I error control under the composite null (left panel of Figure 2). These comparisons highlight the importance of explicitly accounting for unknown mixing proportions in sequential goodness-of-fit testing.

We have also expanded our discussion of the related work by Martínez-Taboada and Ramdas (2025), highlighting the methodological and computational differences between their approach and SeqGoF in the context of composite mixture nulls (Appendix C.2).

Q3. In the revised manuscript (Appendix D, Figure 4), we investigate the sensitivity of SeqGoF to three kernels (Gaussian, Laplace, and IMQ) and four bandwidths. While empirical type-I error control is robust across specifications, power exhibits varying degrees of sensitivity. Aggregating the wealth processes across all twelve kernel–bandwidth combinations substantially mitigates this sensitivity and achieves power comparable to the best-performing individual specifications. Importantly, such aggregation preserves anytime-valid type-I error control by the closure of nonnegative supermartingales under convex combinations. We have incorporated these empirical findings in Appendix D of the revised manuscript.

Q4. In the revised manuscript (page 5), we clarify the roles of ε\varepsilon and λmax⁡\lambda_{\max} and conduct a sensitivity analysis over nine configurations, considering ε∈{10−2,10−6,10−8}\varepsilon\in\{10^{-2},10^{-6},10^{-8}\} and λmax⁡∈{20,5000,10000}\lambda_{\max}\in\{20,5000,10000\} (Appendix E, Figure 5). The empirical type-I error and power curves are nearly indistinguishable across these configurations, suggesting limited sensitivity to these hyperparameters over the range considered. We have also expanded the SMART data analysis to investigate the choice of KK and the construction of reference components. We discuss the details in our response to your next comment. These additional analyses provide practical guidance on constructing reference components using both domain knowledge and data-driven approaches.

W3. In the original analysis, the component distributions were constructed by partitioning healthy drives according to age tertiles (SMART 9). In the revised manuscript (Appendix F.1), we examine alternative age-based partitions with K=2,3,4K=2,3,4, as well as a data-driven clustering approach that selects K^=2\widehat K=2 by maximizing the mean silhouette coefficient. None of the four specifications triggers a false alarm over 5000 held-out healthy observations. Detection delays are comparable across the age-based partitions and somewhat longer under the clustering-based specification. Please see Table 2 in Appendix F.1.

Thank you for addressing my concerns, espically adding the new Section 4 and comparison to the baseline. I will raise my score to 3 since I believe the paper is stronger now. However, could you clarify the following points further?

  1. In figure 1 and 2, it seems to me that the dashed line is for proposed method and the real line is for the baseline? If this is correct, could you make this clearer in the legend? Currently it is not very obvious. Also, the presentation for the figures could be better. For example, in Figure 1, Type I error, I cannot tell which one is the proposed method and which one is for baseline in Scenario 2; in Figure 2, the two methods are indistinguishable.

  2. In figure 2, the two methods are almost indistinguishable in the figure. Does this suggest that the proposed method's main advantage is for Gaussian reference distributions?

  3. The new theoretical results in Section 4 is great. Could you also provide corresponding experiments for the results?

We thank you for your comments that have led to a significantly improved version of the paper. We respond to your remaining comments below.

  1. We have revised Figures 1 and 2 to improve their readability. Figure 1 is now represented by Figures 5 and 6 on page 35 while Figure 2 is now Figures 7 and 8 on page 36. In the revised figures, SeqGoF is represented by solid blue lines and ZL24 by dashed orange lines, with distinct plotting symbols for the different scenarios. Additionally, and to respect the page limit, in Section 5 we now report the numerical results at selected sample sizes in Tables 1 and 2, allowing direct comparisons even when the curves overlap.

  2. We have revised Figure 2, now presented as Figures 7 and 8 (page 36), to improve the distinction between the methods and added Table 2 in Section 5 to facilitate direct numerical comparisons.

Table 2 and Figures 7 and 8 demonstrate that SeqGoF performs well beyond the Gaussian setting. In particular, its empirical-reference implementation maintains accurate type-I error control while exhibiting increasing power against non-Gaussian alternatives outside the composite null. Although ZL24 appears to achieve greater sample efficiency in Figure 8 (and Table 2), this apparent advantage must be interpreted cautiously because ZL24 exhibits substantial type-I error inflation under the composite null (Figure 7 and Table 2). In fact in Figure 7, the type I error curves for ZL24 under scenarios 1 and 3 exceed 0.1 for all values of n and are, therefore, not shown. These results demonstrate that SeqGoF provides reliable testing performance for non-Gaussian reference distributions, with valid inference under the composite null.

  1. We thank you for this suggestion. We have added two numerical experiments (Experiments 3 and 4) in Appendix G to investigate the theoretical results established in Section 4. Experiment 3 considers Gaussian reference distributions, while Experiment 4 considers non-Gaussian (Gamma–Exponential) reference distributions. Each experiment includes two settings.

In Setting 1, we examine type-I error control under time-varying mixing weights, considering both gradually evolving weights and abrupt changes. The results (Figures 9 and 11) demonstrate that SeqGoF maintains type-I error close to the nominal level, whereas ZL24 exhibits substantial error inflation.

In Setting 2, we investigate power under change-point alternatives, where the observations initially follow a time-varying distribution within the composite null before transitioning to a fixed alternative. The results (Figures 10 and 12) show that SeqGoF gains power following the change point, with detection performance depending on the change-point location and the strength of the alternative. In contrast, ZL24 frequently rejects well before the true change point, reflecting its inability to accommodate changes in the null mixing weights.

Together, these experiments provide empirical support for the theoretical results in Section 4, illustrating the robustness of SeqGoF to time-varying mixing weights and its ability to detect subsequent departures from the composite null.

Reviewer_V6AYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies sequential goodness-of-fit testing when the null distribution is an unknown mixture of several reference distributions. The proposed method, SeqGoF, projects the empirical kernel mean embedding onto the convex hull of the reference embeddings and uses the projection residual to construct a betting process. The paper establishes anytime-valid type-I error control, power-one consistency, and stopping-time asymptotics for the oracle procedure, and provides a finite-sample validity correction when the reference distributions are represented by samples. Synthetic experiments and an application to hard-drive telemetry illustrate the approach.

Critical strengths

The problem formulation is practically meaningful. Allowing unknown mixture proportions provides a useful way to accommodate population heterogeneity while testing for departures beyond the reference family. The method has a clear geometric interpretation. The combination of projection-based witness construction and component-wise null calibration leads to a relatively simple procedure. The theoretical analysis provides a coherent account of validity, consistency, and asymptotic stopping behavior. The discussion of sample-based reference distributions also addresses an important practical setting. The paper is generally well organized, and the synthetic experiments and real-data example help illustrate the intended use case.

Critical weaknesses

The empirical evaluation is relatively limited and does not include comparisons with alternative methods, making the practical advantages harder to quantify. In addition, the distinction between the oracle guarantees, the corrected sample-based procedure, and the uncorrected implementation used in the experiments could be highlighted more clearly in the main text. These points limit the breadth of the empirical conclusions, but I still find the focused methodological contribution useful.

Questions for authors
  1. The experiments with estimated reference quantities omit the correction in Appendix D. Could the authors make the scope of the guarantees more explicit in the main text and briefly discuss the expected trade-off between validity and detection speed when this correction is applied? A qualitative explanation would be helpful.

  2. Could the authors report the values of ε\varepsilon and λmax⁡\lambda_{\max} used in the experiments and explain how they were chosen? Were the same values used across the synthetic and real-data experiments?

  3. In practice, a monitoring system may run for a long time before an anomaly occurs. Since SeqGoF uses all past observations to construct its witness, could a long period of normal operation make it slower to detect a later distributional change? It would be helpful to discuss how this might affect its use in continuous monitoring.

3 comments
AuthorsOpenReview ↗

Response to reviewer V6AY

We sincerely thank the reviewer for taking the time to carefully read our manuscript and provide thoughtful feedback. Below, we address your comments point by point.

Q1. Thank you for raising this point. In the revised manuscript (after Theorem 1), we explicitly clarify that the finite-sample anytime-valid guarantee is established either when the population reference quantities are known or when the correction in Appendix A.6 is applied. When these quantities are estimated from reference samples and the correction is omitted, the theoretical guarantee does not directly apply, and validity is assessed empirically.

Regarding the trade-off between validity and detection speed, the correction makes the betting strategy more conservative in two ways: it increases the centering term to account for uncertainty in the estimated reference quantities, thereby reducing the potential payoff, and raises the rejection threshold from 1/α1/\alpha to 1/(α−η)1/(\alpha-\eta). Both adjustments may result in slower detection under the alternative. However, the calibration adjustment decreases at the rate O(m−1/2)O(m^{-1/2}) for fixed η\eta and KK, where mm is the reference sample size. Consequently, the reduction in wealth growth attributable to the calibration adjustment is expected to be less pronounced with larger reference samples. We have added this qualitative discussion to the revised manuscript (after Theorem 1). This also addresses the second weakness.

Q2. In the revised manuscript (Section 5), we explicitly report the default values of ε=10−6\varepsilon=10^{-6} and λmax⁡=5000\lambda_{\max}=5000. The former is chosen to ensure strictly positive betting factors while avoiding an overly restrictive betting strategy, whereas the latter serves as a safeguard against excessively large betting fractions. We use these values across the synthetic and real-data experiments. We have also conducted a sensitivity analysis involving both parameters (Appendix E, Figure 5). Specifically, we consider ε∈{10−2,10−6,10−8}\varepsilon\in\{10^{-2},10^{-6},10^{-8}\} and λmax⁡∈{20,5000,10000}\lambda_{\max}\in\{20,5000,10000\}. The empirical type-I error and power curves are nearly indistinguishable across these configurations, suggesting limited sensitivity to these hyperparameters over the range considered.

Q3. We thank you for highlighting this important practical consideration. We agree that a prolonged period of normal operation may increase detection delay, both because the witness function is dominated by pre-change observations and because the accumulated wealth may be substantially depleted. We now explicitly discuss these limitations in the revised manuscript (Section 7 and Remark 5). Potential remedies include constructing the witness function using a rolling window or exponentially weighted observations and aggregating wealth processes initiated at different time points. These modifications can preserve anytime-valid type-I error control when appropriately constructed. A rigorous and systematic investigation of these extensions for continuous monitoring is, however, left for future work.

Weakness (empirical evaluation). In the revised manuscript, we include the sequential goodness-of-fit test of Zhou and Liu (2024) as an empirical baseline in both simulation experiments (Figures 1 and 2). Since their method is designed for a simple null, we implement it by instantiating our composite null at specific choices of mixing weights (see Appendix C.3 for details). Our numerical results show that Zhou and Liu (2024) controls the type-I error when the specified mixture coincides with the data-generating distribution but exhibits substantial error inflation when the mixing proportions differ (left panels of Figures 1 and 2). In contrast, SeqGoF maintains empirical type-I error at or near the nominal level across all the scenarios considered. Moreover, SeqGoF demonstrates greater sample efficiency in the Gaussian experiment (right panel of Figure 1). Although Zhou and Liu (2024) achieves higher sample efficiency in the non-Gaussian experiment (Figure 2), this apparent advantage should be interpreted with caution given its lack of type-I error control under the composite null (left panel of Figure 2). These comparisons highlight the importance of explicitly accounting for unknown mixing proportions in sequential goodness-of-fit testing.

Thank you for the detailed response—especially the added baseline comparisons addressing the empirical evaluation weakness. I will maintain my current rating.

We thank you for your comments that have substantially improved the quality of the paper.

Reviewer_zb9NOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes SeqGoF, a nonparametric sequential goodness-of-fit test for a composite null hypothesis defined by the convex hull of KK reference distributions with unknown mixing weights. The method projects the empirical kernel mean embedding onto the null polytope, uses the projection residual as an RKHS witness, and constructs an e-process via testing-by-betting. The authors establish anytime-valid Type-I error control, power-one guarantees, and stopping-time asymptotics, supported by synthetic experiments and a real-world application to Backblaze SMART telemetry.

Critical strengths

(1) The establishment of anytime-valid Type-I error control (Theorem 1) and almost-sure power-one guarantees against any fixed alternative (Theorem 2) are solid and non-trivial. (2) Projecting the empirical mean embedding onto the convex hull to construct the witness is geometrically intuitive. Reducing the infinite-dimensional projection to a KK-dimensional Quadratic Program (QP) makes it highly efficient. (3) The method remains valid when reference distributions are only available through samples (Remark 1 and Theorem 4). (4)The application to Backblaze SMART telemetry effectively demonstrates the sequential test's capability in detecting near-failure hard drives amidst heterogeneous healthy drives.

Critical weaknesses

(1) Lack of First-Order Optimality. In Appendix B, the authors note that SeqGoF does not satisfy Assumption A2 of Shekhar (2025) and thus is not first-order optimal. \textit{Question:} Can the authors provide intuition on the efficiency loss (in terms of stopping time) compared to an oracle or the DV e-process when the alternative is close to the null? (2) Scalability w.r.t. KK. Solving a KK-dimensional QP at each time step tt is fast for K=3K=3. \textit{Question:} How does the computational cost scale for very large KK (e.g., hundreds)? Can the QP be warm-started efficiently across time steps? (3) Missing Empirical Baselines. The numerical experiments only show the Type-I error and power of SeqGoF. \textit{Suggestion:} Include at least one baseline (e.g., a batch kernel MMD test adapted sequentially, or an extension of Zhou & Liu 2024) to compare sample efficiency and stopping times. (4) Kernel Tuning. The empirical performance depends on the kernel bandwidth (e.g., Gaussian RBF with h=dh=\sqrt{d}). \textit{Question:} Is there a principled way to choose the bandwidth sequentially without violating the e-process validity?

Questions for authors

(1) A brief comment on the practical choice of ϵ\epsilon (downside risk) and its impact on wealth growth would be helpful. (2) When using empirical estimates for QQ and μFk\mu_{F_k}, does maintaining the anytime-valid property require a union bound over time? A brief discussion on the cost of the finite-sample validity correction in the main text would be useful. (3) How was the standardization of the 7-dimensional SMART features performed? Specifically, was it computed using only the healthy reference samples or the entire data stream? Using the entire stream might violate the sequential i.i.d. assumption.

2 comments
AuthorsOpenReview ↗

Response to reviewer Reviewer zb9N (1/2)

Thank you for your careful reading and positive assessment of our paper, below are our responses to your questions.

W1. Thank you for this question. The DV e-process chooses its test functions by maximizing an empirical log-growth objective. Under its assumptions, this targets the minimum KL divergence from the alternative to the composite null. This is the information-theoretic benchmark: it determines the smallest achievable first-order expected stopping time Shekhar (2025), Fact 1 and Theorem 3.1.

SeqGoF constructs its betting score from the RKHS projection residual and optimizes the betting fraction for that score. Its resulting log-growth rate is determined by this geometric construction and need not attain the KL-optimal rate. The potential efficiency loss therefore comes from the gap between these two growth rates. Since the leading stopping time is the log rejection threshold divided by the log-growth rate, a smaller rate requires more observations.

This choice is motivated by computational simplicity. Although our composite null contains infinitely many mixtures, its RKHS geometry reduces witness construction to a K-dimensional quadratic program. Moreover, linearity in the mixture weights reduces null calibration to the maximum of K component expectations. Together with a one-dimensional betting optimization, this gives a tractable procedure. We therefore use this projection-based witness without requiring optimization over a function class rich enough to recover the KL benchmark. The potential efficiency loss reflects this computational trade-off.

W2. We thank you for raising these questions.

Computational scalability. For a dense Gram matrix QQ, standard QP solvers involving matrix factorization incur an O(K3)O(K^3) cost per factorization, which may become computationally demanding for large KK. Alternatively, projected-gradient algorithms can be employed, with a computational cost of O(K2)O(K^2) per iteration. The overall computational cost depends on the conditioning of QQ and the number of iterations required for convergence. We note, however, that KK represents the number of reference components, which is typically small (often in the single digits) in many applications and need not increase with the number of observations or the data dimension.

Warm-starting. The QP is particularly amenable to warm-starting because the Gram matrix Q remains fixed across time steps, and only the linear term cₜ changes. Moreover, under a bounded kernel,

∥ct−ct−1∥∞=O(t−1).\lVert c_t - c_{t-1} \rVert_{\infty} = O(t^{-1}).

Thus, successive optimization problems become increasingly similar as t increases. The solution from the previous time step can therefore be used to initialize the current QP, potentially reducing the computational burden. We have included this discussion in Remark 3 (Appendix C) of the revised manuscript.

W3. We thank you for this comment. In the revised manuscript, we include the sequential goodness-of-fit test of Zhou and Liu (2024) as an empirical baseline in both simulation experiments (Tables 1 and 2). Since their method is designed for a simple null, we implement it by instantiating our composite null at specific choices of mixing weights (see Appendix C.3 for details). Our numerical results show that Zhou and Liu (2024) controls the type-I error when the specified mixture coincides with the data-generating distribution but exhibits substantial error inflation when the mixing proportions differ (Tables 1, 2 and Figures 5 - 8). In contrast, SeqGoF maintains empirical type-I error at or near the nominal level across all the scenarios considered. Moreover, SeqGoF demonstrates greater sample efficiency in the Gaussian experiment (Table 1 and Figure 6). Although Zhou and Liu (2024) achieves higher sample efficiency in the non-Gaussian experiment (Table 2), this apparent advantage should be interpreted with caution given its lack of type-I error control under the composite null (Table 2 and Figure 7). These comparisons highlight the importance of explicitly accounting for unknown mixing proportions in sequential goodness-of-fit testing.

We have also expanded our discussion of the related work by Martínez-Taboada and Ramdas (2025), highlighting the methodological and computational differences between their approach and SeqGoF in the context of composite mixture nulls (Appendix C.2).

AuthorsOpenReview ↗

Response to reviewer Reviewer zb9N (2/2)

W4. We thank you for raising this question. In Remark 4 of Appendix C, we establish that sequential bandwidth selection is compatible with e-process validity, provided that the bandwidth is chosen predictably and the corresponding reference quantities are updated appropriately.

As a practical alternative to explicit bandwidth selection, we investigate the aggregation of wealth processes across three kernels and four bandwidths (Appendix D, Figure 2). Our numerical results suggest that aggregation mitigates sensitivity to kernel and bandwidth selection while retaining competitive power. We, however, acknowledge in Section 7 that our current work does not investigate explicit sequential bandwidth selection strategies or their power properties and is potentially an avenue for future research.

Q1. We include the following comment on page 5 of the manuscript: ``In practice, smaller values of ε\varepsilon permit more aggressive betting and potentially faster wealth growth under the alternative, at the expense of greater downside risk."

In the revised manuscript, we conduct a sensitivity analysis with ε∈{10−2,10−6,10−8}\varepsilon\in\{10^{-2},10^{-6},10^{-8}\}. The results indicate negligible differences in empirical power and type-I error across these choices, lending support to our default specification of ε=10−6\varepsilon=10^{-6} (Appendix E, Figure 3).

Q2. No union bound over time is required. The reference quantities are estimated once from fixed reference samples independent of the online stream. Appendix A.6 constructs a single event with probability at least 1−η1-\eta that controls the calibration error uniformly over the entire RKHS witness class. Thus, the same event covers all subsequently chosen witnesses, even though they change adaptively over time. The reference-error bound uses a union bound over the KK components.

The correction adds an uncertainty margin to the empirical null calibration and raises the rejection threshold from 1/α1/\alpha to 1/(α−η)1/(\alpha-\eta), allocating η\eta to reference-estimation failure only once. Both adjustments can slow detection. For fixed KK and η\eta, the calibration margin decreases as O(m−1/2)O(m^{-1/2}), where mm is the reference sample size per component. We have added a brief discussion on page 6 of the revised paper.

Q3. Thank you for raising this point. The seven SMART features were first subjected to a signed logarithmic transformation and subsequently standardized using feature-specific means and standard deviations computed exclusively from the healthy reference samples. These preprocessing parameters were held fixed throughout the sequential analysis, thereby avoiding any look-ahead bias. We have clarified this in the revised manuscript (Appendix F). In addition, we have expanded the real-data analysis to examine sensitivity to alternative age-based partitions and data-driven construction of the reference components (Appendix F.1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.