- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The method discovers subgroups defined by baseline covariates and jointly optimizes a temporal window. The empirical experiments primarily provide synthetic recovery demonstrations and internal objective comparisons, without clearly identified comparisons against established methods.
Critical strengths
The problem formulation may be useful. Jointly returning a subgroup rule and a temporal window can be a reasonable goal.
Critical weaknesses
-
Insufficiently established methodological novelty. The paper uses sigmoid predicates, feature gates, and smooth interval boundaries, but does not clearly establish what the joint formulation contributes beyond existing differentiable rule learning and temporal subgroup discovery. The absence of external comparisons makes that contribution especially difficult to assess.
-
Limited empirical validation and reproducibility. The experiments primarily provide synthetic recovery demonstrations and comparisons among internal objective variants. They do not clearly identify or evaluate established competing methods. Furthermore, the optimization settings, baseline definitions, synthetic data generation, and real-world data preprocessing are insufficiently specified to reproduce and assess the results.
Questions for authors
-
Please provide sufficient implementation details for reproducibility, including optimizer settings, stopping criteria, temperature schedules, regularization coefficients, hyperparameter selection, and so on. For the synthetic experiments, please specify the data generation process and covariate and treatment distributions.
-
For the real-world MIMIC data, please provide details for data processing, including treatment definition, time-zero alignment, baseline measurement windows, missing-data handling, and so on. Please also interpret the discovered temporal pattern.
-
Please compare with established subgroup-discovery methods, including relevant approaches discussed in the related work. Where necessary, explain how these methods are adapted to longitudinal outcomes. Such comparisons would help establish the benefit of joint subgroup and temporal-window optimization beyond the internal ablations.
-
Please define the exact objectives and implementations of “Static,” “Global Temporal,” “Window,” and “Behavioral,” as well as “Static SD,” “ATE-SD,” and “Temporal SD.” In particular, how are windows obtained for variants without explicit window optimization, and why do all three clinical comparators return the same subgroup?
4 comments
Q3. Comparisons with established subgroup-discovery methods
We added two conventional beam-search SD/EMM comparators: Beam-Exact, which ranks candidate rules using exact temporal-window scanning, and Static-then-Scan, which selects a rule using a static discrepancy before scanning its trajectory. Both use coverage weighting, minimum support, four quantile thresholds per feature in both directions, beam width 5, and maximum depth 3.
| Method | Transient IoU | Mean F1 | Mean trajectory RMSE | Larger-scale runtime (s) |
|---|---|---|---|---|
| Beam-Exact | 0.315 | 0.728 | 0.528 | 17.838 |
| Static-then-Scan | 0.320 | 0.590 | 0.745 | 17.830 |
| Behavior + Scan | 0.320 | 0.745 | 0.411 | 3.771 |
| TDSD without coverage | 0.875 | 0.601 | 0.462 | 3.932 |
| Revised TDSD | 0.775 | 0.698 | 0.322 | 4.119 |
Across 20 seeds, revised TDSD improves transient IoU over Beam-Exact by 0.460 (paired 95% CI [0.399,0.521]); in the larger regime, IoU is 0.742 versus 0.102. Mean F1/RMSE aggregate five patterns; RMSE excludes infeasible test contrasts (2/100 standard and 7/100 larger runs for revised TDSD). Beam-Exact achieves higher F1, and TDSD is slower in the standard regime. Thus the results support a localization–recovery–runtime trade-off rather than dominance.
These are conventional discrete-search implementations, not reproductions of specific published methods. Comparisons with Xu et al. (2024) on window-averaged outcomes and Wei et al. (2020) remain outstanding.
Q4. Exact baseline definitions and window selection
Let denote the subgroup-specific treatment–control discrepancy, and its mean absolute value within window .
| Original synthetic variant | Optimized quality |
|---|---|
| Static | $ |
| Global Temporal | $\operatorname{mean}_t |
| Window | over eight predefined soft windows |
| Behavioral | |
| TDSD | Soft-window-weighted , with learned boundaries |
Originally, non-TDSD windows were obtained from the test trajectory's half-peak envelope; TDSD stored its training window without restoring the corresponding checkpoint. We acknowledge these evaluation inconsistencies. The revised benchmark selects and freezes both rules and windows on training data before testing.
| Original clinical comparator | Final ranking objective |
|---|---|
| Static SD | Absolute signed mean discrepancy |
| ATE-SD | Mean absolute discrepancy (not a causal ATE estimator) |
| Temporal SD | Maximum mean absolute discrepancy over windows of 3–12 points |
All three clinical variants share the same beam expansion ranking (mean absolute discrepancy), candidate pool, beam width 50, depth 4, and minimum support of 50 overall and 20 per arm. Static SD and ATE-SD obtain windows by post-hoc exact scanning; Temporal SD scans retained candidates before final selection. Their identical output is explained by shared search and similar rankings, not agreement among independent algorithms. We will label them accordingly.
We appreciate the reviewer's suggestions and will incorporate these clarifications and the completed benchmark results into the revision.
Thank you for the candid response. I appreciate the authors' efforts, but my main concerns remain unresolved.
-
Insufficient motivation and methodological novelty. The authors clarify that their contribution lies in the joint formulation rather than the combination of standard techniques. However, they do not convincingly establish why "jointly discovering subgroups and temporal windows" constitutes a meaningful problem beyond identifying increasingly fine-grained patterns in the data. Moreover, the clinical experiment lacks convincing interpretation and validation of the discovered temporal patterns. Therefore, neither the methodological development nor the empirical case study sufficiently demonstrates the significance of this formulation.
-
Missing comparisons with established methods. The additional baselines are custom implementations rather than established published methods. The authors acknowledge that comparisons with relevant existing methods remain outstanding.
-
Serious concerns about experimental rigor and reproducibility. The authors acknowledge substantial problems in the original evaluation, including test-data-dependent window selection, inconsistent checkpoint handling, and discrepancies between the soft and hard subgroup evaluations. Furthermore, hyperparameters were selected heuristically without sufficient justification. These are substantial evaluation issues rather than minor implementation details. Moreover, essential configurations, code, and corrected clinical analyses are still promised for future release rather than provided for assessment.
Overall, while the response provides useful clarifications, it also reveals additional weaknesses in the original evaluation. My concerns about novelty, empirical evidence, and soundness remain. I therefore maintain my original rating.
Thank you for your further assessment. We appreciate your concerns and have revised the manuscript to clarify the contribution and strengthen the empirical evidence. We would like to emphasize three points.
1. Motivation and methodological novelty. The contribution is not simply combining sigmoid predicates with smooth windows, nor discovering increasingly fine-grained patterns. It addresses the interdependence between who exhibits an exceptional discrepancy and when that discrepancy occurs. Under our synthetic model, an episode-specific effect in a subgroup of prevalence is attenuated by when averaged across the population and time horizon. Consequently, selecting a subgroup or temporal interval independently can obscure the very signal being sought. Our joint objective allows the interval to influence rule learning and the evolving subgroup to influence interval learning, while coverage weighting, minimum duration, and arm-support safeguards discourage degenerate solutions (Sections 3.6–3.9 and 4.6). The novelty lies in this coupled, interpretable discovery objective and its optimization, rather than in the individual relaxation techniques.
2. Comparisons with competing methods. Beam-Exact and Static-then-Scan are adaptations of conventional beam-search subgroup discovery to longitudinal outcomes, not additional internal ablations. They explicitly test discrete joint search and sequential discovery, respectively. With 20 paired seeds, TDSD achieves transient-window IoU of 0.775, compared with 0.315 and 0.320, respectively (Appendix G, Tables 5–7). The corresponding paired gains are 0.460 and 0.455, with 95% confidence intervals excluding zero. At 50 covariates and 100 time points, TDSD achieves IoU of 0.742 versus 0.102 for Beam-Exact, with mean runtimes of 4.12 versus 17.84 seconds. These results establish advantages over the specified search alternatives under the tested settings.
3. Experimental rigor and reproducibility. We agree that the original evaluation issues were substantive. The revised paired benchmark addresses them through independent training and test samples, training-only selection of hard rules and windows, consistent checkpoint selection, and evaluation of the frozen discoveries. Ground-truth membership is used only for evaluation in this benchmark. Coverage weighting improves mean hard-rule F1 from 0.601 to 0.698 in the standard setting and from 0.516 to 0.699 in the larger setting. The revised manuscript now provides the data-generating process, optimization settings, initialization, temperatures, penalties, baseline definitions, per-pattern results, paired confidence intervals, support failures, and clinical preprocessing (Appendices G–I). We distinguish these completed experiments from the original component checks and remove the unsupported confirmatory clinical significance claim. We believe that code is not the only evidence to support reproductivity, we provided detailed setting for our experiments and we believe that this can be sufficient to make our method reproducible.
We respectfully ask that the methodological contribution be assessed in light of the new independent benchmark and its direct test of coupled versus sequential discovery, rather than solely on the original ablations. We believe the revised evidence supports a meaningful contribution to interpretable temporal subgroup discovery, within the explicitly stated scope.
We thank the reviewer for recognizing the potential usefulness of jointly discovering subgroups and temporal windows. We address the concerns about novelty, empirical validation, and reproducibility below.
Weakness 1: Methodological novelty
Our novelty lies not in individual sigmoid predicates or feature gates, but in coupling subgroup discovery and temporal localization through the subgroup-specific discrepancy trajectory ΔS(t). Changing the subgroup changes the preferred window, while changing the window changes which subgroup is rewarded. This differs from discovering a subgroup using a fixed temporal summary and subsequently identifying its window. Our new discrete-search comparisons (Q3) support the benefit of this coupling for temporal localization, although they do not establish universal superiority of differentiable search.
Weakness 2: Empirical validation and reproducibility
We agree that the original internal ablations are insufficient. We have completed a new outcome-only benchmark with 20 seeds, five temporal patterns, independent training/test samples, and discrete subgroup-search baselines. We additionally introduce coverage weighting, minimum window duration, arm-support safeguards, and improved hard-rule extraction. The revised benchmark does not use these labels for training. We will release the configurations, code, and per-run results.
Q1. Implementation details and synthetic data generation
The synthetic data follow Yi(t)=μ(t)+AiSi∗τ(t)+ϵi(t), with independent standard-normal covariates, independent Bernoulli(0.5) treatment, and planted rule S∗=1[X1>0.8,X3<0,X6>0.5].
The revised benchmark uses 2,500 training and 2,500 test individuals per seed, with (p,T)=(10,30) or (50,100), μ(t)=0.1t+0.4sin(t/3), and Gaussian noise SD 0.5. For l=⌊0.4T⌋, h=⌊0.7T⌋, the five effect functions are 2.51[l,h](t), 2.51t≥l, 2.5t/(T−1), 2.5(1−t/(T−1)), and 2.5sin(4πt/(T−1)).
Thresholds, gate and window logits start at zero; direction logits start at 0.2N(0,1). Every ten epochs and at the end, feasible hard-rule candidates are scored on training data; the selected rule and window are frozen for testing. Hyperparameters are fixed heuristic settings, not tuned using planted labels. The original experiments used Adam (0.03), fixed budgets of 800/1500/1200/350 iterations, and fixed temperatures; the full original configurations will be provided in the appendix.
Q2. MIMIC-IV preprocessing and interpretation
The discovered lactate ≥2.10 rule covers 6,753 individuals, with a soft window of [4.52,14.06] h after the landmark. Lactate may indicate physiological stress, but this is exploratory, not a causal treatment recommendation. The original figures plot soft trajectories whereas the table evaluates a hard rule extracted using feature medians. We will recompute both using the same frozen rule and window, correct the extractor to use learned thresholds and directions, and withdraw the selection-unadjusted permutation p-value as confirmatory evidence. Independent clinical validation remains outstanding.