- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes a model-assisted diagnostic for conditional dependence between latent event and censoring times. It measures the range of mutual information over censoring-compatible binary completions at landmark times. A deep variational frailty model supplies one completed dataset for within-stratum permutations and a symmetric rank test. The paper establishes exact stratum-wise computation, oracle validity, and calibration-error bounds, and evaluates the method on synthetic and real survival data.
Critical strengths
S1. The statistic has a clear interpretation. It measures how much mutual information remains undetermined by the observations.
S2. The oracle calibration argument is sound under its assumptions. Reusing one compatible completion gives an error bound without a factor proportional to the number of permutations. Algorithm 2 correctly calibrates its Fisher score through symmetric ranks.
S3. The appendices discuss nonidentifiability, model-dependent calibration, and median saturation.
Critical weaknesses
W1. The real-data protocol does not establish practical reliability. Table 6 in the Appendix reports rejection summaries without baseline comparisons or model and stratification sensitivity analyses. Covariate and stratum counts vary across runs without a clear selection rule, so runs may test different conditional-independence hypotheses. Continuous covariates can also yield singleton exact-profile strata with no nontrivial permutations; if all strata are singletons, Algorithm 2 returns . Coarsening instead requires justified within-stratum invariance. These summaries describe behavior across runs, not detection accuracy or power. A clear conditioning target, stratification rule, and sensitivity analysis are needed to interpret the real-data rejections.
W2. The interpretation of rejection remains unresolved. The paper acknowledges observational equivalence but leaves the restricted model class behind "identifiable departures" unspecified. Theorem 3 bounds calibration error using error in the joint conditional completion law for the whole dataset; it does not show that DVFM fitting makes this error small. Under correct marginal specification, Proposition 3 establishes a correct null-compatible optimum, while allowing other completion laws to tie in observed likelihood. The paper needs to explain which assumptions let a rejection distinguish dependent censoring from imputation-model misspecification.
W3. The source of power needs clarification. Proposition 5 gives a population width of when the at-risk probability is . In a single stratum, selecting an observed-time median landmark separately for each dataset saturates the population raw maximum for both original and reconstructed distributions, removing the positive gap required by Theorem 2. Appendix H's grids for include the median. Global landmarks need not be medians within every stratum, so power could arise from separation across strata or finer-scale behavior outside Theorem 2. The paper should clarify which mechanism drives its results.
W4. The experiments do not isolate the statistic's benefit or adequately characterize its scope. Synthetic evaluation fixes and four covariates, leaving behavior across sample sizes, dimensions, and sparse strata unexplored. The copula baselines report single-seed p-values rather than rejection rates under a common replication protocol. Different generation conventions in Appendices C and C.1 also leave comparability unclear. A comparison that changes only the statistic, retaining the same DVFM fits and shared completions, would isolate the value of MI uncertainty. Appendix H provides a useful preliminary null check, but ten independent datasets per setting yield intervals too wide to confirm precise 5% error control.
W5 (minor). The landmark illustration has incorrect labels. Figure 1's post-event and post-censoring should be zero, as its caption correctly states. Table 5 also marks the counterpart indicator as unknown at , contrary to Appendix H.2's no-ties boundary convention.
Questions for authors
Q1. How are covariates and strata selected, and do all runs test the same hypothesis? What calibrates the repeated-run Fisher combination, and how sensitive are real-data rejections to the completion model and stratification?
Q2. Which model restrictions make the null and alternatives distinguishable, and what supports accurate conditional completion by the fitted DVFM?
Q3. Does power reflect population separation across strata or finer-scale behavior? Results by landmark grid and the maximizing landmarks would help resolve this.
Q4. Could you provide a matched comparison of statistics and baselines, and characterize calibration across sample sizes and stratum sparsity?
7 comments
Q3 and W3 Median saturation and the source of power
We agree that choosing each dataset's own median saturates the raw population maximum. The implemented statistic instead fixes the observed cohort's landmarks for all reference datasets and uses the largest absolute standardized width deviation within each stratum:
where and are reference means and standard deviations. An observed-data median need not be a reference median, and maximizing this standardized deviation is different from maximizing raw width.
The eight-variable analysis holds each fitted model, completion, and permutation stream fixed while varying only the landmark grid. The original ten-landmark grid remains primary; no grid was selected using these results. Entries are detections over 20 independent development cohorts.
| Design | K=3 | K=5 | K=7 | K=10 | 10 plus median |
|---|---|---|---|---|---|
| Shared frailty (3,3) | 20/20 | 20/20 | 20/20 | 20/20 | 20/20 |
| Supplied Frank, θ=20 | 20/20 | 20/20 | 20/20 | 20/20 | 20/20 |
For shared frailty, 272/320 standardized stratum maxima occur at quantiles at least 0.6, and 317/320 have negative deviations. Only 4/320 also maximize raw width. Adding the median selects it in only 16/320 strata and leaves the detection count unchanged. An early-only grid gives 1/20 detections and the median alone gives 19/20, identifying the importance of later landmarks in this fitted experiment. Frank gives 20/20 under both additional grids.
A population contrast can nevertheless exist within one stratum. In the shared-frailty class above, let . At a landmark , the original and independently permuted completed distributions have at-risk probabilities for nonconstant . At the population 0.15 quantile, and . The widths then satisfy , where is binary entropy. Thus a nonmedian downward contrast can supply population separation. This establishes a possible mechanism; it does not show that every fitted-model detection is explained by population separation. Completion-reference and finite-stratum effects also contribute, so we will restrict the corresponding power claim.
Q4 and W4. Matched comparisons and calibration
We isolate the statistic by replacing MI width with at-risk entropy, retaining the same eight-variable DVFM fit, completion, landmarks, permutations, normalization, and p-value calculation. LRTs use identical cohorts, Weibull margins, 200 bootstrap samples, and multistart fitting for observed and bootstrap data. Frank is excluded from fitted copula families. The maximum across Gaussian, Clayton, and Gumbel is bootstrap calibrated.
| Design | MI width | Entropy | Gaussian | Clayton | Gumbel | Maximum |
|---|---|---|---|---|---|---|
| Independent null | 0/20 | 0/20 | 0/20 | 1/20 | 1/20 | 0/20 |
| Shared frailty (3,3) | 20/20 | 20/20 | 15/20 | 20/20 | 4/20 | 20/20 |
| Supplied Frank, θ=20 | 20/20 | 20/20 | 20/20 | 20/20 | 0/20 | 20/20 |
Each row uses the same 20 development cohorts (2,000 subjects, four binary covariates); the null row is the prespecified first 20 cohorts, not the full calibration study. Width matches the strongest baseline rejection counts on both alternatives. Width and entropy make identical decisions here, so we claim no power advantage. MI width instead gives an exact uncertainty interval and minimax interpretation. At 5% at risk, event/censoring fractions (0.475,0.475) and (0.90,0.05) yield widths 0.5268 and 0.1985 nats, while at-risk entropy is 0.1985 for both. This illustrates the measured quantity, not additional power. After fixing the eight-variable configuration, we validated on fresh independent cohorts. Independent-Weibull null results were:
| Subjects | Covariates | False rejections | 95% interval | Mean coverage |
|---|---|---|---|---|
| 2000 | 4 | 7/100 | 3.4–13.7% | 100% |
| 4000 | 4 | 2/20 | 2.8–30.1% | 100% |
| 2000 | 6 | 0/20 | 0–16.1% | 99.42% |
All binary covariates form exact profiles. Six-covariate coverage ranges from 97.25% to 100%; all cohorts yield finite results. Wilson intervals reflect cohort uncertainty. The earlier development result was 12/100 null rejections, distinct from fresh 7/100; we do not claim exact or uniform 5% calibration. Separate independent-frailty stress nulls yielded 18/20 and 14/20 false rejections at scales 3 and 1, showing dependence on completion and reference assumptions.
Fresh detection counts were 20/20 for shared frailty (3,3), 7/20 for (1,3), 20/20 for supplied Frank, 18/20 for standard Frank (θ=20), and 17/20 for standard Gumbel (θ=6). Standard generators use uniform copula margins and Weibull times; supplied Frank has different margins and is reported separately. These cohorts are disjoint from matched baseline cohorts. All counts use p < 0.05 and are not power comparisons at matched true error rates.
Short derivation of the identifiable class
Within one stratum, and . Conditional independence given gives If is nonconstant, the two conditional survival functions are strictly positively correlated for positive , so and are dependent. Since the observed law already determines the joint law within the class, an independent member cannot reproduce it. Also,
P(C>c\mid T=t,\delta=1) =\frac{E[M\exp\{-M(rt+(1-r)c)\}]}{E[M\exp(-Mt)]},$$ which proves the completion formula. For fixed known Weibull shapes $k_E,k_C>0$, suppose the conditional survivals are $\exp(-rM u^{k_E})$ and $\exp(-(1-r)M v^{k_C})$. The observed event and censoring subdensities satisfy \$ \frac{f_1(t)}{f_0(t)}=\frac{r k_E}{(1-r)k_C}t^{k_E-k_C}, \$ identifying $r$. With $H(t)=rt^{k_E}+(1-r)t^{k_C}$, \$ P(E>u,C>v) =S_T\!\left(H^{-1}(ru^{k_E}+(1-r)v^{k_C})\right). \$ For $c\geq t$, let $z=H^{-1}(rt^{k_E}+(1-r)c^{k_C})$. The completion law is \$ P(C>c\mid T=t,\delta=1) =\frac{f_T(z)/H'(z)}{f_T(t)/H'(t)}. \$ The missing-event expression follows by symmetry. The restrictions must hold for the true and fitted distributions; this result does not assert that unrestricted DVFM training enforces them.
Thank you for the detailed comments. In the additional analyses below, we fix the DVFM latent dimension at eight and retain our implemented MI-width statistic, single completion, and p-value calculation. We make the conditioning target explicit, provide a concrete identifiable class, and add matched comparisons and independent-cohort calibration experiments.
Q1 and W1 Conditioning target and real data
Our target is E⊥C∣S, where S is a stated covariate profile. For NWTCO, the primary target uses study, stage, and central histology in their original categories. There are 4,028 patients and 16 exact strata, with 32--757 patients per stratum. We fix this target, the ten landmark quantiles from 0.15 to 0.85, minimum stratum size 20, and 200 permutations across all three prespecified fit/completion seeds. We also examine two explicitly different conditioning targets, using the same settings and eight latent variables throughout.
Thus NWTCO remains flagged in every run without combining repeated-run p-values. The age cutoff is the covariate median; three profiles containing 47 patients are below the minimum size. Different targets test different conditional hypotheses and will be labeled separately. These results establish seed and target stability for the examined DVFM configuration, rather than real-data detection accuracy or robustness to all completion-model families. On the same primary cohort and target, Gaussian and Clayton bootstrap LRTs also give p=1/201, while Gumbel gives p=180/201; the bootstrap-calibrated maximum across the three gives p=1/201.
We will remove the Fisher chi-square pooling across runs in Table 6. Seeds on the same patients do not create independent datasets. The Fisher score within an individual test is compared with generated reference scores, not a chi-square distribution. We will also distinguish the fixed-target analysis from the earlier exploratory feature settings.
For continuous covariates, grouping must be stated in advance. Constancy of the conditional event and censoring distributions within each group is a sufficient condition for preserving the original conditional null; otherwise the grouped analysis has its own target. Singleton or undersized profiles cannot provide usable permutation evidence. We report their exclusion and subject coverage, and report no result if no eligible stable stratum remains.
Q2 and W2 Model restrictions and completion accuracy
We replace the phrase ``identifiable departures'' with an explicit example. Within each fixed stratum, suppose E and C are conditionally independent exponentials with rates rM and (1−r)M, where M>0 is shared frailty and 0<r<1. Write T=min(E,C) and δ=1{E<C}. The observed law identifies r=P(δ=1) and ST, and therefore identifies
P(E>e,C>c)=ST(re+(1−r)c).Nonconstant shared frailty has no observationally equivalent independent member of this restricted class. The missing-time law is identified as well: for c≥t>0,
P(C>c∣T=t,δ=1)=fT(t)fT(rt+(1−r)c).The derivation and extension to fixed, known Weibull shapes appear below. These structural restrictions explain when observed-law estimation can determine latent completion. They are not imposed merely by choosing eight latent variables, and independent proxies outside the class are not ruled out.
To assess the fitted DVFM directly, we evaluated its conditional missing-time CDF against known conditional quantiles in 20 independent synthetic null cohorts of 2,000 subjects. We used 64 subjects per missing-time type per cohort, quantiles 0.1, 0.5 and 0.9, and 256 posterior draws. Mean absolute CDF error was 0.0313; the six type-by-quantile errors ranged from 0.0098 to 0.0498. Coverage of true central 90% conditional intervals averaged 88.3% for missing event times and 86.8% for missing censoring times. This directly examines latent completion rather than relying on observed likelihood alone. It is a diagnostic of accuracy, not an estimate of whole-cohort total variation.
The theoretical validity claim requires calibrated joint completion and exchangeable scoring. Our reference-only normalization and ranking are evaluated separately by the calibration experiments below; we do not claim that reducing latent dimension makes them inherit the exact symmetric-rank guarantee. This distinction keeps the theoretical assumptions and the evidence for the fitted procedure explicit.