acceptodds
Under review as a conference paper at ICLR 2027

G3I-Net: A Trimodal Architecture for Tumor Staging and How Much Its Gene Graphs Actually Vary

Abstract

Architectures that condition one modality's structure on another are evaluated by end-task accuracy. That measurement cannot separate a mechanism producing genuinely input-dependent structure from one that has collapsed onto a single solution shared by every input. We build such a mechanism and measure it directly. G3I-Net is a trimodal architecture for non-small-cell lung cancer T-staging: a frozen Swin-UNETR encodes CT volumes, an MLP encodes radiomic descriptors, and a graph neural network encodes pathway genes over a protein–protein interaction prior. A hypernetwork reads the CT embedding and emits one score per gene, from which a per-patient adjacency over that prior is built, so the molecular branch computes over a topology derived from the patient's own tumor rather than one shared across the cohort. On 128 patients with 21 advanced-stage events the model reaches a patient-level AUC of 0.9119, against 0.749 for the strongest classical late-fusion baseline. We then measure the generated graphs. Each sits far from the prior it was built from, so the conditioning is not inert, yet the graphs are nearly identical to one another: the architecture allows sixteen independent ways for patients to differ and the trained model uses about one. Widening the hypernetwork's input leaves that unchanged, and letting it score every edge freely lowers it further. Removing the conditioning entirely leaves every patient's predicted class unchanged, while two implementations of the same model differing only in the order of floating-point operations change two. What limits conditioned graph generation here is not how much the architecture can express, but how much of it the trained model uses.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
multi-modal learning, graph neural networks, hypernetworks, conditional computation, medical imaging

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eCJLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes G3I-Net for distinguishing between early-stage and advanced-stage T-stages in non-small cell lung cancer. The model integrates a frozen Swin-UNETR CT encoder, a radiomics-based MLP, and a gene expression graph network; it employs a CT-conditioned hypernetwork to generate patient-specific edge weights, informed by a fixed protein-protein interaction (PPI) support set.

Critical strengths

1.Rather than interpreting a higher AUROC directly as successful patient-level graph generation, the authors examine the parametric constraints of the graph generation process, inter-patient variability, static surrogates, and modality ablations. This approach helps uncover redundant mechanisms within complex multimodal models that might otherwise be obscured by the final performance metrics.

2.The paper explicitly acknowledges that the gating improvements did not significantly outperform the baseline on the primary endpoint, that the gene branches failed to provide incremental predictive value, that the router is prone to degradation, and that model selection across 52 configurations tends to yield overly optimistic point estimates.

Critical weaknesses

1.Section 3.4.1 reports an effective rank of 1.72, with the first principal component accounting for 84.9% of the variance; based on this, the authors characterize the patient plots as approximating a single, fixed pattern. However, the paper fails to clarify whether the effective rank was calculated using modulation codes, node scores, or a normalized adjacency matrix; it also omits details regarding whether data were centered across patients and which spectral normalization method was applied. Similarly, the claim in the introduction—that the data are "73% from the prior"—lacks an explicit definition of the distance formula and the normalization procedure used.

2.Section 3.3.2 describes a process involving initial ranking via LASSO followed by feature-count selection through cross-validation; however, it remains unclear whether these steps are fully nested within the outer training folds. Furthermore, the specific independent validation set used for early stopping is not specified. Section 5 acknowledges that the final configuration was selected from among 52 candidates; consequently, the reported out-of-fold (OOF) point estimates and bootstrap-based confidence intervals for patients do not inherently account for or eliminate model selection bias.

Questions for authors
  1. What data are used for each of the following steps in ASSO: ranking, feature selection, normalization, early stopping, and final configuration selection? Please provide the complete nested workflow, and ensure that all primary baselines employ a consistent, patient-level out-of-fold (OOF) aggregation method. If the baselines are provided with the same frozen CT features, does the performance gap still persist?

  2. How exactly are the effective rank, 84.9% explained variance, and 73% prior distance calculated? Are the statistics computed using the training patients or the held-out patients? Please provide the centering method, the full spectrum, and the distribution of graph distances between patients; furthermore, please distinguish between the dimensionality of the latent codes, the local degrees of freedom, and the linear effective rank across patients.

  3. When the trained weights are held fixed, how do replacing patient-specific scans—either through random permutation, substitution with the training set’s mean scan, or use of a globally learned static scan—affect the predicted probabilities, loss, and AUROC? Can these results provide evidence for a "lack of patient specificity," and help distinguish between potential explanations such as CT scan compression, parameter saturation, or the absence of incremental genetic signals relevant to the task?

  4. Please explain how the threshold for matching specificity was determined, and provide the experimental basis for the lower bound of approximately 0.0125. Furthermore, in Table 2, both models show a specificity of 0.944 among the 107 negative cases; their respective accuracy values ​​of 0.875 and 0.883 correspond to approximately 11 and 12 true positives, implying an incremental sensitivity of about 1/21≈0.0481/21 \approx 0.048. However, Figure 4 reports an increment of 0.095 for the "mixture-of-gates" approach. Were different thresholds or aggregation rules used in these instances? Please clarify and ensure consistent reporting.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer eCJL

Thank you. Two of your questions exposed errors; see the general response. W/Q number your weaknesses and questions in order.

W2/Q1. The complete workflow.

What the submitted pipeline did:

  • Ranking and feature-count selection: on all 128 patients, with labels (not nested).
  • Normalisation: on all patients, without labels.
  • Early stopping: on the outer held-out fold.
  • Final configuration: chosen after the results were known.

The first three are replaced: selection and normalisation inside each outer training fold, a fixed schedule, final weights. The configuration (e.g. width 64) was still fixed on this cohort, which our CIs do not cover; the revision states this.

Baselines are now aggregated like the model: patient-level, from seed-averaged out-of-fold predictions. With the same frozen CT features (Swin-embedding PCA, fit within each fold), logistic regression on CT alone reaches 0.52–0.57, in line with the deep CT-only arm (0.509). Under corrected evaluation, no gap remains.

Q2/W1. Effective rank, 84.9% and 73%. All values use held-out patients, for each of the 25 trained models f, reported as mean and range.

Representations:

  • mp∈R16m_p \in \mathbb{R}^{16} is the hypernetwork code.
  • ap=σ(Wmp+b)∈R89a_p = \sigma(W m_p + b) \in \mathbb{R}^{89} are the node scores.
  • Adyn(p)=rownorm(P⊙apap⊤+10−8)A_{dyn}(p) = \mathrm{rownorm}(P \odot \sqrt{a_p a_p^\top + 10^{-8}}) is the consumed graph, where PP is the row-normalised STRING prior.

Effective rank:

  • XfX_f stacks one representation per held-out patient, with columns centred across patients.
  • From its singular values sks_k, let qk=sk2/∑jsj2q_k = s_k^2 / \sum_j s_j^2.
  • Effective rank is exp⁡(−∑kqklog⁡qk)\exp(-\sum_k q_k \log q_k). We also report the leading share q1q_1.

Corrected models (untrained network on the same inputs in brackets):

  • Node scores: effective rank 1.95, range 1.08–3.76 [3.94]. Leading share q1q_1 = 0.815 [0.607]. The paper's 1.72 and 84.9% are from the earlier models.
  • Code: effective rank 1.58, range 1.07–2.60 [3.85].
  • Consumed graph: effective rank 2.12 [3.83].

Distance from the prior: S(p)=∥Adyn(p)−P∥F/∥P∥FS(p) = \|A_{dyn}(p) - P\|_F / \|P\|_F.

  • The "73%" was computed on one patient per fold.
  • Over all 640 held-out evaluations: mean 0.83, median 0.84, within-model SD 0.10.
  • A static model's single graph sits 0.86 from PP, so this distance does not show patient-specificity.

Between-patient distances (same normalisation):

  • The median is 0.54 × ‖P‖, with 5–95% range 0.03–1.35.
  • So the graphs change between patients in magnitude, but along about two directions.

The three quantities you distinguish:

  • Latent-code dimensionality: 16, an upper bound.
  • Local degrees of freedom of a↦Adyna \mapsto A_{dyn}: 83, i.e. 89 genes minus one scale per connected component of the prior.
  • Linear effective rank across patients: the values above.

Q3. Fixed-weight replacements (table in our reply to PieR Q2).

  • For the audited model, replacing each patient's graph with the training-mean graph, the prior, a permuted graph or a random patient's graph leaves AUC unchanged (every CI covers 0). Probabilities move by 0.011–0.016 on average, mean log-loss decreases slightly, and at most one patient changes class.
  • We did not swap a learned static graph into the trained model; trained with one (our static control: zero hypernetwork input, so one learned graph), the model is no worse: static − dynamic +0.014 [−0.012, +0.046].

Separating the three causes:

  • Saturation is present (74% of node-score logits on T-stage, 87% on histology) but is not required: without it (non-saturating node scores, per-gene weights; reply to wieH), the patient-specific graph is still unused.
  • Incremental genetic signal for T-stage is not detected: 0 of the 89 genes survive FDR, against 28 of 89 for histology. The audited encoder recovers little gene signal even on histology (0.628; logistic regression 0.905); a per-gene encoder recovers it (0.822), yet adds nothing to CT + radiomics on T-stage: +0.017 [−0.039, +0.079].
  • Missing signal is not required for the collapse. With histology as the label (genes predictive), the variation collapses further (effective rank 1.61) and the patient-specific graph is still unused.
  • CT compression cannot be isolated in our design; we no longer claim it.

Q4. Matched specificity. You found a real bug.

  • The rule lowered only the candidate model's threshold, leaving the reference at 0.5. At its own matched threshold the reference recovers one more true positive.
  • So every Figure 4 value was inflated by exactly 1/21; under the old rule, the reference compared with itself scores +0.048.
  • With a symmetric rule, mixture-of-gates is +0.048 [0, 0.158], i.e. one patient, as you computed from Table 2.
  • The 0.0125 bound is the patient-bootstrap interval of two equivalent trainings, not a training-variance floor. Repeated complete trainings vary more (SD 0.019 in patient AUC; reply to wieH W2).

Thank you for your rebuttal; it addresses some of my concerns. However, the paper still has some issues: the training set includes only 128 patients, and this dataset may contain some noise, which could skew the results. Furthermore, the paper does not include an appendix, which leads me to believe that the research is not yet thorough. I will unchange my score.

Thank you for the follow-up. We agree that this is a real limitation. The cohort is small (128 patients, 21 with advanced T-stage), comes from one public collection without external validation, and has the input-pipeline problems disclosed in our general response. The revision states this in the Limitations (§5) and makes no accuracy claim.

We use NSCLC Radiogenomics because our question, whether a graph generated from a patient's CT is patient-specific and used, needs CT, tumour segmentation, gene expression and clinical labels for the same patients, which this public collection provides (Bakr et al., 2018).

Within it, we tried to make the most of the data:

  • Fewer parameters to fit: a frozen CT trunk pretrained without labels, 89 node scores per patient instead of 7,921 adjacency entries, a fixed protein-interaction prior, and 10 radiomic descriptors.
  • Paired evaluation of every patient: five-fold cross-validation over five seeds on splits shared by all arms, with a patient-level bootstrap. Pairing narrows the comparisons: static − dynamic is +0.014 [−0.012, +0.046], against [0.547, 0.791] for the AUC itself.
  • No held-out label in training: feature selection inside each training fold and a fixed schedule. For the full model, the shortcuts this removes inflated AUC by +0.221 on this cohort.
  • A positive control on the same patients: histology, where genes are predictive (logistic regression 0.905), shows that our tests detect graph and gene effects at this size (e.g. −0.106 when the graph is replaced by the prior).
  • Training variation measured by repeating complete trainings.

Hyperparameters were still chosen on this cohort, so some optimism remains. Testing generality needs a larger, external cohort; the tests can be rerun on one unchanged.

Reviewer_wieHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

G3I-Net combines CT, radiomics, and gene expression for binary NSCLC T-staging, with a hypernetwork generating CT-conditioned reweightings of a gene graph. On 128 patients, including 21 advanced cases, the gated model reaches an AUC of 0.9119. The main contribution is an audit showing limited variation across generated patient graphs, little predictive effect from conditioning, and sensitivity to numerical implementation.

Critical strengths
  1. The paper examines whether a conditioning mechanism actually varies across inputs, rather than using downstream accuracy as its only validation. Comparing graph variation with the static-graph control makes this a useful methodological case study.
  2. The modality and gating ablations support a restrained account of predictive benefit. The authors acknowledge that the gene branch provides no demonstrated incremental staging benefit and that the gate does not separate from the backbone on the primary endpoint. This transparency gives the negative findings value.
Critical weaknesses
  1. The explanation for the observed collapse is not isolated. Section 4.5 concludes that the limitation lies in compression of the CT embedding because adding radiomics does not increase effective rank and freeing edge weights lowers it. Neither result rules out optimization or saturation, which the Future Work paragraph itself identifies as unresolved. The weak incremental molecular signal also leaves open whether this task provides an incentive to learn patient-specific graphs at all. I appreciate the narrower interpretation in Section 5, but it does not support the stronger localization in Section 4.5. A bounded edge generator that avoids the reported saturation, tested on a task with demonstrated molecular signal, would help distinguish these explanations. Without such evidence, the supported contribution is a description of collapse in this configuration, with limited explanatory reach.
  2. The claimed numerical measurement floor is not established by the reported uncertainty analysis. Section 4.3 turns the upper end of a patient-bootstrap interval for the AUC difference between two equivalent implementations into a general resolution limit of approximately 0.0125. Under Section 3.7, patients are resampled after averaging out-of-fold predictions across seeds. That interval describes uncertainty in a particular difference between fitted predictors. It does not estimate the distribution of changes caused by independently repeating the training pipeline, despite the repeated cross-validation design. Because this floor is used to interpret subsequent architectural comparisons, the distinction matters to the paper's audit. Please separate variation across repeated complete training runs from patient sampling uncertainty, or restrict the claim to the observed implementation comparison.
Questions for authors
  1. Can the low variation be reproduced with a non-saturating edge generator on a task with an established molecular signal? If you don't have such a control, will you withdraw the compression-specific explanation?
  2. Do repeated complete training runs support the proposed AUC resolution floor when training variation is separated from patient-bootstrap uncertainty?
1 comment
AuthorsOpenReview ↗

Reply to Reviewer wieH

Thank you. Both points hold. We ran the control you proposed and repeated complete trainings, and both changed what we claim. Corrected results are in the general response. All numbers below use the corrected protocol, are patient-level, and are paired where they compare two models.

W1/Q1. Collapse on a task with molecular signal.

We withdraw the compression-specific explanation of §4.5.

We re-trained with histology as the primary label: adenocarcinoma vs squamous, 125 patients (94 / 31), NOS excluded. On the same folds, logistic regression on the 89 genes reaches 0.905 [0.822, 0.967], and 28 of the 89 genes pass FDR (0 for T-stage). Swaps keep all trained weights fixed.

Audited encoder.

  • AUC 0.628; without the gene branch, 0.521 (−0.107 [−0.226, +0.006]).
  • Only the shared part of the graph is used. Replacing each patient's graph by the STRING prior: −0.106 [−0.182, −0.040]; by the training-mean graph: +0.008 [−0.007, +0.023]; by another patient's: +0.006 to +0.012. A trained static graph matches the dynamic one: +0.007 [−0.023, +0.040].
  • Effective rank of the node scores 1.61 (T-stage 1.95; untrained 3.95); 87% of logits saturated.

A limitation this exposed. The encoder passes every gene through one shared Linear(1, h), so gene identity enters only through the graph. Logistic regression without gene identity (values sorted within each patient) also reaches only 0.649. We therefore trained a variant with one Linear(1, h) per gene, otherwise unchanged:

  • Histology 0.822 [0.716, 0.914]: +0.194 [+0.082, +0.307] over the audited encoder, +0.301 [+0.164, +0.440] over no gene branch. Swapping the gene branch's output for its training mean: −0.356.
  • Patient-specific graphs still add nothing. Static − dynamic: +0.015 [−0.017, +0.048]. Training-mean graph swap: +0.011 [+0.002, +0.021], so AUC rises. Effective rank 1.62 (untrained 4.23); 78% saturated.
  • T-stage 0.688. Genes add +0.017 [−0.039, +0.079] to CT + radiomics; static − dynamic −0.012 [−0.048, +0.027]. Here the training-mean swap changes AUC by −0.024 [−0.044, −0.008] (one patient changes class): the trained network uses the patient graph slightly, but one trained without it does as well.

Non-saturating generator (closest to your test). You asked for a bounded edge generator. We bounded the node scores that set every edge: their logits are standardised across genes within each patient, so the sigmoid cannot saturate. A non-saturating per-edge generator remains untested; our per-edge variant saturates (93% of logits). Histology AUC:

EncoderSaturatingNon-saturating
Shared input weight (audited)0.6280.511
One weight per gene0.8220.847
  • Per-gene weights, where the model demonstrably uses the genes (+0.326 [+0.197, +0.455] over no gene branch). No logit saturates, yet the effective rank of the node scores is 2.17, against 4.26 untrained and 1.62 with saturation. The patient-specific part is still unused: the training-mean graph raises AUC by +0.012 [+0.001, +0.025], and static − dynamic is −0.010 [−0.029, +0.009] (vs. the per-gene static model).
  • Shared weight: removing saturation lowers AUC (−0.116 [−0.189, −0.048]), and the gene branch is no longer used (output swap +0.015 [−0.058, +0.083]).

So, to Q1, for this node-score generator: yes. Without saturation, on a task where the model uses the genes, the variation stays well below its initial level and the patient-specific graph is still not used. Saturation deepens the collapse (1.62 against 2.17) but is not required for it.

W2/Q2. The 0.0125 floor.

You are right. 0.0125 is the upper end of a patient-bootstrap interval for two numerically equivalent trainings (+0.0004 [−0.0124, +0.0125]). It is not training variation.

We trained the corrected backbone twice more on identical splits, changing only the random seed (initialisation, data order, dropout):

  • Patient AUC (five-seed average): 0.676, 0.714 and 0.688; SD 0.019, range 0.038.
  • Pairwise differences: +0.038 [−0.014, +0.098], +0.012 [−0.038, +0.064] and −0.025 [−0.084, +0.031].
  • Per patient: the predicted probability varies with a mean SD of 0.035 (maximum 0.167). Between two trainings, 4 or 5 of 128 patients change class; across all three, 7 do.
  • Single runs: one complete 5-fold run without seed averaging changes AUC by 0.028 on average between trainings (maximum 0.074).
  • Other arms: three trainings each of five further arms give SDs of 0.016 to 0.047, the largest for radiomics only (0.576, 0.662, 0.652).

Training variation alone therefore exceeds the claimed floor, and it covers every audit difference we report:

  • static − dynamic: +0.014;
  • mixture-of-gates − backbone: −0.021;
  • gene branch: +0.005.

We have removed the "resolution floor"; the revision reports training variation and patient-sampling intervals separately (revised §4.7).

Reviewer_PieROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents G3I-Net, a trimodal model for binary NSCLC T-staging that combines CT, radiomics, and gene expression. A CT-conditioned hypernetwork reweights a protein–protein interaction graph used by the molecular branch. Beyond predictive performance, the paper audits whether the generated graphs meaningfully vary across patients. On 128 patients, including 21 advanced-stage cases, the reported model achieves a patient-level AUC of 0.9119. However, the generated graphs exhibit low effective rank, conditioning provides no statistically resolved improvement over a static graph, and radiomics accounts for most of the predictive signal. The central contribution is therefore an empirical audit of conditional graph generation rather than demonstrated superiority of patient-specific molecular modeling.

Critical strengths

The paper distinguishes predictive performance from mechanism utilization. Directly examining graph variation and comparing conditional and static graphs addresses an important limitation of evaluating architectural claims solely through downstream accuracy. The finding that a competitive model can make little use of its intended conditioning mechanism is informative.

The experimental analysis is unusually transparent about negative results. Modality ablations, alternative graph parameterizations, gating comparisons, and implementation sensitivity help contextualize the headline AUC. The authors also explicitly acknowledge configuration-selection optimism and the absence of external validation.

Critical weaknesses

The evaluation protocol does not yet establish a reliable, directly comparable performance estimate. Section 3.3.2 describes LASSO feature ranking and selection of the number of radiomic features, but does not clearly establish whether these operations, together with standardization and early stopping, are performed strictly within each outer training fold. This is a potential source of leakage that requires clarification, rather than evidence that leakage definitely occurred. Moreover, Table 2 compares patient-level AUC from seed-averaged out-of-fold predictions for G3I-Net against mean fold-level AUC for the classical baselines, which also exclude the CT input. These differences prevent a clean attribution of the reported performance gap. Selection among 52 configurations on the same cohort further limits the interpretation of the best reported result.

The descriptive evidence for low graph variation is stronger than the explanation of why it occurs. The paper attributes the limitation to compression of the CT embedding, but the tested alternatives do not isolate this cause from optimization, saturation, or weak task-relevant molecular signal. The effective-rank calculation is also insufficiently specified: the analyzed representation, centering, normalization, and rank definition should be stated. A 16-dimensional latent code does not by itself establish 16 independently usable graph directions after nonlinear transformation and normalization. Finally, the measurements characterize models trained on this small cohort; their being computed directly from weights does not make the learned behavior independent of cohort size.

Questions for authors
  1. Were LASSO feature ranking, feature-count selection, and standardization performed independently within each outer training fold? Please clarify which data were used for early stopping and selection among the 52 configurations, and whether the reported performance was evaluated independently of these choices.

  2. With all other trained weights held fixed, how do predictions change when each patient’s generated graph is replaced by a shared graph derived only from training patients, or when patient–graph assignments are shuffled? These controls would help distinguish whether the model uses patient-specific graph information from whether it primarily learns a shared reweighting of the prior.

1 comment
AuthorsOpenReview ↗

Reply to Reviewer PieR

Thank you. Your Q1 identified the most serious problem in the paper. The corrected results are in the general response; here are the answers point by point.

Q1. Nesting, early stopping and configuration selection.

  • LASSO ranking and feature-count selection were not nested: they used all 128 patients and their labels.
  • Standardisation was fit on all patients but is label-free. Refitting it per fold changes AUC by 0.000.
  • Early stopping and checkpoint selection used the outer held-out fold; there was no inner split.
  • Selection among configurations. The mixture-of-gates arm was promoted after we had seen the results, against our own decision rule, and "52" undercounts the search: it leaves out 45 two-stage cells and at least 19 earlier configurations on the same patients. Choosing the configuration inside the cross-validation gives 0.883–0.902 under the old protocol, rather than 0.912.
  • Was the reported performance independent of these choices? No. Under the corrected protocol the full model scores 0.676 [0.547, 0.791]; two further complete trainings on identical splits give 0.714 and 0.688. The mixture-of-gates model scores 0.021 below the backbone (CI [−0.069, +0.022]). Some optimism remains from hyperparameters, such as the width of 64, that were fixed during earlier development on this cohort.

Q2. Graph swaps with all other weights fixed. These are measured on the corrected models: 25 models and 640 held-out evaluations. ΔAUC is patient-level, paired against each patient's own graph.

Replacement graphΔAUC [95% CI]Mean |Δp|Patients changing class
Mean graph of the fold's training patients+0.016 [−0.003, +0.043]0.0110 / 128
Prior graph (no conditioning)+0.012 [−0.020, +0.050]0.0160 / 128
Graphs shuffled among held-out patients (20×)+0.010 (range −0.005 to +0.023)0.011≤ 1
A random training patient's graph (20×)+0.010 (range −0.004 to +0.020)0.012≤ 1
Whole gene-branch output replaced by the training mean+0.004 [−0.023, +0.037]0.0271 / 128
  • Mean log-loss decreases slightly under every swap.
  • For the mixture-of-gates model, replacing the patient graphs by the prior improves AUC by +0.023 [+0.004, +0.047]. This is one result among 12 comparisons, uncorrected for multiplicity.
  • In single runs (before averaging over seeds), 19 of the 640 predictions move by more than 0.1 under the prior graph. So individual predictions do react to the graph, but discrimination does not.
  • Conclusion: the model does not use the patient-specific graph for discrimination.
  • With one input weight per gene (see our reply to wieH), the T-stage model does use it slightly: the training-mean swap changes AUC by −0.024 [−0.044, −0.008], and one patient changes class. A model trained with a static graph is no worse (static − dynamic −0.012 [−0.048, +0.027]).
  • The same holds with histology as the label, where the genes are predictive. There, replacing each patient's graph by the training mean changes AUC by +0.008 [−0.007, +0.023], while the prior costs 0.106 (see our reply to wieH).

W1. Table 2 scales, and CT for the baselines. All rows are now on the patient-level scale; see our reply to eCJL Q1 for baselines on the same frozen CT features. Under corrected evaluation, the model does not differ from any baseline. One example: the maximum 3D diameter of the radiomics mask, with no fitting, scores 0.642, and the model's margin over it is +0.034 [−0.136, +0.203].

W2. The explanation for low variation.

  • Compression explanation. We withdraw it. On the corrected models 74% of node-score logits are saturated (none at initialisation), and the rank of the modulation code falls when only the downstream mask changes. Optimisation and saturation therefore cannot be separated from compression. We then tested saturation and molecular signal directly: with histology as the label and a non-saturating generator, the patient-specific graph is still unused (see our reply to wieH).
  • Mask and input capacity (§4.5, Table 3; revised Table 3), re-run under the corrected protocol. Widening the hypernetwork input to include radiomics changes neither AUC (+0.003 [−0.017, +0.024]) nor the consumed graph's effective rank (2.20, against 2.12). Scoring the 440 edges independently lowers the rank to 1.25 (AUC −0.010 [−0.042, +0.023]). Both observations stand; the compression conclusion drawn from them does not.
  • Effective rank. It is now fully defined in our reply to eCJL Q2.
  • "Sixteen directions". We agree that a 16-dimensional code does not give 16 usable directions. The right reference is an untrained network on the same inputs, whose code has effective rank 3.85; the trained models use 1.58.
  • "Do not depend on the size of the cohort". We delete this claim.

Comments on the paper

3 comments
AuthorsOpenReview ↗

To all reviewers (part 1 of 2): we found label leakage; corrected results

We thank the three reviewers. Your questions about the evaluation protocol (PieR Q1, eCJL W2/Q1) led us to re-audit the whole pipeline. You were right, and the problem is larger than a caveat. We report it here first, with corrected numbers. Part 2 covers what survives and the new controls, and we then answer each review separately. Section, table and figure numbers refer to the submitted paper unless marked "revised".

1. Two label leaks, confirmed.

  • Radiomics selection was not nested. The LASSO ranking and the choice of k were computed once, on all 128 patients, using the T-stage labels. The z-score was also fit on all patients; it uses no labels, and refitting it inside each fold changes AUC by 0.000. The same script has two further errors:
    • LassoCV chose α = 0.1, which keeps 2 features, so the reported α = 0.005 is a hard-coded fallback, not a tuned value.
    • An indexing bug made the k-search score the wrong columns, so the k-curve in §3.3.2 (0.713 → 0.505) does not describe the selected features.
  • Early stopping used the outer held-out fold. Checkpoints were selected on the same fold whose predictions we report. There was no inner split.

2. What we did. We re-ran every arm with two changes:

  • the LASSO ranking and the z-score are refit inside each outer training fold, with k = 10 fixed as an architectural constant;
  • training follows a fixed 100-epoch schedule, and we report the final weights, so no held-out label is used for any choice within a run.

Each run also logs held-out predictions at every epoch, so we can replay the original protocol on the same trained runs. With the original radiomics features, the replay reproduces the submitted predictions exactly in all 25 folds (patient AUC 0.8972 and 0.9119). The differences below therefore come from the protocol alone.

One residual optimism remains: hyperparameters such as the width of 64 were fixed during earlier development on this cohort.

3. Corrected results. Patient-level AUC from seed-averaged out-of-fold predictions, with 95% CIs from a patient bootstrap. n = 128, of whom 21 are advanced-stage.

ArmSubmittedCorrectedInflation (paired)
Full model (dynamic graph)0.8970.676 [0.547, 0.791]+0.221 [+0.127, +0.327]
Mixture-of-gates0.9120.655 [0.513, 0.779]+0.257 [+0.152, +0.377]
Static graph0.8890.690 [0.563, 0.802]+0.198 [+0.113, +0.296]
CT + radiomics0.8950.671 [0.539, 0.789]+0.224 [+0.133, +0.326]
Radiomics + gene0.8720.619 [0.496, 0.732]+0.253 [+0.150, +0.370]
Radiomics only0.8690.576 [0.442, 0.700]+0.292 [+0.167, +0.434]
CT only0.6770.509 [0.368, 0.653]+0.168 [+0.050, +0.291]
Gene only0.6670.500 [0.371, 0.628]+0.167 [+0.068, +0.271]
CT + gene0.6610.541 [0.398, 0.677]+0.120 [+0.028, +0.216]
  • How the full model's inflation splits. On its own, the radiomics leak adds +0.154 [+0.053, +0.266] and checkpoint selection +0.043 [−0.027, +0.117]; each adds more when the other is present (+0.178 and +0.067), and together they add +0.221. In the CT-only and gene-only arms, which use no radiomics, checkpoint selection alone lifted chance-level models to about 0.67.

  • Simple baselines, evaluated the same way (patient-level, no leakage):

    • LASSO refit per fold, then logistic regression: 0.626;
    • logistic regression on 14 shape features chosen a priori: 0.615;
    • maximum 3D diameter of the radiomics mask, no fitting: 0.642.

    The full model does not differ from any of them; against diameter, the difference is +0.034 [−0.136, +0.203].

  • Training variation. The table shows one training per arm. Three complete trainings of the full model on identical splits give 0.676, 0.714 and 0.688; radiomics only varies most (0.576, 0.662 and 0.652). See part 2, section 7.

Input pipeline (also found in this audit; it affects every result, including the corrected ones). The radiomics mask is the full-field segmentation rescaled to the crop, not cropped with it. For 55 of 117 segmentations with fewer frames than CT slices, the crop is centred near mid-scan (tumour outside it in 28). The 11 patients without a segmentation were cropped from the recorded lobe with the vertical axis inverted. Re-running on corrected inputs is not possible before the deadline; the revision states all of this in §3 and Limitations.

4. What this withdraws. Two contributions no longer hold, and every absolute AUC in the paper is optimistic.

  • Contribution 1: the AUC of 0.9119 and the margin over the baselines.
  • Contribution 3: that radiomics carries the signal, and that our gene-only model is above chance. It scores 0.500; logistic regression on the same 89 genes reaches 0.633 [0.505, 0.754], so any signal in gene expression alone is weak.
AuthorsOpenReview ↗

To all reviewers (part 2 of 2): what still holds, other corrections, new experiments

5. What survives: the audit, which PieR identified as the central contribution. Results on the corrected models, as paired patient-level CIs. These intervals cover patient sampling only; training variation is in section 7.

  • Static minus dynamic graph: +0.014 [−0.012, +0.046]. No benefit from patient conditioning is detected; the CI excludes a dynamic advantage above 0.012.
  • Mixture-of-gates minus backbone: −0.021 [−0.069, +0.022].
  • Adding the gene branch to CT + radiomics: +0.005 [−0.038, +0.045].
  • Fixed-weight graph swaps (PieR Q2, eCJL Q3). Replacing each patient's graph by the training-mean graph, by the prior, or by another patient's graph changes AUC by +0.010 to +0.016. Every CI covers 0, and at most one patient's predicted class changes.
  • Graph variation. The effective rank of the node scores is 1.95, against 3.94 for an untrained network on the same inputs. 74% of node-score logits are saturated (|z| > 4). The graphs do differ between patients in magnitude, but along about two directions (graph effective rank 2.12).

6. Other corrections (details in the individual replies):

  • Figure 4. The matched-specificity rule re-thresholded only the candidate model, which added exactly +1/21 to every arm. Mixture-of-gates is +0.048 [0, 0.158], not +0.095. Figure 4 also showed only 8 of the 17 width-64 gated arms; the revision removes it, since those arms were not re-run under the corrected protocol.
  • Contribution 2. The CI had its sign reversed: dynamic minus static is +0.0085 [−0.007, +0.028].
  • "73%". It was computed on one patient per fold. Over all 640 held-out evaluations, it is 0.83 on the corrected models. Distance from the prior is not evidence of patient-specificity, since a static model's graph sits at the same distance.
  • "6.4 × 10⁻⁶". This is the 10⁻⁸ numerical stabiliser, not a measurement.
  • §4.5. We withdraw the CT-compression explanation.
  • Pre-registration. The primary endpoint was fixed on 26 Aug 2026, after several gating arms had already been run, not "before the experiments were run".
  • "52 configurations". This undercounts the search: it leaves out 45 two-stage cells and at least 19 earlier configurations on the same patients.

7. New controls (details in our reply to wieH):

  • Histology as the primary label, where logistic regression on the 89 genes reaches 0.905. The patient-specific graph is still unused:

    • a trained static graph matches it: +0.007 [−0.023, +0.040];
    • with weights fixed, the training-mean graph replaces it at no cost: +0.008 [−0.007, +0.023];
    • the prior, by contrast, costs 0.106.

    The collapse is stronger than on T-stage: effective rank 1.61, and 87% of logits saturated.

  • A limitation of the encoder we audit. It shares one input weight across all genes, so gene identity enters only through the graph; on histology it reaches 0.628, against 0.905 for logistic regression. With one input weight per gene:

    • histology rises to 0.822 [0.716, 0.914], and the gene branch adds +0.301 [+0.164, +0.440];
    • a trained static graph still matches the dynamic one (histology +0.015 [−0.017, +0.048]; T-stage −0.012 [−0.048, +0.027]);
    • T-stage genes still add nothing to CT + radiomics: +0.017 [−0.039, +0.079]. Added to radiomics alone they give +0.102 [+0.004, +0.204] over three trainings per arm, but +0.083 [−0.026, +0.193] in the two trainings run after we singled this comparison out. CT gives +0.071 [−0.042, +0.186] there;
    • with weights fixed, replacing the T-stage model's patient graphs by the training mean changes AUC by −0.024 [−0.044, −0.008]: a slight use, which a model trained with a static graph does not need.
  • Non-saturating generator (closest to wieH's test; it bounds node scores, not edges). With node-score logits standardised per patient, no logit saturates. With per-gene weights (histology 0.847), the effective rank still halves (2.17, against 4.26 untrained) and the patient-specific graph is still unused (training-mean swap +0.012 [+0.001, +0.025]; static − dynamic −0.010 [−0.029, +0.009], against the per-gene static model with the standard generator). Saturation deepens the collapse but is not required for it. A non-saturating per-edge generator was not run. With the audited shared weight, the same change lowers histology AUC to 0.511.

  • Repeated complete trainings.

    • Training variation (SD 0.019 in patient AUC; 0.016 to 0.047 in five further arms) exceeds the claimed 0.0125 floor, which we withdraw.
    • Its spread across three trainings (0.038) exceeds every audit difference in section 5 above.

We understand that these corrections change the paper substantially. We have uploaded a revised PDF that applies all of them, with the same title and structure; a separate comment lists the changes by section.

AuthorsOpenReview ↗

To all reviewers: summary of changes in the revised PDF

We have uploaded a revised PDF; numbers in this comment refer to it. The title and section structure are unchanged, and the main text is within 10 pages. Every model result now comes from the nested protocol described in our general response, except the non-nested comparison in §4.1. Changes by section, with the review points they address:

Abstract and §1. Corrected numbers throughout. Contribution 1 now makes no accuracy claim; the model is in line with simple baselines. Contributions 2 and 3 state the audit results in five settings, including the one setting where a trained model leans slightly on its patient graphs. Contribution 4 reports the effect of the two evaluation shortcuts and the measured training variation.

§2. Added sample-specific gene networks (SSN, LIONESS), the attention-weight audits that used the same fixed-weight and fixed-structure tests (Jain & Wallace; Wiegreffe & Pinter), and evaluation leakage (Ambroise & McLachlan; Varma & Simon; Cawley & Talbot; Kapoor & Narayanan).

§3 Method.

  • §3.3.2: radiomic selection is described as run, inside each training fold. The text reports how often descriptors are selected across the 25 folds, and the k-search curve is removed (PieR Q1, eCJL W2/Q1). §3 and Limitations now describe the input pipeline as run, including where the crop and radiomics mask miss the tumour.

  • §3.4: the bias of the node-score logits is added to the equations. The shared-weight gene encoder, the per-gene variant and the non-saturating generator are defined (wieH Q1).

  • §3.4.1: full definitions as requested by eCJL (W1/Q2) and PieR (W2):

    • the representation measured, centring and the entropy effective rank of variance shares;
    • held-out patients per model, and the untrained reference;
    • 83 local degrees of freedom against the 16-dimensional code;
    • distance from the prior, with the caveat that a static graph sits just as far;
    • saturation.

    The "6.4e-6" value is removed.

  • §3.2 and §3.6: the static control and the gated fusion are defined precisely. §3.5: mixture-of-gates is an ablation, not the reported model.

  • §3.7: a fixed 100-epoch schedule with final weights, a 5,000-draw patient bootstrap, intervals that cover patient sampling only, and the residual optimism from at least 116 configurations explored on this cohort (PieR W1, eCJL W2).

§4 Results.

  • §4.1 (new): nested against non-nested evaluation for all nine arms, with the paired inflation and the split between the two shortcuts (PieR Q1, eCJL Q1).
  • §4.2: all baselines are patient-level and nested, including logistic regression on frozen CT features (PieR W1, eCJL Q1).
  • §4.3: Figure 3 is regenerated from the nested runs. "Radiomics carries the signal" and "gene-only is above chance" are removed. The gene screens now use a nested imaging covariate.
  • §4.4: mixture-of-gates is −0.021 [−0.069, +0.022] against the backbone. The old Figure 4 and the matched-specificity comparison are removed (eCJL Q4).
  • §4.5: the CT-compression explanation is removed (PieR W2, wieH W1). Table 3 adds fixed-weight replacements: training-mean graph, prior, permuted and random-patient graphs, and the whole gene-branch output, with log-loss and reclassification (PieR Q2, eCJL Q3).
  • §4.6 (new): histology as a positive control, the per-gene encoder and a non-saturating node-score generator, summarised over five settings in Table 4 (wieH W1/Q1, eCJL Q3).
  • §4.7: the "measurement floor" is replaced by repeated complete trainings of six arms (Table 5; wieH W2/Q2, eCJL Q4).

§5 and §6. Rewritten limitations: the input pipeline (crop and radiomics mask), one cohort, hyperparameters chosen on it, and "no difference detected" rather than equality; the per-gene T-stage nuance is stated in §4.6. "Independent of cohort size" is removed (PieR W2). The conclusion is limited to the five settings tested.

References and statements. The bibliography is corrected (two duplicated keys had resolved to stale entries) and cites seven verified additional references. The reproducibility statement now describes the nested protocol.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.