acceptodds
Withdrawn from ICLR 2027

MM-HAGT: Hierarchical Graph Transformers Conditioned on Incomplete Clinical Data for Whole-Slide Images

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pathological staging from gigapixel WSIs requires modeling spatially organized tissue patterns together with complementary patient context. Existing WSI models typically ignore structured clinical evidence, whereas conventional multimodal fusion is vulnerable to incomplete clinical records. We introduce MM-HAGT, a hierarchical graph Transformer that accounts for missing information and incorporates clinical context. MM-HAGT represents each WSI as a graph combining topological and semantic relationships, adaptively reweights uninformative patches, and hierarchically aggregates patch features into representations at the region and slide levels. A missing aware clinical encoder transforms variables with a fixed schema and their observation mask into a token controlled by a reliability gate, which interpolates between observed clinical evidence and a learned prior for missing information. Self attention subsequently enables the clinical token to condition localized WSI graph representations. Clinical modality dropout and auxiliary supervision further promote robustness to incomplete records and discourage excessive reliance on structured variables. This framework provides a unified approach to hierarchical WSI representation learning with partially observed clinical context for pathological staging at the patient level.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
hierarchical graph Transformer, Incomplete Clinical Data, Whole-Slide Images

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Bx3NOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents MM-HAGT, a hierarchical graph Transformer for pathological staging that jointly models gigapixel whole-slide images (WSIs) and partially observed clinical data. The method comprises: (1) an adaptive WSI graph encoder with a learnable semantic gate that suppresses background patches and edge weighting combining a Gaussian spatial kernel with cosine feature similarity; (2) multi-query hierarchical attention that aggregates patch features into region- and slide-level representations, combined via gated cross-level fusion; and (3) a missing-aware clinical encoder (observation masks, reliability gating, a learnable missing-information prior, modality dropout, and an auxiliary classification head) whose token conditions WSI representations through self-attention. The model is evaluated with five-fold cross-validation on TCGA-COAD (461 WSIs) and TCGA-KICH (113 WSIs), reporting improvements in Accuracy, F1, and AUC over six baselines (SSM, DTFD-MIL, MambaMIL, HDMIL, HEAT, CLAM), along with ablations of three modules and attention heatmap visualizations.

Critical strengths
  1. Genuine and clinically relevant motivation. Fusing WSIs with structured clinical records while explicitly handling missing fields addresses a real pain point in pathology workflows. The missing-aware conditioning design is more principled than simple imputation with late fusion, and the overall framework is internally coherent.

  2. Reasonably sound evaluation protocol. Patient-level splitting prevents leakage, five-fold cross-validation is used, and checkpoint selection relies on validation data. The authors also show welcome restraint in interpreting attention heatmaps and ROC curves (explicitly stating no pathologist validation and no calibration claims).

Critical weaknesses
  1. The core contribution lacks an evidence chain. The title, abstract, and contributions all emphasize conditioning on incomplete clinical data, yet the ablation study covers only pathology-side modules — the clinical encoder, modality dropout, and reliability gate are never ablated. Moreover, no multimodal baseline and no WSI-only variant of the model are compared, so the performance gains cannot be attributed to the claimed mechanism. In effect, the paper's central claim remains unverified.
  2. Questionable reliability of the results. The anomalously large AUC gain (+11.5 points) together with an anomalously small variance (±0.5) strongly suggests inconsistent metric computation across methods. Combined with an undefined ablated module (GAPE) and duplicated references listing the same paper twice (Chan et al. 2023a/b and Dong et al. 2025a/b are the same works), the overall rigor is concerning.
Questions for authors
  1. How exactly is AUC computed (macro-averaged one-vs-rest?). Is the identical evaluation code used for all methods and all metrics? Can you provide paired significance tests?
  2. Please define GAPE and indicate where it sits in the architecture — it appears to be absent from the main text.
  3. What are the actual missing rates of each clinical field in the two cohorts? How does performance vary under artificially imposed missing rates (e.g., 0–80%), compared against simple imputation or mean-fill clinical encoders?
  4. What is the performance of the model without the clinical branch (WSI-only)?
Reviewer_mYX2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MM-HAGT, a hierarchical graph Transformer for pathological staging using WSIs and incomplete clinical data. The method integrates adaptive graph learning, hierarchical feature aggregation, and clinical information fusion, and is evaluated on two TCGA datasets.

Critical strengths

The paper combines hierarchical graph modeling with incomplete clinical information, addressing a practical challenge in computational pathology. The proposed method achieves promising performance on two public datasets.

Critical weaknesses
  1. Limited novelty. The method mainly combines existing techniques, including graph convolution, hierarchical attention, and multimodal fusion, with limited methodological innovation. 2.Insufficient experimental validation. The paper lacks experiments under different clinical missingness rates and comparisons with other multimodal fusion methods, weakening the evidence for the proposed mechanism.
Questions for authors

1.Given that only a single Transformer layer is used, could the authors further explain how clinical information effectively conditions localized pathological representations? 2.Given the limited novelty and insufficient validation, what distinguishes MM-HAGT from existing methods?

Reviewer_XFLpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MM-HAGT, a multimodal framework for patient-level pathological staging that conditions a hierarchical graph Transformer on incomplete clinical data from whole-slide images (WSIs). The authors first apply a learned retention gate to softly select informative patches. They then connect the patches in a radius graph whose edge weights combine spatial distance and feature similarity, and encode it with a weighted GCN. The node features are aggregated from patch to region to slide level through hierarchical multi-query attention pooling, and a gated cross-level fusion module combines the fine- and coarse-grained representations. In parallel, ten clinical variables and their observation mask are encoded into a single clinical token. A reliability gate, scaled by the fraction of observed fields, interpolates this token with a learned missing embedding. Finally, the clinical token is fed together with the region and slide tokens into a Transformer to predict the stage, with clinical modality dropout and an auxiliary clinical head used during training. The method is evaluated against six WSI-only MIL baselines on TCGA-COAD and TCGA-KICH with 5-fold cross-validation, and three modules are examined through ablation studies.

Critical strengths

The framework addresses a relevant practical problem by allowing incomplete clinical records to interact with localized, hierarchical WSI representations.

Critical weaknesses
  1. The methodological novelty and necessity of the proposed components are not sufficiently established. Adaptive graph weighting, learned-query pooling, and hierarchical feature fusion are closely related to existing techniques [1,2]. The current ablations do not clearly isolate the contributions of graph adaptation, hierarchical aggregation, multiple queries, and gated fusion. Comparisons with simpler alternatives and closely related graph-based WSI methods [2,3] are needed.

  2. The experimental comparisons do not adequately control for the additional clinical information available to MM-HAGT. WSI-only and clinical-only controls, together with WSI baselines equipped with the same clinical inputs, are needed for a fair assessment. The robustness of the clinical encoder should also be evaluated under different missingness rates and patterns and compared with standard missing-data handling strategies.

  3. The empirical evaluation and reporting are incomplete. Several central components lack dedicated ablations, while important dataset statistics, patient-level evaluation details, and metric definitions are insufficiently specified. More transparent reporting of fold-level results and statistical calculations is also needed to establish the reliability and reproducibility of the reported improvements.

References

[1] Lee, J., Lee, Y., Kim, J., et al. “Set Transformer: A Framework for Attention-Based Permutation-Invariant Neural Networks.” Proceedings of the International Conference on Machine Learning, 2019, pp. 3744--3753.

[2] Guo, Z., Zhao, W., Wang, S., et al. “HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis.” Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2023, pp. 755--764.

[3] Chen, R. J., Lu, M. Y., Shaban, M., et al. “Whole Slide Images Are 2D Point Clouds: Context-Aware Survival Prediction Using Patch-Based Graph Convolutional Networks.” Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2021, pp. 339--349.

Questions for authors
  1. The “adaptive graph construction” claim is overstated. The radius-based topology remains fixed. Adaptation is limited to soft node gating and reweighting existing edges. The paper does not establish an advantage over standard graph construction.
  2. The need for hierarchical multi-query aggregation remains unclear. Learned-query pooling is already established in Set Transformer [1]. The HAP ablation does not isolate the benefits of hierarchy or multiple queries. Comparisons with single-query hierarchical pooling and flat multi-query pooling are needed under comparable parameter budgets.
  3. Cross-level fusion is not new to WSI analysis. HIGT [2] already combines coarse- and fine-grained features. The contribution of the proposed gate remains unclear. Removing fusion altogether does not establish its value over simple summation or concatenation.
  4. The clinical encoder uses record completeness as a proxy for reliability. Eq. 7 caps every gate channel by the fraction of observed fields, regardless of their informativeness. This may suppress sparse but useful records. Experiments across missingness rates and patterns are needed, with imputation plus missingness indicators as a baseline.
  5. The comparisons do not control for access to clinical information. WSI baselines should receive the same clinical inputs through standard fusion schemes. Clinical-only and WSI-only controls are also missing. Graph comparisons should include HIGT and Patch-GCN [3], with the latter adapted from survival prediction to staging.
  6. The COAD AUC standard deviation of 0.5 percentage points is unusually small relative to the baselines’ 5.4–7.1 points. The model’s accuracy and F1 also vary substantially more across folds. Please provide per-fold scores and clarify the AUC averaging and standard-deviation calculations.
  7. GAPE appears in the ablation table but is never defined in the methodology. Several central components also lack ablations: semantic selection, topology–semantic edge weighting, query diversity, clinical reliability gating, modality dropout, and auxiliary clinical supervision.
  8. The dataset description needs more detail on patient and slide counts, stage definitions and distributions, inclusion criteria, and per-variable clinical missingness. Fold-level class counts are especially important for KICH. The authors should also explain how multiple slides are combined for patient-level evaluation.

[1] Lee J, Lee Y, Kim J, et al. Set transformer: A framework for attention-based permutation-invariant neural networks[C]//International conference on machine learning. PMLR, 2019: 3744-3753.

[2] Guo Z, Zhao W, Wang S, et al. Higt: Hierarchical interaction graph-transformer for whole slide image analysis[C]//International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland, 2023: 755-764.

[3] Chen R J, Lu M Y, Shaban M, et al. Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolutional networks[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2021: 339-349.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.