- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AutoMarker addresses outcome-guided discovery of pathology biomarker candidates rather than outcome prediction alone. It learns protein-aligned morphology representations from paired H&E–mIF images, constructs cell, microenvironment, tissue, and tissue-conditioned cell views, and uses adaptive representation-space neighborhoods to identify outcome-enriched responses. Direction-consistent graph refinement consolidates these responses into candidate components with WSI abundance and tissue back-tracing. Orion-CRC candidates are frozen and evaluated in TCGA-COAD; two private cohorts provide additional within-cohort analyses. The paper also reports search/refinement ablations, patient-level sensitivity analyses, and computational scalability.
Critical strengths
-
A useful formulation of outcome-guided biomarker discovery. The framework jointly specifies candidate membership, scale, enrichment direction, and spatial support. Connecting clinical outcome differences to inspectable cellular and tissue patterns gives the discovery task a concrete and interpretable output.
-
An adaptive approach to candidate localization. Selecting neighborhood scales from outcome-conditioned divergence trajectories provides a motivated alternative to a single global scale. The stopping and refinement ablations support practical improvements in search efficiency and candidate redundancy.
-
A traceable discovery and validation workflow. The method consolidates local responses into candidates and maps them back to supporting tissue. Freezing candidates before external evaluation strengthens the evidence for transfer, while H&E-only target inference increases the workflow's practical applicability.
Critical weaknesses
-
The advantage over established differential abundance methods remains unclear. DA-seq is discussed but not experimentally compared, and the separate Milo analysis does not provide a matched discovery comparison. Since outcomes guide candidate selection, stronger discovery-set significance alone cannot establish better discovery quality. A comparison using the same representations and patient splits, assessed through external replication, would better isolate the search contribution.
-
The value of mIF supervision is insufficiently isolated. H&E-only external associations do not establish accurate protein prediction or improvement over ordinary morphology features. A matched alignment ablation and held-out channel-prediction assessment are missing. A strong H&E encoder such as Virchow2 or UNI would also provide a useful tissue-level baseline for assessing the added complexity.
-
The complementarity of the representation views is not demonstrated. Cellular, microenvironmental, and tissue candidates may reflect overlapping composition differences. Finding associations in each view does not show that the hierarchy adds information. A focused comparison against cell-only discovery and simple regional pooling would clarify whether the additional views yield distinct, externally reproducible findings.
-
Abundance is defined differently during discovery and validation. Discovery counts retained query-center component members, whereas external evaluation counts entities covered by frozen neighborhoods. This acknowledged difference complicates interpretation of cross-cohort replication. Applying the external assignment rule back to the discovery cohort would establish whether the principal associations persist under a consistent measurement.
-
The main external results need stronger patient-level reporting. Primary tests treat WSIs as independent observations, while the patient-aggregation sensitivity analysis covers only a separate cell-level workload. Patient-level external inference and reporting the complete frozen candidate family, with effect directions and multiplicity adjustment, would clarify the reliability and scope of validation.
Questions for authors
- Under matched representations and patient splits, does AutoMarker improve external discovery replication over DA-seq or Milo?
- What discovery benefit comes specifically from mIF alignment and the additional representation views?
- Do the principal external associations persist with consistent abundance definitions, patient-level inference, and complete-family reporting?
5 comments
3. Do the Hierarchical Views Discover Complementary Phenotypes Beyond Cell-Only Representations?
We thank the reviewer for this insightful suggestion. The hierarchical views retrieved distinct phenotype candidates beyond Cell-only discovery, and a Tissue-level candidate demonstrated additional external clinical relevance. Matched comparisons with Cell-only discovery and simple regional mean pooling revealed differences in both candidate membership and patient-level abundance patterns.
(1) Candidate distinctiveness. Environment, Tissue, and Tissue–Cell showed median maximum Cell-overlap Jaccard values of 0.0019, 0.0984, and 0.2038, respectively, with corresponding maximum absolute abundance correlations of 0.2545, 0.4391, and 0.4626.
| View | Median max Cell Jaccard ↓ | Median max |r| ↓ |
|---|---|---|
| Mean pooling | 0.0012 | 0.2387 |
| Environment | 0.0019 | 0.2545 |
| Tissue | 0.0984 | 0.4391 |
| Tissue–Cell | 0.2038 | 0.4626 |
These results were obtained through matched patient-disjoint Orion-CRC discovery and frozen candidate assignment. We assessed spatial membership overlap using maximum Cell-candidate Jaccard and patient-level abundance redundancy using maximum absolute correlation, summarized by their medians.
The four views reflect distinct levels of tissue pathology: Cell captures individual phenotypes, Environment local cellular context, Tissue higher-order morphology, and Tissue–Cell cells conditioned on tissue context. Their purpose is to discover complementary pathological phenotypes across tissue scales, rather than merely improve outcome prediction through feature fusion.
(2) External clinical relevance. One frozen Tissue candidate demonstrated a direction-consistent association in TCGA-COAD (Cliff's δ = −0.264, 95% CI [−0.446, −0.087]; BH q = 0.0183).
Importantly, this association persisted after adjustment for an Orion-frozen Cell-only score (adverse OR = 5.09, 95% CI [1.58, 16.34]). In a further prescreening control adjusting for both the Cell-only score and regional mean-pooling information, the Tissue candidate remained associated with outcome (OR = 4.99, 95% CI [1.60, 15.57]).
Together, these findings provide evidence that hierarchical representations retrieve distinct phenotype populations and can reveal externally associated tissue-level information beyond the evaluated cellular and regional-pooling summaries, supporting AutoMarker's pathology-motivated multiscale discovery design.
4. Do the Principal Associations Persist Under Consistent Abundance Definitions?
Sorry for the confusion. 35/47 candidates (74.5%) preserved their outcome-association directions when we reapplied the frozen external membership rule to Orion-CRC.
The reviewer correctly observes that Eqs. 12 and 20 count different entities. This reflects an intentional distinction between phenotype-core discovery and transferable phenotype coverage.
Eq. 12 measures the retained query-center component identified through outcome-guided search and graph refinement, characterizing the discriminative phenotype core. Eq. 20 uses the union of frozen adaptive center–radius neighborhoods to assign broader populations of phenotypically related entities.
The distinction is pathology-motivated: identifying a representative outcome-associated phenotype and quantifying the prevalence of its broader population address different but related analytical objectives.
Following the reviewer's suggestion, we preserved the original query centers, radii, and component structures and reapplied the frozen Coverage definition to the discovery cohort without repeating outcome-guided search.
The observed 74.5% direction preservation indicates that most evaluated candidates retained their outcome-association directions under the alternative membership definition.
We will clarify the distinction between Center and Coverage abundances and report the corresponding candidate-level measurements in the supplementary material. Together, these analyses make the discovery-to-transfer measurement relationship explicit while preserving AutoMarker's intended two-stage design.
5. How Reliable Are the External Results Under Patient-Level Evaluation?
We thank the reviewer for emphasizing patient-level inference and complete-family reporting. Two AutoMarker candidates retained significant external associations after BH, Bonferroni, and permutation-based MaxT correction, with both remaining significant across all 162 leave-one-patient-out evaluations and after clinical covariate adjustment.
| Metric | Candidate 1 | Candidate 2 |
|---|---|---|
| Cliff's δ (Favorable − Adverse) | 0.389 | 0.458 |
| 95% CI | [0.189, 0.576] | [0.287, 0.616] |
| BH q | 0.00894 | 0.00136 |
| Bonferroni p | 0.01788 | 0.00136 |
| MaxT permutation p | 0.0180 | 0.0019 |
| LOPO Bonferroni significance | 162/162 | 162/162 |
| Clinically adjusted adverse OR | 0.444 | 0.293 |
| Clinically adjusted Bonferroni p | 0.0405 | 0.000914 |
(1) Patient-level inference and complete-family correction. Both candidates remained significant after BH and Bonferroni correction across the complete family of 47 frozen Cell candidates, including non-estimable entries. We evaluated 162 independent TCGA-COAD patients (36 favorable, 126 adverse) under the specified 24/60-month prognostic criteria. Abundances were first calculated per WSI and then averaged within patients, ensuring one observation per patient.
(2) Permutation and influence robustness. Both candidates retained MaxT-adjusted significance (p = 0.0180 and 0.0019) and remained Bonferroni-significant in 162/162 LOPO evaluations. We performed 19,999 patient-label permutations, preserving outcome-group sizes and jointly permuting the complete candidate family. LOPO analyses recalculated the full family after each patient deletion.
(3) Clinical covariate adjustment. Both associations remained significant after adjustment for pathological stage, age, and sex in 157 complete-case patients, with adjusted adverse-prognosis ORs of 0.444 and 0.293, respectively.
Together, these results demonstrate robust patient-level external associations under complete-family multiplicity correction, permutation calibration, patient-influence analysis, and clinical adjustment. Complete candidate-level statistics and protocols will be provided in the supplementary material.
1. How Does AutoMarker Compare with Established Differential Abundance Methods?
We extended our matched comparison to DA-seq (Zhao et al., 2021), Milo (Dann et al., 2022), and Kompot (Otto et al., 2025). AutoMarker achieved the highest membership-reconstruction Jaccard (0.8994) and the lowest minimum external BH q under common transfer (0.0171). Under native transfer, AutoMarker and Kompot shared the lowest external BH q (0.0076).
The comparison uses matched CellID-19 representations from 23,000 Orion-CRC cells (23 patients), followed by evaluation on 162 TCGA-COAD patients. Each method retains its native discovery procedure. We examine both native transfer and a common source-trained kNN assignment protocol, with patient-level external abundance testing.
AutoMarker's distinguishing capability is that its discovery output directly includes frozen adaptive center–radius geometry, enabling deterministic assignment of unseen representations without training an additional membership model.
Ground-truth discovery quality. AutoMarker achieved the highest ground-truth-best-match Jaccard (0.138) among the evaluated configurations, outperforming fixed-k (0.113) and fixed-radius (0.086). Moreover, removing sign-consistent graph pruning reduced recovery at Jaccard ≥ 0.10 from 63.3% to 3.9%, demonstrating its contribution to structured phenotype discovery. The semi-synthetic benchmark used Orion-CRC embeddings, three geometries, four signal conditions, ten seeds, and patient-disjoint splits.
Discovery-ranked recovery without oracle selection. AutoMarker achieved the highest median held-out Top-1 Jaccard (0.1122), compared with DA-seq (0.0040) and Milo (0.0015). The corresponding Top-5 union Jaccard values were 0.1356, 0.1080, and 0.0050, respectively. Unlike ground-truth-best matching, this evaluation selects candidates using discovery-based ranking without oracle information, assessing their recovery under the evaluated frozen-transfer protocols.
These experiments support AutoMarker's adaptive search, direction-aware refinement, and transferable membership. Full protocols will appear in the supplement.
2. Does mIF Supervision Improve Protein Prediction Beyond Morphology Features?
Yes. Three experiments demonstrate the benefits of mIF supervision for cell-level protein decoding, alignment objectives, and patch-level representations.
(1) Held-out cell-level protein decoding. CellID-19 achieved the highest Pearson (0.2435), Spearman (0.2723), and lowest MAE (0.2929) among the evaluated representations.
This evaluation used 40,192 matched H&E–mIF cells from 23 Orion-CRC patients, covering 16 protein channels, with identical patient-disjoint Ridge probes.
(2) Controlled alignment ablation. Combining cell-level and patch-level supervision achieved the highest Pearson (0.1809) and lowest MAE (0.2817), outperforming the evaluated morphology-only controls on both metrics.
Using an identical frozen CTransPath backbone, we evaluated seven patient-disjoint folds and three matched seeds (63 training runs). All mIF-supervised configurations improved protein-decoding point estimates over morphology-only controls.
(3) Foundation encoder comparison. CellID-19 achieved the highest Pearson (0.4826) and lowest MAE (0.2188) among the evaluated representations, compared with UNI (0.4354/0.2461) and Virchow2 (0.4770/0.2340).
This separate patient-disjoint evaluation used 466 protein-labeled Orion-CRC patches and a shared Ridge probing protocol. The measurements assess protein decodability from patch-aggregated targets.
These results support mIF supervision for learning compact, protein-aligned representations useful for outcome-guided phenotype discovery.