- Rating
- 2
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes FRAME, a pathology framework for recognizing structural pathological events beyond appearance-based modeling. FRAME uses paired H&E–mIF data during training to learn protein-grounded spatial representations, then performs constituent parsing and boundary-centric relational reasoning to model how tissue components are organized around anatomical targets. At inference, the method requires only H&E images. The framework is evaluated on molecular grounding, structural-event recognition, and slide-level cancer diagnosis/grading tasks, showing improvements over several pathology foundation models and conventional MIL/segmentation baselines.
Critical strengths
- Well-motivated problem formulation. The paper clearly identifies the limitation of appearance-centric pathology models and motivates the need to model target-relative spatial organization for structural pathological events.
Critical weaknesses
- Unclear ViT encoder. The paper only mentions a ViT encoder, but does not specify the exact backbone (e.g., UNI, CONCH, DINOv3 ViT) or whether it is frozen or fine-tuned.
- Missing molecular-grounding data scale. The authors list five internal H&E–mIF datasets, but do not report the number of patients, WSIs, or paired patches. This weakens the claimed data efficiency compared with large-scale PFMs.
- Unclear fairness of PFM comparisons. Table 1 compares FRAME directly with UNI, CONCH, and CHIEF, although FRAME is a task-specific framework with molecular grounding, constituent parsing, and structural reasoning. Without a unified frozen-encoder + same-MIL-head protocol, these results do not show that FRAME learns better tile representations.
- Insufficient details for Table 6 baselines. The tile encoder used for MIL-based methods and the aggregation strategy used for PFM-based methods are not clearly specified.
- Ambiguous training paradigm. Based on the current description, FRAME appears closer to task-specific molecularly supervised adaptation/fine-tuning of an existing ViT backbone than to pretraining a new general-purpose pathology foundation model. The paper should clarify this distinction.
- Figures 1 and 2 are unclear.
Questions for authors
Please respond to the Critical Weaknesses. If the authors fully address my concerns, I will increase my score.
6 comments
Response to W3-W4
W3. Fairness of PFM Comparisons
We apologize for the lack of clarity. Table 1 aims to evaluate the complete diagnostic capabilities of different methods under identical H&E WSI splits and task-specific labels (Sec. 4.1; Appendix A.4).
We fully agree with your suggestion of a more controlled comparison using frozen encoders and an identical MIL head. To this end, we freeze the encoders used in the Morphology Encoding stage and compare them with frozen HIPT, CTransPath, UNI, and CONCH encoders under a shared ABMIL head. CHIEF is not separately included because it uses CTransPath as its encoder.
| Backbone / feature representation | MIL head | PANDA AUC | CPTAC AUC | TCGA-CRC AUC |
|---|---|---|---|---|
| HIPT | ABMIL | 0.832 | 0.715 | 0.837 |
| CTransPath | ABMIL | 0.921 | 0.763 | 0.842 |
| UNI | ABMIL | 0.937 | 0.822 | 0.857 |
| CONCH | ABMIL | 0.941 | 0.829 | 0.862 |
| FRAME | ABMIL | 0.956 | 0.844 | 0.883 |
Under this unified aggregation protocol, FRAME consistently outperforms the strongest PFM baseline (CONCH) by 1.5, 1.5, and 2.1 percentage points on PANDA, CPTAC, and TCGA-CRC, respectively. These results further support the effectiveness of FRAME's molecularly grounded representations beyond the choice of WSI aggregation architecture.
W4. Implementation Details of Baselines
Thank you for pointing out this missing information. For conventional MIL baselines, we use ImageNet-pretrained ResNet50 as the tile encoder, followed by each method's native MIL aggregator. For patch-level PFM baselines, we use their frozen pretrained encoders with ABMIL for WSI-level prediction, whereas models with native WSI-level architectures retain their original aggregation strategies.
To further clarify the implementation protocol, we expand Appendix A.4:
Benchmarking protocol. Our objective in Table 6 is to evaluate complete WSI-level diagnostic capabilities rather than isolated tile representations. We therefore prioritize implementation fidelity:
- Complete WSI pipelines: For methods with publicly available end-to-end implementations, we retain their original encoders and aggregation architectures, including TEA-Graph, Patch-GCN, ZoomMIL, HIPT, and CHIEF.
- Partial implementations: For methods providing only patch encoders or WSI aggregators, we retain the original components and use standardized CLAM/TRIDENT-based WSI preprocessing and feature extraction to complete the pipeline.
- Paper-based reimplementation: For methods without publicly available implementations, we extract 1024-dimensional tile features using an ImageNet-pretrained ResNet50 following the CLAM preprocessing pipeline, and implement the original aggregation architecture based on the published descriptions.
Response to W5-W6
W5. FRAME's Training Paradigm and Methodological Positioning
Thank you for raising this important point! Your comment prompted us to clarify both FRAME's training paradigm and its methodological contribution.
FRAME extends morphological representation into biologically grounded structural reasoning by explicitly modeling tissue constituents and their target-relative relationships (Sec. 2.1, Line 122).
For an intuitive analogy, in visual scene understanding, detecting a person and a bicycle identifies what is present, but distinguishing "riding a bicycle" from "standing beside a bicycle" requires modeling how they are spatially related. Similarly, identifying tumor cells and a nerve is insufficient to determine perineural invasion (PNI); the diagnosis depends on how tumor cells are organized relative to the nerve boundary.
(1) What is FRAME and how is it trained?
FRAME progressively transforms H&E morphology through three stages: Morphology Encoding learns protein-grounded spatial representations; Constituent Parsing identifies Anatomical Targets and their surrounding Constituent Attributes; and Relation Reasoning models local constituent–boundary interactions and integrates them into global contour configurations for Structural Pathological Event Recognition.
Training paradigm. We retain a frozen CTransPath backbone, without additional large-scale pretraining or backbone fine-tuning. Five internal cohorts comprise 20 patients and 40 paired H&E–mIF samples (8 pairs per cancer type, ~7.58M registered patches), providing dense, spatially aligned, multi-channel protein supervision. Internal Parsing objectives enable the model to learn target masks and constituent attributes as intermediate representations, without requiring exhaustive manually annotated constituent maps. Dataset-provided task-specific labels supervise final event prediction. Importantly, mIF is used only during training, while inference requires H&E images alone.
(2) Why does FRAME matter?
-
Molecular supervision. A limited number of spatially registered H&E–mIF pairs provides dense token-level protein supervision, enabling the model to learn biologically grounded representations of epithelial, stromal, vascular, and nerve-associated constituents while reducing reliance on exhaustive manual semantic annotations.
-
Structural reasoning. General-purpose pathology foundation models learn rich morphological and contextual representations, but their conventional prediction pipelines do not explicitly model the event-defining spatial relationships between tissue constituents and specific anatomical targets. FRAME addresses this gap through Constituent Parsing and Boundary-centric Relation Reasoning, enabling the model to recognize not only what constituents are present, but also how they are organized to form pathological events.
To further facilitate understanding, we provide a pathologist-guided H&E–mIF biological correspondence schematic (Fig. 9; see W2) and visualizations on ORION-CRC illustrating constituent recognition and structural reasoning under molecularly supervised training (Appendix Fig. 10; anonymous link).
W6. Clarification of Figures 1 and 2
Thank you for highlighting this issue. Your feedback, together with the preceding comments (W1–W5), helped us provide sufficient pathological context and biological foundations.
We have made three complementary revisions:
- Pathological motivation (Fig. 1). We replace the abstract illustrations of structural pathological events with representative H&E sections of PNI invasion, selected and annotated by pathologists, with enlarged views provided in Appendix Fig. 11.
- Biological and methodological context (Fig. 2). We supplement the framework explanation with a pathologist-guided H&E–mIF correspondence schematic (Fig. 9) and ORION-CRC visualizations illustrating constituent recognition and structural reasoning (Fig. 10). These additions clarify the biological basis and information flow underlying FRAME.
- Clinical relevance (Related Work). Following your suggestion, we expand the discussion of histological isomerism with representative clinical prevalence data for PNI, LVI, and EMVI across multiple malignancies, establishing the clinical relevance of the structural events addressed by FRAME.
The revised figures and expanded discussion are included in the updated manuscript and Appendix (pp. 23–24), and are available through our anonymous supplementary link.
Response to W1–W2
Sincerely thank you for careful review ! We organize our responses into W1–W6, presented in three response windows (W1–W2, W3-W4, and W5-W6). Supplementary figures are collected in the revised Appendix (starting from p. 23) and are also available through this anonymous link: https://anonymous.4open.science/r/FRAME_r3/. Apologize for any inconvenience.
W1. Backbone Specification and Training Strategy
Apologize for the lack of clarity. We use CTransPath as the pathology-pretrained backbone, selected for its relatively lightweight architecture and manageable training and deployment costs. Its parameters remain frozen throughout FRAME training.
About frozen or fine-tuned. Our design considers the difference between the two modalities: H&E provides high-dimensional morphological information, whereas mIF provides spatially aligned continuous protein-expression signals. We therefore retain the pretrained morphological representations and introduce a trainable Spatial Adapter (Sec. 3.2, Eq. 2) to facilitate local feature adaptation and molecular information fusion, rather than directly fine-tuning the entire backbone with protein-recovery supervision.
To validate this design choice, we additionally compare direct CTransPath fine-tuning against our frozen-backbone + Spatial Adapter design using the protein-recovery setting in Sec. 4.2.1 (Fig. 3; Appendix Table 4). Direct fine-tuning reduces PCC, Spearman, AUROC, and AP by 0.174, 0.167, 0.208, and 0.146, respectively. These results support retaining the pretrained backbone while learning molecular adaptation through the Spatial Adapter under our protein-grounding setting.
We further provide a direct visual comparison between protein recovery using the fine-tuned CTransPath and our original design in Appendix Fig. 12 (revised Appendix, p. 24; anonymous supplementary link).
W2. Molecular-Grounding Data Scale and Composition
Thank you for this helpful suggestion. Based on Appendix A.2, we provide a more transparent account of the scale and composition of our paired H&E–mIF training data.
These paired samples are used specifically for training-time molecular grounding, providing dense, spatially aligned protein supervision while retaining a frozen pathology-pretrained backbone and H&E-only inference (see W1). Thus, FRAME does not rely on large-scale pretraining of a new foundation model; instead, it exploits limited paired molecular data to obtain dense biological supervision for downstream structural reasoning.