- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces PathMem, a memory-centric multimodal framework for pathology MLLMs that integrates structured domain knowledge as a long-term memory graph and uses a novel Memory Transformer to dynamically transition it into working memory for context-aware reasoning. Extensive evaluations demonstrate that PathMem achieves sota performance across various pathological benchmarks, significantly improving both report generation and open-ended diagnosis.
Critical strengths
Effectively mimics the hierarchical memory retrieval and diagnostic reasoning process of human pathologists by linking visual patterns with structured medical knowledge graphs.
Consistently achieves superior performance across multiple benchmarks (such as WSI-Bench) and diverse downstream tasks like report generation, diagnosis, and treatment planning.
Employs an efficient hashing-based deduplication and probabilistic multi-evidence fusion mechanism to maintain a high-precision, non-redundant long-term memory bank.
Critical weaknesses
The overall reasoning capability is heavily bounded by the completeness and precision of the initial biomedical literature retrieval and LLM-based triple extraction, which may still carry noise or missing relations despite filtering thresholds.
The sparse top- selection and token-level integration within the Memory Transformer introduce additional computational complexity during inference, potentially impacting scalability for ultra-large-scale slide sets.
While evaluated on standard tumor types within benchmark suites, the framework's adaptability to rare or atypical pathological variants—where literature evidence is scarce—remains underexplored.
Questions for authors
See weakness
9 comments
Response to Reviewer kEiv on knowledge reliability, inference cost, and rare variants.
-
-
Table 1. Inference efficiency analysis on WSI-Bench.
-
Metrics Quilt-LLaVA WSI-LLaVA SlideChat MLLM-HWSI PathMem Time (s/WSI) ↓ 0.75 0.93 0.69 294 0.69 GPU Memory (GB) ↓ 12.62 16.70 11.58 25.82 16.3
-
-
Table 2. Efficiency and practicality analysis of PathMem.
-
Metric Value Inference Time ~1.45 WSIs/s GPU Utilization 1 × GPU GPU Memory Usage 16.3 GB Image Feature Size 1728 × 4096 tokens KG Construction Cost Offline embedding + indexing KG Update Cost Re-embedding + re-indexing
-
-
-
Importantly, our sparse top- design restricts memory integration to selected relevant entries,
- while KG embedding and indexing are performed offline, avoiding additional knowledge construction costs during inference.
- Overall, PathMem achieves competitive inference efficiency with a manageable memory overhead, demonstrating a favorable trade-off between computational cost and memory-enhanced representation.
- while KG embedding and indexing are performed offline, avoiding additional knowledge construction costs during inference.
Weakness 3 :
While evaluated on standard tumor types within benchmark suites, the framework's adaptability to rare or atypical pathological variants—where literature evidence is scarce—remains underexplored.
Authors’ Response:
-
We would like to clarify the scope of PathMem: a pan-cancer, multi-organ memory-augmented MLLM for general whole-slide question answering, pathological analysis, and report generation.
-
Its central contribution is context-dependent activation of structured long-term knowledge for these tasks.
-
The available dataset metadata shows the following breadth:
-
Task Tumor categories Organ/tissue-site groups Morphology understanding 30 25 Diagnostic assessment 30 25 Treatment-related reasoning 12 12 Report generation 30 25 - The 25 site groups include adrenal/paraganglionic, bladder, breast, cervix, bile duct, colon, lymphoid tissue, esophagus, head and neck, kidney, brain, liver, lung, mesothelium, ovary, prostate, rectum, soft tissue, skin, stomach, testis, thyroid, thymus, uterus, and uvea.
- These counts describe train/test metadata coverage using the stated organ/tissue grouping.
- Our KG additionally organizes 490 disease entities.
- These counts describe train/test metadata coverage using the stated organ/tissue grouping.
- The 25 site groups include adrenal/paraganglionic, bladder, breast, cervix, bile duct, colon, lymphoid tissue, esophagus, head and neck, kidney, brain, liver, lung, mesothelium, ovary, prostate, rectum, soft tissue, skin, stomach, testis, thyroid, thymus, uterus, and uvea.
-
-
Dedicated validation on rare or atypical variants is outside the present experimental scope.
- It requires subtype-defined cohorts and targeted evaluation of literature-scarce conditions, addressing a more specific generalization question than the cross-organ, multi-task objective studied here.
- Broad anatomical coverage and rare-variant robustness are distinct evaluation dimensions.
- It requires subtype-defined cohorts and targeted evaluation of literature-scarce conditions, addressing a more specific generalization question than the cross-organ, multi-task objective studied here.
-
Accordingly, we will explicitly delimit our empirical claims to the reported tasks and cohorts,
- and identify rare/atypical-variant validation as a subsequent extension.
Dear Reviewer kEiv,
We sincerely appreciate your time and effort in providing such thoughtful reviews and insightful comments.
Could you please kindly let us know if our rebuttal has addressed your concerns?
If you have any further questions regarding our work, we would be delighted to address them.
If our responses have addressed your concerns, we would greatly appreciate your reconsideration of your overall assessment.
Best regards,
Authors
Response to Reviewer kEiv on knowledge reliability, inference cost, and rare variants.
Weakness 1 :
Authors’ Response:
Thank you for highlighting the importance of knowledge quality. Our KG construction combines source-grounded extraction, explicit quality controls, and expert supervision.
1. Rigorous construction.
2. Pathologist-supervised validation.
Construction and validation were supervised by four pathologists from three medical centers across two countries.
They performed article-level screening to determine the included literature and reviewed each constructed KG for the validity and pathological plausibility of its knowledge and relations.
The curated resource includes 1,998 accepted articles, 490 disease entities, and 19 relation types (As shown in the table below), providing substantial coverage with domain-expert oversight.
3. Demonstrated downstream utility. Table as listed below shows that activating this curated knowledge improves report generation:
Weakness 2 :
Authors’ Response:
Thank you for raising this important concern.
Our efficiency analysis demonstrates that this overhead is well controlled.