- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper investigates the visual representation challenges faced by medical vision-language models in understanding lesions with multiple phenotypes. The authors note that existing semantic alignment-based methods struggle to preserve the hierarchical relationships of anatomical structures and the continuous relationships between different pathological phenotypes within the visual embedding space; furthermore, sparse lesion information is easily diluted by the abundance of normal tissue patches. To address this, the authors propose "PureVision," which comprises two main modules: "PureEyes," which supervises anatomical and phenotypic visual representations using ideal geometric distributions, and "PureNeurons," which selects lesion-relevant patches from the structured visual space and fuses anatomical and phenotypic evidence before passing it to the language model. Experiments conducted on the LIDC-IDRI, CBIS-DDSM, and 3DReasonKnee datasets, covering tasks such as VQA and radiology report generation, demonstrate that this method enhances capabilities in lesion localization and the description of pathological phenotypes.
Critical strengths
-
The issues the paper focuses on are more practical. The author did not just continue to strengthen vision-text alignment, but further considered whether the visual feature space itself retains the medical anatomical hierarchy and pathological phenotype relationships. This starting point has certain significance.
-
The designs of PureEyes and PureNeurons in Section 3.2 and Section 3.3 have a clear complementary relationship: the former is responsible for building a structured visual representation space, and the latter is responsible for filtering and delivering sparse lesion evidence.
-
The experiment covered three different imaging modalities: CT, mammography and MRI. Multiple medical VLM backbones were tested at the same time and verified on two tasks: VQA and report generation. The scope of the experiment was relatively comprehensive.
Critical weaknesses
-
I feel the paper's framing regarding "multi-phenotype lesion interpretation" is slightly overstated. The 3DReasonKnee dataset described in Section 4.1 actually involves only a single phenotype dimension; thus, it serves more to validate cross-modal and cross-disease generalization rather than strictly testing a multi-phenotype setting.
-
In Appendix A, the methods for calculating phenotype accuracy differ between the VQA and RRG tasks, yet Tables 1 and 2 use identical metric names. This could mislead readers into thinking the phenotype accuracy metrics for the two tasks are identical and directly comparable; I suggest the authors provide further clarification.
-
The terminology used to describe core concepts in the paper is somewhat inconsistent; terms such as "ideal spatial distribution," "ideal visual semantic distribution," and "ideal geometry" are all used to refer to similar supervision targets. While the authors' intent is generally clear, for a paper whose primary contribution is geometry supervision, it would be best to standardize the terminology to avoid confusing readers about the specific objects of the constraints.
Questions for authors
Could the authors clarify what aspect of the multi-phenotype lesion interpretation setting is intended to be validated by 3DReasonKnee, given that Section 4.1 includes only one phenotype dimension for this dataset?
Response to the Program Chairs
To provide greater clarity about my review process, I have included my original review comments below for reference:
1, Med Caption 任务现在临床上都不用,不知道为啥还有继续坚持这些任务,性能再好有什么意义呢,如果注入模型之后 反过来改善数学或者coding 甚至意义更大一些;
2,Med + VLM 这类文章最大的问题就是他到底在解决医学问题还是在解决AI问题,这文章关注multi-phenotype lesion,说白了就是长得不一样的病灶,对医学影响比较大,对智能的增益 我觉得没啥用,关键是他测的也不是multi-phenotype lesion,3DReasonKnee 也就一种款式啊?
3,写的有点乱,感觉前后不一致问题比较大, ideal spatial distribution ,ideal visual semantic distribution,感觉怪怪的,像中式英语,不是专有的学术表达,感觉指标的计算上 对比的setting也有点问题;
To avoid having my review flagged as AI-generated, I revised the wording and then translated it using Google Translate. However, it was still flagged as AI-generated.