- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ΔRepresentation, a representation-learning framework for medical vision-language models that characterizes pathological phenotypes as representation increments relative to estimated counterfactual normal anatomy. The paper introduces BaseAnatomy to organize anatomical representations using geometric supervision and ΔPhenotype to learn lesion-specific changes relative to these anatomical references. The overall perspective is constructive and intuitive, and I believe the high-level idea is promising.
However, the paper’s current level of completion is limited. Many important implementation details remain insufficiently explained, and the logical development of the method is not always rigorous. The visual presentation is also below the standard expected of an academic paper. Several figures are low-resolution and contain poorly standardized visual elements. Figure 1, in particular, appears to contain obviously AI-generated illustrative assets with numerous imprecise details. I do not object to the use of generative models for figure creation in principle, but the resulting figures must still satisfy academic standards. Similarly, the method figure lacks careful alignment and even basic horizontal and vertical organization (It appears that the authors have not received standardized training in academic literacy). Finally, the empirical analysis relies heavily on scatter plots and low-dimensional projections, which are not always appropriate or sufficient to substantiate the claimed representation properties (but not a big deal).
Given the space constraint, I focus below on one critical strength and two critical weaknesses.
Critical strengths
The paper’s central perspective, "representing a pathological phenotype as a deviation from its corresponding normal anatomical representation," is both intuitive and potentially valuable. It offers a conceptually meaningful alternative to learning pathology representations solely through absolute visual features or image-text alignment. The proposed direction is broadly consistent with clinical reasoning, in which abnormalities are often recognized as changes relative to expected anatomy. Although the present implementation requires substantial improvement, the underlying research direction is promising.
Critical weaknesses
1: Mismatch Between the 3D Methodological Motivation and the 2D Evaluation
The paper motivates the method in terms of volumetric anatomical geometry, three-dimensional spatial continuity, and reasoning over 3D CT anatomy. However, the actual benchmark treats an individual axial CT slice as the unit of analysis. The reported downstream tasks operate on resized 2D slices, and lesion grounding is reduced to selecting a cell from a (4 x 4) two-dimensional grid. Slice-position encoding does not, by itself, constitute a convincing evaluation of volumetric reasoning.
Consequently, the experiments do not demonstrate that the proposed representation preserves meaningful 3D anatomical relationships, supports cross-slice reasoning, or improves volume-level lesion localization and characterization. The observed gains could instead arise from learning 2D spatial patterns or slice-position priors. To support the paper’s broader claims, the authors should evaluate on complete CT volumes using genuinely 3D tasks and baselines, for example, volume-level lesion localization or segmentation, cross-slice consistency, and volumetric report generation, and should ablate the contribution of slice-position information.
2: Heavy Dependence on Localization Priors and Dense Annotations
The proposed framework depends heavily on strong localization priors during representation learning. BaseAnatomy requires anatomical annotations and lesion masks, while ΔPhenotype uses known pathological patches, anatomically matched mask-negative neighbors, lesion-level aggregation, and explicit localization supervision. Moreover, the phenotype VQA setup directly provides the target grid cell in the question. These design choices substantially simplify the learning problem and make it difficult to determine whether the improvements originate from the proposed counterfactual representation mechanism or from the extensive spatial supervision provided to the model.
Although the paper states that masks are not required during inference, this does not eliminate the framework’s substantial training-time dependence on accurately localized anatomical and pathological regions. This dependence may limit scalability to datasets or clinical settings without dense annotations and raises concerns about robustness when annotations are noisy or produced by imperfect automatic segmentation models (such issue always happens). The paper needs stronger ablations that remove or weaken these priors, comparisons against baselines receiving equivalent localization supervision, and experiments using predicted or noisy masks. Without such evidence, the practical generalizability of the proposed approach remains unclear.
Questions for authors
See weaknesses.