- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes Knowledge Driven Reasoning, an inference time framework for improving agreement between mammographic findingsn and diagnostic assessments generated by a vlm. It samples multiple candidate interpretations, parses their desctiptions into BI-RADS-aligned features, and selects a candidate using a combination of generative likelihood and clinical-rule consistency. The paper also introduces a Reasoning Consistency Score to assess whether generated findings support the associated diagnosis.
Critical strengths
- Tackling inference-time dissonance between diagnostic labels and textual rationales is crucial in safety-critical medical imaging. Post-hoc symbolic filtering offers a clean, modular plug-and-play solution that avoids expensive domain finetuning.
- Experiments span five diverse mammography datasets covering different modalities and patient populations.
Critical weaknesses
- The paper is not well organized, with typos, grammer errors, very lengthy paragraphs, hard to follow, for example, the first two paragraphs of Section 3 largely repeat the same generation–parsing–verification–selection pipeline, but the clinical rules, feature parser, distance function, and feature-weight construction are not adequately specified.
- There is mismatch between subsection headings and their content in Section 3.4, “Iterative Hypothesis Refinement.” This subsection consists of three high-level sentences about stochastic search and expected benefits, without describing an iterative update, feedback mechanism, or stopping criterion.
- The main verification step is not clearly explained, the method checks whether the generated description agrees with the diagnosis, but it is unclear how it checks whether that description actually matches the image. Figure 4 introduces image-faithfulness scores without explaining how they are calculated. The clinical rules, parser, and feature weights are also not properly described, making the method difficult to understand or reproduce.
- For the experiment results, RCS uses the same rules that guide output selection, so a higher RCS alone does not show better visual grounding. Also, if all rule-breaking outputs are rejected, why is RCS still below 1? The paper mixes up accuracy and macro-F1, does not explain how the binary diagnosis labels are obtained, and does not clearly establish whether the baselines use the same sampling budget. These gaps make it hard to judge what the reported improvements actually demonstrate.
Questions for authors
Questions for Authors
-
How are the and scores in Figure 4 calculated? Does the verifier check whether the described findings are actually present in the image? Please provide the clinical rules and a concrete example of how they work.
-
Do candidate selection and RCS use the same rules? If invalid candidates are always rejected, why is RCS below 1? What happens when none of the candidates passes the checks?
-
Is “ACC” accuracy or macro-F1? How are the benign/malignant labels obtained for each dataset, especially VinDr-Mammo? Please also clarify the data splits and which expert annotations are used to calculate CIG.
-
How does KDR compare with likelihood-only selection and majority voting when they use the same sampled candidates? Is there independent evidence that the selected descriptions match the images, including cases where the diagnosis is correct but the explanation could be wrong?