acceptodds
Under review as a conference paper at ICLR 2027

Logic-Driven Concept Bottleneck for Interpretable Radiology Report Generation

Abstract

Automated radiology report generation helps alleviate radiologists' workload and enhance diagnostic accuracy. However, current deep learning-based radiology report generation models often lack interpretability and logical consistency. The semantically agnostic nature of their reasoning processes often fails to align visual findings with clinical diagnoses, leading to factual inconsistencies. To address these challenges, we propose LoCoGen, a novel model for radiology report generation, which leverages a logic-driven concept bottleneck to perform diagnostic inference without relying on ground-truth annotations for the intermediate clinical concepts. Specifically, LoCoGen learns to predict intermediate concepts by optimizing high-level disease classification through logic-driven formulas linking concepts and diseases. The resulting inference paths are then integrated with clinical context and visual features as prompts to guide a pre-trained language model in generating accurate and interpretable reports. Extensive experiments on the MIMIC-CXR dataset demonstrate that our model achieves superior performance compared to state-of-the-art models. Qualitative analysis further validates the interpretability and diagnostic accuracy of LoCoGen. Code is available at https://anonymous.4open.science/r/LoCoGen-zj8snwin2j0d/.

Primary area:
interpretability and explainable AI
Keywords:
Radiology Report Generation, Concept Bottleneck Model, Neuro-Symbolic AI, Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8CWMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LoCoGen, a logic-driven concept bottleneck framework that uses intermediate clinical concepts and predefined rules to guide radiology report generation. Experiments on MIMIC-CXR show improvements over backbone baselines, but the validity of the learned concepts and the contribution of the reasoning mechanism remain insufficiently established.

Critical strengths

1.The paper addresses an important problem in radiology report generation: improving clinical consistency and interpretability beyond conventional end-to-end generation. 2.Competitive Performance and Component Analysis LoCoGen achieves strong results across most linguistic and clinical efficacy metrics on MIMIC-CXR. Ablation studies further examine the contributions of the proposed components.

Critical weaknesses

1.The learned concepts are not directly evaluated. Disease-level supervision does not guarantee that the predicted concepts correspond to actual clinical findings, and no independent validation is provided. 2.The logic-driven inference mechanism is insufficiently specified. The construction of the rule matrix and the alignment of learnable region queries with anatomical regions remain unclear. 3.While the performance improvements reported in the paper are quite significant, it is not fully clear whether LoCoGen was compared against the baseline models under identical experimental settings; therefore, it is difficult to ascertain whether these gains are entirely attributable to the proposed method.

Questions for authors

1.How are the eight learnable region queries initialized and aligned with specific anatomical regions, and how are the Chest ImaGenome region-level annotations used to supervise this alignment? 2.How is the rule matrix constructed in practice? What do its weights represent, and how does Equation (6) guarantee the claimed regional gating behavior when the disease bias is added outside the gate? 3.Were LoCoGen and the baseline models evaluated under identical experimental settings, including data splits, preprocessing, and evaluation protocols? If not, how were these differences addressed to ensure a fair comparison? 4.The paper reports no parameter count, computational cost, or inference latency, making it difficult to assess the feasibility of deploying the model in clinical settings. 5.How are the clinical context and any references to previous radiographs obtained at inference time? Are prior reports or images available for every evaluated case, and are the same inputs provided to the corresponding baseline models?

Reviewer_myj4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a framework for radiology report generation that introduces intermediate clinical concepts and a predefined concept to disease mapping. A visual encoder with an adapter and learnable regional queries predicts regional abnormalities and 37 clinical concepts, each represented by three states. A manually constructed rule matrix maps these concepts to 13 diagnostic categories, allowing the concept predictor to be trained through regional abnormality and disease supervision derived from Chest ImaGenome without direct concept annotations. elected concepts and disease predictions are then converted into textual inference paths, which condition a report generator alongside visual features and clinical context. Experiments show that the proposed method have improvements over the baselines.

Critical strengths
  1. the approach can reduce the need for concept annotations
  2. consistent ipmrovements across multiple backbones, demonstrate that the framework adds predictive value across the tested architectures.
Critical weaknesses
  1. The paper does not verify that the generated reports actually rely on the intermediate concept-based explanation. Because the language model is directly conditioned on continuous visual features, it can bypass the discrete concept bottleneck entirely during generation, meaning that high report accuracy does not guarantee that the provided explanation faithfully reflects the model's actual reasoning.
  2. The experiments do not clearly isolate the benefit of the clinical rules. The rule matrix and baseline settings are not fully described, making it difficult to separate the effects of the rules, clinical context, and regional supervision.
  3. Because the concepts are trained purely by backpropagating gradients from disease-level labels through a linear rule matrix, concepts mapped to the same disease receive proportional gradient updates. Without direct concept-level supervision or evaluation, it remains unclear how the model actually distinguishes between co-occurring signs rather than learning entangled representations.
Questions for authors
  1. How do you verify that the learned concepts represent the intended clinical findings? Can you report concept accuracy on a small, independently annotated test set?

  2. Can you provide the full concept list and rule matrix, and explain how the weights were chosen? How does the weighted sum in Eq. (6) implement the claimed clinical logic?

  3. How do you check that the discrete concept states at inference are correct when training uses only their expected values? Please also clarify why red means “present” in Figure 2 but “absent” in Figures 3–4.

4.What exactly is included in the clinical context, and do the paired baselines receive the same context and regional supervision? Does “w/o Concept” remove only the concept text or also change the disease predictor?

  1. Have you tested whether correcting a concept changes the relevant report content appropriately while keeping unrelated content unchanged? This would help establish whether the displayed concepts faithfully explain the generated report.
Reviewer_FbejOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper is about how to use Deep learning (LLMs, pre-trained image encoders) to get radiology reports. At test time, input will be radiology images and output will be the assessment. It is well motivated: End-to-end models are black-boxes and so there is little interpretebility. Concept-Bottleneck-Models create a two-step inference: first mapping the image to human defined concepts, and then from there to disease classification and report generation. However, they need concept-annotated images. This method LoCoGen claims to have the same level of interpretebility as CBMs but do not need label-annotated images. Similar to other works, they use pre-trained & frozen visual-encoders (ViT) to get to a useful embedding. An MLP network 'adapts' these embeddings. These adapter patch level embeddings are then used to get embeddings for different anatomical regions. The authors then predict abnormality score for each region. For each region, the authors predict absent/mild/present states for different concepts. They make use of a W_rule matrix, created from radiology domain knowledge to map the relevance of each of the 37 pre-defined concepts to diseases. The formulation of logit computation for a disease being gated by the abnormality probablity, and by the role of different concepts makes logical sense.

Critical strengths

There is a fair bit of engineering that has gone into this method. Additionally, they have done ablations to justify the use of several of the involved components. In terms of performance, Table 1 shows that the performance is generally good over the baselines.

Critical weaknesses
  • Evaluation on only one dataset: MIMIC-CXR, when there are multiple publicly available datasets.
  • A relevant work PromptMRG (from 2024, 327 citations) has been left out. PromptMRG seems to have some overlap with the current manuscript: disease classification, converting prediction into prompts and conditioning the decoder on these prompts. So, the question of novelty remains unanswered.
  • The performance of the model is dependant on the given matrix W_rule. The sensitivity of performance to fluctuations in W_rule would be beneficial to know.
  • 'logic-driven' does not seem appropriate. We don't have AND, OR, or other operations here. It is a linear layer that is doing the work. So, the title may need to be appropriately adjusted.
  • How easy/hard is to have such annotations where different regions are annotated with disease and region abnormality? the reviewer is not from this field, however it looks more plausible that image level annotations would be more common.
Questions for authors
  • Line 130: Regarding work by other group, the authors claim 'their label-correlated concepts often result in “predicting results from results” and fail to reveal genuine diagnostic processes'. There is no citation nor any study done in this work that supports this claim. Without this, it looks unprofessional to say the least.
  • The word 'region' is not defined, but seems to be loosely associated with things like different lung zones, spine etc.
  • Minor details:
    • MHA's full-form is not mentioned (multi-head attention) in equation 2.
    • Notation issues: Q_r is matrix with N_R rows, and so r is not an index. But in f_r, has r as an index.
    • L170: It need to be mentioned that fr is one row of Fr.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.