acceptodds
Under review as a conference paper at ICLR 2027

Ontology Concept Bottleneck Models: Towards Human-Aligned Reasoning with Graded Logic

Abstract

As autonomous systems increasingly make consequential decisions on behalf of humans, interpretability alone is insufficient. Their reasoning should not only be explainable but also align with the way humans evaluate evidence and make decisions. This motivates reasoning over intuitive concepts through concise and human-readable compositional processes. We introduce Ontology Concept Bottleneck Models (OCBMs), which jointly learn task-functional concepts and class-specific graded-logic reasoning trees, leveraging the reasoning structure itself as an inductive bias on concept formation. Across standard visual and tabular benchmarks, OCBMs achieve competitive or leading predictive performance while producing compact, auditable reasoning structures with controllable granularity. Moreover, controlled head ablations show that these concepts transfer substantially better across domains than those learned with linear or MLP heads. These results suggest that graded-logic ontologies can serve simultaneously as an interpretable reasoning layer and a structural constraint on concept learning, providing a practical foundation for machine reasoning that can be inspected, intervened upon, and progressively aligned with human reasoning.

Primary area:
interpretability and explainable AI
Keywords:
Concept Bottleneck Models, Interpretable Machine Learning, Human-Aligned Reasoning, Graded Logic, Concept Learning, Explainable AI, Ontology-Based Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8C2pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study proposes OCBM, which uses graded logic for the final prediction layer with one tree per class. These trees are pruned to be small, and they use soft AND/OR operators with learned weights and strictness. Main claims of the authors are: accuracy is competitive, five concepts have been learned on MNIST without concept labels, the concepts are better transferred to USPS than those from linear or MLP heads, and all of this make the model’s reasoning more human-aligned.

Critical strengths
  1. The idea is interesting and well motivated in the paper. Graded logic can express relationships that boolean heads could not.

  2. 99% accuracy on MNIST that has been achieved with only five concepts and without concept labels shows that this idea could potentially improve CBMs.

Critical weaknesses
  1. The only comparison to assess whether graded logic is better and gives more transferable concepts is against a linear and an MLP head. Multiple factors are not controlled (that could explain the results, e.g., sparsity, one tree per class, training with extra losses, etc.). They could run other experiments with simpler logic operators, such as min/max, or other tests with the matched sparsity, like a sparse MLP, or more controlled tests with simplifying parts of the method to show that the gains are because of the graded logic and the full method. Overall, the idea is good, but the tests are not sufficient to support the claim.

  2. Considering that the abstract, and even the title, mention human-aligned reasoning, a reader would expect to see some sort of user study or experiment that would further support this. Instead, the authors only use an LLM to revise the concept names until the score passes a threshold, and the explanations are not checked against the model predictions.

Questions for authors
  1. Could you add other experiments that would control for different factors and isolate the effect of graded logic, for example, simpler logic operators, sparse MLP heads, or removing individual components of the method? Would the advantages of your method persist over these controls as well?

  2. Are there any results or tests that would further support the human-aligned reasoning? Also, can LLM-written explanations reproduce the model predictions?

Reviewer_xDuGOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a new Concept Bottleneck Model that learns graded-logic trees as task predictor. The method is able to learn concepts and tasks end-to-end. Experiments evaluate a.o. the model's task accuracy and ability to learn meaningful concepts in the absence of concept supervision.

Critical strengths

Combining graded logic with Concept Bottleneck Models seems a novel idea that has not been explored yet.

Experiments are performed on multiple CBM datasets like CUB, AwA2, and CIFAR100.

Critical weaknesses

Gap

The gap this work is trying to solve is already addressed by combining existing works. The gap appears to be a method combining concept acquisition and reasoning in a single end-to-end framework. It is unclear what this method contributes that is not already provided by others, and why this being end-to-end is useful. Regarding the concept reasoning, this is, as admitted by the paper, already done by other papers (i.e. there are multiple neurosymbolic CBMs), and to me, they seem more interpretable than this CBM (see weakness below of graded logic vs other logics).

Concept Acquisition

The central concept acquisition claim is insufficiently supported, and I personally do not believe the method can make this claim real. The paper claims that it is possible to learn both the task predictor (C -> Y) and to learn meaningful concepts C at the same time thanks to learning graded logic trees.

Learning meaningful concepts, in practice, rarely happens without semantic grounding mechanisms like concept supervision or distant task supervision through human-provided rules. Regular task supervision does not identify a particular human-interpretable concept representation, as many alternative representations can give the same task predictions. The biases introduced by the method (e.g. using trees for task prediction) seem weak to try to encourage learning meaningful concepts. For instance, prototype-based networks attempt to discover concepts by using as mechanism visual reconstruction / visual similarity, which is absent here.

The authors seem to provide an explanation in lines 141-143 saying that e.g. sharing the learned concepts among multiple classes helps learning meaningful concepts instead of shortcuts. I would like to refer the authors to reasoning-shortcuts literature (e.g. Marconato et al., 2023), where it is indeed well-known that this helps. However, it is also known that this is typically insufficient to prevent reasoning shortcuts, even when the logic rules are fixed and already provided beforehand. In this paper, both the concepts and the rules are learned jointly, creating an even larger identifiability problem.

The only experiment that shows concept acquisition is on MNIST, which is a very trivial dataset. So it is not the case that the experiments provide real evidence for the claim.

Insufficient mathematical specification

Many essential parts of the method are only explained in prose (Section 3.1) instead of being mathematically formalized. It is therefore difficult to understand the actual computational graph / inference procedure of the model, as well as how the model is exactly trained.

It is unclear whether in the experiment with concept supervision the task gradient backpropagates through the concepts or is detached. In the former (like in standard joint training), there can be concept leakage, which would weaken the task accuracy results.

CIFAR-100 does not have concept labels. It is unclear how the authors train a standard CBM on it without concept labels.

Metrics

The self-defined G_ci and G_s score by construction favor the proposed method, as they cannot really be computed on competitors and the paper seems to assign a score of 0 for them for this reason.

Interpretability claim

The paper claims the introduced CBM is interpretable because it uses concepts and its inference is built on the GCD operator (line 187), which the paper considers interpretable. I'm not convinced this operator would be considered interpretable by most people. I note that the authors provide a more "human-like interpretation" of specific instantiations of this operator in Table 1, but this seems like some kind of explanation. It seems like the operator is too complicated to be really interpretable, and thus, it is necessary to try to give some intuition in text about it. The semantics of the operator are complicated, unlike in boolean or probabilistic logic, where the logical connectives are interpretable on their own. This is reinforced by Table 4: the graded logic expression is very complicated, and the "verbalization" is provided by an LLM, so this verbalization is a post-hoc approximation.

Questions for authors

See the weaknesses above.

Reviewer_1okyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces Ontology Concept Bottleneck Models (OCBM), a method that learns ontologies via graded logic trees based on concept representations to provide an expressive, yet human-aligned reasoning head for the CBM. OCBM can provide concise, compositional and human-readable logic predictions. Furthermore, it is also possible to use the ontology prediction head as a guiding signal to find suitable concepts in settings without concept supervision.

Critical strengths
  • While using logic predictors on CBMs is not a new direction, most of these are limited to specific, rigid structures in what they can learn. In contrast, the ontology head using graded logic, which is presented in this work, can compose concepts in a hierarchical yet readable fashion to make good predictions, which is something most CBM predictors currently miss.
Critical weaknesses
  • This work makes several claims regarding the proposed method, including its interpretability, utility for human interactions, and its use in settings without concept annotations. However, these claims do not match what is presented in this work and overstate its contributions.
  • While this paper contains some references to related work, it would benefit from a more thorough connection to related work. Large parts of the text lack references, and it is often unclear which statements are findings from previous works and which are the authors' own claims.
Questions for authors

Overstating claims of the method:

This is a more general point that is somewhat tied to the previous two points. In my opinion, this paper makes a lot of claims about different things (mostly interpretability of the proposed method, but also some other things). I think these are not sufficiently supported through the work. This can partly be due to missing connections to things that have already been established in prior work, but partly is also due to overstatements of the results provided in this work.

  1. One main motivation of this work is its improved interpretability compared to other CBM approaches. This is largely based on the argument that graded logic is more interpretable than other types of logic predictors that current CBM approaches have been using (introduction and Sec. 2.1), and is not sufficiently supported (either by evidence or sufficient established results from prior work). Further claims that the results are interpretable in the experimental section are only supported by showing a few qualitative examples, which is insufficient.
  2. Using OCBM without concept supervision: Throughout the paper, it mostly sounds like OCBM is exclusively used without explicit concept supervision and can achieve all its benefits without such annotations (this includes, for example, the introduction, contributions, parts of the method section, and the experimental results). However, the experiments performed in this work mostly use supervised concept annotations. The primary experiment that investigates concept learning without concept supervision with OCBM is on MNIST. And while I think this is a worthwhile and interesting experiment, it is not sufficient to claim that OCBM is going to achieve such concept decomposition generally.

References to other works:

Throughout the paper, references to prior work are missing nearly completely (except for related works and the beginning of the experimental section). This makes it partly more difficult to follow the text (e.g., which CBM methods were referenced in the introduction), and particularly difficult to distinguish between claims of the authors and established findings of prior work. For this, a full pass through the paper is needed to improve this. In the following, I have some specific pointers/remarks about references:

  • The authors refer to both MNIST and USPS, however the references to both datasets should be clearer (particularly in the experiments in Sec. 4.4)
  • The baseline methods which are used in the experiments should also be mentioned in the related work.
  • For CBMs with logical predictors, DCR [1] should also be mentioned (and probably also be part of the experimental comparison)
  • The discussion on concept acquisition would benefit from a bit more fine-grained perspective, in particular for the mode of OCBM that does not use concept annotations. Many works have tried to reduce the reliance on concept annotations in different ways, some coming from a top-down perspective (defining what concepts should be found and then relying on pretrained models like CLIP to ground these concepts in the input data), and others going a bottom-up way (first grounding abstract symbols in the input data and defining/identifying their semantics afterward). Discussing this difference and where human knowledge comes in would be helpful to clarify the approach of OCBM even more. See [2] for a detailed discussion on this topic and an overview of related works in both areas.

Further Questions:

  • l.51: What are compensatory and non-compensatory reasoning patterns?
  • How computationally expensive is it to train OCBM, and how does this depend on the size of the dataset, the size of the concept space, and the depth of the resulting trees?
  • Can OCBM generally also be adapted to tasks beyond classification?

Questions about the method:

  • l.161: What does merge onto a shared parent mean exactly?
  • Can the authors explain how the definition of GCD in this work differs from the previous introduction by Dujmović?
  • Is there a specific reason why you start with a jointly trained model as a starting point for OCBM, if the predictor is not used later on anyway? Also, can the authors comment on whether this joint training setting can potentially lead to concept leakage [3]?
  • Can you provide more details on OvR term? From the description, it remains unclear what exactly this term does and how it functions.
  • Can you explain how the variant for left-associative binary tree learning works in more detail? Also I would recommend describing this in the method section instead of the experimental evaluation (currently under 4.3)
  • I would strongly suggest moving the full concept semantics description either into the method section or into its own new section. Having this in the middle of the experimental evaluations is a bit confusing and breaks the flow of the section.
    • Internal activation coherence - This assumes the concept is always present for a class, which does not have to be the case (as e.g. motivated for the CUB evaluations, there might be different explanations/versions of a class), can you comment on this?
    • External alignment: Where does the vocabulary of human strokes come from, and is this over the full dataset?
    • Logical coherence: What is the proposer? And how should they be able to provide the world knowledge memberships if they do not know yet what the concept represents?

Questions about the experimental evaluation:

  • 4.1: What are the concepts for the base CBM and OCBM for CIFAR100?
  • 4.2:
    • I assume you use class-level concepts for CUB as it is typically done? In this case, the connection between concepts and classes is extremely easy, as there is the same GT concept pattern for each sample of the same class, and the main limitation of a CBM typically lies in the prediction of the concepts. Thus, using CUB to show that different reasoning widths do not affect performance is not an ideal setting. How does the same comparison look for the other datasets?
    • How exactly are the different oracle settings designed?
    • I would argue that adjusting the width of the tree does not really qualify as incorporating human knowledge into the reasoning process. Can you discuss this point?
  • 4.3:
    • All results in this experiment(Appendix A) are done with the reduced OCBM variant but not with the default OCBM variant. Furthermore, the compactness between both versions of the method is not investigated. Can you provide results on how this would work with the default variant of OCBM or discuss why this is not applicable here?
  • 4.4:
    • Connected to the overstatement of the papers claim above: Does this setting for OCBM also work for more complex datasets than MNIST (both in terms of the data represented, as well in terms of potentially required concepts)?
    • Can you explain the setting of the quick-draw experiment a bit more in detail? What is the setting there? What are the task labels, and how well does the model perform? Just showing some qualitative images is a bit confusing.

Minor points

  • Fig.1 is never mentioned in the text, I would recommend integrating it into the introduction text.
  • For Tab. 1, I assume the second-to-last column represents the value of alpha. It would be nice to note this somewhere (either in a header row or in the caption)
  • The weight of the OvR term should probably be framed as a hyperparameter rather than being stated as a fixed value in the method section.
  • l.225: I think it would make more sense to introduce the combined L ontology loss in the previous sections formally and just reuse it here.
  • An initial section describing the experimental setup would improve readability of the experimental section. This would allow us to introduce baselines, datasets, and the evaluation setting all in one shared space.

Overall, while I currently lean towards rejection for this paper, I think if the integration of prior work into the text is improved and the claims are more aligned with what is provided in the paper, I am happy to reconsider my score.

[1] Barbiero, Pietro, et al. "Interpretable neural-symbolic concept reasoning." International Conference on Machine Learning. PMLR, 2023. [2] Knab, Patrick, et al. "What’s in the bottle? a survey and roadmap of concept bottleneck models." Transactions on Machine Learning Research (2026). [3] Havasi, Marton, Sonali Parbhoo, and Finale Doshi-Velez. "Addressing leakage in concept bottleneck models." Advances in Neural Information Processing Systems 35 (2022): 23386-23397.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.