acceptodds
Under review as a conference paper at ICLR 2027

Gradient-Cancelled Concept-Based Models

Abstract

Concept-based models are designed to support human intervention: an expert can correct an incorrect concept at test time and improve task prediction, a capability particularly important in high-stakes domains. Joint training typically achieves the strongest task performance, but this can come at the cost of information leakage, whereby concept representations encode task-relevant information not captured by their intended semantic meaning. We provide an information-theoretic characterisation of task-driven leakage in joint training: under task-head convergence, the encoder update contains an explicit mutual-information maximisation term that encourages concept representations to capture task-relevant information beyond the annotated concepts. We introduce Gradient-Cancelled Concept-Based Models, which use a Gradient Reversal Layer to cancel the task-driven encoder gradient, leaving the concept encoder to learn from concept supervision alone while training the task head concurrently. Our model requires a simple training-time gradient modification and applies to any concept-based model where task gradients reach a shared encoder via concept representations. Our experiments show that Gradient Cancelled (GC) training results in significant task improvements upon intervention over vanilla concept-based models, while maintaining strong concept and task baseline performance, making these models better suited to in the human-in-the-loop settings they were originally designed for.

Primary area:
interpretability and explainable AI
Keywords:
interpretable models, concept bottleneck model, leakage

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_A3raOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a information theory-based mathematical formulation for leakage caused by the joint optimization of CMs. Based on their formulation, the authors prove that the joint training scheme optimizes concept to task leakage, a fact supported by theirs and previous work's empirical results. Then, they introduce the Gradient Reversal Layer into CBMs, yielding Gradient-Cancelled Concept Based Models (GC-CBMs), which provide an end-to-end equivalent of sequential CMs. Based on their analysis GC-CBM achieves the same minima as sequential CBMs, while being end-to-end and avoiding concept to task leakage. Lastly, they empirically show that GC-CBM outperforms jointly trained CBMs and CEMs in concept accuracy and intervenability, and performs similarly to sequential training while being end-to-end.

Critical strengths

Thank you so much for submitting this work! I enjoyed reading the analysis, and I appreciate the time taken to write it up and submit it to ICLR. Below are what I believe are this paper’s main strengths:

  • S1: Mathematical analysis and proof of concept-to-task leakage created by joint training. Also the authors mathematically explain the interaction between concept incompleteness and leakage. Both of these phenomena have been observed in previous empirical works and this work complements them by adding mathematical rigor.
  • S2: They propose new metrics RTL and RCL, to account for the bias of estimating MI in high dimensions seen in the previous metrics concept-to-task (CTL) and Interconcept Leakage (ICL) [1].
  • S3: They show that it is possible to train sequential Concept-based Models in one go, avoiding the need for two steps. Then they show that sequential CBMs and GC-CBMs (and GC-CEMs) can outperform their jointly trained counterparts in concept accuracy and intervenability. Lastly, by tackling concept to task leakage, they also reduce interconcept leakage.

[1] Parisini, Enrico, et al. "Leakage and interpretability in concept-based models." Journal of Machine Learning Research 27.174 (2026): 1-40.

Critical weaknesses

In contrast, I believe the following are this work's major limitations:

GRL

  • W1: Given that gradient cancellation happens only for λ=1\lambda=1 and fϕ=hψf_\phi=h_\psi, both of which are strictly enforced, this leads to a zero gradient. Unless I am missing something, isn't is possible to just set that gradient to zero in the code (e.g. using detach in PyTorch) and achieve the same result? This eliminates the need of using GRL, thus avoiding software, architectural and optimization overhead and complexity of having two task heads (e.g., GC-CBM training takes longer to converge (Line 1375)). Also, given that the authors don't evaluate partially allowing for leakage (i.e. λ<1\lambda<1), I really don't understand the use of GRL.

Experiments

  • W2: Unless I missed it, the authors did not empirically investigate GC-CBM in a concept incomplete dataset, even though the mathematical analysis of concept incompleteness and leakage. Toy concept incomplete datasets include incomplete FMNIST [2], and a synthetic tabular dataset [3,4].

  • W3: Given that one of the main comparisons in this work is GC training against sequential training, I would expect in the CEM part of the experiments to see a sequential CEM baseline.

Presentation

  • W4: One of the main presentation issues is the fact that the figures in the main text are generally too small and difficult to read. Fig. 1, for example, could simply have been presented at a larger size to improve readability. Fig. 2 also leaves a substantial amount of unused space (due to wrapfigure). More generally, the figures could be decluttered. Finally, the resolution (DPI) of some of the figures appears to be rather low and should be improved for readability and presentation quality.

Minor Comments

  • Please provide some interpretation of the RTL and RCL metric values and what they imply in practice. For example, in Fig. 6, CEM appears to have a lower RCL on CUB, but the absolute values are quite small. It would be helpful to discuss whether this difference is practically significant, or whether it is negligible and therefore does not represent a meaningful disadvantage of GC-CBM.

Clarity

  • Please state in the main text that the interventions are group-interventions (as stated in App. J1). Also, for Fig. 4 and Fig. 7, I think it would help to show points instead of continuous lines since the x-axis is discrete.
  • Line 108: "Hard CBMs by construction cannot encode any leaked information: they are trained independently, ..." there are works in which hard CBMs are being trained end-to-end (e.g., [2, 5]). Please specify only in the experiments that the hard CBMs are trained independently.

Experiments

  • Lines 343-346. "Concept accuracy is competitive with joint and sequential CBM across all datasets and in some cases outperforms them (for AwA2 and CUB against both references for all λc). ". In all cases, Seq. CBM and GC-CBM seem to have the same concept accuracy (as also seen in the tables of App. K) and as also stated in Proposition 1. Please rephrase this to make the distinction between joint and sequential clearer.
  • Lines 360-362: there is a bigger drop in CTL in TabularToy but accuracy remains almost the same. Please be more accurate in the wording, or further clarify on why this statement does not hold in TabularToy.
  • Lines 375-377: As seen in Fig. 4, all but one joint CBM performs equally good to the rest of the models in interventions. Please rephrase and specify that the comparison is against the worst-perfoming Joint CBM.

Typos

  • The captions of Figures 2,3,9 and Tables 2, 5, 6, 7 are missing a period at the end.
  • Missing reference to Fig. 4 in paragraph "Gradient cancellation makes concept interventions more effective."

[2] Kalampalikis, N., Gupta, K., Vitanov, G., & Valera, I. (2026). Towards Reasonable Concept Bottleneck Models. Transactions on Machine Learning Research. Retrieved from https://openreview.net/forum?id=0TxwJMhXBY

[3] Dominici, G., Barbiero, P., Espinosa Zarlenga, M., Termine, A., Gjoreski, M., Marra, G., & Langheinrich, M. (2025, May). Causal concept graph models: Beyond causal opacity in deep learning. In International conference on learning representations (Vol. 2025, pp. 55102-55131).

[4] Marcinkevičs, R., Wolfertstetter, P. R., Klimiene, U., Chin-Cheong, K., Paschke, A., Zerres, J., ... & Vogt, J. E. (2024). Interpretable and intervenable ultrasonography-based machine learning models for pediatric appendicitis. Medical image analysis, 91, 103042.

[5] Vandenhirtz, Moritz, et al. "Stochastic concept bottleneck models." Advances in Neural Information Processing Systems 37 (2024): 51787-51810.

Questions for authors

Clarity

  • Unless I missed it, it is not stated what λ\lambda value is used. I assume it is λ=1\lambda=1. Regardless of its value, please state it in the main text.
  • What is the concept representation used in GC-CBM? Fig. 1 shows that the logits are propagated to the task. Furthermore, what intervention values are used? The ground truth binary values, or (e.g.) 5th and 95th activation percentiles?
  • In line 295, Rm2R^2_m I assume mm is the concept embedding's dimensionality?
  • In App. J.2 I noticed that different number of training epochs is used per model. I assume that this is due to early stopping or due to the models converging earlier?

Methodology

  • Why is there a clamp to zero for the RTL and RCL metrics? What would negative Rm2R^2_m mean?

  • Why is GC-CBM's CTL lower than an independently trained hard CBM (and the Sequential CBM)? Given lines 108-109 "Hard CBMs by construction cannot encode any leaked information: they are trained independently...", and previous works, I would expect it to be equal at worst. On top of that, why aren't the CTL values of hard, sequential and GC-CBM zero?

Experiments

  • This work introduces alternatives to the CTL and ICL metrics intended for higher dimensionalities (e.g., concept embeddings). As a sanity check and as a comparison to the pre-existing metrics, can you provide both sets of metrics for both the CBM and CEM experiments?

  • An easy experiment, where I think GC-CBM would really shine is a comparison on TabularToy, against a joint CBM that achieves almost 100% concept accuracy, to show that leakage may still happen for the joint CBM.

  • An interesting ablation would be different λ\lambda values, and quantifying the benefit of allowing any leakage, and thus providing a use case of GRL other than complete gradient cancellation.

My final rating is based on the use of GRL, the overall clarity of the manuscript, and the incompleteness of the experimental evaluation. I believe the strongest aspect of this work is its mathematical formulation of concept-to-task leakage. I would be happy to increase my score if the authors address the concerns raised above.

4 comments

We sincerely thank the reviewer for a thorough and constructive review, and for recognising the mathematical analysis, the new metrics and the single-stage training. We ran every suggested experiment; all new figures and tables are in App. R of the revised PDF.

Why a GRL rather than a detach? The reviewer is right. At λ=1\lambda = 1, GC is exactly a stop-gradient on the bottleneck for the task loss, and a detach implementation produces identical gradients without any overhead. All our runs use λ=1\lambda = 1 and a single shared task head. For λ<1\lambda < 1, GC is equivalent to joint training with concept weight λc/(1−λ)\lambda_c/(1-\lambda), so λ\lambda only re-parameterises the joint-training family. We will present GC as a stop-gradient in the final version and keep the GRL only as this interpretation.

We do not claim the operator as the contribution. The contribution, which the reviewer lists as strengths S1–S3, is:

  1. the analysis showing why joint training leaks, through an MI-maximisation term in the task gradient into the encoder;
  2. metrics that detect leakage in concept embeddings;
  3. evidence that removing this one gradient is enough to remove the leakage, in a single training stage, for both CBMs and CEMs.

The new experiments strengthen point 3:

  1. High concept weight. Jointly trained CEMs keep leaking at high concept weight: even at λc=100\lambda_c = 100 they reach 99.9% concept accuracy with RCL 0.195, against 0.003 for GC-CEM (see "joint CBM with near-100% concept accuracy" below).
  2. Dedicated leakage-reducing methods. GC-CBM matches or beats the information-bottleneck model CIBM on leakage and intervenability (Table R1, Fig. R1).
  3. RandInt. With RandInt, CEM's accuracy without interventions collapses (54.8% → 3.3% on CUB), while GC-CEM stays at 69.1–69.4% (Tables R2–R3, Fig. R2).

Value of λ\lambda, and ablation of λ\lambda. λ=1\lambda = 1 in all experiments; we will state this in the main text. We have added a λ\lambda sweep on CUB, λ∈{0,0.2,…,1}\lambda \in \{0, 0.2, \dots, 1\} (Table R8, Fig. R7). For GC-CBM, intermediate λ\lambda raises concept accuracy (70.6% → 88.0%) and intervention AUC (55.2 → 78.0), like a larger concept weight, while CTL stays at the joint level (0.038–0.044); only λ=1\lambda = 1 removes it (CTL 0.024, concept accuracy 94.7%, AUC 87.9). By the equivalence above, intermediate λ\lambda trades leakage for task accuracy exactly as a larger λc\lambda_c does, so the benefit of "allowing some leakage" is that of joint training with a re-weighted concept loss. What only λ=1\lambda = 1 provides is the limit λc→∞\lambda_c \to \infty, which joint training cannot reach.

Sequential CEM baseline. At λ=1\lambda = 1 the GC-CEM encoder is trained exactly as Sequential stage 1, so on each frozen GC-CEM encoder (CUB, 5 folds) we fitted a Sequential CEM head with exactly GC's hyperparameters, and compared the two in a pre-specified test (Table R7, Fig. R6). The results:

  1. Leakage (CTL, ICL, RTL, RCL): identical by construction, since both share the encoder.
  2. Task accuracy: equivalent within ±1\pm 1 pp at every λc\lambda_c (GC minus Sequential: +0.2+0.2, +0.6+0.6, +0.1+0.1 pp).
  3. Interventions: the Sequential head is better by 1.5–2.8 AUC points.
  4. Calibration: the GC head is better calibrated (ECE lower by 0.8–2.8 points).

GC-CEM thus attains Sequential-CEM leakage and accuracy in one end-to-end stage.

Concept-incomplete data. We added a concept-incomplete experiment on the synthetic tabular setting suggested by the reviewer. In TabularToy, y=1{c0+c1+c2≥2}y = \mathbb{1}\{c_0 + c_1 + c_2 \ge 2\}, and we withhold c2c_2 (2 of 3 concepts observed) or c1,c2c_1, c_2 (1 of 3); 3 seeds (Table R6, Fig. R5). Since concepts are binary, the best accuracy attainable from the observed ground-truth concepts is exact: 80.4% on our test set. Any accuracy above it requires the withheld concept to pass through the bottleneck.

  1. Jointly trained models exceed this ceiling by about 20 pp (99.5–99.8% with 2 of 3 concepts), through leakage. Joint CBM encodes the withheld concept in its probabilities: its concept accuracy drops to as low as 49.6%. CEM hides it in its embeddings at 99.8–99.9% concept accuracy: RTL 0.05–0.15.
  2. GC-CBM and GC-CEM stay at the ceiling (79.6–82.3%, within the ceiling's sampling error of about 1.3 pp), with GC-CEM's RTL ≤0.001\le 0.001, like Hard and Sequential CBMs.

Joint CBM with near-100% concept accuracy on TabularToy. We trained joint models at λc∈{5,20,100}\lambda_c \in \{5, 20, 100\} (c2c_2 withheld, 3 seeds; Table R5, Fig. R4).

  1. Scalar CBMs: a high concept weight does suppress leakage. Joint CBM reaches 99.8% concept accuracy and stays at the ceiling, like GC-CBM.
  2. CEMs: it does not. CEM reaches 99.9% concept accuracy yet 93.9% task accuracy (13.5 pp above the ceiling), with RCL 0.17–0.20 that grows with λc\lambda_c.
  3. GC-CEM: RCL 0.003 and RTL 0.001.

High concept accuracy is therefore not evidence of low leakage for embedding models, which is exactly where RTL/RCL are needed.

Q. Both metric sets for CBMs and CEMs. Tables R5 and R6 report CTL and ICL for all models, and RTL and RCL for the CEMs. The two families agree in direction (CEM > GC-CEM), but CTL and ICL on CEM concept probabilities vary far more across seeds: at λc=20\lambda_c = 20, CEM's CTL ranges from 0.02 to 0.27 over 3 seeds, while its RCL stays within 0.14–0.21 across all 9 runs. This is the estimation problem in high dimensions that motivated RTL and RCL.

Q. Why is GC-CBM's CTL below Hard/Sequential CBM, and why is it not zero?

  1. Lower than Sequential. GC-CBM's encoder is trained exactly as Sequential stage 1. Under matched hyperparameters the two therefore have identical CTL. The gap in the paper therefore reflects optimisation differences between separately trained runs, not the training scheme; on CUB, for example, our Sequential CBM used a different optimiser and learning rate. We will report the matched comparison.
  2. Not zero. CTL is a KSG estimate of task information in the soft concept probabilities beyond the ground-truth concepts. A concept-only encoder is not rewarded for carrying task information, but it is not prevented from carrying it either: its probabilities depend on the input, e.g. its confidence varies with the class. KSG also has a positive finite-sample bias. Hard and Sequential CBMs, which receive no task gradient at all, have the same non-zero floor (CTL 0.17 on concept-incomplete TabularToy, Table R6). CTL should therefore be read relative to this floor: joint CBMs reach 0.69–0.73 there.

Q. Why clamp RTL/RCL at zero? A negative held-out R2R^2 means the probe predicts a residual dimension worse than its mean, i.e. finite-sample noise rather than "negative leakage"; the conditional MI that RTL/RCL approximate is non-negative. We clamp so that noise on uninformative dimensions cannot cancel genuine leakage on others when averaging.

Minor. Interpreting RTL/RCL values. After removing the ckc_k-conditional mean, RTLk\mathrm{RTL}_k is the residual variance of concept kk's embedding that the task label explains, per embedding dimension and in units of the embedding's total variance: a variance-explained analogue of I(c^k;Y∣Ck)I(\hat c_k; Y | C_k). RCLkjRCL_{kj} is the analogue of I(c^k;Cj∣Ck)I(\hat c_k; C_j | C_k). For Gaussian residuals, I=−12log⁡(1−R2)I = -\tfrac12 \log(1 - R^2) per dimension. On CUB (Fig. 6), RCL is 0.007–0.008 for CEM and 0.009–0.012 for GC-CEM: about 1% of the embedding variance, i.e. below 0.01 nats per dimension. As the reviewer suggests, this difference is practically negligible: it is an order of magnitude below CEM's RCL on TabularToy (0.09–0.12), and it comes with a 40–51% lower RTL and higher concept accuracy for GC-CEM on CUB. We will add this interpretation.

Clarity questions.

  1. Concept representation in GC-CBM: the sigmoid concept probabilities are passed to the task head. Fig. 1 shows logits, and we will correct it.
  2. Intervention values: interventions set the intervened concepts to their ground-truth binary values (0/1).
  3. Intervention type: all interventions are group interventions (App. J.1); we will state this in the main text.
  4. mm in line 295: yes, mm is the concept-embedding dimensionality.
  5. Epochs in App. J.2: the numbers differ because the models converge at different rates on CUB.

W4 and presentation. We thank the reviewer for the detailed suggestions and will implement all of them in the final version:

  1. larger, decluttered and higher-resolution figures (Fig. 1 enlarged, the wrapfigure in Fig. 2 removed);
  2. markers instead of lines for the discrete xx-axes of Figs. 4 and 7;
  3. the missing reference to Fig. 4, and the missing periods in the captions of Figs. 2, 3 and 9 and Tables 2, 5, 6 and 7.

Wording. We will revise the passages the reviewer points out:

  1. Line 108: we will state that Hard CBMs are trained independently in our experiments, noting that end-to-end hard CBMs exist [2, 5].
  2. Lines 343–346: we will state that GC-CBM matches Sequential CBM in concept accuracy, as Proposition 1 predicts, and outperforms joint CBMs.
  3. Lines 360–362: we will qualify the claim for TabularToy, where CTL drops more while task accuracy is unchanged.
  4. Lines 375–377: we will specify that the intervention gain is relative to the worst-performing Joint CBM.

We are grateful for the reviewer's offer to reconsider the score. We believe the main concerns are now addressed:

  1. GRL: we agree it reduces to a stop-gradient, and the contribution is the analysis, the metrics and the evidence that this gradient is the source of leakage.
  2. Completeness: we added concept-incomplete data, a Sequential CEM baseline, high-concept-weight joint models and a λ\lambda ablation on CUB, plus CIBM, MixCEM and RandInt.
  3. Clarity: we answered each question above.

I thank the authors for their detailed responses. Unfortunately, as the rebuttal was submitted somewhat late, I have only had time to skim through it. Nevertheless, it appears that most of my questions have been addressed and that the added experiments showed positive results. Here are my few remaining concerns:

GRL vs Detach and lambda values

As the authors themselves point out, when λ<1\lambda<1, joint training can achieve similar results. Furthermore, when λ=1\lambda=1, a simple stop-gradient operation in the code would suffice to achieve the same effect. Therefore, I believe the authors should consider removing the GRL entirely. I understand that this would constitute a major change to the proposed method.

I would also suggest the authors discuss why a practitioner might prefer to set λ<1\lambda < 1, if there are any practical reasons for doing so.

Lastly, I believe an ablation study on different values of λ\lambda, in incomplete settings, would have been very informative.

Comments on presentation

I noticed that almost none of the presentation-related comments I raised were addressed. Unfortunately, I still believe that the current presentation of the work falls below the standard expected at ICLR.

Conclusion

Some of the experiments introduced in the ablation studies could have been incorporated into the main paper. ICLR allows an additional page during the rebuttal period, and I would strongly recommend that the authors use this opportunity to highlight the changes they have made in the revised PDF.

I believe this work has the potential to become a seminal contribution to the CBM literature. However, I remain deeply concerned about the presentation of the work and the use of GRL. I am confident that a subsequent version of this work could be significantly stronger.

Finally, I would encourage the authors to place greater emphasis on incomplete settings, where information leakage is an even more significant concern. It would also be valuable to provide a clearer explanation of why λc→∞\lambda_c \to \infty in a CBM is worse compared to GC-CBM.

Reviewer_8RaWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on information leakage in concept-based models. These models map the input to the concept space, and a classifier maps from the concept space to the final classes. The training that this paper considers is joint training with concept-level and class-level classification losses. As the gradients from the class-level loss are transferred to the concept encoding layer, the concept representations can learn task-relevant information that could be different from their associated concepts. This is called information leakage, which hurts the interpretability goal these models promise to achieve. The authors propose an information-theoretic analysis and show that, under task-head convergence, joint training pushes the model to maximize mutual information between concept representations and task labels for better classification performance. This is identified as the main reason for the problem. They propose Gradient-Cancelled Concept-Based Models to cancel task-driven gradients reaching the concept encoder. The authors also introduce two leakage metrics: Residual Task Leakage and Residual Inter concept Leakage. Experiments show that their method generally reduces information leakage while maintaining competitive concept and class classification performance.

Critical strengths
  1. Information-theoretic analysis explaining how joint training can lead to information leakage in the concept space.
  2. Improved intervention performance by reducing information leakage.
Critical weaknesses
  1. The proposed method is the same as training the concept encoder based on concept annotations, detaching the concept encoder, and training only the classifier head using the classification loss to prevent gradients from reaching the concept encoder. This can be achieved using a simple stop-gradient operation. The authors also acknowledge this in the paper. I am not convinced why this approach is needed if a similar method can achieve the same result. The authors mention that one can balance information leakage and classification accuracy by adjusting how strongly the leakage is removed, but I do not think this is a fundamental achievement that motivates the paper.

  2. The information-theoretic claim relies on task-head convergence. How can we determine whether the task head has converged in real-world models? How do the claims and method change if we are near convergence?

  3. Missing comparisons to recent leakage removal methods, such as information bottleneck regularizers [1, 2], kernel-based MI minimization [3], and SupCBM/hierarchical supervision [4].

[1] ConceptsInformation Bottleneck Models. [2] There Was Never a Bottleneck in Concept Bottleneck Models. [3] Towards Faithful Multimodal Concept Bottleneck Models. [4] Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning.

Questions for authors
  1. Please answer the questions I raised in the weaknesses section.

  2. RTL/RCL details: What is the MLP's architecture? Do you use any regularization or early-stopping protocol? Is there any mechanism to avoid overfitting?

3 comments

We thank the reviewer for a clear summary and for recognising the information-theoretic analysis and the improved intervention performance. We respond to each point below; all new results are in App. R of the revised PDF.

GC can be implemented as a stop-gradient. We agree: at λ=1\lambda = 1, GC is equivalent to a stop-gradient on the bottleneck for the task loss, with the concept encoder and the task head trained simultaneously. We see this simplicity as a strength rather than a weakness, and we do not claim the operator itself as the contribution. The contribution is:

  1. A diagnosis. We show why joint training leaks: the task loss contains an MI-maximisation term −∇θI(C^;Y)-\nabla_\theta I(\hat C; Y) that drives the encoder to store task information beyond the concepts. This identifies the task gradient into the encoder as the source to remove.
  2. Metrics that detect leakage existing metrics miss. RTL and RCL measure leakage in concept embeddings, where CTL and ICL are not applicable. They reveal that CEMs leak heavily even with near-perfect concept accuracy (below).
  3. Evidence that removing the source is enough. The simplest intervention implied by the diagnosis matches or outperforms dedicated leakage-reducing methods (Table R1, R9, Figure R1) and is robust where jointly trained models fail:
    1. With intervention-aware training (RandInt) on CUB, CEM's accuracy without interventions collapses from 54.8% to 3.3% as pintp_{\text{int}} grows, while GC-CEM stays at 69.1–69.4% (Tables R2–R3).
    2. Jointly trained CEMs keep leaking at any concept weight: at λc=100\lambda_c = 100 on TabularToy they reach 99.9% concept accuracy with RCL 0.195, against 0.003 for GC-CEM (Table R5, Fig. R4).

GC also applies where a plain stop-gradient pipeline has no clean counterpart, e.g. hybrid CBMs, where the task gradient can be removed from the concept units only while an unsupervised residual is still trained end to end.

Task-head convergence? The mechanism does not rely on task-head convergence; only its interpretation as pure MI maximisation does. For any task head fϕf_\phi, the task loss decomposes exactly as

Ly=H(Y)−I(C^;Y)+E[KL(pθ(y∣c^)∣∣fϕ(y∣c^))]L_y = H(Y) - I(\hat{C};Y) + E[ KL( p_\theta(y|\hat{c}) || f_\phi(y|\hat{c}) ) ]

where pθ(y∣c^)p_\theta(y \mid \hat c) is the conditional induced by the encoder. Since this holds for every θ\theta at fixed ϕ\phi,

∇θLy=−∇θI(C^;Y)+∇θE[KL(pθ(y∣c^)∣∣fϕ(y∣c^))]\nabla_\theta L_y = -\nabla_\theta I(\hat{C};Y) + \nabla_\theta E[ KL( p_\theta(y|\hat{c}) || f_\phi(y|\hat{c}) ) ]

where EE is an expectation.

  1. The MI-maximisation term is present at every step of joint training, not only at convergence.
  2. Convergence is needed only for the second (head-mismatch) term to vanish.
  3. Near convergence, the mismatch term is small and the MI term dominates, so our interpretation holds approximately.
  4. GC removes ∇θLy\nabla_\theta \mathcal{L}_y entirely, hence both terms, at every step. Its effect therefore does not depend on whether, or how closely, the head has converged.

We will make this convergence-free argument explicit in the final version.

Comparison with recent leakage-removal methods. We added the information-bottleneck baseline and MixCEM, run with the authors' official code on CUB with our concept-loss weights λc∈{0.1,0.5,1}\lambda_c \in \{0.1, 0.5, 1\} and 5 seeds.

CIBM (fine-tuned ResNet-18; Table R1, Fig. R1). GC-CBM leaks less than CIBM at λc=0.1\lambda_c = 0.1 (CTL 0.024 vs. 0.033, p<10−7p < 10^{-7}) and as little at higher λc\lambda_c. It is clearly more intervenable at every λc\lambda_c:

λc\lambda_cModelTask acc.CTL ↓\downarrowAcc., 50% groups intervenedIntervention AUC
0.1CIBM68.00.03386.683.9
GC-CBM65.90.02492.388.0
1CIBM66.40.02889.685.6
GC-CBM66.50.02692.088.1

Bold: significantly better (Welch, p<0.05p < 0.05). CIBM's accuracy even drops when its first 10–15% of concept groups are corrected.

MixCEM (Espinosa Zarlenga et al., 2025), a recent leakage-aware CEM. MixCEM, which is designed to stop leakage from poisoning interventions, also brings embedding leakage on CUB down to GC-CEM's level (Table R9): RTL 0.19–0.31 against 0.20–0.28 for GC-CEM and 0.42–0.47 for CEM, with similar CTL (0.03–0.04) for all three and the lowest RCL (0.005–0.008). GC-CEM reaches this level with no change to the architecture and with higher concept accuracy (94.8–95.5% vs. 88.6–93.2%, p<0.001p < 0.001). MixCEM's dropout of its input-dependent context makes it more intervenable (AUC 93.9–95.5 vs. 90.0–90.4).

RTL/RCL implementation details. For each concept kk, the embedding is normalised (global mean and standard deviation, fitted on the probe training set). Its own-concept signal is removed with a ridge regression ck→c_k \to embedding (α=1\alpha = 1), and the residual rkr_k is probed from the one-hot task label (RTL) or from each other concept cjc_j (RCL).

  1. MLP probe: scikit-learn MLPRegressor, one hidden layer of 64 ReLU units, Adam, standardised inputs, up to 500 iterations.
  2. Regularisation and early stopping: L2 penalty α=10−3\alpha = 10^{-3}; early stopping on a 15% validation split (patience 20).
  3. Overfitting : probes are trained and evaluated on data disjoint from the model's training data (the model's validation and test predictions, pooled and split with a fixed seed); R2R^2 is always computed on the held-out probe test set. Per-dimension R2R^2 is clipped at zero, so an overfitted probe (negative held-out R2R^2) can only lower RTL/RCL, never raise them.

We hope these clarifications and the new baselines address the reviewer's concerns.

Reviewer_8RaWOpenReview ↗

Response to Rebuttal

Thank you for the rebuttal and for the clarifications and new experiments. After reading the response and the other reviews, I have decided to keep my score.

My main concern remains the motivation for the method. It is still not clear to me why we should train jointly in the first place, and what GC brings that makes removing leakage from joint training worthwhile. The paper lacks evidence that motivates joint training over disjoint training: in its own results (Table 3), Joint CBM matches Sequential CBM on TabularToy and AwA2 and is better only on CUB. Disjoint (independent or sequential) training is a simple and intuitive baseline: by construction it does not cause task-driven leakage, and it already performs well.

Reviewer_PX2rOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a training strategy for concept-based models that mitigates the well-known problem of task leakage in concept representations. The method (partially or fully) cancels the task gradient received by the encoder, according to a parameter λ\lambda, through a gradient reversal layer. The idea is that the higher the cancellation of the task signal, the less leakage is encoded in the concept representation, up to becoming a variant of sequential training when λ=1\lambda=1, which completely prevents the task-loss gradient from flowing to the concept encoder. Since it is a modification of the training procedure, it is applicable to any concept-based model learning concept representations through a gradient (thus limited only for language-guided schemes, to the best of my understanding). Results on three datasets show that the gradient-cancelling variants improve the performance of the jointly trained baselines (CBM and CEM) when intervening on the concepts, although slightly limiting the performance on clean data without interventions on CUB.

Critical strengths
  • Simple approach to reduce leakage: the paper proposes a simple modification to the joint learning policy that considerably reduces the task leakage for concept-based models like CBM and CEM, with a single added hyperparameter λ\lambda and little downside on the standard performance (the clean accuracy decreases only on CUB, by a few points).
  • Two metrics to compute leakage: the paper proposes two metrics to compute task and inter-concept leakage in concept representations, which seem to be stable with respect to the concept representation dimensions (as they work also on CEM) and on synthetic data they are monotonic with respect to the injected leakage.
Critical weaknesses
  • Limited evaluation: the paper evaluates the proposed method on two real-world datasets against two standard concept-based models only: among the many methods already existing for limiting task leakage in concept-based representations (some of which also reported by the same authors in the related work), none is considered in the experimental evaluation. The main hyperparameter λ\lambda controlling the behavior of the proposed approach is also only tested on synthetic data, with no ablation reported on the main datasets.
  • Limited contribution and soundness: the proposed approach at λ=1\lambda=1 is equivalent to a standard bottleneck with a detached concept representation (as also noted by the same authors). In practice it is a sequential CBM trained in a single cycle with the task head trained at the same time as the encoder. The λ\lambda used in the experiments is not reported, but is likely λ=1\lambda=1. The experimental results are indeed very similar to Sequential, particularly on CUB, 98.1 against 96.6 with full intervention, and almost identical task and concept clean accuracy. On AwA2 I guess that the difference in the reported performance is more likely attributed to the different hyperparameters than to a different training strategy (where both epochs and learning rate differ).
Questions for authors
  1. Can the authors report the λ\lambda value employed in the experiments? Can they also provide an ablation on this hyperparameter on at least one real dataset?
  2. Can you repeat the Sequential experiments with the same hyperparameters (epochs and learning rate) used by GC?
  3. Can you provide some comparison against the methods that you also cite in Sec. 2, e.g., Galliamov et al. 2026, Almudévar et al. 2026 but also MixCEM [1]?
  4. Does CEM use Random Intervention in training as proposed in the original paper to improve intervention responsiveness?
  5. For λ∈(0,1)\lambda \in (0,1), the encoder gradient of Eq. 3 is (1−λ)[∇θLy+λc1−λ∇θLc](1-\lambda)\left[\nabla_\theta L_y + \frac{\lambda_c}{1-\lambda}\nabla_\theta L_c\right]. Is it not the same as a joint model with concept weight λc/(1−λ)\lambda_c/(1-\lambda) and a rescaled encoder learning rate?

[1] Espinosa Zarlenga, et al. 2025. "Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts." In Proceedings of the 42nd International Conference on Machine Learning, 15564–15595. PMLR.

2 comments

We thank the reviewer for a detailed review and for recognising the simplicity of the approach and the value of the two leakage metrics. We ran every requested experiment; all new figures and tables are in App. R of the revised PDF. In short: GC reaches or beats the leakage of recent leakage-reducing methods (CIBM, MixCEM) with higher concept accuracy and no change to the architecture, stays robust where jointly trained CEMs collapse (RandInt), and removes embedding leakage that no concept weight removes in joint training.

Comparison with leakage-reducing methods. We added the requested baselines, run with the authors' official code.

CIBM (Galliamov et al., 2026). CUB, fine-tuned ResNet-18, the authors' training settings, our λc\lambda_c, 5 seeds (Table R1, Fig. R1):

λc\lambda_cModelTask acc.CTL ↓\downarrowAcc., 50% groups intervenedIntervention AUC
0.1CIBM68.00.03386.683.9
GC-CBM65.90.02492.388.0
0.5CIBM67.30.02489.485.6
GC-CBM66.10.02393.488.8
1CIBM66.40.02889.685.6
GC-CBM66.50.02692.088.1

Bold: significantly better (Welch, 5 vs. 5, p<0.05p < 0.05). GC-CBM leaks less than CIBM at λc=0.1\lambda_c = 0.1 (p<10−7p < 10^{-7}) and as little at higher λc\lambda_c. It is more intervenable at every λc\lambda_c: CIBM's accuracy drops when its first 10–15% of concept groups are corrected (Fig. R1), while GC-CBM's improves from the first one.

MixCEM (Espinosa Zarlenga et al., 2025). Official code with the paper's CUB hyperparameters, our λc\lambda_c, 5 seeds. MixCEM, which is designed to stop leakage from poisoning interventions, also brings embedding leakage on CUB down to GC-CEM's level (Table R9): RTL 0.19–0.31 against 0.20–0.28 for GC-CEM and 0.42–0.47 for CEM, with similar CTL (0.03–0.04) for all three and the lowest RCL (0.005–0.008). GC-CEM reaches this level with no change to the architecture and with higher concept accuracy (94.8–95.5% vs. 88.6–93.2%, p<0.001p < 0.001). MixCEM's dropout of its input-dependent context makes it more intervenable (AUC 93.9–95.5 vs. 90.0–90.4). The two are complementary: GC only changes which gradient reaches the concept encoder, so it can be applied to MixCEM's encoder as well.

RandInt. The paper's CEMs used pint=0p_{\text{int}} = 0. With RandInt on CUB, jointly trained CEM collapses and GC-CEM does not (Tables R2–R3, Fig. R2):

CEMGC-CEM
Task acc., pint=0.25/0.5/0.75p_{\text{int}} = 0.25 / 0.5 / 0.75 (1 fold)54.8 / 12.5 / 3.369.1 / 69.4 / 69.1
Task acc., pint=0.25p_{\text{int}} = 0.25, 5 folds, λc=0.1/0.5/1\lambda_c = 0.1 / 0.5 / 156.0 / 62.5 / 66.169.4 / 69.7 / 70.6
Concept acc., same runs85.7 / 90.2 / 91.994.8 / 95.3 / 95.5

All 5-fold differences are significant (p≤0.008p \le 0.008). With RandInt, the task loss on non-intervened samples pushes a jointly trained CEM's embeddings to carry the information that intervened samples withhold. Removing the task gradient from the concept encoder blocks this.

GC-CEM is also far more robust to incorrect interventions (Table R4, Fig. R3). Setting one correctly predicted concept to a wrong value, which leaves the image's own class as the closest match, changes CEM's prediction in 38–66% of cases but GC-CEM's in only 3.8–4.0%. Both models still follow 89–100% of warranted edits to another class's full concept signature. The RandInt CEM thus follows any edit: having learned to rely on the intervened values (its accuracy without interventions falls to as low as 3.3%, Table R3), it no longer weighs an edit against the rest of its concept evidence. GC-CEM's concept representations carry no task information beyond the concepts, so its prediction changes only when the edited concepts as a whole point to another class. Its interventions are faithful: a change in the prediction reflects a change in the concepts.

Response to Reviewer (2/2)

The role of λ\lambda. The reviewer's derivation is correct, and it sharpens our contribution. For λ<1\lambda < 1, GC is equivalent to joint training with concept weight λc/(1−λ)\lambda_c/(1-\lambda). The method's essential setting is therefore the limit λ=1\lambda = 1, which corresponds to λc→∞\lambda_c \to \infty and cannot be reached by joint training. All our experiments use λ=1\lambda = 1; we will state this explicitly in the final version.

A new experiment shows that this limit is necessary. On TabularToy, jointly trained CEMs with a very large concept weight reach near-perfect concept accuracy yet still leak; GC-CEM at the same λc\lambda_c does not (Table R5, Fig. R4; c2c_2 withheld, concept-only accuracy ceiling 80.4%, 3 seeds):

λc\lambda_cModelTask acc.Concept acc.RTL ↓\downarrowRCL ↓\downarrow
5CEM93.999.90.0510.167
GC-CEM81.999.60.0010.004
20CEM89.699.90.0500.192
GC-CEM81.599.70.0010.003
100CEM80.399.90.0500.195
GC-CEM81.499.60.0010.003

CEM exceeds the ceiling by up to 13.5 pp, which is possible only through leakage, and its RCL grows with λc\lambda_c. With all concepts observed, CEM's RCL is still 0.065–0.107 against 0.002 for GC-CEM. High concept accuracy is not evidence of low leakage, and no concept weight removes it. The CUB ablation over λ∈{0,0.2,…,1}\lambda \in \{0, 0.2, \dots, 1\} confirms this on real data (Table R8, Fig. R7; λc=0.1\lambda_c = 0.1, one fold). For GC-CBM, increasing λ\lambda acts like a larger concept weight: concept accuracy rises from 70.6% (λ=0\lambda = 0, joint) to 88.0% (λ=0.8\lambda = 0.8) and intervention AUC from 55.2 to 78.0, but CTL stays at the joint level (0.038–0.044). Only at λ=1\lambda = 1 does CTL fall, to 0.024, with 94.7% concept accuracy and AUC 87.9; the 5 pp of task accuracy given up is the accuracy that leakage provided. As the equivalence predicts, λ=0.8\lambda = 0.8 (effective concept weight 0.5) closely matches a joint CBM trained with λc=0.5\lambda_c = 0.5 (concept accuracy 88.0% vs. 86.3%, CTL 0.038 vs. 0.045).

Relation to Sequential training. We agree that at λ=1\lambda = 1 the GC encoder is trained as Sequential stage 1. This is precisely what gives GC Sequential-level leakage, which GC obtains in a single end-to-end stage. As requested, on each frozen GC encoder we fitted a Sequential head with exactly GC's hyperparameters (CUB, 5 folds, 3 λc\lambda_c) and ran a pre-specified test (Table R7, Fig. R6):

  1. task accuracy is equivalent within ±1\pm 1 pp in all six settings, and leakage is identical by construction;
  2. the heads differ modestly: the Sequential head gains 1–3 AUC points under interventions, and GC-CEM's head is better calibrated (ECE lower by 0.8–2.8 points).

On AwA2 the reviewer is right that the reported gap came from encoder hyperparameters: a matched Sequential head equals GC (85.19% vs. 85.19%). We will correct this comparison.

GC also extends to architectures without a clean Sequential counterpart. In a hybrid CBM, whose bottleneck adds unsupervised residual units to the concept units, Sequential training has no stage in which the residual can be learned without changing the concepts. GC can remove the task gradient from the concept units only, keeping the residual as an explicit, end-to-end-trained side channel.

Changes for the final version (all new results are already in App. R):

  1. λ=1\lambda = 1 stated, plus the CUB ablation and the λ<1\lambda < 1 equivalence;
  2. the CIBM, MixCEM and RandInt baselines, including incorrect interventions;
  3. the high-λc\lambda_c experiment;
  4. the matched-Sequential analysis and the corrected AwA2 comparison.

We hope these results show that GC is a simple, single-stage method that matches or exceeds recent baselines on leakage and intervenability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.