acceptodds
Under review as a conference paper at ICLR 2027

Harmful Partial Interventions and Benign Leakage in Concept-based Models

Abstract

Artificial intelligence systems deployed in high-risk settings increasingly require models that support meaningful human oversight. Concept-based models (CMs) offer a promising framework by enabling users to inspect and intervene on intermediate, human-understandable concepts. However, concept interventions can counterintuitively worsen downstream predictions, a failure mode we formalise as intervention harm. We distinguish harm after all concepts have been intervened on from harm under partial interventions, the more likely practical use case. Modern CMs often achieve high accuracy by encoding, or leaking, additional task-relevant information in their intermediate concept representations. Recent work argues that such leakage can be benign with respect to intervention harm when concept representations satisfy two conditions: sufficiency and localisation. We prove that these criteria are not sufficient to guarantee that concept interventions improve downstream performance. Across controlled and real-world settings, we find that concept supervision incompleteness associates with increased intervention harm. Importantly, optimising the model when all concepts are intervened on, a previously proposed leakage mitigation objective, reduces endpoint harm but does not reliably prevent harmful partial corrections. Hence, we introduce a complementary monotonic intervention objective that directly penalises increases in task loss along partial intervention trajectories. Combining endpoint and monotonic intervention training substantially reduces the prevalence and magnitude of partial intervention harm while preserving concept and task predictive performance.

Primary area:
interpretability and explainable AI
Keywords:
Concept based models, leakage, interventions, human-in-the-loop

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PqbPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why correcting some of a model's predicted concepts can worsen its final predictions, even when correcting all concepts helps. A theoretical example shows that corrections can be harmful even when the task predictor is optimal and correction does not introduce unseen inputs. The authors add a training penalty for increases in prediction loss as concepts are corrected one by one. In synthetic tasks and bird classification, combining this penalty with training on fully corrected concepts reduces the frequency and severity of harmful partial corrections while largely preserving uncorrected accuracy.

Critical strengths
  • The counterexample shows that corrections can hurt even with an optimal task predictor and no change in its input distribution. It isolates a failure that cannot be blamed on poor training or unfamiliar inputs.

  • The experiments make a strong case for evaluating partial corrections, not just full correction. On CUB, the proposed penalty reduces harm during partial correction while largely preserving uncorrected accuracy and the benefit of full correction.

Critical weaknesses
  • The experiments do not separate the penalty's benefit from the benefit of training on partial corrections. IntCEM includes a variant trained on random correction paths without a learned sampling policy. The real-data baselines here receive no comparable training on partially corrected states. A control that directly minimizes task loss at the same sampled states would establish what the pairwise penalty adds. It should use the same architecture, block the same gradients through concepts, and match computation. Linear task predictors and a single real dataset with frozen features also limit how broadly the results apply.

  • Lower reported harm does not establish lower absolute loss after correction. Harm is measured against each model's own uncorrected loss. The penalty can decrease either by lowering loss after a correction or by raising loss before it. The task and full-correction objectives discourage the latter, but nearly unchanged accuracy does not rule out changes in cross-entropy. Absolute loss and accuracy curves across correction budgets would clarify whether the reduction in harm preserves useful correction.

  • The related-work discussion omits Disentangled Concept-Residual Models (Zabounidis et al., TMLR 2026). D-CRM studies incomplete concept sets and uses mutual-information minimization to reduce concept-residual overlap and improve intervention effectiveness, making it directly relevant to this paper's setting. Its main positive-intervention metric replaces all concepts simultaneously, so it does not resolve harm along partial-correction paths. Discussing this distinction and comparing MI-based disentanglement with the endpoint and monotonic losses under the same architecture and partial-correction protocol would clarify what the proposed objective adds.

Questions for authors
  1. How does the penalty compare with direct task-loss training on the same sampled partial corrections, using the same architecture, blocked concept gradients, and computation?
  2. What are the absolute cross-entropy and accuracy curves as more concepts are corrected, including the uncorrected and fully corrected states, for each training regime?
  3. How does partial-correction harm compare between D-CRM's mutual-information regularization, the proposed penalty, and their combination under the same architecture and correction paths?
2 comments
AuthorsOpenReview ↗

Response to Reviewer PqbP

Thank you for taking the time to read and carefully assess our manuscript. We appreciate your positive comments on our counterexample proof and on the importance of evaluating partial corrections. Your thoughtful suggestions have helped us strengthen the experimental comparisons and clarify how the monotonic loss affects performance across correction budgets.

Below we address all concerns and questions raised in this review. If you have further questions or anything remains unclear, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score in light of our replies.

1. Separating the penalty’s benefit from training on partial corrections

Thank you for suggesting this important control, which we now include. As detailed in point 2 of our global reply, we compared the monotonic penalty with a matched plain task-loss baseline evaluated at the same sampled correction states. The monotonic penalty reduced average maximum partial harm and harmful path prevalence in synthetic data and CUB more than the plain task-loss. The experimental details and results are reported in Appendix C.1.7 (synthetic) and Appendix C.2.7 (CUB) of the revised manuscript.

2. Comparison with IntCEM

Thank you for pointing out that evaluations against IntCEM may be worth exploring. Please see our answer to concern 1 in our general reply for details on this point. There we explain how our new results in Section 5.3 show that Lint\mathcal{L}_{\mathrm{int}} and Lmono\mathcal{L}_{\mathrm{mono}} together reduce harmful paths and interventions for CEM more so than what the original IntCEM objective does for CIFAR10, AwA2, and our CUB variants. These results therefore strongly suggest that Lmono\mathcal{L}_{\mathrm{mono}} provides additional harm-reducing benefits that are not induced by IntCEM’s training pipeline.

3. Absolute loss and accuracy curves across correction budgets

We thank this reviewer for raising this point and provide a detailed response in point 4 of our global reply. Briefly, adding Lmono\mathcal{L}_{\mathrm{mono}} preserved or improved Intervention AUC for CEMs and yielded similar Intervention AUC for Hybrid CBMs across three real world datasets (Table 1 of the revised manuscript). We also provide the requested absolute cross-entropy and top-1 accuracy curves for several incomplete CUB settings in Appendix C.2.8 of the revised manuscript. At λmono=10\lambda_{\mathrm{mono}}=10, monotonic training reduced partial harm, with slightly higher cross-entropy before correction, comparable cross-entropy at full correction, and slightly higher top-1 accuracy from 15/50 corrections onwards.

4. Comparison with D-CRM’s mutual-information regularisation

Thank you for suggesting this comparison. We evaluated D-CRM’s mutual-information penalty on CUB using the same Hybrid CBM architecture, concept sets, and 512 correction orderings (Appendix C.2.5 of the revised manuscript). Adding the penalty to endpoint-only training reduced concept predictability from the residual (probe F1 fell from 0.189 to 0.157), but increased harmful-path prevalence from 91.2% to 98.2% and maximum partial harm from 0.0054 to 0.0072.

Adding the same penalty to endpoint-plus-monotonic training reduced harmful-path prevalence from 51.3% to 48.1%, with a marginal reduction in  maximum partial harm and a 0.43 percentage point drop in accuracy.

These results suggest that D-CRM’s mutual-information penalty may complement Lmono\mathcal{L}_{\mathrm{mono}} by reducing concept–residual overlap and further lowering the prevalence of harmful correction paths, with a modest accuracy cost in our CUB experiments.

AuthorsOpenReview ↗

Follow-up before the discussion period ends

Dear Reviewer PqbP

Thank you once more for all your feedback! As the discussion period is coming to an end, we wanted to ask whether you have any further questions? If so, we would be more than happy to provide further clarifications while the discussion period is open. Otherwise, if our rebuttal addressed your main concerns and questions, we would sincerely appreciate it if you could let us know. Thank you in advance!

Reviewer_mJjxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses intervention in concept based models that allow information leakage. Recent work suggested that benign leakage can help models preserve concept and final accuracies while not negatively affecting intervention effectiveness. The paper observed that the criteria for benign leakage suggested in the previous work is incomplete when considering partial concept intervention scenario and suggests monotonicity as additional criteria for minimizing partial intervention harm. The monotonic intervention loss, inspired by the monotonicity criteria, when used along with previously suggested intervention loss for training concept-based models, is shown to mitigate partial intervention harm.

Critical strengths
  1. Intervention is a crucial aspect of concept-based models, and analyzing the affect of information leakage on intervention and whether potential intervention harm can be mitigated is an important topic. The paper theoretically and empirically shows that partial intervention harm can occur even when the two notions of benign leakage proposed in previous work are satisfied. The monotonic intervention loss for mitigating intervention harm, motivated by the monotonicity criteria, seems to be principled and effective method for addressing intervention harm.
Critical weaknesses
  1. Currently, the paper lacks thorough evaluation. The results on Hybrid CBM for CUB dataset looks good, but further experiments on various architectures that allow leakage (like ), and datasets typically used in literature (like AwA2, CIFAR10, SUN) would further validate the effectiveness of monotone intervention loss.
Questions for authors
  1. [1] defines concept representation exhibiting benign leakage as satisfying sufficiency and localization criteria. As the paper suggested that such definition can be incomplete in the partial intervention scenario, how would the paper formally define benign leakage?
  2. While I think Theorem 1 is meaningful result, it does seem to only consider the simplest case where there is only one concept. Do you think the theorem can be expanded to analyze the general partial intervention scenario?

[1] Espinosa Zarlenga, In Defense of Information Leakage in Concept-based Models, ICML 2026.

3 comments
AuthorsOpenReview ↗

Response to Reviewer mJjx

Thank you for taking the time to carefully assess and read our manuscript. We appreciate your positive assessment of the monotonic loss as a principled and effective way to reduce partial intervention harm, as well as your thoughtful questions about the scope of our experiments and theory.

Below we address all concerns and questions raised in this review. If you have further questions or anything remains unclear, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score in light of our replies.

1. Evaluation on further datasets and using further leakage-permitting architectures.

Thank you for raising this point! Please see our answer to concern 1 in our general reply for details on this. There we explain how our new results in Section 5.3 show that Lmono\mathcal{L}_{\mathrm{mono}} reduces harmful paths and interventions for Concept Embedding Models (CEMs), a popular family of architectures that enable and exploit leakage by construction, without jeopardizing other properties of this model users care about (e.g., unintervened accuracy, intervenability, concept accuracy).

2. Extension of the theory to multiple concepts and partial harm

We thank the reviewer for suggesting this extension to our theory. As detailed in point 3 of our global reply, the revised Theorem 1 (Appendix A) now establishes arbitrarily large harm from a strictly partial correction under exact localisation and arbitrarily approximate sufficiency.

3. Updated definition of benign leakage

We agree with the reviewer that this is a great opportunity to re-define benign leakage. As we mention in the Discussion, the conditions of sufficiency and localisation pertain to the representation of information without any reference to intervenability. On the other hand, in the present paper, we focus on the behavioural aspect of benign leakage, which is the lack of intervention harm under partial concept correction. We use this behavioural definition to motivate Lmono\mathcal{L}_{\mathrm{mono}}. However, it is an interesting and open question whether some set of purely representational conditions could guarantee the lack of intervention harm in all cases.

To formalise and prove such conditions is an exciting challenge, but outside the scope of this work. However, we discuss what we consider to be an interesting avenue in the Discussion paragraph on out-of-distribution concept states. Concept corrections may shift the joint distribution of Y, Z, and C^\hat{C}, rendering a frozen task head inaccurate. In the partial intervention case, we may have a distinct distribution shift for each subset of corrected concepts.

A new representational definition of benign leakage could therefore focus on these distribution shifts - more specifically, whether P(Y∣Z,C^)P(Y \mid Z,\hat{C}) changes as a result of correction, which could degrade the performance of a task head trained solely in the uncorrected regime.

We thank the reviewer for raising this point, and have extended our section on Limitations/future work to include this perspective in the revised manuscript.

Thank you for the detailed response.

I appreciate the additional experiments, which seem to address the weakness concerning lack of thorough evaluation. On Theorem 1, I appreciate the extension to the partial intervention scenario, which makes the theoretical result more realistic.

I appreciate the authors added discussion about the formal definition of benign leakage in the revised version, but I do think the lack of a formal revised definition is a weakness of the paper.

Overall, my concerns, especially the major one on experiments, are mostly addressed and will keep my score (weak acceptance).

Dear Reviewer mJjx,

Thank you for taking the time to engage in this discussion and for your encouraging feedback! We are glad you appreciated our new additions and that we addressed your major concerns. If further concerns arise before the discussion period ends, please do not hesitate to let us know.

Reviewer_FNCPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies intervention harm in leaky concept-based models, where correcting a concept prediction worsens the task prediction. It separates endpoint harm (all concepts corrected) from partial harm (a subset corrected). Experiments on synthetic linear-Gaussian data and on CUB with 80 concept sets of varying incompleteness show three things. First, more incomplete concept sets go with more of both kinds of harm. Second, the existing endpoint objective L_int removes endpoint harm but leaves partial harm on most intervention paths. Third, the proposed L_mono, a hinge penalty on loss increases along sampled intervention paths, combined with L_int, sharply reduces partial harm with little cost to task or concept accuracy.

Critical strengths
  1. The studied problem is important and interesting.
  2. The key ideas are formalized with strong theoretical guarantees.
Critical weaknesses
  1. Experiments: i. the paper shows main results on CUB and synthetic data only. It is unclear whether the findings hold for other datasets, for nonlinear task heads, or for other leaky architectures such as CEMs. ii. Missing baselines. The most natural alternative to the hinge is to add the plain task loss at randomly sampled partial-intervention states. An ablation is needed to show that the hinge structure (penalising increases) matters beyond partial-state data augmentation. (Espinosa Zarlenga et al., 2023b) should also be compared.

  2. The connection of presented theory with the experiments is unclear. The theoretical result concerns a single concept, so it covers only endpoint harm, while the paper’s focus is partial harm.

Questions for authors
  • How does L_mono affect the benefit of interventions, not just their harm?
  • How sensitive are the conclusions to the harm threshold and to label noise in the concept annotations? This may need additional evaluation.
2 comments
AuthorsOpenReview ↗

Response to Reviewer FNCP

Thank you for taking the time to carefully assess and read our manuscript. We are glad to receive your positive assessment of the theoretical guarantees, as well as your thoughtful questions about further experiments and theoretical work, and we are encouraged that you found that this work addresses an “important and interesting” problem.

Below we address all concerns and questions raised in this review. If you have further questions or anything remains unclear, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score in light of our replies.

1. Experiments on wider datasets and baselines

Thank you for raising this point! Please see our answer to concern 1 in our general reply to all reviewers. There we explain how our new results in Section 5.3 show that Lmono\mathcal{L}_{\mathrm{mono}} reduces harmful paths and interventions across datasets and also for different, more leaky and non-linear baselines (such as CEMs and IntCEMs).

2. Plain task-loss baseline

Thank you for suggesting this comparison. As detailed in point 2 of our global reply, we compared the monotonic penalty with a matched plain task-loss baseline evaluated at the same sampled correction states. The monotonic penalty reduced average maximum partial harm and harmful path prevalence in synthetic data and CUB more than the plain task-loss. The experimental details and results are reported in Appendix C.1.7 (synthetic) and Appendix C.2.7 (CUB) of the revised manuscript.

3. Extension of the theory to multiple concepts and partial harm

Thank you for this important suggestion. As detailed in point 3 of our global reply, the revised Theorem 1 (proof in Appendix A) shows that arbitrarily accurate satisfaction of the sufficiency criterion, together with exact localisation places no finite upper bound on the harm caused by strictly partial ground-truth concept corrections.

4. How does the monotonic objective affect the benefit of interventions?

Thank you for this important question, which we address in detail in point 4 of our global reply. Briefly adding Lmono\mathcal{L}_{\mathrm{mono}} preserved or improved Intervention AUC (a measure of the benefit of interventions) across 3 real world datasets and various CM architectures, while Greedy Harm AUC (a measure of partial harm) was lower or comparable (Table 1 of the revised manuscript).

5. Sensitivity to the harm threshold and label noise in concepts

We thank the reviewer for raising these questions. Appendix E of the revised manuscript reports both of these sensitivity analyses on synthetic data and CUB, comparing endpoint-only training with endpoint-plus-monotonic training at λmono=10\lambda_{\mathrm{mono}}=10.

Harm threshold. We count a path as harmful if its mean task loss at any intermediate state exceeds its uncorrected mean loss by more than the harm threshold τ\tau. The threshold is used only for evaluation of harmful-path prevalence. We rescored the same paths from the trained models at six thresholds from 00 to 10−310^{-3} and found lower mean harmful-path prevalence with monotonic training at every threshold on both synthetic and CUB datasets. At the default tolerance of 10−810^{-8}, prevalence fell from 44.7% to 41.8% on synthetic data and from 57.6% to 49.6% on CUB. At 10−510^{-5}, it fell from 42.3% to 0.0% and from 56.9% to 48.6%, respectively. The choice of harm threshold within this range did not qualitatively impact our results.

Noise in concept labels. Our CUB models were trained on image-level concept annotations, which are known to be naturally noisy [1]. To assess how models respond when the concept corrections themselves are noisy, we held the training annotations fixed and added zero-mean Gaussian noise to the concept values supplied at test time. We tested five noise levels, with standard deviations of 0, 0.1, 0.25, 0.5 and 1 times each concept’s training-set standard deviation. Monotonic training reduced mean maximum partial harm at every tested noise level on both synthetic and CUB datasets. At the highest noise level, harm fell from 13.349×10−313.349 \times 10^{-3} to 0.360×10−30.360 \times 10^{-3} on synthetic data and from 7.502×10−37.502 \times 10^{-3} to 4.433×10−34.433 \times 10^{-3} on CUB.

[1] Koh et al., "Concept Bottleneck Models". Proceedings of the 37th International Conference on Machine Learning, PMLR 119, 5338–5348, (2020).

AuthorsOpenReview ↗

Follow-up before the discussion period ends

Dear Reviewer FNCP,

Thank you once more for all your feedback! As the discussion period is coming to an end, we wanted to ask whether you have any further questions? If so, we would be more than happy to provide further clarifications while the discussion period is open. Otherwise, if our rebuttal addressed your main concerns and questions, we would sincerely appreciate it if you could let us know. Thank you in advance!

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Reply (to all reviewers) - Part 1

We thank all reviewers for their consideration and for taking the time to carefully read our manuscript and provide helpful feedback. We believe your feedback significantly improved our work and we thank you for that.

To simplify the rebuttal process, in this global reply we will include answers to questions or concerns that were shared by two or more reviewers (each concern tagged with the IDs of the reviewers that raised that point). Then, in replies posted to individual reviewers, we will address concerns or questions that were not shared by others.

We have updated our manuscript accordingly to include all new results, discussions, and clarifications presented here and in the individual replies. To make our results easier to find in the paper, we have coloured them in blue.

1. Experiments on wider datasets and baselines (FNCP, mJjx, PqbP)

As suggested by all reviewers, we expanded our evaluation to include new datasets (CIFAR10 and an incomplete version of AwA2) and new baselines (namely very leaky models such as CEMs [1] and models that already include some intervention-aware losses, such as IntCEMs [2]). We chose CIFAR10 as an example of a dataset where concept labels are obtained using a label-free pipeline. In contrast, we chose AwA2, specifically the incomplete version used in [3], as an example of an incomplete real-world dataset with class-level concept annotations. For both datasets and the new baselines, we followed the same training and setup configurations as in [3].

These new results are shown in the newly added Section 5.3 of the updated manuscript. There, specifically in Table 1, we show for all models: (1) the task accuracy, (2) the mean concept ROC-AUC (Concept AUC), (3) the area under the task-accuracy-vs-concepts-intervened curve (Int. AUC), (4) the estimated fraction of intervention steps that increased the model’s loss (Harmful Steps), and (5) the cumulative harm the worst greedy intervention path can cause on a model’s loss (Greedy Harm AUC). We computed the latter metric by estimating the area under the loss-vs-interventions curve centred at the loss for the unintervened model, where concepts are selected by greedily choosing the concept that yields the largest loss increase at each step. As such, this estimates how much a model’s test loss may change when one intervenes along the worst possible greedy path (therefore, lower, negative values are better).

Our results in Table 1 show that (1) training both Hybrid-CBMs and CEMs with Lint\mathcal{L}_\mathrm{int} and Lmono\mathcal{L}_\mathrm{mono} leads to a reduction in the fraction of harmful steps as well as a reduction of the harm caused by the worst greedy path compared to the same baselines trained only with Lint\mathcal{L}_\mathrm{int} or without either regularizer, (2) introducing Lmono\mathcal{L}_\mathrm{mono} does not come with significant sacrifices to key properties of CMs (e.g., intervenability, accuracy, interpretability), (3) Lmono\mathcal{L}_\mathrm{mono} can reduce the presence of intervention harm for different leaky architectures (e.g., CEMs), and (4) previous intervention-aware losses such as those used by IntCEM still lead to models that have worse adversarial harmful paths or higher fraction of harmful steps than CEMs trained with both Lmono\mathcal{L}_\mathrm{mono} and Lint\mathcal{L}_\mathrm{int}.

[1] Espinosa Zarlenga et al. "Concept embedding models: Beyond the accuracy-explainability trade-off." NeurIPS (2022).

[2] Espinosa Zarlenga et al. "Learning to receive help: Intervention-aware concept embedding models." NeurIPS (2023).

[3] Espinosa Zarlenga. "In Defense of Information Leakage in Concept-based Models." arXiv ICML (2026).

AuthorsOpenReview ↗

Global Reply (to all reviewers) - Part 2

2. Matched plain task-loss control (FNCP, PqbP)

We thank Reviewers FNCP and PqbP for suggesting this control. We compared Lmono\mathcal{L}_{\mathrm{mono}} with a plain task loss, denoted Lplain\mathcal{L}_{\mathrm{plain}}, evaluated at the same sampled correction states. For each sampled step, Lplain\mathcal{L}_{\mathrm{plain}} averages the task losses before and after the correction, while Lmono\mathcal{L}_{\mathrm{mono}} penalises loss increases between those two states. Both arms retained the endpoint objective and used the same architecture, initialisation, minibatch order, concept gradient blocking, optimiser steps, and number of task-head evaluations.

We evaluated both objectives on synthetic data and CUB at matched auxiliary weights of 1, 10, and 100, in line with the conditions used in the manuscript, and we fixed λint=10\lambda_{\mathrm{int}}=10 for synthetic data and λint=1\lambda_{\mathrm{int}}=1 for CUB. For the synthetic comparison, we used five worlds with five supervised and five unsupervised concepts (ku=kk=5k_u=k_k=5) under the independent covariance condition. We evaluated each world at ω=0.50\omega=0.50 and 0.900.90, yielding ten world-incompleteness conditions. On CUB, we used five initialisations for each of the three concept sets in Figure 2, which span different levels of concept incompleteness.

The table below reports the mean [observed range] of paired metric differences between models trained with Lmono\mathcal{L}_{\mathrm{mono}} and those trained with Lplain\mathcal{L}_{\mathrm{plain}}. For synthetic data, we averaged the differences across weights and both ω\omega values within each world, then across the five worlds. For CUB, we averaged across weights and initialisations within each set, then across the three sets.

DatasetMaximum partial harm (×10−3\times 10^{-3}) (Lmono−Lplain\mathcal{L}_{\mathrm{mono}}-\mathcal{L}_{\mathrm{plain}})Harmful-path prevalence (pp) (Lmono−Lplain\mathcal{L}_{\mathrm{mono}}-\mathcal{L}_{\mathrm{plain}})
Synthetic−0.722 [−2.445, −0.107]−0.86 [−18.47, +23.47]
CUB−1.181 [−1.456, −0.748]−4.20 [−7.11, −0.99]

Compared with Lplain\mathcal{L}_{\mathrm{plain}}, Lmono\mathcal{L}_{\mathrm{mono}} reduced average maximum partial harm in all five synthetic worlds and all three CUB concept sets. Harmful-path prevalence was lower in all three CUB sets and lower on average across the synthetic worlds. These results show that penalising loss increases between successive correction states with Lmono\mathcal{L}_{\mathrm{mono}} can reduce partial harm beyond the benefit of training on those same states with ordinary task loss. The experimental details and results are reported in Appendix C.1.7 (synthetic) and Appendix C.2.7 (CUB) of the revised manuscript.

3. Extension of the theory to strictly partial interventions (FNCP, mJjx)

We thank Reviewers FNCP and mJjx for raising this important point on the extension of the theory to multiple concepts. We now show in an updated theorem 1 (proved in Appendix A) that strictly partial interventions can be arbitrarily harmful under benign leakage. To show this, we have extended our result, keeping the construction similar, by adding a second known concept C2C_2 that is task-relevant (i.e., non-trivial), mispredicted with probability pp, but is not intervened on. Hence, during  intervention we only correct C^1\widehat{C}_1, while leaving C^2\widehat{C}_2 and the residual ZZ fixed, and show that, as before, arbitrarily accurate satisfaction of the sufficiency criterion, together with exact localisation places no finite upper bound on the harm caused by the (strictly) partial ground-truth concept correction. The previous result which observes the same phenomena in the case of full interventions is now in Appendix B.

AuthorsOpenReview ↗

Global Reply (to all reviewers) - Part 3

4. Absolute performance and intervention benefit across correction budgets (FNCP, PqbP)

We thank Reviewers FNCP and PqbP for asking whether reducing partial intervention harm with Lmono\mathcal{L}_{\mathrm{mono}}, preserves the benefit of correction. Table 1 in Section 5.3 of the revised manuscript reports (for various datasets and architectures) Intervention AUC, the area under the curve of top-1 task accuracy against the number of concepts corrected, and Greedy Harm AUC, the area under the curve of intervention harm along a path that greedily selects the correction yielding the largest harm increase at each step. Higher Intervention AUC indicates more benefit from intervention and lower Greedy Harm AUC indicates less harm.

Adding Lmono\mathcal{L}_{\mathrm{mono}} to endpoint training preserved or improved Intervention AUC for CEMs and yielded similar Intervention AUC for Hybrid CBMs across CUB, CIFAR10, and incomplete AwA2. For CEMs, Intervention AUC improved on CUB and CIFAR10 and was comparable on incomplete AwA2. For Hybrid CBMs, it changed from 78.70 to 78.53 on CUB, from 86.07 to 85.84 on CIFAR10, and from 92.43 to 92.52 on incomplete AwA2. Greedy Harm AUC was lower or comparable in every dataset–architecture comparison with endpoint-only training. Thus Lmono\mathcal{L}_{\mathrm{mono}} broadly preserves accuracy across correction budgets while reducing partial harm.

Harm is measured relative to each model’s own uncorrected loss, so raising that loss can reduce measured harm. To assess this directly, Appendix C.2.8 of the revised manuscript plots absolute cross-entropy and top-1 accuracy on CUB from zero to all 50 concepts corrected as requested by Reviewer PqbP. The curves cover baseline, endpoint-only, and endpoint-plus-monotonic training at weights 10 and 100.

At λmono=10\lambda_{\mathrm{mono}}=10, uncorrected cross-entropy was 0.0029 higher than with endpoint-only training. The gap narrowed as more concepts were corrected, and mean cross-entropy fell below that of endpoint-only training once 37 of 50 concepts were corrected. At full correction, the paired cross-entropy difference was not significant at −0.0003 (95% CI −0.0007 to 0.0001), and top-1 accuracy was 0.07 percentage points higher (95% CI 0.04 to 0.11). At λmono=100\lambda_{\mathrm{mono}}=100, the curves show a flatter response to additional corrections. The stronger penalty reduced partial harm further, but increased absolute task loss and limited the benefit of correction. These results highlight the importance of balancing the two objective weights so that Lint\mathcal{L}_{\mathrm{int}} improves performance at full correction and Lmono\mathcal{L}_{\mathrm{mono}} reduces harm along the way while preserving the benefit of correction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.