- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies why correcting some of a model's predicted concepts can worsen its final predictions, even when correcting all concepts helps. A theoretical example shows that corrections can be harmful even when the task predictor is optimal and correction does not introduce unseen inputs. The authors add a training penalty for increases in prediction loss as concepts are corrected one by one. In synthetic tasks and bird classification, combining this penalty with training on fully corrected concepts reduces the frequency and severity of harmful partial corrections while largely preserving uncorrected accuracy.
Critical strengths
-
The counterexample shows that corrections can hurt even with an optimal task predictor and no change in its input distribution. It isolates a failure that cannot be blamed on poor training or unfamiliar inputs.
-
The experiments make a strong case for evaluating partial corrections, not just full correction. On CUB, the proposed penalty reduces harm during partial correction while largely preserving uncorrected accuracy and the benefit of full correction.
Critical weaknesses
-
The experiments do not separate the penalty's benefit from the benefit of training on partial corrections. IntCEM includes a variant trained on random correction paths without a learned sampling policy. The real-data baselines here receive no comparable training on partially corrected states. A control that directly minimizes task loss at the same sampled states would establish what the pairwise penalty adds. It should use the same architecture, block the same gradients through concepts, and match computation. Linear task predictors and a single real dataset with frozen features also limit how broadly the results apply.
-
Lower reported harm does not establish lower absolute loss after correction. Harm is measured against each model's own uncorrected loss. The penalty can decrease either by lowering loss after a correction or by raising loss before it. The task and full-correction objectives discourage the latter, but nearly unchanged accuracy does not rule out changes in cross-entropy. Absolute loss and accuracy curves across correction budgets would clarify whether the reduction in harm preserves useful correction.
-
The related-work discussion omits Disentangled Concept-Residual Models (Zabounidis et al., TMLR 2026). D-CRM studies incomplete concept sets and uses mutual-information minimization to reduce concept-residual overlap and improve intervention effectiveness, making it directly relevant to this paper's setting. Its main positive-intervention metric replaces all concepts simultaneously, so it does not resolve harm along partial-correction paths. Discussing this distinction and comparing MI-based disentanglement with the endpoint and monotonic losses under the same architecture and partial-correction protocol would clarify what the proposed objective adds.
Questions for authors
- How does the penalty compare with direct task-loss training on the same sampled partial corrections, using the same architecture, blocked concept gradients, and computation?
- What are the absolute cross-entropy and accuracy curves as more concepts are corrected, including the uncorrected and fully corrected states, for each training regime?
- How does partial-correction harm compare between D-CRM's mutual-information regularization, the proposed penalty, and their combination under the same architecture and correction paths?
2 comments
Follow-up before the discussion period ends
Dear Reviewer PqbP
Thank you once more for all your feedback! As the discussion period is coming to an end, we wanted to ask whether you have any further questions? If so, we would be more than happy to provide further clarifications while the discussion period is open. Otherwise, if our rebuttal addressed your main concerns and questions, we would sincerely appreciate it if you could let us know. Thank you in advance!
Response to Reviewer PqbP
Thank you for taking the time to read and carefully assess our manuscript. We appreciate your positive comments on our counterexample proof and on the importance of evaluating partial corrections. Your thoughtful suggestions have helped us strengthen the experimental comparisons and clarify how the monotonic loss affects performance across correction budgets.
Below we address all concerns and questions raised in this review. If you have further questions or anything remains unclear, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score in light of our replies.
1. Separating the penalty’s benefit from training on partial corrections
Thank you for suggesting this important control, which we now include. As detailed in point 2 of our global reply, we compared the monotonic penalty with a matched plain task-loss baseline evaluated at the same sampled correction states. The monotonic penalty reduced average maximum partial harm and harmful path prevalence in synthetic data and CUB more than the plain task-loss. The experimental details and results are reported in Appendix C.1.7 (synthetic) and Appendix C.2.7 (CUB) of the revised manuscript.
2. Comparison with IntCEM
Thank you for pointing out that evaluations against IntCEM may be worth exploring. Please see our answer to concern 1 in our general reply for details on this point. There we explain how our new results in Section 5.3 show that Lint and Lmono together reduce harmful paths and interventions for CEM more so than what the original IntCEM objective does for CIFAR10, AwA2, and our CUB variants. These results therefore strongly suggest that Lmono provides additional harm-reducing benefits that are not induced by IntCEM’s training pipeline.
3. Absolute loss and accuracy curves across correction budgets
We thank this reviewer for raising this point and provide a detailed response in point 4 of our global reply. Briefly, adding Lmono preserved or improved Intervention AUC for CEMs and yielded similar Intervention AUC for Hybrid CBMs across three real world datasets (Table 1 of the revised manuscript). We also provide the requested absolute cross-entropy and top-1 accuracy curves for several incomplete CUB settings in Appendix C.2.8 of the revised manuscript. At λmono=10, monotonic training reduced partial harm, with slightly higher cross-entropy before correction, comparable cross-entropy at full correction, and slightly higher top-1 accuracy from 15/50 corrections onwards.
4. Comparison with D-CRM’s mutual-information regularisation
Thank you for suggesting this comparison. We evaluated D-CRM’s mutual-information penalty on CUB using the same Hybrid CBM architecture, concept sets, and 512 correction orderings (Appendix C.2.5 of the revised manuscript). Adding the penalty to endpoint-only training reduced concept predictability from the residual (probe F1 fell from 0.189 to 0.157), but increased harmful-path prevalence from 91.2% to 98.2% and maximum partial harm from 0.0054 to 0.0072.
Adding the same penalty to endpoint-plus-monotonic training reduced harmful-path prevalence from 51.3% to 48.1%, with a marginal reduction in maximum partial harm and a 0.43 percentage point drop in accuracy.
These results suggest that D-CRM’s mutual-information penalty may complement Lmono by reducing concept–residual overlap and further lowering the prevalence of harmful correction paths, with a modest accuracy cost in our CUB experiments.