- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PACE which is an early-exit framework for concept bottleneck models. It freezes the visual backbone and shares a fixed concept-to-class classifier across intermediate exits. Its training component, Concept-Preference Guided Distillation (CPGD), constructs concept-specific teachers by combining outputs from deeper exits according to estimated depth preferences. Its inference component, Readiness-Calibrated MaxProb, augments classification confidence with a class-conditioned concept-readiness gap. Experiments on five recognition datasets show improved accuracy–computation trade-offs over adapted early-exit baselines, supplemented by component ablations, multi-seed evaluations, and concept-level experiments.
Critical strengths
-
The paper addresses a meaningful problem: accelerating CBMs while retaining intermediate concept representations, rather than producing only early class predictions.
-
The comparisons control several important factors, including the backbone, concept vocabulary, exit locations, exit-head capacity, and shared classifier. The forced-exit and matched-computation ablations help separate training-side and gating-side effects.
-
CPGD consistently improves shallow predictive accuracy over final-only distillation across the reported dataset–seed pairs. The transfer experiments also suggest that its training-side benefit is not restricted to the main CBM implementation.
Critical weaknesses
- CPGD closely resembles the preference estimation and sparse multi-layer fusion mechanisms in MVP-CBM, which is cited by the paper. Group-level preferences, their combination with concept similarities, cross-layer weighting, and adaptive sparse thresholding already appear in that work.
- The preference scores are derived from intermediate visual–text similarities. Higher similarity or a larger activation does not necessarily imply a more accurate concept prediction, particularly when intermediate features are mapped through the frozen final-layer projection.
- The reported speedups are cumulative-MAC ratios obtained from offline exit decisions, rather than measured latency improvements. The practical deployment is a central motivation, such as dynamic batching, sample compaction, memory access, and the execution of preceding exit heads can affect actual speedups.
Questions for authors
-
How does CPGD compare with MVP-CBM-style teacher fusion under the same distillation protocol?
-
Do the preference scores and fusion-concentration weights correlate with actual teacher concept accuracy?
-
Does cumulative-MAC accounting include all preceding exit heads, and what latency improvement is achieved by online execution?
7 comments
AW1: Methodological Novelty and Substantive Differences from MVP-CBM
Our design for concept-depth preference estimation and sparse fusion was indeed inspired by MVP-CBM, and we have explicitly cited this work in Section 2 of the main text and Appendix B. It is worth noting that the two methods differ substantively in their core computational paradigms and optimization objectives: MVP-CBM adopts bottom-up multi-layer feature aggregation to serve a single final classifier, still requiring execution of the full visual backbone at inference; in contrast, PACE adopts top-down multi-exit knowledge transfer to build shallow concept bottlenecks capable of independent prediction and early exiting. Specifically, this is reflected in the following three aspects:
(1) From cross-layer aggregated prediction to heterogeneous, independent multi-exit concept bottlenecks. MVP-CBM aggregates concept activations from different visual layers into a single fused concept representation for final classification. In contrast, PACE constructs six independently predictive concept exits; tailored to the characteristics of visual representations at different depths, it employs Local Perception Heads (LPH) at shallow layers and Global Aggregation Heads (GAH) at deeper layers, while sharing a single frozen sparse linear classifier. This design enables intermediate layers to independently reconstruct interpretable concept bottlenecks compatible with the shared classifier without relying on deep feature fusion. Table 11 in the original paper further validates the critical role of heterogeneous exit heads in shallow concept recovery.
(2) From end-to-end activation fusion to decoupled knowledge transfer for shallow students. MVP-CBM adopts end-to-end joint optimization, directly learning a fused representation for classification via classification cross-entropy, concept cross-entropy, and sparsity constraints. In contrast, PACE adopts a decoupled training framework that freezes the visual backbone, final exit, and shared classifier, training shallow students via CLIP concept KL, final-layer class KD, and CPGD concept distillation. More importantly, CPGD utilizes deeper candidate teachers solely during the training phase to construct concept-specific distillation targets for each shallow exit, rather than directly using cross-layer fused representations for inference classification. Building upon the concept-preference idea, CPGD further introduces a candidate teacher count constraint, differentiable selection, normalized concept fusion, and supervision weighting based on fusion concentration, thereby achieving stable knowledge transfer from deep teachers to independent shallow bottlenecks.
(3) From fixed full-depth inference to concept-supported dynamic early exiting. MVP-CBM must compute all network layers to complete the final fused prediction; in contrast, PACE proposes the RCMax gate, which introduces a class-conditioned concept-support gap on top of MaxProb classification confidence to enable sample-wise dynamic exit decisions. Once the current shallow bottleneck satisfies the exit condition, neither subsequent visual layers nor deeper concept exits need to be executed, reducing inference computation overhead while preserving the concept bottleneck at the actual exit location for semantic evaluation and test-time intervention.
Controlled comparison with MVP-CBM-style teacher fusion under the same protocol. Following your suggestion, we implemented an MVP-CBM-style Teacher Fusion baseline under the exact same multi-exit distillation protocol. The experiment keeps the frozen visual backbone, six-exit architecture, heterogeneous LPH/GAH concept heads, shared classifier, and base distillation protocol identical, replacing only CPGD's teacher construction and corresponding fusion weights with MVP-CBM-style concept preference computation, threshold filtering, and exponential sparse weighting, thus eliminating interference from differences in exit-head capacity and base training settings.
| Method | Avg. Early Acc. (%) ↑ |
|---|---|
| MVP-CBM-style Teacher Fusion | 60.42 |
| Full CPGD (Ours) | 61.68 |
On CIFAR-100, directly adapting MVP-CBM's exponential sparse fusion achieves an average shallow accuracy of 60.42%, whereas full CPGD reaches 61.68%, improving over MVP-CBM-style Teacher Fusion by 1.26 percentage points. Because both methods adopt the exact same student architecture and base distillation protocol, this controlled experiment demonstrates the effectiveness of CPGD in normalized teacher construction and concentration-weighted supervision.
AW2: Verification of the Relationship Between Preference-Guided Fusion, Concentration Weighting, and Teacher Concept Accuracy
Yes, our supplementary experiments provide direct evidence from two dimensions: concept-depth preferences guide the construction of fused teachers with higher ground-truth concept accuracy, while higher fusion concentration consistently corresponds to lower teacher concept prediction error rates. In PACE, preference scores are not used to directly single out an individual teacher, but rather to construct sparse multi-teacher fusion targets as auxiliary supervision signals for shallow concept learning, while fusion concentration is used to dynamically adjust the strength of this auxiliary supervision. To directly verify the relationship between these two mechanisms and ground-truth concept quality, we conducted two sets of supplementary analyses on the CUB dataset based on human-annotated attributes.
AW2(a): Concept Semantic Quality of Preference-Guided Fused Teachers
Using 107 deduplicated human-annotated concept targets from CUB, we constructed the corresponding deep fused teachers for the three shallow student exits L2, L4, and L6, and directly evaluated the concept balanced accuracy of these fused teachers against ground-truth annotations. The experiment used the pre-fixed seed 42, and simultaneously computed the expected performance of random single-teacher selection and random sparse fusion strategies.
| Teacher Construction | Concept BAcc. (%) ↑ |
|---|---|
| Random Single Teacher | 65.34 |
| Random Sparse Fusion | 65.54 |
| Preference-guided Sparse Fusion | 66.16 |
The results show that preference-guided sparse fusion clearly outperforms both random single-teacher selection and random sparse fusion strategies in ground-truth concept balanced accuracy. This indicates that its advantage does not stem solely from multi-teacher information aggregation; concept-depth preferences provide useful teacher selection and fusion information, thereby constructing concept supervision targets with higher semantic quality.
AW2(b): Relationship Between Fusion Concentration and Concept Prediction Error Rate
We further analyzed the relationship between fusion concentration and concept prediction error rates by dividing sample–concept pairs into Low and High groups according to fusion concentration and measuring the prediction error rates of the corresponding fused teachers on ground-truth concepts.
| Fusion Concentration | L2 Error (%) ↓ | L4 Error (%) ↓ | L6 Error (%) ↓ |
|---|---|---|---|
| Low | 34.55 | 34.54 | 35.17 |
| High | 34.14 | 34.01 | 34.49 |
| Error Reduction (pp) | 0.41 | 0.53 | 0.68 |
Across all three exits L2, L4, and L6, the high fusion concentration group exhibits lower ground-truth concept prediction error rates, confirming a consistent positive association between fusion-weight concentration and teacher concept quality, which directly supports CPGD's design of using fusion concentration to adjust distillation supervision weights. These two concept-level analyses also corroborate the final classification gains observed on CIFAR-100.
We thank the reviewer for recognizing the importance of the research problem addressed in this paper, as well as our experimental efforts in controlled evaluation settings, improved shallow predictions, multi-seed validation, and transfer across CBM frameworks. To address your three technical concerns, we have conducted an additional comparison with MVP-CBM-style teacher fusion under the same distillation protocol, verified the relationship between preference scores and fusion concentration with teacher concept accuracy, and measured online inference latency on real GPU hardware.