acceptodds
Under review as a conference paper at ICLR 2027

ATCG: Structural Trait Molding for Combinatorial Auditing of Deep Classifiers

Abstract

Deep classifiers map learned embeddings to class labels through a flat prediction head and achieve high accuracy. However, when classes contain heterogeneous latent subpopulations, the opaque, unstructured embedding space makes the internal basis of predictions difficult to audit. Inspired by the combinatorial richness of modern biological taxonomy, we investigate the feasibility of representing intra-class heterogeneity through the combinatorial composition of “root elements.” We introduce *Artificial-Trait Configurative-Generation* (ATCG), a structured framework that retrofits pretrained backbones with a *Pinball head*: a per-class binary hierarchy that learns a coordinate system of basis traits and induces a combinatorial barcode space that encodes intra-class heterogeneity. To structure this combinatorial space for auditing, ATCG aggregates class evidence from basis traits bottom-up via *Max-Dominance* through a hierarchy of intermediate-layer traits, while a geometric *Molding Bonus* discourages trait collapse by encouraging sibling specialization into complementary patterns, together yielding sparse basis-trait barcodes. The induced barcodes reveal whether each prediction's evidence is *consistent*, *conflicting*, or *insufficient*, and detect intra-class shifts that may be missed by class-label frequencies at the cohort level. We validate this on controlled Stacked-MNIST experiments where barcode-based monitoring identifies intra-class shifts, demonstrate it on Camelyon17-WILDS where barcode auditing relates prediction accuracy to patient-share profiles of basis traits, and show it generalizes to non-vision settings via TCGA RNA-seq classification. ATCG further integrates with Concept Bottleneck Models as a structural backend, exposing multiple latent realizations under a single concept.

Primary area:
interpretability and explainable AI
Keywords:
Interpretable machine learning, Intra-class heterogeneity, Hierarchical representation learning, Prediction auditing, Intra-class distribution shift

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6VfEOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduced a method of explainable AI exploiting biological taxonomy. The method proposes to use pinball head as an interpretable hierarchical classifier, and the produced barcode as an auditable feature. Also some theoretical derivations are provided. Different downstream tasks like stacked MNIST, CIFAR generation, etc. are evaluated.

Critical strengths

This paper provides method to train pinball head as an interpretable classifier, based on top-down mold bonus and bottom-up max-dominance. Theoretical derivation is provided to guarantee strict depth monotonicity for hierarchical trait distance. Several different benchmark results are provided, including MNIST classification and CIFAR generation. In addition, the proposed method is applied to prior Concept Bottleneck Models as a backend to demonstrate its high compatibility.

Critical weaknesses

The proposed hierarchy is functionally equivalent to a classical flat Maxout classifier. Its structural contribution comes primarily from the regularization and interpretation imposed on the terminal units. The paper lacks sufficiently strong competing methods for its central auditing claims. There is no comprehensive comparison against simpler auditing alternatives, such as clustering pretrained embeddings, learned mixture models, prototype methods, or subclass discovery approaches. The experiments do not convincingly establish that barcode monitoring detects hidden intra-class shifts better than simpler alternatives. Table 2 reports an OOD test accuracy increase from 87.12% to 92.48%. However, ATCG involves additional training stages, backbone fine-tuning, extra parameters, and two regularizers. Semantic interpretability of learned traits remains insufficiently validated. The central claim is that basis traits expose meaningful intra-class factors. Yet most structural evaluations measure activation overlap, sparsity, factor purity, or barcode prevalence. Different training runs may discover equivalent prediction functions with different trait identities or hierarchies. The paper reports seed-level accuracy and barcode-category statistics but does not provide reproducibility of specific discovered traits, their semantics, or their correspondence across runs. Barcode-based error detection is not evaluated. The paper does not evaluate error-detection AUROC/AUPRC, selective-classification risk–coverage curves, or improvements over confidence-based rejection and other uncertainty measures. The CBM extension demonstrates limited improvement. The inference latency, memory overhead, monitoring throughput, and the practical tradeoff between hierarchy depth and auditing resolution are not systematically evaluated

Questions for authors

Since recursive Max-Dominance is mathematically equivalent to taking the maximum over terminal traits, what benefit does the hierarchical structure provide over a flat Maxout classifier? Could the authors compare against a parameter-matched Maxout model with similar regularization? How can the learned traits correspond to meaningful latent factors rather than arbitrary partitions of the embedding space? Are trait identities and semantic interpretations stable across different training seeds? What additional detection or localization capability does ATCG provide over conventional embedding-based clustering? Could we compare against a conventional classification head trained with the same computational budget and consistency regularization? Although barcode categories correlate with prediction errors, does barcode-based auditing outperform conventional confidence and uncertainty measures in detecting incorrect predictions? Could the authors report AUROC, AUPRC, or risk–coverage comparisons against maximum softmax probability and entropy-based baselines? The depth-monotonicity and bi-Lipschitz results characterize a weighted hierarchical distance. How do these guarantees translate into improved auditing performance? Can we empirically demonstrate that the proposed distance provides advantages over Hamming or conventional embedding distances?

7 comments

Response to Reviewer 6VfE

We thank the reviewer for recognizing our purpose of using the Pinball head as an interpretable hierarchical classifier and the produced barcode as an auditable feature, as well as for pointing out our key strengths: providing theoretical derivations to guarantee strict depth-monotonicity for hierarchical trait distance; providing several benchmark results including synthetic MNIST and CIFAR generation; and demonstrating high compatibility through application to Concept Bottleneck Models (as a backend).

Before addressing the weaknesses and responding to specific questions, we would like to clarify three points in our manuscript that may have contributed to the reviewer's criticism regarding presentation:

First, there appears to be some confusion about whether the Pinball head is designed to compete with post-hoc auditing tools for auditing a standard deep classifier. The answer is no. Because the standard class-label decision step from the feature layer to the logit layer compresses intra-class heterogeneity too tightly, our goal is to introduce a new classification head that surfaces intra-class heterogeneity relevant to class prediction directly from the feature layer. Our approach inserts multiple structured layers of representation between the feature layer and the logit layer.

Through this construct, we make the decision logic at the classification head, how trait scores are aggregated into class scores, explicit for auditing. We use the term combinatorial auditing to highlight the diagnostic richness of these resulting barcode patterns.

Second, we clarify how our hierarchical trait distance and heatmap visualization directly operationalize our Diagnostic Utility claim—namely, using barcodes as auditing signals at both the sample and cohort levels to flag predictions lacking consistent evidence and to localize intra-class distribution shifts through changes in barcode prevalence. Clustered heatmaps are a standard diagnostic tool in genomics. In ATCG, their rows represent trait-activation patterns, and their columns follow the hierarchy used for class-score computation. By pairing ATCG's discrete multi-level trait representations with our proposed hierarchical trait distance, heatmaps become a powerful visual diagnostic aid: (1) At the sample level: A heatmap row makes a single prediction's decision logic immediately inspectable across trait layers, allowing an auditor to visually spot missing or conflicting trait activations (inconsistent evidence) without digging into black-box embeddings; (2) At the cohort level: Using our hierarchical trait distance to drive row clustering (as shown in Figure 3 for Camelyon17-WILDS and Figure 12 for generative structure fidelity) by shared decision pathways, visually exposing shifts in barcode prevalence across subpopulations or domains.

In responding to the reviewer’s last question concerning Hamming and embedding distances, terminal Hamming compares leaf-level activation patterns, while Euclidean embedding distance measures proximity in the encoder representation. HBD explicitly weights disagreements across trait layers. Its depth-monotonicity and conditional bi-Lipschitz guarantees characterize this hierarchical geometry, which we use to organize heatmaps for medical imaging (Figure 3) and generative models (Figure 12). The empirical comparison with terminal Hamming is reported below.

Third, we clarify the criticism regarding a "lack of sufficiently strong competing methods for its central auditing claim" and the absence of comparisons against "simpler alternative approaches." Because ATCG is an architectural classification head rather than a standalone post-hoc diagnostic, it does not replace existing auditing tools—instead, it provides a structured, multi-level representation space upon which those very tools can operate. Various existing auditing methods (such as clustering, distance-based outlier detection, or confidence calibration) can be directly applied to the basis traits and intermediate layers of ATCG to audit different types of distribution shift. While applying simpler post-hoc alternatives directly to a standard flat-head classifier requires auditing either a compressed logit layer or an entangled penultimate feature layer, ATCG exposes intermediate trait representations that are explicitly aligned with the decision logic, making downstream auditing with existing tools more informative.

Further responses to questions and weaknesses

1. Hierarchy versus flat Maxout.

We agree that recursive Max-Dominance and a terminal maximum give the same class score. The hierarchy contributes through Molding Bonus's sibling relationships and recursively defined contexts, which are retained for inspection.

We compared parameter-matched flat Maxout heads using the same backbone, terminal capacity, consistency loss, and two-stage training protocol. The flat alternatives omit Molding Bonus or replace it with orthogonality or Information Maximization. Three-seed means on Camelyon17-WILDS are:

MethodDead winner (%)OOD test accuracy (%)
ATCG8.992.48
Flat + consistency96.988.53
Flat + consistency + Orth96.988.49
Flat + consistency + InfoMax60.491.53

A dead winner is never selected as the within-class argmax on clean training images of its own class; this measure is threshold-free. InfoMax improves utilization, while ATCG achieves substantially broader utilization with higher mean accuracy. This supports the proposed structural training relative to the tested non-hierarchical alternatives.


2. Additional localization capability over clustering.

We thank the reviewer for suggesting a direct comparison with embedding clustering. We added matched-k-means and GMM baselines to evaluate how many affected and distinct changed modes can be identified within an inspection budget. We report the results on Stacked-MNIST.

Clustering is fitted to baseline training embeddings within each predicted class, with KK matched to the training-observed Stage 2 barcode count. The reference and target are separate held-out generated cohorts of 60,000 images each, distinct from the training population used to fit clustering and determine KK. All grouping methods use the paper’s Z statistic and method-specific Bonferroni correction at α=0.05\alpha=0.05 over groups observed in either cohort. Inspection quotas are proportional to significant positive frequency changes; within each group, images are ordered by the corresponding model's own low MSP. Exhausted candidate pools are not replenished. This additional comparison uses occupied-group correction; the original Table 3, including its stricter correction and Novel-10 Recall@3 of 0.6, remains unchanged.

Here, a mode is a Red–Green digit combination; Recall@3 measures the fraction of manipulated modes appearing among the three most frequent modes in at least one flagged group. Under this comparison protocol, both ATCG stages and both clustering baselines achieve 100% mode Recall@3 on Missing-1/2/10 and Novel-1/2. For Missing-10, Stage 1 and Stage 2 localize the changed modes through 24.3 ± 2.9 and 22.0 ± 2.6 Loss groups, respectively, compared with approximately 60 for either clustering method. For Novel-10, Stage 1 achieves 73.3 ± 11.5% mode Recall@3, versus 83.3 ± 15.3% for k-means and 70.0 ± 10.0% for GMM; Stage 2 increases the mean to 86.7 ± 15.3%. Thus, useful localization is already present in Stage 1 with the backbone weights frozen and improves on average after Stage 2 refinement.

The table reports affected images identified in Novel-10, which contains 6,042 affected images among 60,000 target images. Values are mean ± sample SD across three seeds. Parentheses indicate actual inspections when candidate pools are exhausted; otherwise, actual inspections equal the budget.

BudgetBaseline MSPBaseline k-means + MSPBaseline GMM + MSPStage 1 Barcode + MSPStage 2 Barcode + MSP
10017.3 ± 4.762.3 ± 7.554.7 ± 12.953.7 ± 1.261.3 ± 6.7
30060.3 ± 3.8182.0 ± 20.5168.3 ± 26.0163.0 ± 6.9181.3 ± 9.5
500106.3 ± 7.2291.0 ± 39.0280.3 ± 39.0264.7 ± 14.0301.7 ± 11.0
1,000223.7 ± 15.9554.3 ± 69.9537.3 ± 66.9502.3 ± 16.6592.3 ± 35.6
3,000653.3 ± 41.91,284.3 ± 244.2 (2,677.3 ± 387.2)1,080.0 ± 274.0 (2,296.0 ± 457.8)1,314.7 ± 218.51,668.0 ± 93.6
5,0001,035.3 ± 83.11,368.3 ± 361.1 (2,919.0 ± 747.7)1,080.0 ± 274.0 (2,296.0 ± 457.8)1,740.7 ± 508.0 (4,469.3 ± 919.1)2,454.7 ± 155.2

Stage 2 is comparable to k-means at small budgets and identifies more affected images on average at 500–1,000 actual inspections. At larger budgets, clustering exhausts its significant candidate pools, while barcode-guided inspection continues to identify affected images. The larger 5,000-budget difference therefore reflects broader coverage together with additional inspection, rather than an equal-cost advantage.

At 500 inspections, the selected images represent 9.7 ± 0.6 of the 10 novel modes for Stage 2, compared with 9.3 ± 0.6 for k-means and 7.7 ± 0.6 for GMM, counting a mode as represented when at least one image is selected. At 1,000 inspections, Stage 2 and k-means both represent 9.7 ± 0.6 modes. This measures breadth of coverage, not balanced recovery across modes.

3. Semantic meaning and stability across seeds.

We thank the reviewer for requesting semantic evidence on real-world data and across training runs. We added Waterbirds experiments with a SigLIP2 vision–language backbone, using depth-4 hierarchies throughout. We examine the training population to characterize the organization learned by ATCG, using semantic descriptions assigned afterward by the unchanged pretrained image–text encoder. These descriptions do not supervise the traits.

Across three seeds, Stage 2 achieves 94.67% test accuracy and 85.41% worst-group accuracy, compared with 93.76% and 79.60% for the 10-epoch fully fine-tuned baseline. ATCG starts separately from the original pretrained encoder, with five frozen-backbone epochs followed by five joint fine-tuning epochs. Worst-group accuracy uses the four bird-class × background groups.

Semantic organization is already present in Stage 1. Eight of the nine examined background and appearance descriptions have significantly enriched landbird groups in both stages across all three seeds. These associations are assessed within the same true bird class using two-sided Fisher exact tests, with Holm correction over the full trait–concept family.

We additionally test whether sibling branches have different semantic distributions, rather than only testing individual traits against their surroundings. We compare images activating only one sibling, report jointly activated images separately, and use permutation tests of the complete semantic distribution. Holm correction covers all 540 planned comparisons across both class trees, three semantic families, both stages, and three seeds.

The table gives the number of significant sibling contrasts among all 15 sibling pairs in the landbird tree. Each entry is Stage 1 → Stage 2.

Semantic familySeed 123Seed 456Seed 789
Background7 → 77 → 78 → 4
Belly colour10 → 137 → 119 → 12
Wing pattern10 → 137 → 98 → 11

Thus, sibling-level semantic differentiation occurs in every run before backbone fine-tuning. Stage 2 yields more significant appearance contrasts in all three runs, although background contrasts do not consistently increase.

A concrete example illustrates shared background association with contrasting appearance associations. In seed 123, two child branches predominantly contain broadleaf-labelled images but differ in wing pattern:

StageLeft-only imagesSpotted / solidRight-only imagesSpotted / solid
17742.9% / 33.8%2339.0% / 79.0%
215165.6% / 23.2%3432.9% / 92.1%

The complete wing-pattern distributions differ in both stages (Holm-adjusted Monte Carlo p=0.0054p=0.0054); Cramér's V increases from 0.446 to 0.752 in this example. At Stage 2, broadleaf labels account for 93.4% and 88.9% of the two exclusive groups. Joint activation remains possible: 216 and 48 shared images in Stages 1 and 2 are reported separately.

Elsewhere in the same tree, upper branches exhibit bamboo versus broadleaf associations, while deeper siblings exhibit brown-belly/striped-wing versus yellow-belly/solid-wing associations. This provides an inspectable account of shared and differentiated descriptions within the class-score hierarchy. The waterbird tree shows weaker wing-pattern differentiation, which we retain in the complete results.


4. Hierarchical barcode distance

In response to the question “Can we empirically demonstrate that the proposed distance provides advantages over Hamming distance?”, we additionally examined the learned organization on CIFAR-100 Superclass across three runs initialized from the same pretrained checkpoint. We assessed whether distances distinguish images from the same versus different fine classes within each superclass. We distinguish the empty barcode (no basis trait meets the activation threshold, ≥5.5) from nonempty barcodes (one or more basis traits meet this threshold). An empty barcode indicates absent above-threshold trait evidence, rather than a shared semantic identity.

Excluding empty barcodes, our hierarchical barcode distance performs better than terminal Hamming in all three runs of both stages, on both the training and test sets:

Fine-class separation AUROCTraining: Stage 1Training: Stage 2Test: Stage 1Test: Stage 2
Terminal Hamming0.815 ± 0.0010.798 ± 0.00050.833 ± 0.0010.822 ± 0.002
HBD0.832 ± 0.0030.826 ± 0.0040.855 ± 0.0030.850 ± 0.004

AUROC is first computed within each superclass and macro-averaged across superclasses; values are mean ± sample SD over three runs. Within each stage and split, both distances use identical image pairs from the common cohort whose true-superclass barcodes are nonempty in all three runs. These results support fine-class organization in the decision hierarchy among images with activated trait evidence.

Reviewer_Sf8vOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This work aims to make deep embeddings of deep neural networks more transparent. The work proposes to solve this by replacing the model's latent representation with a per-class binary embedding of "traits". Each trait has an associated value where the class logit is defined by the maximal value among a hierarchy of the per-class traits, and the activation state of the trait is defined through a global threshold. The approach requires fine-tuning a pre-trained models in a two stage process, where the first stage fixes the model weights and only learns the per-class embedding on top, and the second stage fine-tunes the model weights with a small learning rate. The work claims that the approach improves architectural transparency and can be used as an auditing tool to find predictions "without consistent evidence" and localize intra-class distribution shift by grouping traits by type (single trait activation, multiple per class) or some label of the data (e.g., patient) and counting the number of activated traits per group. The approach is verified quantitatively by its test accuracy on various vision tasks with ResNet-18 on MNIST, ResNet-50 on STL, ImageNet, and Camelyon-17, as well as an MLP on the RNA-sequence dataset TCGA-10. The work verifies the learned hierarchy in a toy experiment based on MNIST, and observes higher accuracy in real-world settings for predictions that only show single traits or only traits for a single class activated. The distribution shift is analyzed on Camelyon17-WILDS, a histopathology dataset, where the work observed that the rates of traits used exclusively for single patients and traits used among multiple patients shifted between the validated and test set.

Critical strengths
  1. The paper deals with an important and difficult problem, and proposes an actionable framework to audit deep neural network embeddings.
  2. The writing and structure of the paper are good.
  3. The details of the experiments are very well documented in the appendix, which also features further ablations and details of the method.
Critical weaknesses
  1. The analysis does not follow established benchmarks for auditing, and results are not compared to a baseline that does not require fine-tuning the proposed embedding. It is unclear whether the observations in Section 4 made while auditing are a result of the proposed embedding, or whether the same analysis could simply be done on the original embedding layer, e.g., through clustering, or simply magnitude of the (uncalibrated, or better yet: calibrated) class predictions. One concrete example would be to evaluate whether the method can find biases and/or spurious correlations in the dataset.
  2. The evidence for why this approach specifically helps in auditing seems weak. The grouping by trait type in Table 1 are reported as "significant in three of the five datasets" (l.323). It is not discussed why it is insignificant in the two cases. It seems to me hard to argue with this success rate that the auditing based on this grouping is meaningful.
  3. Experiments focus on somewhat dated vision datasets and ResNet. The lack of more modern transformer-based model architectures questions the relevance to the current state of machine learning research.
Questions for authors
  1. In Table 1, why does the "None" trait group sometimes have higher accuracy than the "Multi-Inter"? This somewhat seems to go against the argument that the hierarchy carries audit-relevant information.
  2. Do the different traits carry any semantic meaning? For instance, the meaning of the traits is somewhat clear in the toy experiment in Section 4.2, but did you see any semantic meaning in the traits of the real world experiments?
  3. How much does the number of traits influence the auditability? If I understand correctly, Table 10 in the Appendix analyzes the hyper-parameter combinations, but somehow, the results do not seem to differ too much?
  4. There is some evidence that intermediate layers may carry more detailed representations than the final layer, such as observed by Skean et. al (2025) in LLMs. This could mean that the concepts that traits can capture could be limited in the final layer. Do you think this could be an issue for ACTG?
  5. I did not understand what ATCG adds combined with CBMs. Since we already see the raw used concepts, adding ACTG appears redundant.
  6. Have you tried simpler methods to audit the model, such as, e.g., the calibrated softmax predictions over patients, or clustering the final layer embeddings?

References

Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann Lecun, Ravid Shwartz-Ziv. Layer by Layer: Uncovering Hidden Representations in Language Models. ICML 2025.

Minor Issues

  • the text in Figures 1, 3, and 4 is too small to read in print
  • Figure 4 describes the figures as top and bottom, but the two subfigures are left
  • "barcode" seems to me like an odd naming for the binary trait embedding
4 comments

Response to Reviewer Sf8v

We thank the Reviewer Sf8v for recognizing the problem's importance, the actionable framework, and the clear presentation. The concrete auditing challenges motivated our new experiments and clarified ATCG's contribution and scope.

First, there appears to be some confusion about whether the Pinball head is designed to compete with post-hoc auditing tools for auditing a standard deep classifier. The answer is no. Because the standard class-label decision step from the feature layer to the logit layer compresses intra-class heterogeneity too tightly, our goal is to introduce a new classification head representation that surfaces intra-class heterogeneity relevant to class prediction directly from the feature layer. Our approach inserts multiple structured layers of representation between the feature layer and the logit layer. Through this construct, we make the decision logic—how features are used to declare the class label—transparent for auditing. We use the term combinatorial auditing to highlight the diagnostic richness of these resulting barcode patterns. The analysis reported in the paper is about auditing ATCG instead of the given model backbone.

Second, following the reviewer’s suggestion, we demonstrate that ATCG can use modern transformer-based encoders, SigLIP2 and ViT, as backbones.

Third, we report results on a concrete example, Waterbirds, and apply ATCG to examine how trait-activation patterns change under a known shift in bird–background correlations.


1. Structural transparency and barcode types.

ATCG provides structural transparency through a trait hierarchy that directly computes class scores and exposes within-class variation. Continuous scores determine predictions; barcodes summarize their activation patterns. We aim to expose the organization of supporting evidence and monitor changes in its use across populations, including changes that do not cause errors. This structure accompanies MSP: a scalar confidence does not explicitly identify the active traits or distinguish within-class support from cross-class conflict.

The four barcode types distinguish one or multiple active traits within a class from conflicting class evidence (Multi-Inter) and no above-threshold response (None). These suggest different types of diagnostic checks, rather than ordered uncertainty levels. Images in the "None" group have no above-threshold trait responses, but relative differences among their continuous trait scores can still support correct predictions. In contrast, "Multi-Inter" indicates strong conflicting class-supporting evidence. Logically speaking, "None" is not necessarily inferior to "Multi-Inter." Table 2 reports their accuracy associations without imposing an ordering between "None" and "Multi-Inter".

Multi-Intra remains distinguishable from Single after confidence calibration. Following the reviewer's suggestion, Table 16 now uses five-fold cross-fitted temperature-scaled MSP. Entries are confidence-adjusted error odds ratios, 95% CIs, and two-sided p-values; the broad contrast compares average category log-odds for Single/Multi-Intra versus Multi-Inter/None.

DatasetMulti-Intra vs. Single: OR [95% CI]pSingle/Multi-Intra vs. Multi-Inter/None: OR [95% CI]p
MNIST0.195 [0.103, 0.369]5.33e−70.799 [0.442, 1.445]0.458
F-MNIST0.421 [0.364, 0.487]1.57e−311.045 [0.866, 1.261]0.644
STL-100.210 [0.085, 0.522]7.78e−40.923 [0.502, 1.699]0.798
CIFAR-100.412 [0.334, 0.509]1.56e−161.070 [0.835, 1.372]0.591
CIFAR-1000.748 [0.669, 0.836]3.33e−70.979 [0.855, 1.122]0.764

Using the calibrated MSP, the Single/Multi-Intra vs. Multi-Inter/None contrast is not significant. Multi-Intra retains lower error odds than Single after adjustment for calibrated MSP in all five datasets. We will replace Table 16 and remove its previous three-of-five significance claim.

2. Semantic meaning.

Real-data traits also expose semantic variation. Table 11 and Figure 7 show differentiation within CIFAR-100 superclasses. In Camelyon17-WILDS, Figure 9 shows complementary trait-localized regions, informally interpreted by a practicing pathologist as lymphocyte-rich tumor tissue versus infiltrating tumor cells. We will add the representative-patch hierarchy in the revision.

We further quantify semantic organization on Waterbirds (Sagawa et al., 2020), a binary landbird/waterbird classification task, using the pretrained SigLIP2 model google/siglip2-base-patch16-256 (Tschannen et al., 2025). Its vision encoder supplies a 768-dimensional pooled representation to a depth-four Pinball head, with 16 basis traits per class. Using only bird-class labels as supervision, Stage 1 trains the head for five epochs with the original pretrained encoder frozen; Stage 2 jointly fine-tunes the encoder and head for another five epochs. We use final checkpoints from three seeds.

For semantic annotation, we separately use the unchanged original image–text encoder to compare images with fixed bird-attribute and background descriptions. Each image receives the highest-similarity description within each attribute family. These labels are held fixed across stages and seeds and are not used to train the classifier; the text encoder is used only for annotation. We assess how the hierarchy organizes the training population through trait–label enrichment and sibling-composition tests. The seed-123 examples below report semantic-label percentages among landbird training images activating only the left or only the right sibling:

LevelSemantic labelStage 1: left / rightStage 2: left / right
1Bamboo background67.5% / 11.3%78.3% / 3.2%
1Broadleaf background27.6% / 84.8%18.7% / 88.5%
3Brown belly45.8% / 2.1%35.2% / 2.1%
3Yellow belly4.4% / 67.8%4.2% / 55.3%
4Spotted wings42.9% / 9.0%65.6% / 2.9%
4Solid wings33.8% / 79.0%23.2% / 92.1%

Semantic distinctions are already present in Stage 1 and occur at multiple depths. The displayed background, belly-color, and wing-pattern distribution contrasts remain significant in both stages (conditional permutation tests; Holm-adjusted p = 0.0054 across 540 planned comparisons). Each of these three semantic families also has significant sibling contrasts in the landbird tree in both stages across all three seeds. These are encoder-derived semantic associations, rather than uniquely identified concepts or independent expert labels.


3. Distribution shifts and auditing comparisons.

For the suggested bias/spurious-correlation example, Waterbirds provides known bird–background conflicts: these constitute 5% of training images but 50% of test images. Stage 1 trains the head on the frozen downloaded SigLIP2 encoder, whereas the baseline separately fine-tunes that encoder with a linear classifier; Stage 1 therefore does not start from the already fine-tuned baseline. After joint fine-tuning, Stage 2 achieves 94.67% test accuracy and 85.41% worst-group accuracy, versus 93.76% and 79.60% for the baseline (three-run means; worst group is defined over bird class × background). On the same 2,897 conflicting test images, accuracy is 80.51% in Stage 1, 90.08% in Stage 2, and 88.44% for the baseline.

The shift monitoring provides a specific structural observation without using background annotations for selection. The two class-specific None barcodes are flagged as Gain in both stages in all three runs. In Stage 2, their combined prevalence increases from 2.82–3.27% in training to 12.12–13.19% in test. Conflicting backgrounds account for 78.9–80.5% of these images, compared with 45.4–46.0% outside None. Thus, the analysis connects the known background shift to an increased prevalence of images without above-threshold trait responses, including images that remain correctly classified.

We also evaluated confidence-based inspection, globally and within groups with increased prevalence. We fit k-means and diagonal GMM to baseline training embeddings within each baseline-predicted class, matching K to the Stage 2 training-observed barcode count, and fix the assignments thereafter. All grouped methods use the paper's Z statistic with Benjamini-Yekutieli FDR (BY-FDR) at 5% over groups observed in training or test. Quotas are proportional to positive frequency changes, capped by group size, with each model's own uncalibrated image-level MSP ranked from low to high within groups; background and correctness labels are used only for evaluation. At 1,200 actual inspections out of 5,794 test images, no method exhausts its candidates. The tables report the fraction of each audited model's errors on conflicting-background images identified, as mean ± sample SD over three seeds.

Global MSP only

Selection methodAudited modelConflict errors identified
Baseline MSPBaseline68.95 ± 1.99%
Stage 1 MSPStage 165.00 ± 4.42%
Stage 2 MSPStage 274.82 ± 2.31%

Inspection within Gain groups

Selection methodAudited modelConflict errors identified
Baseline k-means+MSPBaseline32.88 ± 2.42%
Baseline GMM+MSPBaseline37.83 ± 3.71%
Stage 1 barcode+MSPStage 158.81 ± 2.49%
Stage 2 barcode+MSPStage 258.89 ± 3.96%

Stage 2 global MSP covers a larger fraction of its remaining conflict errors than Stage 1 global MSP, while the mean conflict-error count decreases from 564.7 in Stage 1 to 287.3 in Stage 2. Within the Gain-screened comparison, barcode-guided inspection already provides higher own-error coverage in Stage 1 than the tested clustering pipelines, and retains this in Stage 2. Global MSP remains stronger than barcode-guided inspection within each model; restricting inspection to Gain groups therefore carries a cost when the sole objective is finding errors.

We conducted a similar analysis on Camelyon17-WILDS and found a similar pattern at 1,200 actual inspections out of 85,054 test images. Global MSP covers 6.90 ± 1.22% of Stage 1's errors and 9.17 ± 1.94% of Stage 2's errors. Within the Gain-screened comparison, Stage 2 barcode+MSP covers 4.76 ± 1.27% of its errors, versus 3.35 ± 0.08% and 2.74 ± 0.21% of baseline errors for matched k-means+MSP and GMM+MSP.

The shift analysis additionally retains the changed pattern and its sources: in seed 123, one predicted-tumor barcode increases from 3.16% to 6.67%; its training profile is concentrated in patient 012 at hospital 0, while 5,637 of its 5,674 test images come from new patient 051 at hospital 2. Source annotations are examined after Gain selection. We will include the full comparisons in the revision.

Separately, the Camelyon17-WILDS analysis in the current manuscript examines learned trait activations across patients and data splits. Single-/Cross-Patient categories are fixed from training profiles, with Single defined by a maximum patient-normalized activation share of at least 50%. Recomputing with actual patient IDs rather than the slide identifiers used in the original analysis, the illustrative seed-123 run shows mean Single-Patient usage among tumor images increasing from 8.47% on validation to 24.12% on test. Test accuracy, averaged equally over traits activated in at least one test image, is 99.22% versus 97.35% for ATCG versus baseline among Single-Patient traits, and 92.08% versus 91.03% among Cross-Patient traits.

These observations do not establish causal correction of shortcuts; Camelyon Gain screening is exploratory because patch-level tests do not account for within-patient dependence.

4. The number of traits and the auditability.

The number of traits controls resolution as well as sample support. Within the same Stacked-MNIST hierarchy, Stage 2 Red activation purity rises from 23.91% at depth one to 73.61% at depth four. Similar classification accuracy across depths therefore does not imply identical subgroup organization; finer groups also require sufficient observations.


5. Intermediate layers.

Regarding intermediate layers, we agree that the final embedding may omit useful information. Stage 2 updates the encoder, including its intermediate layers, jointly with the structured head. This does not make those intermediate representations directly inspectable or guarantee recovery of all useful information. Extending the structural design to intermediate layers is a direction for future work.


6. CBMs.

For CBMs, a named concept score identifies which concept is used; ATCG additionally exposes within-concept variation through multiple traits aggregated into that score. The red-eyes example illustrates distinct visual realizations of one concept. CUB accuracy remains comparable (75.53% versus 75.54%); the contribution here is added structure behind concept scores, rather than a substantial accuracy gain or unique semantics for every trait.


7. Transformer-based backbones.

We additionally evaluated ATCG with ViT backbones on CIFAR-10 and CIFAR-100, using hierarchy depths four and two, respectively. Starting from each task-trained baseline checkpoint, Stage 1 trains the structured head for 20 epochs with the encoder frozen, followed by 20 epochs of joint fine-tuning in Stage 2. Final-checkpoint test accuracies are reported as mean ± population SD over three runs.

DatasetBaselineStage 1Stage 2
CIFAR-1098.64 ± 0.08%98.43 ± 0.09%98.27 ± 0.16%
CIFAR-10090.45 ± 0.02%89.22 ± 0.08%90.03 ± 0.10%

Stage 2 remains within 0.43 percentage points of the baseline mean accuracy while providing the trait hierarchy and activation patterns for inspection. Together, the ViT and SigLIP2 experiments demonstrate that ATCG’s structural classification head extends to transformer-based encoders. We will include these additional results and implementation details in the revision.


8. Presentation.

We will enlarge the text in Figures 1, 3, and 4, correct the left/right description, and clarify that "barcode" refers to the binary trait-activation pattern used for inspection, distinct from the continuous scores used for prediction.


References

Tschannen, M., Gritsenko, A., Wang, X., et al. (2025). SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. arXiv:2502.14786.

Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. (2020). Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. ICLR.

Reviewer_hmVcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The Authors introduce Artificial-Trait Configurative-Generation (ATCG), a framework meant to aid in classification problems with intra-class heterogeneity.

The framework involves adding a mapping gϕg_{\phi} on top of a pretrained backbone, which projects embeddings into a basis layer of a balanced, binary hierarchy of depth DD, for each class cc. The basis trait activations advance in the hierarchy through max-out activations and the final activations of each class constitute the class logits. The projection gϕg_{\phi} is trained using a standard classification loss, a consistency loss for perturbation invariance, as well as a structural specialization loss which promotes diversity among each class’ basis traits. ATCG uses a two-staged training pipeline. The first stage trains only the Pinball head, keeping the backbone frozen, while the second stage also refines the backbone.

On a synthetic StackedMNIST dataset, the Authors assert qualitative behaviours of the learned hierarchy

On simple datasets (MNIST, FashinMNIST, CIFAR-10/-100) and an RNA sequence classification dataset, the Stage 2 networks match the performance of the pretrained backbones, while on the OOD test set of Camelyon17 it surpasses it by ~5% (from 87.1% to 92.5%).

By thresholding the activations in the binary hierarchy, the Authors produce binary barcodes (activation patterns at each hierarchy level). These barcodes can be used to split the data into multiple groups based on their activation patterns. These groups exhibit varying degrees of difficulty, which are in line with their design.

The barcodes can be used to identify distribution shifts (loss or gain of subpopulations, relative to the training set) (Sec. 5.1).

In Sec. 5.2, the Authors also show that the proposed Pinball head can be used to replace the units in a Concept Bottleneck Layer of a CBM, resulting in comparable (slightly lower) performance.

While I do not consider there to be major/critical weaknesses, as highlighted by the Authors, the scalability of the proposed framework is constrained by the number of per-class samples.

Critical strengths

I think that the structural specialization loss is the main contribution of this work. While the binary hierarchy has the same effect as a maxout on the basis traits, the Authors provide a working strategy for training the Pinball head. For the purpose of Interpretability & Explainability, the proposed binary hierarchy is effective for identifying data subpopulations and tracking distribution shifts.

Critical weaknesses

None

Questions for authors

Comments/Suggestions

The two-stage approach is similar in nature to the LP-FT proposed in [Kumar et al., 2022]. This strategy is known to improve OOD generalization for simple finetuning. The Authors should acknowledge individual properties of framework components. From the description in Appx. B.3, the baselines do not seem to be trained with the same LP-FT strategy. I am not highlighting this as an issue. I just think that it should be a standard approach when using pretrained weights, and it has been practiced even before the work of Kuma et al. [2022].

Something seems to be wrong with the citations in the paper. They are in contextualized format [Author et al., (year)], but not integrated in the sentences. Also, unlike other manuscripts, the citations do not have hyperlinks to the corresponding entries in the References section. This problem is also present for Figure/Table references. There is likely an error in the LaTeX code that the Authors should fix.

Further applications of the binary barcodes may exist, e.g. OOD detection.

[Kumar et al., 2022] Kumar, Ananya, et al. "Fine-tuning can distort pretrained features and underperform out-of-distribution." arXiv preprint arXiv:2202.10054 (2022).

Q:

Have the authors considered using a single linear projection for gϕg_{\phi} ? Appx. B mentions that gϕg_{\phi} is a two-layer MLP, which by itself might give the Pinball head an advantage over the simple linear projectors in modeling heterogenous data.

1 comment

Response to Reviewer hmVc

We thank the reviewer for the thoughtful and constructive assessment, and for recognizing the structural specialization loss as a central contribution of ATCG. We also appreciate the observation that the learned hierarchy supports the identification of data subpopulations and the tracking of distribution shifts. ATCG provides structural transparency through a trait hierarchy that directly computes class scores and exposes within-class variation. The suggestions help us clarify the respective roles of the training protocol, projection architecture, and structural specialization mechanism.


1. Relationship to LP–FT

We thank the reviewer for pointing us to Kumar et al. (2022). We agree that training the head before updating the backbone is an established strategy that can improve OOD generalization, and we will explicitly acknowledge this relationship. Our contribution is the structural specialization mechanism and the resulting trait-based classification and auditing structure, rather than the two-stage optimization schedule itself.

To examine the contribution of additional staged training, we performed an LP–FT-style continuation from the same three task-trained ResNet-50 Base checkpoints used by ATCG. The flat linear classifier was trained with cross-entropy for five epochs with the backbone frozen, followed by five epochs of full fine-tuning. This matches ATCG’s additional training epochs in both stages. All comparisons use the final checkpoint of each stage. On Camelyon17-WILDS, **LP–FT improves mean OOD test accuracy from 87.12% to 90.60 ± 1.35%, while ATCG achieves 92.48 ± 1.28%, with higher test accuracy in all three matched runs. We will add these training controls to Appendix Table 8 and clarify the training schedules in Appendix B.3.


2. Citation formatting and cross-reference links

We will correct the LaTeX citation and hyperlink configuration and verify that citation formatting, bibliography links, and figure/table cross-references are correctly rendered in the revised PDF.


3. Potential OOD detection applications

We thank the reviewer for suggesting this important direction. ATCG is proposed as a general framework for classification and trait-based auditing; adapting it to a specific auditing objective, such as OOD detection, may require additional design and evaluation. We will discuss this application as future work.


4. Single linear projection versus two-layer MLP

We appreciate this suggestion, which motivated an additional projection-sensitivity ablation on Camelyon17-WILDS. We replaced the two-layer MLP in the Pinball head with a single affine projection followed by the same scaled sigmoid activation, retaining the hierarchy, Molding Bonus, consistency regularization, and two-stage training protocol.

Pinball projectionOOD validation accuracy (%)OOD test accuracy (%)Mean dead basis traits (%)
Single affine projection94.05 ± 0.6792.82 ± 0.583.1
Two-layer MLP94.38 ± 0.2892.48 ± 1.285.7

Accuracy is reported as mean ± population standard deviation over three seeds. Dead basis traits are reported as the mean percentage of terminal-depth basis traits that are never activated on the clean training split, using the same threshold as Table 8.

The single affine projection achieves a slightly higher mean test accuracy and a lower mean proportion of unused basis traits. It therefore provides a viable simpler alternative, showing that the additional nonlinear projection layer is not required for the observed benefits on Camelyon17-WILDS. We will include this projection-sensitivity result alongside the original component ablations in Appendix Table 8.


We thank the reviewer again for the helpful suggestions. They help clarify the contribution of structural specialization in relation to the training protocol and projection architecture.


Reference

Kumar, A., Raghunathan, A., Jones, R., Ma, T., and Liang, P. (2022). Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. ICLR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.