- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a hard-threshold binarization bottleneck into the layer of a ResNet-50, representing each ImageNet image using only about 25 active bits, while maintaining a Top-1 accuracy of 78.5%. The authors observe that the usage frequency of these bits follows a Zipf-like heavy-tailed distribution. Also, images belonging to the same class exhibit highly overlapping bit-activation patterns, and removing critical bits significantly impairs recognition. So, the authors learn an embedding for each bit and feed these into a frozen Stable Diffusion model. They find that individual bits can consistently trigger the generation of specific content, while combinations of two bits can produce novel, composite imagery.
Critical strengths
I think the idea is interesting and the generative probing offers a new approach: by mapping each discrete bit to an input embedding for CLIP, which is then fed into a frozen Stable Diffusion decoder, one can directly visualize the semantics of individual visual units. This includes concepts that lack corresponding terms in the CLIP vocabulary, providing an intuitive new tool for the interpretability of current vision models.
Critical weaknesses
-
The first thing comes into my mind is none of the emergent properties are compared against a standard ResNet-50, for example, a ResNet-50 without retraining, simply by taking the top-k or applying thresholding. So, it cannot be proven that these properties stem specifically from the binary bottleneck rather than being general attributes of the penultimate layer of a CNN.
-
I think the bit embeddings in the paper are, in essence, a form of Textual Inversion trained using diffusion loss. The resulting semantics and compositional effects are likely derived from Stable Diffusion/CLIP itself, rather than demonstrating that bit inherently possesses these semantics or compositional capabilities. Also, the semantic evidence consists of only a few selected bits and lacks comprehensive, quantitative statistics.
Questions for authors
-
I am thinking if taking a standard, pre-trained ResNet-50 (without a binarization bottleneck), apply the same top-25 selection or thresholding to its GAP features, and then conduct the exact same analyses, and the same Stable Diffusion-based generative probe, how would the results compare to those of the authors' model? If the differences are minimal, how do the authors justify the claim that these properties emerge specifically from the binarization bottleneck, rather than being inherent to the CNN representations themselves?
-
The recognition component performs a linear summation of a set of unordered active bits, which essentially serving as a bag of features. Furthermore, the evidence in the paper that comes closest to demonstrating compositionality, for example, "statue cake" from a "statue" and a "cake", is derived from Stable Diffusion. Based on this, I feel this compositional capability may be an inherent property of Stable Diffusion’s own pretraining. Given that the paper’s claim is the emergence of symbolic organization within visual recognition systems, the evidence should be sought within the recognition model itself, rather than by leveraging the capabilities of Stable Diffusion. Could the authors provide evidence from the recognition model itself that supports the assertion of symbolic organization?
Response To Reviewer FsTZ
Response To Reviewer FsTZ
We invite you to read the "To all reviewers" responses first, which clarify the motivation, organization, and contributions of this work and present additional experiments quantifying bit generation success rates.
1. Comparison with post-hoc binary bottlenecks
We thank you for suggesting this important comparison. The suggested top-k and hard-thresholding procedures are also binary bottlenecks. The key distinction is that they apply binarization after training, whereas our method trains the recognition network with the binary bottleneck in place.
We provide the following direct comparison:
At comparable active-bit counts, both post-hoc binary bottlenecks achieve approximately 10% lower Top-1 accuracy. Directly converting a pretrained continuous representation into a binary set therefore reduces discriminative quality, whereas end-to-end training under the binary constraint preserves strong recognition performance.
The post-hoc procedures also produce different organizational structures. Top-25 selection activates a fixed number of bits for every image, whereas our threshold-based representation allows the set size to vary with the input. Furthermore, their rank–frequency curves are shifted, with elevated rather than suppressed high-frequency heads and substantially lower fitted slopes (0.45). The overlap gap between same-class and different-class images is also reduced. These results indicate that post-hoc binarization changes both the quality and organization of the resulting binary vocabulary. We will include these comparisons and clarify this distinction in the final version.
Our contribution is therefore not simply to convert continuous features into binary sets, but to learn an effective binary bottleneck that preserves strong recognition performance and provides a meaningful basis for studying the organization of sparse visual units.
2. Textual Inversion and the role of the generator prior
The learned embeddings resemble Textual Inversion in that they are optimized through a diffusion objective with the generator frozen. The distinction lies in how the training images are associated with each embedding. Rather than assigning a token to a manually selected concept image set, our probe uses bit identities and activation patterns determined by the independently trained recognition model. Each embedding is shared across images in which its bit is active, allowing it to capture the bit's existing visual associations.
Our representation shares the order-independent aggregation of bag-of-features representations, but records binary membership of learned global units rather than conventional local-word frequency counts.
Stable Diffusion and CLIP provide the pretrained visual prior through which these associations are rendered. However, this prior alone does not determine a stable bit-specific semantic effect. If a bit is activated by statistically heterogeneous visual content, its embedding receives inconsistent signals and is not expected to converge to a stable, concept-specific direction. Stable generation therefore depends on the statistical regularity of the visual content associated with the bit, while the frozen generator provides the mechanism for making that association visually inspectable.
3. Recognition-model evidence and the role of the probe
The semantic-generative probe is not the sole source of evidence for symbolic organization. Sections 4.3 and 4.4 provide large-scale statistical and intervention analyses directly on the recognition representation, including vocabulary usage, category-structured active-set combinations, and counterfactual deletion. These experiments reveal structured frequency usage, greater active-set overlap within classes than across classes, and larger recognition changes under targeted deletion than under random deletion, all without using Stable Diffusion.
Section 4.5 addresses a complementary question at the individual-unit level. Its selected examples are intended to visualize and inspect the semantic effects of particular bits, rather than to replace the population-level analyses in Sections 4.3 and 4.4. The same probing procedure can be applied to every visual bit, while the reported examples cover distinct probing regimes, including the highest activation frequencies, the strongest lexical matches, and interpretable visual effects without matching lexical neighbors. Thus, the recognition-model experiments provide the global structural and functional evidence, while the semantic-generative probe provides a new individual-unit readout. The two sources of evidence are complementary, and the compositional images from Stable Diffusion are not the sole basis for our claims about the organization of the recognition representation.