- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether a network trained on many tasks at once can be made to recover the discrete factors that generated its inputs. Each input is produced from a handful of categorical factors (for example shape, colour, position) by an unknown nonlinear mixing, and each task is a binary classification whose label probability depends on those factors through a logistic-linear rule. The proposed network has a bottleneck made of several winner-take-all (WTA) groups: in each group exactly one unit is active, so the bottleneck is a set of discrete choices, and a linear layer reads the task predictions off it (Sec. 3, Fig. 1a).
The main result is Theorem 1 (proof in App. A.2). If the network solves all tasks exactly, there are at least as many linearly independent tasks as categories, and the WTA groups have the same sizes as the true factors, then the bottleneck must equal the true factors up to relabelling: each group encodes one factor and each unit one category.
The experiments test this, and then what happens when the assumptions are relaxed, with 20 training runs per setting. On synthetic data with five factors of five categories each and a matching bottleneck, most runs solve the tasks essentially perfectly (15 of 20 with 30 tasks), and every such run recovers the factors as the theorem predicts (Sec. 5.1). When the bottleneck does not match the factors, which is the realistic case since the number of categories is not known in advance, the paper looks for a weaker "symbolic" code in which several units may jointly stand for one category. This often appears but not reliably: in 17 of 20 runs with five groups and 6 of 20 with seven groups, rising to 14 of 20 when the readout weights are -regularised (Sec. 5.2). A coloured variant of dSprites with 1920 images gives a similar picture (9 of 20 runs with four groups, 15 of 20 with six; Sec. 5.4). Finally, a new task is learned from fewer examples, and generalises better to held-out factor combinations, when a classifier is trained on the bottleneck code than when it is trained on the raw inputs (Sec. 5.3, Fig. 2b–c).
Critical strengths
1. An exact, checkable identifiability statement for categorical latents. Under its assumptions, Theorem 1 gives recovery of the latents themselves up to relabelling, which is stronger than the linear decodability that the same multi-task argument yields without the bottleneck (Eqs. 8–11). I followed Lemma 1 and the proof in App. A.2 and found no error: the steps , , one block per column, bijection between blocks, and the final counting argument on block sizes all hold. This matters for my assessment because it is the paper's main claim and it is correct as stated.
2. Transparent reporting that lets the reader see where the claim stops. Every setting is run with 20 seeds and the per-run MAE and number of recovered categories are tabulated (Tables A3–A7, A14–A16, A20). The counts and percentages quoted in the text agree with those tables. The paper also states its failure modes openly (L246–249, L372–373, L442–443), and it reports the non-WTA control for the generalisation experiment in the appendix (Tables A8–A10).
3. Very clear presentation. A reader can reconstruct exactly what was done - the setup is fully defined, the theorem is stated precisely, and the text states its own failure modes.
Critical weaknesses
1. The guarantee holds in a narrow regime in which the supervision already determines , and the evidence outside that regime is small-scale and mixed.
- Strength of supervision. The targets are exact, noise-free posteriors (L269–270, L942–946), i.e. generic real-valued linear functions of the one-hot code, from which follows by a fixed linear map (Eq. 10). The linear-decodability step is taken from Vafidis et al. (L199), so the new part is the combinatorial refinement in Theorem 1, where enters only through injectivity. The theorem further needs an exactly optimal predictor on all latent combinations and known numbers of latents and categories.
- No guarantee where it would be used. The paper notes that category counts are unknown in practice (L287–288), and there the result is empirical only. Perfect task performance does not imply a symbolic code in that case (Table A5 runs 9 and 19; Table A6, where 12 of 18 runs with MAE are not fully symbolic). Success rates range from 6/20 to 17/20 with the number of heads, and the runs that help have MAE , outside the paper's own "perfectly solved" criterion (Tables A7, A16).
- Scale. One synthetic MLP mixing and a 1920-image dSprites subset, with learning rate and temperature set per setup (Table A1) and no sensitivity analysis or competing method.
2. Nothing in the paper shows a benefit that is specific to the WTA bottleneck or to the symbolic format. The abstract closes by claiming "insights into the generalization capabilities of deep neural networks with WTA-like components" and an interface to symbolic AI. Sec. 5.3 and Fig. 2b–c compare a readout on only with a readout on the raw input . The readout on the multi-task representation of the same network without WTA heads is in the appendix only, and it performs the same: on the random split with 100 samples, test AUC 0.992 [0.988–0.994] (Table A8) versus 0.992 [0.985–0.995] with WTA (Table A11); Tables A9–A10 versus A12–A13 give 0.990 vs 0.989 and 0.965 vs 0.959 (the captions identify A9–A10 as ; the headers in the test block of A9 and in A10 look like a typo). What the experiments establish is therefore parity: the generalisation gain comes from multi-task pre-training (as in Johnston & Fusi, 2023), and discretising the bottleneck does not lose it. No experiment uses a task for which an axis-aligned discrete code is needed, so the neurosymbolic motivation is untested. Likewise, the softmax/attention motivation of the abstract and introduction is at odds with L246–249, where softmax gates are reported to give "blurry" outputs that were insufficient.
Questions for authors
Clarifications
- Is there any experiment in which the WTA/symbolic representation outperforms the non-WTA multi-task representation (Tables A8–A10 vs A11–A13)?
- In Sec. 5.3 and 5.4, was the multi-task encoder trained on latent combinations that later appear in of the downstream task (App. A.3 suggests uniform sampling over all of )? If so, combinations described as "never been observed during training" (L402–403) and "unseen" (Fig. 2c) were seen by the encoder; does the symbolic code still emerge, and generalise, when they are withheld from encoder training as well?
- Do the results hold when the targets are binary labels sampled from , as a learner would observe them, instead of the exact posterior probabilities?
- The theory requires , hence (L456–458). Because each of the one-hot blocks of sums to one, the codes span only dimensions, so injectivity would need only 21 tasks, yet structured permutations emerge with 20 and 15 tasks (Table A4), below both bounds. Is recovery in practice driven by the discreteness of the bottleneck rather than by the linear-decoding step (Eqs. 8–11), and can the theorem be stated under the weaker condition?
- How entangled is ? Please report the accuracy of a linear probe for from , and the softmax-gate results mentioned at L246–249, as baselines.
Suggestions for future work
- A characterisation of when a symbolic representation is guaranteed in the unmatched case (for example via the readout sparsity explored in Sec. 5.2), and for partially observed latent combinations.
- A positioning against work that uses a discrete bottleneck as the inductive bias for disentanglement, which the related-work section does not cover (e.g. Hsu et al., "Disentanglement via Latent Quantization", NeurIPS 2023, arXiv:2305.18378, where each latent dimension is quantised to a small learned codebook). Earlier empirical work on disentanglement emerging from multi-task training (e.g. Maziarka et al., ECML PKDD 2022, arXiv:2110.03498) could also be discussed alongside Johnston & Fusi (2023) and Vafidis et al. (2025) at L084–093.
13 comments
R3W1c: Scale. One synthetic MLP mixing and a 1920-image dSprites subset, with learning rate and temperature set per setup (Table A1) and no sensitivity analysis or competing method.
Answer to R3W1c: We note that our experiments were performed to test our main contribution, the theoretical result, and to investigate empirically how this result extends to scenarios that do not match its preconditions. Of course, it is an important question how the approach performs on more complex data sets, but we leave this largely to future work. Nevertheless, we agree that a sensitivity analysis is called for, and we performed additional simulations with the visual entanglement (dsprites). Results are shown in the Table below. The results show that symbolic representations emerge over a large range of learning rates and Gumbel softmax temperatures . Low values of and performed clearly worse, indicating the need for stochasticity in the learning dynamics in order to find a good representation in the WTA bottleneck.
We report these results in Appendix A.13 in the revised version and refer to it at the end of Section 5.4.
Table 1: Results for different values of and learning rate for visual entanglement (dSprites). : mean number of recovered categories; : mean number of recovered categories over perfect runs (). Each row summarizes runs with different seeds, and the total number of latent categories was .
| mean MAE | # symb. rep. | # perf. runs | ||||
|---|---|---|---|---|---|---|
| 10 | 0.001 | 0.02609 | 22.65 | 2 | 0 | NA |
| 20 | 0.001 | 0.03857 | 16.45 | 3 | 0 | NA |
| 30 | 0.001 | 0.06299 | 11.55 | 2 | 0 | NA |
| 1 | 0.005 | 0.12496 | 4.5 | 0 | 0 | NA |
| 5 | 0.005 | 0.00425 | 26.35 | 10 | 6 | 27.83 |
| 10 | 0.005 | 0.00339 | 26.9 | 13 | 8 | 28.5 |
| 20 | 0.005 | 0.00796 | 25.9 | 9 | 7 | 28.57 |
| 30 | 0.005 | 0.01590 | 23.85 | 9 | 6 | 28.5 |
| 40 | 0.005 | 0.01542 | 24.25 | 8 | 3 | 29 |
| 1 | 0.01 | 0.23698 | 1.65 | 0 | 0 | NA |
| 5 | 0.01 | 0.00467 | 25.8 | 10 | 10 | 28.4 |
| 10 | 0.01 | 0.00086 | 27.4 | 12 | 11 | 28.73 |
| 20 | 0.01 | 0.00127 | 27.05 | 11 | 14 | 27.86 |
| 30 | 0.01 | 0.00382 | 26.15 | 9 | 16 | 27.25 |
| 40 | 0.01 | 0.00858 | 25.75 | 12 | 11 | 28.55 |
R3W2: Nothing in the paper shows a benefit that is specific to the WTA bottleneck or to the symbolic format. The abstract closes by claiming "insights into the generalization capabilities of deep neural networks with WTA-like components" and an interface to symbolic AI. Sec. 5.3 and Fig. 2b–c compare a readout on only with a readout on the raw input . The readout on the multi-task representation of the same network without WTA heads is in the appendix only, and it performs the same: on the random split with 100 samples, test AUC 0.992 [0.988–0.994] (Table A8) versus 0.992 [0.985–0.995] with WTA (Table A11); Tables A9–A10 versus A12–A13 give 0.990 vs 0.989 and 0.965 vs 0.959 (the captions identify A9–A10 as ; the headers in the test block of A9 and in A10 look like a typo. What the experiments establish is therefore parity: the generalisation gain comes from multi-task pre-training (as in Johnston & Fusi, 2023), and discretising the bottleneck does not lose it. No experiment uses a task for which an axis-aligned discrete code is needed, so the neurosymbolic motivation is untested. Likewise, the softmax/attention motivation of the abstract and introduction is at odds with L246–249, where softmax gates are reported to give "blurry" outputs that were insufficient.
R3Q1: Is there any experiment in which the WTA/symbolic representation outperforms the non-WTA multi-task representation (Tables A8–A10 vs A11–A13)?
Answer to R3W2 and R3Q1:
We thank the reviewer for raising this important point. The space of symbolic methods that can be utilized is huge, and thorough investigation is left for future work. As a first plausibility check, we have performed new experiments and used decision trees, a simple symbolic and interpretable method, as the downstream model. It allows to clearly determine how the model arrives at a given result based on specific factors.
We found that, when using a decision tree as downstream model, the generalization performance is clearly better when trained on than when trained on the bottleneck representation from a network without a WTA bottlenck. The results for both data sets are listed in the tables below.
We briefly summarize the results at the end of Section 5.3 and 5.4 and added the new Table 1 in the revised version. Details and further tables are given in Appendix A.9 and A.10.
Table 2: Test performance (AUC; random train-test split) on visual entanglement (Sec. 5.4) of a decision tree (DT) downstream model which uses the WTA bottleneck representation or the abstract representation of a multi-task learning model without a WTA bottleneck. The WTA bottleneck consisted of 4 WTA heads, each with 10 neurons. # train: number of training examples for the downstream model; med.: median of runs.
| # train | med. | Q1 | Q3 | med. | Q1 | Q3 |
|---|---|---|---|---|---|---|
| 100 | 0.721 | 0.674 | 0.767 | 0.843 | 0.819 | 0.862 |
| 300 | 0.767 | 0.750 | 0.797 | 0.889 | 0.876 | 0.915 |
| 500 | 0.800 | 0.761 | 0.831 | 0.909 | 0.900 | 0.924 |
| 1000 | 0.807 | 0.789 | 0.851 | 0.939 | 0.929 | 0.957 |
| 1500 | 0.854 | 0.829 | 0.873 | 0.959 | 0.945 | 0.969 |
Table 3: Test performance (AUC; random train-test split) on generic MLP entanglement (Sec. 5.2) of a downstream decision tree (DT) model which uses the WTA bottleneck representation or the abstract representation of a multi-task learning model without a WTA bottleneck. The WTA bottleneck consisted of 5 WTA heads, each with 10 neurons. # train: number of training examples for the downstream model; med.: median of runs.
| # train | med. | Q1 | Q3 | med. | Q1 | Q3 |
|---|---|---|---|---|---|---|
| 100 | 0.686 | 0.652 | 0.723 | 0.820 | 0.765 | 0.865 |
| 500 | 0.770 | 0.751 | 0.788 | 0.890 | 0.867 | 0.922 |
| 1000 | 0.810 | 0.787 | 0.817 | 0.919 | 0.895 | 0.942 |
| 2000 | 0.820 | 0.806 | 0.848 | 0.931 | 0.919 | 0.952 |
| 4400 | 0.853 | 0.830 | 0.868 | 0.952 | 0.946 | 0.963 |
We split the remarks into sub-points to facilitate readability of the response and pair them with related points in the "Questions For Authors" section.
R3W1a: Strength of supervision. The targets are exact, noise-free posteriors σ(Wz) (L269–270, L942–946), i.e. n≥l generic real-valued linear functions of the one-hot code, from which z follows by a fixed linear map (Eq. 10). The linear-decodability step is taken from Vafidis et al. (L199), so the new part is the combinatorial refinement in Theorem 1, where Φ enters only through injectivity. The theorem further needs an exactly optimal predictor on all p=∏ili latent combinations and known numbers of latents and categories.
R3Q3: Do the results hold when the targets are binary labels sampled from P(Ai∣z), as a learner would observe them, instead of the exact posterior probabilities?
Answer to R3W1a and R3Q3: We agree that the influence of noise on the supervision targets is an interesting question. Binary labels as proposed in Q3 are also quite natural and can be viewed as a very specific noise on the targets (i.e., quantization noise). We therefore conducted new experiments with the visual entanglement (dsprites) where the supervision was performed with binary classification targets instead of posterior probabilities (binarization threshold 0.5). We found that in 9 out of 20 runs, a fully symbolic representation emerged. On average 24.8 out of 29 categories were encoded in the bottleneck. 28.57 categories were encoded in the bottleneck on average over runs with an MAE of at most 0.04.
The mean MAE on the test set was high (0.054) as expected, since the 0 and 1 target values could not be represented with the sigmoid activation functions. These results were achieved using 200 tasks instead of the 50 tasks in the main experiment, indicating that the information loss due to target binarization can be compensated for by an increased number of tasks.
We report these results in Appendix A.12 and incorporated a short summary at the end of Section 5.4.
R3W1b: No guarantee where it would be used. The paper notes that category counts are unknown in practice (L287–288), and there the result is empirical only. Perfect task performance does not imply a symbolic code in that case (Table A5 runs 9 and 19; Table A6, where 12 of 18 runs with MAE ∼10−8 are not fully symbolic). Success rates range from 6/20 to 17/20 with the number of heads, and the ℓ1 runs that help have MAE ≈10−3, outside the paper's own "perfectly solved" criterion (Tables A7, A16).
Answer to R3W1b: It is correct that perfect task performance does not always imply the emergence of a complete symbolic representation in the unmatched setting where our theory does not directly apply. However, we found that it does lead to at least a nearly complete symbolic representation in which almost all latent categories are correctly encoded. We noted this at lines 320-322 (recovery rate of categories 94.33% over all runs and 99.3% for perfect runs; perfect run if MAE$<10^{-6}$ after multi-task learning) and lines 357 (recovery rate 87.17% over all runs and 97.04% for perfect runs; the latter number was added in the revised version). Excluding runs with an MAE greater than 0.01, the average category recovery rate across Tables A5 (generic MLP entanglement, 5 WTA heads), A6 (generic MLP entanglement, 7 WTA heads), A14 (visual entanglement, 4 WTA heads), and A15 (visual entanglement, 6 WTA heads) is 97.5%, with the lowest rate being 95.4%. Notably, here, recovery is evaluated using a strict exact-match criterion: a latent category is considered recovered only if a set of WTA bottleneck output neurons or one neuron encodes it correctly across all samples. Even a single mismatch causes the entire category to be counted as not recovered and prevents a complete symbolic representation.
An universal threshold guaranteeing a nearly complete symbolic representation is difficult to establish in practice due to noise in inputs and targets, as well as regularization. Thus, a rule of thumb is more appropriate (see our answer to R2W1). First, the maximum achievable performance is identified. Then, the smallest possible bottleneck is selected without substantially compromising performance.