- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper edits the vq-vae architecture to use holographic reduced representations in the latent space of the autoencoder. They claim this vq-vae variant has good disentanglement property. They compare the proposed method with other classical VAE variants and QLAE in disentanglement tasks using disentanglement metrics.
Critical strengths
-
Combining VSA like HRR into modern popular deep learning structure has big potential. Symbolic system can provide strong representation and reasoning ability, and improve the explainability of modern deep learning structure.
-
As a method for disentanglement, this paper enforces disentanglement by editing network structure and uses new latent representation, instead of directly adding disentanglement-specific loss term. Actually this paper's loss, except for the original vq-vae loss, is just regularization of the HRR. disentanglement is enforced indirectly and naturally. It is elegant and I like this kind of work.
Critical weaknesses
All the tasks of this paper are traditional disentanglement tasks and can be done by many disentanglement methods like vq-vae, the authors' method just has better performance. As a method combining VSA and vq-vae, the author seems to have not fully utilized HRR's compositional reasoning and compositional representation ability.
If I ignore the potential advantage of VSA and only assess this paper as a disentanglement method, it seems this paper's comparison experiments are not enough and it is hard to evaluate the contribution of this paper. In the main experiment results in Table 2 in the main text, the author compares HRR with beta-VAE (2017), beta-TCVAE (2018), VQ-VAE (2017) and QLAE (2023). The first three methods are very classical methods which were published 8-9 years ago. QLAE is the only one recent work in Table 2 and Table 2 is the main experiment results of this paper. More recent work comparison is needed. For example EncDiff, Bayes-QLAE and so on.
Yang, Tao, et al. "Diffusion model with cross attention as an inductive bias for disentanglement." Advances in Neural Information Processing Systems 37 (2024): 82465-82492.
Patil, Vaishnavi, et al. "A Bayesian Nonparametric Framework For Learning Disentangled Representations." International Conference on Learning Representations. Vol. 2026. 2026.
Questions for authors
In recent years, for common disentanglement metrics, some work has pointed out that a high disentanglement metric doesn't mean truly disentanglement or controllable generation. Do you think these issues of disentanglement metrics affect your paper?
2 comments
Thanks for the additional experimental results. It address my concern. I will raise my score from 2 to 3.
Response to Reviewer 7UKJ
We thank the reviewer for their time and feedback. We provide our responses to the reviewer's question and concerns below.
Question 1
In recent years, for common disentanglement metrics, some work has pointed out that a high disentanglement metric doesn't mean truly disentanglement or controllable generation. Do you think these issues of disentanglement metrics affect your paper?
We agree with the reviewer that performance on disentanglement metrics alone is not always indicative of performance on downstream tasks, and it is a gap that applies across works in the field. We measure performance on disentanglement metrics in our paper as they are still important quantitative metrics that the field uses to assess separation of factors in a representation [1-5]. This serves a primary motivation of the paper, which is investigating how VSA encoders, which are typically hand-designed per application, can be learned from data. Such encoders could map high dimensional data to compositional representations that can be manipulated within VSA frameworks. However, a learning algorithm that wishes to do this needs to be able to recognize meaningful structure in the data in order to properly represent it in a compositional representation. We chose disentanglement metrics as they measure what we are interested in, which is the degree to which factors of generation are meaningfully separated across the m pairs in our learned representations. Thus, our results on the disentanglement metrics are a step towards the broader goal of learning representations that take better advantage of the compositional structures that VSAs can create. We expect that learning to map data to these richer compositional structures from data may require imposing similar VSA-related inductive biases to the ones we have shown in our paper. In particular, modifying the process through which the latent representation gets reconstructed could lead to learning representations with nested structure, such as a filler that is itself a bound set of role-filler pairs, by unbinding in a specific sequence. In such a scenario, disentanglement metrics still provide a way to measure if the representation separates factors in a meaningful way.
[1] Hsu, K., Dorrell, W., Whittington, J. C. R., Wu, J., & Finn, C. (2023). Disentanglement via latent quantization. Thirty-Seventh Conference on Neural Information Processing Systems.
[2] Hsu, K., Hamid, J. I., Burns, K., Finn, C., & Wu, J. (2024). Tripod: Three complementary inductive biases for disentangled representation learning. Proceedings of the 41st International Conference on Machine Learning, ICML’24.
[3] Yang, T., Lan, C., Lu, Y., & Zheng, N. (2024). Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement. Advances in Neural Information Processing Systems 37, 82465–82492.
[4] Patil, V. S., Patil, S., Evanusa, M., Kundu, A. K., Fermuller, C., & JaJa, J. (2026). A bayesian nonparametric framework for learning disentangled representations. The Fourteenth International Conference on Learning Representations.
[5] Sun, B., Pagnucco, M., & Song, Y. (2024). Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products. Advances in Neural Information Processing Systems, 37, 139470–139560.
Comparison to Other Recent Works
We note the reviewer's comment that a comparison with more recent work may need to be made in order to evaluate the contributions of our paper. We collected data for three more baselines and post them as a separate comment in the discussion for other reviewers to see as well.