- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Common ICA methods have identifiability results of the generative factors up to some transformations such as permutation or element-wise diffeormorphism, the paper aims to identity the generative factors up to different transformations with a normalized iVAE [1], additionnaly they want these transformations to be effectively achieved by existing models. The prove the necessity of normalization (otherwise the identified functions collapse) and they prove that under specific assumptions some equalities concerning identified functions. Finally they empirically study their claims on synthetic dataset.
Critical strengths
None
Critical weaknesses
- The main issue with this paper is that it is really hard to read: concepts are introduces without any example of what they could represent, some notations are not explained or formally defined, some notations are not used which add unnecessary complexity, some notations are reused for different objects, some different notations represent exactly the same objects, ...
- Another issue is that the paper do not ensure that a representation learned by a iVAE respect the very strict assumption require for the theory (see Question 1), additionally the experiments are limited and did not convince me that the method works (see Question 2)
Questions for authors
- My main concern lies here: assume having learned an instance and one of its sufficient statistic with a normalized iVAE, what guarantees that belongs to the chart-pinned class of the ground-truth instance ? Particularly, what guarantees that there exists such that $ T^J=g_J\circ T^I$ which is necessary for Theorem 5 and 10 ? In the original paper introducing iVAE [1], Theorem 4 guarantees that a representation that maximizes the ELBO belong to their studied equivalence classe, is there any analogous result here ? If I understood well we have while the set of instances maximizing the ELBO seems infinite (even restricting to those satisfying A1-A6), it feels odd to imagine that a learned representation will naturally fall into
- Experiments in Section 7 are very light as there are no interval confidence or correlation between and to verify if the functions are correctly approximated. It seems that more interesting results are in Appendix I, however it is really hard to read as the results are really poorly presented, could you present those results in complete sentences ? It could be interesting to know if those properties are still true for other non linear ICA methods, why is there no ablation ? The experimental setting assume to know the family of the law , would it still work in a more general parameterization of exponential family ?
- Section 3 aim to explain the necessity of normalization by using while in the rest of the paper is used. Additionnaly Theorem 3 uses the total realization of the symmetries as only assumption while the rest of the paper requires several assumption except normalization (as stated section 4.2). Could you explain how this section is related to the necessity of normalization with under A1-A5 ?
- I don't understand why requires to have , this property seems to be implied by and by the assumption of witnesses between two equivalent instances. Therefore why not define directly as ? In such case would be a direct property from the assumptions and the results of the paper would be the same ?
The following issues concern unclear aspect of the text:
- In the definition of an instance, what represent and in real life dataset ?
In Theorem 8, become skewness do you assume that skewness is an observational variable or is it another concept with same notation ? - I don't understand what represents , seems totally defined from an instance, therefore why introduce ?
- In the assumptions what is the difference between and ? More generally, when the chart are assumed to be the sufficient statistics and when it is two separated things ?
- You use the same notation both for equivalence relation and for the chart-pinned class, this is highly confusing. For instance in Theorem 8, is a chart-pinned class or an equivalence class ?
- In Theorem 8: as several charts are admissible for the same instance, some satisfying the rule and some not, I don't understand how is defined, could you give a formal definition ?
- In the proof of Theorem 9: you use for skewnesses while in the main text skewnesses are represented by and represents means, why ?
- In the proof of Theorem 9: what are the ? Are they related to to the chart image ?
- In Theorem 9: what is a representation error ? Could you give a definition ?
[1] Variational Autoencoders and Nonlinear ICA: A Unifying Framework (Khemakhem et al, AISTATS 2020)