- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces entropy-ordered flows (EOFlows), which augment normalizing-flow training with a regularizer encouraging orthogonality of the decoder Jacobian. The learned coordinates are ranked by manifold entropy after training, enabling an adjustable inference-time bottleneck. The experiments investigate feature reproducibility, sparsity, disentanglement, and reconstruction.
Critical strengths
-
Clear presentation and motivation: The paper is very well written, with clean formalization and a convincing explanation of the proposed approach.
-
Strong experimental study: The experiments are thoughtfully designed and provide compelling evidence for the method’s benefits.
-
Faithful baseline comparisons: I appreciate that the reported baseline results are consistent with independent findings, which lends credibility to the empirical study.
Critical weaknesses
-
The nonlinear PCA framing needs qualification: Reconstruction-based nonlinear autoencoders are also commonly interpreted as generalizations of PCA. The abstract should more precisely distinguish what EOFlows generalize, including the role of entropy ordering versus the identification of principal directions.
-
The evidence for statistical independence could be stronger: Decoder-Jacobian orthogonality does not directly measure statistical independence between latent features. Since orthogonality is also an explicit training objective of EOFlows, superiority on this metric alone does not establish superiority on the independence objective pursued by methods such as β-TCVAE. A complementary latent-dependence evaluation like the LPS score from Fruytier et al.'s "Sculpting Latent Spaces" would strengthen these claims.
-
Broader baseline evaluation would be useful: Despite the strong experimental study, I would appreciate more evaluation of the baselines across the different experimental settings.
Questions for authors
-
1. Polar coordinates and remaining ambiguities: The introduction states that rotating the latent coordinate system destroys orthogonality of the induced coordinates. However, polar coordinate systems admit different choices of angular reference and origin while remaining orthogonal. Could you clarify how these transformations differ from the latent rotations discussed in the paper, and which ambiguities orthogonality resolves or leaves intact?
-
2. Connection to weakly supervised disentanglement: Varying one coordinate while holding the others fixed, as in Equation (3), seems related in spirit to the factor-wise variation considered in Locatello et al.’s "Weakly-Supervised Disentanglement Without Compromises". Could you briefly discuss this connection and the differences in assumptions?
-
3. Soft orthogonality: Does “soft orthogonality” simply mean encouraging orthogonality through a finite-weight regularizer rather than enforcing it as an exact constraint?
-
4. Entropy ordering: My understanding is that coordinates are rearranged post hoc according to their estimated manifold entropies, rather than occupying entropy-ordered positions during training. Is this correct? How does this ranking compare with approaches such as Matryoshka representations (Kusupati et al.'s "Matryoshka Representation Learning"), which encourage an ordering of information through the architecture and nested training objectives?
-
5. Statistical independence: Does greater Jacobian orthogonality also produce more statistically independent features on encoded data? Could you evaluate this using a complementary measure such as Fruytier et al.’s Latent Predictability Score (LPS)?