acceptodds
Under review as a conference paper at ICLR 2027

Dynamics Hessian Penalty for Disentangled Representation Learning

Abstract

Disentangled representation learning seeks to represent distinct factors of variation in separate latent dimensions. Some methods encourage disentanglement by restricting interactions between latent dimensions during generation. When imposed on the complete generative mapping, these restrictions can favor additive structure and limit the nonlinear composition of latent effects in observation space. We introduce the Dynamics Hessian Penalty (DHP) for iterative generators, which applies this interaction prior to the individual conditional predictions, penalizing their mixed second-order derivatives with respect to the latent representation. Our analysis shows that the complete generative mapping need not collapse to an additive form, even when all such local mixed derivatives vanish. We implement DHP using stochastic finite differences and apply it to diffusion and rectified-flow models, without unrolling the sampler during training or adding inference cost. Experiments show strong disentanglement on synthetic benchmarks and improvements in both disentanglement and generation quality on real-world images. Qualitative results further illustrate controllable generation through latent manipulation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
disentangled representation learning, unsupervised learning, diffusion models, rectified flow

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_16QnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the Dynamics Hessian Penalty (DHP), which penalizes mixed latent derivatives of single-step conditional predictions in diffusion and rectified-flow models using stochastic finite differences. The goal is to encourage disentanglement while retaining the capacity for nonlinear composition through iterative generation.

Critical strengths
  1. Clear motivation and practical implementation. Appendix A explains why vanishing local mixed derivatives need not make the complete generative mapping additive; the authors also explicitly state that this does not provide an identifiability guarantee. The method avoids explicit Hessian computation and sampler unrolling during training, and adds no prediction branches at inference.
  2. Controlled evidence for incremental value. Table 3 uses the same ten training seeds and controls for BatchNorm. DHP-C improves mean DCI from 0.716 to 0.747 for noise prediction and from 0.686 to 0.726 for rectified flow. DHP-F shows improvements in the same direction. These results provide useful evidence that the regularizer contributes beyond normalization in the tested setting.
Critical weaknesses
  1. The novelty is incremental, and the relationship to prior work could be positioned more precisely. The authors should distinguish DHP more explicitly from the Hessian Penalty, COMET, DisDiff, and Decomp Diffusion, particularly by clarifying the relationship between DHP-C and the original HP estimator. The more compelling contribution is the choice of regularization target in per-step conditional training and its empirical effect, rather than independent novelty in the finite-difference estimator.
  2. Attribution of the largest gains remains incomplete. Some baselines in Tables 1 and 2 use settings from their original papers. Table 3 provides useful matched evidence, but does not fully explain the large gains on datasets such as Cars3D. A matched BN-only control on Cars3D would help isolate the contribution of DHP.
  3. CelebA provides limited evidence about cross-run stability. The authors clearly state that each model is represented by a single trained checkpoint and that the error bars come from repeated evaluation. These results therefore do not establish stability across independent training runs. Additional independent runs, or a more qualified robustness claim, would strengthen the real-image evidence.
Questions for authors
  1. Could the authors clarify the relationship between DHP-C and the original Hessian Penalty estimator, and focus the novelty claim on the regularization target and its empirical effects?
  2. Could a matched BN-only control be provided on Cars3D or another setting with large reported gains to better isolate DHP's contribution?
  3. Do the main CelebA findings persist across independently trained checkpoints? If additional runs are not feasible, could the limitation regarding training stability be stated more explicitly?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 16Qn

Thank you for your thoughtful and encouraging review. We appreciate your positive assessment of DHP's motivation, practical implementation, and gains beyond BN1d in the controlled comparisons, as well as your constructive suggestions for strengthening the paper.

The relationship between DHP and HP

We agree that DHP’s main contribution lies in the regularization target and its empirical effects.

Section 4.7 (Table 7) now directly compares DHP-C fine-tuning (DHP-FT) with HP applied to the final latent of a five-step DDIM sampler (HP-FT), using the same initialization and 30,000 fine-tuning steps. DHP-FT achieves higher DCI on both datasets while reducing local Hessian energy and retaining 200-step sampling-map energy of the same order of magnitude as EncDiff w/BN1d. Appendix D.6 provides the corresponding Hessian visualizations.

We have clarified the comparison with COMET, DisDiff, and Decomp Diffusion in Section 2.2. COMET generates images by gradient descent on a sum of concept-conditioned energies. DisDiff adds factor-conditioned guidance fields to a pretrained unconditional diffusion model. Decomp Diffusion averages denoising predictions, each conditioned on a single component, during reconstruction and recombination.

At each generation step, the combined gradient or denoising prediction is additive across component codes, which may be multidimensional. With the current state held fixed, mixed derivatives between distinct component codes therefore vanish wherever they exist. DHP instead jointly conditions its denoising network on all latent coordinates and softly penalizes mixed derivatives between scalar coordinates.

Appendix B.3 now derives the relationship to the original HP estimator. DHP-C can be obtained from the original two-direction (k=2k=2) HP construction by reparameterizing the sampled directions and accounting for normalization. We have also added an explicit citation to Peebles et al. (2020) in Section 3.3.

DHP-F uses a forward mixed-difference construction not provided in the original HP formulation. It reuses the base-loss prediction and requires three additional predictions, compared with four for DHP-C. Appendix B discusses this construction and its approximation-accuracy trade-off.

Matched BN-only control on Cars3D

Thank you for suggesting this control. We report the requested comparison below. All four methods use the same training budget, seed set, VQGAN, and evaluation data. Results are means and standard deviations over ten training runs with different seeds.

MethodDCI disentanglementFactorVAE
EncDiff, reproduced0.539 ± 0.0570.950 ± 0.025
EncDiff w/BN1d0.637 ± 0.0550.992 ± 0.012
EncDiff w/BN1d + DHP-F0.752 ± 0.0600.994 ± 0.011
EncDiff w/BN1d + DHP-C0.780 ± 0.0630.997 ± 0.008

BN1d increases mean DCI by approximately 0.098. With BN1d held fixed, DHP-F and DHP-C provide additional mean DCI gains of approximately 0.115 and 0.143, respectively. FactorVAE is already close to its ceiling with BN1d, so its additional gains are small.

CelebA results across independent training runs

Following your suggestion, we evaluated five independently trained models for each DHP variant on CelebA. The following results are means and standard deviations across these five models.

MethodTAD ↑FID ↓
DHP-F1.0638 ± 0.25287.2426 ± 0.1088
DHP-C1.0007 ± 0.25697.1340 ± 0.1249

Thank you for the detailed responses. The additional experiments and clarifications have addressed a lot of my concerns. However, some limitations remain (e.g., quantitative semantic controllability and its effect on generation quality). I will maintain my recommendation.

Thank you for taking the time to consider our responses and additional experiments. We appreciate your thoughtful review and constructive suggestions, which have helped us clarify the contribution and strengthen the evaluation.

Reviewer_WK6tOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Standard Hessian Penalty (HP) promotes disentanglement by penalizing mixed second-order derivatives with respect to latent dimensions. When applied to the complete generative mapping, this prior encourages additivity in the generated output, potentially restricting nonlinear composition of latent effects.

This paper proposes Dynamics Hessian Penalty (DHP) for iterative generative models. Instead of applying HP to the complete mapping, DHP penalizes mixed latent derivatives of individual conditional predictions while holding the current state (x_t) fixed. This encourages local additivity with respect to (z), without necessarily imposing additivity on the complete generative mapping.

The authors theoretically establish this distinction and implement DHP using stochastic finite differences without unrolling the sampler. Experiments on synthetic benchmarks and CelebA report improvements in disentanglement and generation quality, with additional evaluations on rectified flow, downstream factor prediction, and qualitative latent manipulation.

Critical strengths
  1. Well-motivated and interesting formulation.

The paper identifies a limitation of applying HP to the complete generative mapping: semantically disentangled factors need not compose additively in observation space. DHP provides a simple but clever solution by encouraging locally additive latent contributions while allowing nonlinear composition through successive generative updates. The theoretical analysis also supports this distinction.

  1. Encouraging empirical results across different settings.

The experiments show improvements in disentanglement on several synthetic benchmarks and in both disentanglement and generation quality on CelebA. The extension to rectified flow and additional evaluations of downstream factor prediction and latent control further support the method's applicability.

Critical weaknesses
  1. Limited empirical validation of the central motivation.

The main motivation of DHP is to encourage latent disentanglement without restricting nonlinear composition in the complete generative mapping. While the theoretical analysis shows that local additivity does not necessarily imply global additivity, it does not establish that the preserved nonlinear interactions are beneficial for disentanglement or generation quality.

The experiments lack a direct comparison between global HP and DHP, or an analysis of nonlinear latent interactions in the complete generative mapping. Therefore, the claimed advantage of balancing local additivity and global nonlinear composition is not sufficiently validated empiricall

  1. Limited evidence connecting representation disentanglement to semantic controllability.

The quantitative disentanglement metrics (FactorVAE, DCI, and TAD) evaluate the learned representation (z), but do not directly measure whether manipulating individual dimensions of (z) enables independent semantic control during generation. The qualitative latent swapping and traversal results are encouraging, but lack systematic quantitative validation.

Questions for authors

See Critical Weakness 1. Could the authors provide a controlled comparison between global HP and DHP, or quantify nonlinear latent interactions in the complete generative mapping, to validate the claimed advantage of DHP?

See Critical Weakness 2. Could the authors quantitatively evaluate whether changing one dimension of (z) consistently modifies the intended semantic attribute while preserving other attributes?

4 comments
AuthorsOpenReview ↗

Response to Reviewer WK6t

We appreciate your constructive feedback and your positive assessment of DHP's motivation and theoretical analysis. Following your suggestion, we have added a direct comparison with global HP and estimates of local and global off-diagonal Hessian energy in Section 4.7. We also explain below the scope of our CelebA evaluation.

Global HP comparison and Hessian energy

Section 4.7 (Table 7) now provides the requested comparison. Applying HP to EncDiff's standard 200-step sampler requires backpropagation through the full trajectory for each perturbed representation. We use a five-step DDIM sampler to make the comparison computationally tractable. DHP-FT applies DHP-C to the denoising prediction; HP-FT applies HP to the final diffusion latent of this five-step sampler. On each dataset, both start from the same EncDiff w/BN1d checkpoint and retain Lbase\mathcal L_{\mathrm{base}} during 30,000 fine-tuning steps.

DHP-FT reduces local Hessian energy while retaining 200-step sampling-map energy of the same order of magnitude as the starting model. Local energy decreases from 2.22959 to 0.04607 on MPI3D-toy and from 0.74302 to 0.07386 on Cars3D. The coordinate-wise Hessian visualizations in Appendix D.6 show this distinction as well.

On Cars3D, DHP-FT increases DCI from 0.6880 to 0.8224, whereas HP-FT yields 0.6684 despite reducing its five-step energy to 0.001068. DHP-FT also achieves higher DCI on MPI3D-toy (0.7304 versus 0.7174). In this comparison, DHP improves disentanglement while retaining off-diagonal Hessian energy in the sampling map.

Quantitative semantic control under latent interventions

Thank you for this helpful suggestion. Our quantitative metrics assess representation disentanglement, while our latent traversals and swapping experiments provide qualitative evidence of semantic control. We agree that directly measuring the effects of latent interventions would strengthen this evaluation.

On CelebA, we follow prior disentangled representation learning studies, including InfoDiffusion, EncDiff, and DyGA, in using TAD. As discussed in InfoDiffusion (Wang et al., 2023), TAD accounts for correlated and imbalanced attributes when assessing representation disentanglement on CelebA. This choice also allows comparison with prior results.

We are not aware of a widely adopted metric in unsupervised representation learning on CelebA that directly measures whether changing one latent coordinate modifies the intended attribute while preserving the others. We would be grateful for recommendations of suitable metrics or references and would be happy to supplement our evaluation with the corresponding experiments.

AuthorsOpenReview ↗

Response to Reviewer WK6t

Thank you again for your helpful suggestions. To further address your first concern, we have added paired generation results in Appendix F (Figures 5–7), complementing the direct comparison in Section 4.7.

Using matched source images and initial noise, these figures show substantial generation degradation for HP-FT under the directly regularized five-step sampler, while EncDiff w/BN1d and DHP-FT retain visually coherent outputs.

These results provide additional evidence about the practical effects of the regularization target beyond the representation metrics. We would appreciate your thoughts on this addition.

Thanks to the authors for the additional experimental evidence. Although quantitative evaluation of semantic controllability is still missing (and TAD is an established proxy for representation disentanglement), I appreciate the added comparison between DHP-FT and HP-FT. The results provide initial empirical evidence that DHP suppresses local latent interactions (measured by reduced local Hessian energy) while retaining global non-additivity in the complete sampling mapping (measured by nonzero global Hessian energy), associated with improved DCI disentanglement.

This addresses my main concern about the empirical validation of the proposed motivation, although the causal benefit of retaining global nonlinearity is not fully established. I hope the authors can further investigate this connection and provide more direct evaluation of semantic controllability in future work, beyond conventional metrics such as TAD.

Therefore, I would raise my score to 3 (Weak Accept).

Thank you for taking the time to review the additional results. Your suggestion to compare DHP and HP directly helped us strengthen the empirical evaluation. We also appreciate your recommendations for further investigating the role of global nonlinearity and evaluating semantic controllability more directly.

Reviewer_yL4EOpenReview ↗
Rating
2
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds on EncDiff: A trainable representation encoder embeds input images into 20-dim latents zz, which are used to condition the denoiser of a latent diffusion model (LDM). In practice, the latter is a diffusion or rectified flow model with noise or velocity prediction. A BatchNorm1d (BN1d) layer is applied to the latents before token embedding. At training time, a stochastic finite difference estimate of the Hessian Penalty (HP) is applied between the output of the LDM and the latents at each denoising step, coined Dynamic Hessian Penalty (DHP). DHP is implemented via central (DHP-C) and forward (DHP-F) differences and trained on several toy-data disentanglement benchmarks as well as CelebA.

This work builds upon recent approaches to disentanglement based on diffusion models, distinguishing itself from classical methods rooted in autoencoders (e.g., β\beta-VAEs). Here the disentangled representation does not have to reside within the same latent space used for generation. Instead, the denoising process—which transitions from noise to data—is conditioned using latent variables that are obtained separately. This decoupling enables significantly improved generation quality.

Critical strengths

The primary motivation for this paper is that applying a HP to the entire generative mapping (x1x_1 to x0x_0) would severely limit the expressive power of the generator. DHP, by contrast, applies this regularization indirectly.

The paper correctly argues that constraining the second-order mixed partial derivatives of a generative process would effectively reduce the model to a sum of functions of single latent dimensions. This would be overly restrictive, making it impossible to learn general mappings while simultaneously ensuring the emergence of disentangled representations.

Regarding its novelty, this work extends the Hessian penalty to the domain of diffusion models: it regularizes the denoiser at each individual step with respect to the conditioning; an approach that has not been explored previously.

Critical weaknesses

Two of the mentioned contributions are not new:

  • 1st contribution: The original work (Peebles et al., 2020) applied HP in practice to intermediate activations rather than to the entire generative mapping. The motivation for DHP—that “local additivity does not imply global additivity”—was thus already implicit in the practical application of HP and is not, in itself, new. In this regard, this work does not offer any novel theoretical analysis.

  • 2nd contribution: (Peebles et al., 2020) implemented HP using stochastic finite-difference methods and Rademacher noise. This therefore does not constitute a novel contribution and must be cited accordingly.

The paper does not provide empirical evidence for DHP to be a meaningful regularization force over baselines. Overall, the degree of novelty is limited, and the motivation is never directly examined:

  • The central claim of the paper is that applying the Hessian penalty per step (“additivity in the conditional dynamics”) is beneficial over applying it on the full trajectory. This, however, is not tested in the same pipeline and remains open. A necessary ablation would for example be to apply HP to a one-step prediction at high noise.

  • The claim that the learned generator remains non-additive after training is never verified. The paper does not measure the Hessian energy HH nor the raw regularization loss LDHPL_{DHP}. It also does not report the base loss as a function of λ\lambda, so the trade-off between denoising and regularization effects is invisible. Therefore, table 4 carries little to no insight into how DHP regularization affects the generative process in practice.

The experimental results do not conclusively support the claim that the DHP regularization has a significant effect on Disentanglement performance. Overall, reported gains over prior work are small:

  • The most conclusive experiment over the direct baseline, table 3 on MPI3D-toy, shows that the effects of the Hessian penalty (DHP-F and DHP-C) are not that significant: they are on the same order as the 0.03 (FactorVAE) and 0.02 (DCI) gap between the authors’ EncDiff re-run and the published number in table 1. The addition of BN1d alone has an effect of comparable size and is absent from the other datasets. The authors do not frame BN1d as an integral part of their approach and do not motivate why they added it over the EncDiff baseline. This confounding factor calls into question the motivation behind the approach, given that DHP regularization itself offers only small and unattributed effects on the only dataset where it is isolated—so why go to the trouble?

  • zz is not trained to follow any prior distributions, yet at generation time it is sampled from a standard normal to condition the LDM. Thus there is a distribution mismatch between training and inference. The addition of BN1d forces the latents to be normalized during training, which could mitigate the mismatch to some degree and ultimately explain the drastically improved FID scores over the baseline EncDiff in Table 2. There is however no indication that DHP drives this improvement, on the other hand it would be quite surprising that an additional regularization force improves generation quality and not hurt it. In any case, sampling latents without a trained prior is not a principled approach at generative modelling.

  • Table 4 investigates the effect of varying DHP regularization by sweeping λ\lambda from 0.0160.016 to 0.40.4. One would expect that it would have a direct effect on disentanglement (measured via FactorVAE and DCI), but the smallest tested regularization strength already shows the optimum, so it remains inconclusive if and how DHP affects the disentanglement performance.

Questions for authors
  1. Can you report the Hessian penalty metric HH and the loss LDHPL_{DHP} of trained models?
  2. Did you train a EncDiff model on CelebA with BN1d without DHP regularization? If so, what is the resulting FID score?
7 comments
AuthorsOpenReview ↗

Response to yL4E (1/3): Relationship to HP and direct comparison

We thank you for your detailed review and for recognizing the extension of HP to diffusion models. Your comments helped us clarify the distinction from the original HP, examine the regularization effects directly, and better separate DHP's contribution from that of BN1d.

From intermediate activations to conditional dynamics

The original HP study by Peebles et al. (2020) primarily evaluated GANs, including ProGAN and BigGAN. Our work considers diffusion models, whose generation process differs substantially. A GAN generates an image through a feedforward mapping, whereas diffusion repeatedly updates a noisy state. In the diffusion formulation used here, training evaluates predictions at noisy states constructed directly from data; sampling recursively updates the state using the model's predictions.

Applying HP to a GAN's intermediate activation hℓ(z)h_\ell(z) constrains its dependence on zz through all preceding layers. In diffusion, the analogous target would be an intermediate sampling state xt(z)x_t(z), with derivatives taken through the preceding sampling steps. DHP instead evaluates all latent perturbations at the same noisy state xtx_t and penalizes mixed latent derivatives of the conditional prediction Fθ(xt,t,c(z))F_\theta(x_t,t,c(z)). It therefore regularizes how zz influences the current update, rather than its accumulated effect on the current state, which is why we call it Dynamics Hessian Penalty. This target allows regularization during per-step training without unrolling the sampler.

During sampling, the evolving state also depends on earlier updates. Appendix A analyzes how this dependence can preserve nonlinear interactions in the complete mapping even when the fixed-state mixed derivatives vanish. The Hessian visualizations in Appendix D.6 further show that the 200-step sampling map, which outputs the final diffusion latent, retains nonzero off-diagonal Hessian energy after DHP regularization.

Estimators and attribution to HP

Appendix B.3 now derives the relationship between DHP-C and the original HP estimator. We have also added an explicit citation to Peebles et al. (2020) in the DHP-C description in Section 3.3.

DHP-F uses a forward mixed-difference construction not provided in the original HP formulation. It compares the effect of a joint perturbation with the sum of the separate effects and reuses the prediction computed for the base loss. It requires three additional predictions, compared with four for DHP-C. Appendix B discusses the construction and its approximation-accuracy trade-off.

Direct comparison with global HP

Following your suggestion, Section 4.7 (Table 7) directly compares DHP-C fine-tuning (DHP-FT) with HP applied to the final diffusion latent of a five-step DDIM sampler (HP-FT). Both start from the same EncDiff w/BN1d checkpoint on each dataset and retain Lbase\mathcal L_{\mathrm{base}} during 30,000 fine-tuning steps. We use five sampling steps to make full-trajectory HP training tractable.

DHP-FT reduces local Hessian energy while retaining 200-step sampling-map energy of the same order of magnitude as the starting model. On Cars3D, it increases DCI from 0.6880 to 0.8224. HP-FT reduces its five-step energy to 0.001068, but its local energy remains at 0.76823 and DCI falls to 0.6684. DHP-FT also achieves higher DCI on MPI3D-toy (0.7304 versus 0.7174). Thus, a smaller five-step sampling-map penalty does not consistently yield better disentanglement. Table 7 reports the full comparison, and Appendix D.6 provides the Hessian visualizations.

AuthorsOpenReview ↗

Response to yL4E (2/3): BN1d controls and generation quality

The role of BN1d and DHP's additional gains

We have expanded the explanation of BN1d in Section 3.1. We apply it without affine parameters before perturbation and tokenization to control representation scale. Otherwise, a fixed perturbation has different relative magnitudes as the latent scale changes. Rescaling zz, together with a compensating change in the tokenizer, can also alter the Hessian penalty while leaving predictions unchanged.

The gap between EncDiff's published and reproduced mean scores does not by itself show that DHP's gains are due to randomness. Locatello et al. (2019, Section 5.3) found that hyperparameters and random seeds could affect disentanglement scores more than the choice of objective in their evaluated settings. To assess DHP's contribution, we compare it with the BN-only baseline using the same ten training seeds for each method and report means and standard deviations across runs.

The original MPI3D-toy ablation in Table 3 uses the same ten training seeds and holds BN1d fixed when assessing DHP's contribution. Adding DHP-F/DHP-C increases mean DCI from 0.716 to 0.740/0.747 for noise prediction and from 0.686 to 0.731/0.726 for velocity prediction.

To extend this comparison beyond MPI3D-toy, we have added a matched BN-only control on Cars3D. All four methods below use the same training budget, seed set, VQGAN, and evaluation data. Results are means and standard deviations over ten training runs with different seeds.

MethodDCI disentanglementFactorVAE
EncDiff, reproduced0.539 ± 0.0570.950 ± 0.025
EncDiff w/BN1d0.637 ± 0.0550.992 ± 0.012
EncDiff w/BN1d + DHP-F0.752 ± 0.0600.994 ± 0.011
EncDiff w/BN1d + DHP-C0.780 ± 0.0630.997 ± 0.008

On Cars3D, BN1d increases mean DCI from 0.539 to 0.637. Adding DHP-F and DHP-C further raises it to 0.752 and 0.780, respectively, yielding additional gains of approximately 0.115 and 0.143. FactorVAE is already close to its ceiling with BN1d, so its additional gains are small.

These controlled comparisons show that DHP improves mean DCI beyond BN1d on both datasets.

Generation quality and the CelebA BN-only control

We agree that the comparison with published FID values alone does not establish that DHP improves generation quality. We evaluated the requested BN-only model alongside EncDiff and both DHP variants. In this additional control, all four methods encode the same 10,000 randomly sampled dataset images to obtain zz, then generate images starting from independent diffusion noise.

MethodFID ↓, image-derived representations
EncDiff, reproduced5.4106 ± 0.0908
EncDiff w/BN1d5.5758 ± 0.0552
EncDiff w/BN1d + DHP-F5.8142 ± 0.1188
EncDiff w/BN1d + DHP-C5.7713 ± 0.0837

We have revised the abstract, introduction, and experimental discussion to describe DHP as improving disentanglement while maintaining generation quality.

AuthorsOpenReview ↗

Response to yL4E (3/3): Measured losses and regularization weight

Measured losses and sensitivity to the regularization weight

Alongside the Hessian energy estimates in Section 4.7, we report below Lbase\mathcal L_{\mathrm{base}}, LDHP\mathcal L_{\mathrm{DHP}}, and disentanglement scores. Entries are means and standard deviations over ten independent training runs for each setting. EncDiff w/BN1d provides the λ=0\lambda=0 control.

MPI3D-toy

Methodλ\lambdaLbase\mathcal L_{\mathrm{base}}LDHP\mathcal L_{\mathrm{DHP}}DCIFactorVAE
EncDiff w/BN1d00.07963 ± 0.00021—0.7161 ± 0.03010.9138 ± 0.0294
DHP-C0.0160.08013 ± 0.000180.006744 ± 0.0009340.7449 ± 0.01260.9563 ± 0.0109
DHP-C0.080.08113 ± 0.000190.002775 ± 0.0002930.7466 ± 0.02090.9236 ± 0.0364
DHP-C0.40.08187 ± 0.000240.001079 ± 0.0001210.7455 ± 0.01510.9300 ± 0.0418
DHP-F0.080.08248 ± 0.000260.003901 ± 0.0002400.7397 ± 0.01400.9140 ± 0.0426

Cars3D

Methodλ\lambdaLbase\mathcal L_{\mathrm{base}}LDHP\mathcal L_{\mathrm{DHP}}DCIFactorVAE
EncDiff w/BN1d00.04941 ± 0.00100—0.6374 ± 0.05460.9916 ± 0.0122
DHP-C0.0160.05905 ± 0.002720.038516 ± 0.0253840.7450 ± 0.08290.9993 ± 0.0006
DHP-C0.080.06912 ± 0.004870.017875 ± 0.0099800.7804 ± 0.06280.9965 ± 0.0077
DHP-C0.40.06778 ± 0.003840.004598 ± 0.0022290.7968 ± 0.05710.9998 ± 0.0002
DHP-F0.080.06354 ± 0.009130.017740 ± 0.0087190.7518 ± 0.06010.9935 ± 0.0107

For DHP-C on MPI3D-toy, mean DCI increases from 0.7161 at λ=0\lambda=0 to 0.7449–0.7466 across the tested nonzero weights. Over this 25-fold range of λ\lambda, LDHP\mathcal L_{\mathrm{DHP}} decreases by approximately 84%, while Lbase\mathcal L_{\mathrm{base}} ranges from 0.08013 to 0.08187. The DCI gains over the BN-only control remain stable across these weights. One possible explanation is that these weights already provide sufficient local regularization, so further suppressing the interactions yields little additional DCI improvement.

On Cars3D, DHP-C improves mean DCI from 0.6374 at λ=0\lambda=0 to 0.7450–0.7968. Increasing λ\lambda from 0.016 to 0.4 reduces LDHP\mathcal L_{\mathrm{DHP}} from 0.038516 to 0.004598, while mean DCI increases from 0.7450 to 0.7968. Sensitivity to the regularization weight therefore also depends on the dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.