acceptodds
Under review as a conference paper at ICLR 2027

Learning Asymmetric Laplacian Representations in Hilbert Space

Abstract

Laplacian representation learning is a highly promising generally and in reinforcement learning, where an important challenge is to retain the asymmetries of directed and stochastic dynamics in the environment. In the self-adjoint setting, prior methods use ordered orthogonality and a smoothness objective to learn a set of spectral representations corresponding to lowest-magnitude eigenvalues. Such ordering dynamics does not extend to the asymmetric Laplacian. Unlike a self-adjoint Laplacian, an asymmetric Laplacian can have complex eigenvalues and non-orthogonal complex eigenfunctions; that is, its right and adjoint-left eigenfunctions can differ and form bi-orthogonality. These properties can complicate approaches that attempt to learn an ordered eigenbasis. One might therefore expect an asymmetric solution to require a substantially more complicated approach. We instead keep the representation learning problem to a similar complexity of prior symmetric approaches and find that, by targeting an unordered subspace, we can learn vectors that approximate mixtures of the asymmetric Laplacian eigenvectors we seek. Careful unmixing then yields good approximations of low magnitude asymmetric eigenfunctions. We compare the obtained asymmetric eigenfunctions with oracle solutions using cosine similarity across multiple domains. Additionally, we show that the learnt eigenfunctions can be used in domains to obtain hitting time estimates and compare them to the prior symmetric Laplacian based commute-time distance, showing improved distance learning in directed and stochastic settings.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Laplacian representations, asymmetric Laplacian, eigenvectors, eigenvalues, Hilbert space, representation learning, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1UuVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors study representation learning in reinforcement learning based on asymmetric Laplacian operators. Many existing Laplacian representation-learning approaches assume symmetric (self-adjoint) operators, whereas asymmetric Laplacians may have complex eigenvalues and non-orthogonal eigenfunctions, potentially making direct extensions of existing approaches challenging. The authors propose to address this problem through a two-phase approach. First, a neural network learns real-valued feature vectors whose span is approximately invariant under the forward transition operator. Second, reduced transition matrices are estimated for different horizons, and the eigenvectors of a weighted combination of these matrices are used to construct approximations of the asymmetric Laplacian eigenfunctions. In the experiments, the authors evaluate the proposed framework in simulated grid and drone environments.

Critical strengths
  • The paper addresses an interesting and technically challenging problem. Learning representations that capture asymmetric properties of transition dynamics is not only practically relevant but also interesting in its own right.
  • The proposed two-phase approach appears to be nontrivial, well motivated, and sensibly designed. The idea of avoiding the complexity of directly learning complex-valued eigenfunctions of asymmetric Laplacians could potentially find applications in other domains.
  • The paper is well written. While I am not an expert in this particular domain, I was able to follow the main ideas reasonably well.
Critical weaknesses
  • While the motivation from the subspace iteration literature is reasonable, the paper does not seem to provide a rigorous theoretical analysis of the quality of the spectral approximation obtained through the proposed scheme, which encourages approximate invariance under the forward transition operator. This is not necessarily a serious limitation in itself and could be compensated for by extensive experiments and validation in real-world settings. However, the experiments are limited to simulated environments. Given the limited theoretical analysis and the absence of real-world experiments, I feel that there is considerable room to strengthen the contribution.

  • Since the asymmetric Laplacian is defined as L=I-P, its eigenfunctions are identical to those of the Markov transition operator P. Furthermore, the proposed learning scheme appears to operate directly on P. In this sense, I was wondering what additional advantage is gained by formulating the problem in terms of the asymmetric Laplacian rather than directly considering the spectral properties of the transition operator. In this respect, I think the paper would benefit from a more detailed discussion of how the proposed framework compares with existing approaches based on Markov or Koopman operators.

Minor comment

Some tables report numerical values to five decimal places, resulting in rather small font sizes. The authors could consider reducing the number of decimal places to improve readability.

Questions for authors

Please see Weaknesses section.

1 comment

We thank the reviewer for their thorough reading of the paper and address their concerns below.

Weakness 1.

The application of asymmetric spectral methods to real-world settings is part of the motivation for this paper. Notably, the symmetry assumption in prior Laplacian methods can be problematic in real-world settings. Whilst evaluation on physical robots remains a future direction for this line of research, we find it important to follow the same evaluation procedures as prior symmetric Laplacian methods, such as GDO, GGDO, and ALLO. Specifically, one experimental limitation is that, to evaluate the quality of our learned spectral representations directly, we need environments where we can compute or estimate oracle eigenvectors.

There are two ways in which we can think about the quality of our representations: first, how accurately they represent the underlying spectrum; and second, how they can be used for downstream tasks. For the first, we follow the direction of related papers, including GGDO and ALLO, by evaluating how well the learned representations approximate oracle eigenvectors in environments for which we can obtain them.

That said, in light of the reviewer’s comment, we have included the application of ALR based hitting-time distance functions in simulated robotic control domains. We consider variations of Ant and Humanoid control tasks as start-and-goal tasks, where we use ALR distance functions for zero-shot goal tasks with policies from OGBench trained on a different domain than evaluated. Specifically, the distances are used to find intermediate waypoints for the pretrained agents to follow. The full details are provided in Section 4 and Appendix F.

In these experiments, we show clear improvements in using ALR hitting-time distance estimates to solve start to goal tasks compared with oracle RA-LapRep, an oracle version of the symmetric commute-time-based distance approach from Wang et al. [1]. Hitting time is the natural directed version of their commute-time distance, as such their comments on the challenge of estimating such as distance from the spectrum in directed settings is worth noting. Table 1 reports mean success rates of 54–96% for ALR across the six domain–controller combinations, compared with 0% for oracle RA-LapRep in this waypoint-planning setup. Here, RA-LapRep is given oracle eigenvectors (to ensure approximation isn't the reason they fail), whilst ALR distances are obtained from learned image representations. This comparison shows the downstream usefulness of learning spectral distances that retain the asymmetry of the environment.

Laplacian representations have already been found useful in reinforcement learning for distance estimation, bottleneck discovery, planning, and option discovery. We therefore consider this a non-trivial step in the right direction. We hope the reviewer will agree that these additional experiments strengthen the contribution by demonstrating the applicability of our method and making progress on the limitation identified in prior symmetric Laplacian distance methods.

Weakness 2.

Thank you for the suggestion. We have updated the manuscript to include a range of Koopman operator-learning baselines: Neural Power Iteration (NPI) [2], DPNet and its variation DPNet Relaxed [3], EncoderOps [4], and inexact subspace iteration (ISI) [5]. For ISI, we include both the Müller–Brown and AIB9 parameter settings, together with the released implementation and a corrected loss-scaling described in Section 4. We show these comparisons in Figure 3 of the updated manuscript. Specifically, in the four drone domains, we show that the ALR method obtains substantially higher mean cosine similarities for the first 20-50 eigenvectors associated with the lowest-magnitude Laplacian eigenvalues. We hope that these comparisons resolve the reviewer's minor concerns on the soundness of the proposed approach.

References:

[2] Berger, G. O., and Jungers, R. M. Data-driven Koopman mode approximation: A neural power iteration algorithm. arXiv:2608.26943, 2026.

[3] Kostic, V. R., Novelli, P., Grazzi, R., Lounici, K., and Pontil, M. Learning invariant representations of time-homogeneous stochastic dynamical systems. ICLR, 2024.

[4] Turri, G., Bonati, L., Zhu, K., Pontil, M., and Novelli, P. Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems. ICLR, 2026.

[5] Strahan, J., Guo, S. C., Lorpaiboon, C., Dinner, A. R., and Weare, J. Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction. The Journal of Chemical Physics, 159(1):014110, 2023.

Reviewer_KW74OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Laplacian representation learning is a widely useful approach for representation learning. However, most prior work on both Laplacian representation learning and the Laplacian operator itself focuses on the symmetric case, where an orthogonal eigenbasis is guaranteed—a property that does not hold for the asymmetric Laplacian. Consequently, despite the many benefits of asymmetric Laplacian-based methods, techniques developed for the symmetric case—including representation learning algorithms—generally do not transfer well to the asymmetric setting.

This paper proposes an efficient method for learning eigenfunctions in the asymmetric case. It achieves this by first learning an approximately invariant subspace and then unmixing the learned representations to recover the true eigenfunctions, particularly those associated with low-magnitude eigenvalues.

Critical strengths
  • Proposes an efficient method to learn asymmetric Laplacian eigenfunctions, a problem previously considered substantially more difficult.

  • Provides reasonable theoretical justifications; although not strictly rigorous, they are still beneficial for explaining the underlying mechanism of the proposed method.

  • Demonstrates strong empirical performance against baselines across several numerical experiments.

Critical weaknesses
  • Hypothetically, the learned approximately invariant subspace might only capture a small portion of the entire space. However, I expect it generally spans the necessary subspace in real-world applications.

  • Some explanations are not entirely precise. For example, the paper states: "We therefore learn with both the invariance loss from equation 12 and an orthogonality loss that discourages the span from collapsing and encourages a sufficient span to overlap with the desired spectral directions of P associated with low magnitude eigenvalues." While likely true in practice, the explanation for why this specifically promotes overlap with the spectral directions of low-magnitude eigenvalues is not completely convincing. Also, some of the terminology could be more accurate. On page 3, the authors state: "The off-diagonal terms discourage duplicate directions; the diagonal terms preserve the mean-square norm of each function, preventing the norm from collapsing to zero." This indicates that the "orthogonality loss" also includes components that regularize the norm of each f_i, rather than solely promoting orthogonality.

Questions for authors
  1. The paper notes that learning asymmetric distances and asymmetric Laplacian representations that scale to practical settings (such as robotic domains) is a "highly non-trivial challenge," further citing the difficulty of learning complex-valued eigenvalues. Could the authors provide more concrete explanations or examples to elaborate on these specific difficulties?

  2. The paper states that the problem setting is continuous and high-dimensional, yet it repeatedly relies on analogies and notation from the finite-dimensional setting. Is the underlying formulation still assumed to be finite-dimensional? In the infinite-dimensional case, the existence and uniqueness of the stationary distribution are not guaranteed, which could theoretically pose an issue (though perhaps not for the empirical results or practical applications of this paper).

  3. Are the eigenvalues of the Laplace operator assumed to be all distinct in general, while they can be complex-valued? This is just for clarification, not an objection.

  4. Pages 4 and 5 contain helpful explanations regarding the problem setup. The overall presentation might benefit from moving some of this context into the introduction.

1 comment

Weakness 1.

In response to Reviewer 3, we have included comparative results against related baselines in Figure 3 and compared the first d Laplacian eigenvectors for dimensions 20–50 (ordered in lowest magnitude associated eigenvalue). Within experiments evaluated across seeds, ALR obtains a clear performance improvement compared to the related methods obtaining good approximate recover up to 50 dimensions. With neural network learning there are many reasons why a model could perform poorly, however, based on competitive evaluation I would be inclined to suggest that ALR is relatively high performing in terms of it's ability to capture a larger portion of the spectrum.

Weakness 2.

One important consideration is the effect of discounted multi-step sampling on the invariant loss. To illustrate this, consider an extreme example in which we optimize

∥∣Fθ(Xi)K−sg⁡(Fθ(Yi))∥∣2,\||F_\theta(X_i)K-\operatorname{sg}(F_\theta(Y_i))\||^2,

where XiX_i is the current-state sample and YiY_i is sampled uniformly at random from the entire state space, independently of XiX_i. With a sufficiently large batch, the prediction Fθ(Xi)KF_\theta(X_i)K would be encouraged to approximate the average of sg⁡(Fθ(Yi))\operatorname{sg}(F_\theta(Y_i)) across the entire state space. Consequently, predictions exhibiting substantial variation across states would incur a larger loss, as they would be required to approximate the same global average.

Although this is an extreme example, similar reasoning applies to discounted multi-step sampling. Under this sampling procedure, a given state can be associated with a range of future states, depending on the sampling discount factor and the environment dynamics. Across training updates, Fθ(Xi)KF_\theta(X_i)K is therefore encouraged to approximate the expected feature values over these future states estimating toward an average of their associated values. As the sampling horizon increases, this can discourage temporal variation in the learned representations, since rapidly varying features may be more difficult to predict consistently across multiple future horizons.

This provides an intuitive explanation for why ALR may favor more slowly varying spectral modes, which are generally associated with lower-magnitude Laplacian eigenvalues. Empirically, we observe that ALR achieves high recovery accuracy for eigenvectors associated with the lowest-magnitude eigenvalues, with recovery accuracy generally decreasing as the number dd of recovered eigenvectors increases, ordered by increasing eigenvalue magnitude. For the other question, we have revised the manuscript to distinguish more clearly between the cross-feature orthogonality term and the norm term in the training objective.

Q1.

First, we would like to clarify that the phrase “highly non-trivial challenge” is taken directly from Wang et al. (2023, RA-LapRep, Section 5.1) [1]. Their concerns include both how to define an appropriate directed version of RA-LapRep and how to design an optimization objective to approximate the eigenvectors of a directed Laplacian, which may have complex-valued eigenvalues.

In response to Reviewer 3’s comments, we have included applications of ALR-based hitting-time distance functions in simulated robotic control domains, where we consider variations of Ant and Humanoid control tasks as start-and-goal tasks. The full details are provided in Section 4 and Appendix F. We also expect that there will be further ways to extend existing approaches to estimating directed distances from a truncated asymmetric Laplacian spectrum. These could improve on the distance-learning results we present, including through alternatives to hitting time that use the same representations obtained by ALR.

Q2. We consider both finite and continuous state spaces, but focus on approximating a finite set of eigenfunctions associated with individual eigenvalues of the asymmetric Laplacian. We do not consider continuous spectral components.

Q3. We assume distinct eigenvalues which may be complex-valued, however, repeated eigenvalues would not in theory prevent our unmixing solutions. Repeated eigenvalues are found only in very distinct settings, such as special symmetries which would be quite surprising to observe in the type of sampling settings we consider.

[1] Wang et al. Reachability-Aware Laplacian Representation in Reinforcement Learning, ICML, 2023

Reviewer_qiXCOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends well established approaches for symmetric Laplacians to non-symmetric Laplacians.

Critical strengths

The paper does not have any theory but has a substantial empirical section.

Critical weaknesses

The paper is quite poorly written. Notation is often not introduced or first used and then introduced later in the paper. The main method seems quite straightforward but is poorly explained and motivated.

Detailed comments:

The preliminary section uses a lot of space for standard results. Do you really need to inline what a Hilbert space is and a Markov kernel. There is also a technical mistake here: Radon-Nikodym doesn't give you the density; you need to combine that argument with a countable generated target space and an approximation process. It was also not clear to me if operators with continuous spectrum are considered. I imagine not, but it would be useful to state this. Overall it feels to me like the intuition was developed for finite states and then it was claimed that it holds in much more generality; e.g. the notation of mu^top etc works well in finite spaces but is weird when working with bigger state spaces. Also phi^r seems to have changed to phi at multiple places. Notation is also messy; first f_i are the components of F_theta then f_theta is defined F_theta^top.

Section 3: The discussion of the spectral overlap and the difficulties with DNNs is extremely vague.

Questions for authors

Section 2: What is X_i and Y_i in the GDO definition?

Section 3: What's K? Can you be more precise.

What is K_gamma K before (11)? At this point one also does not know what unmixing coefficients are meant to be.

I also don't understand why you have an orthogonality loss. The eigenvectors are now not orthogonal anymore so why aim for orthogonality?

Something is wrong in (13). You seem to optimize for K but J_orth is not depending on K. Is that meant to learn any basis of an invariant subspace?

1 comment

We thank the reviewer for their constructive questions and thorough reading of the paper.

Weaknesses

Weakness 1. Notation is often not introduced or first used and then introduced later in the paper.

We have updated the notation throughout the paper and improved the introduction of notation prior to its use.

Comments

Comment 1. The preliminary section uses a lot of space for standard results.

Thanks for the feedback. We have moved the Markov decision process section to Appendix B, with a reference in the main text.

Comment 2. Radon–Nikodym density.

Thanks for spotting this issue. Following the absolute-continuity assumption in GDO, we assume that Pπ(u,⋅)P^\pi(u,\cdot) is absolutely continuous with respect to ρ\rho for every u∈Su\in S. Additionally, we explicitly assume that these Radon–Nikodym derivatives admit a jointly measurable version k(u,v)k(u,v), as now stated in Appendix B.1.

Comment 3. It was also not clear to me if operators with continuous spectrum are considered.

We consider both finite and continuous state spaces, but focus on approximating a finite set of eigenfunctions associated with individual eigenvalues of the asymmetric Laplacian. We do not consider continuous spectral components.

Comment 4. Also ϕr\phi^r seems to have changed to ϕ\phi at multiple places. Notation is also messy; first fif_i are the components of FθF_\theta then fθf_\theta is defined Fθ⊤F_\theta^\top.

Thanks for the feedback. We have updated the notation.

Questions for Authors

Q1. What is XiX_i and YiY_i in the GDO definition?

XiX_i is the current state sample, and YiY_i is the future state sample for the iith training pair. This description is in the updated manuscript.

Q2. Section 3: What's KK? Can you be more precise?

Assume P\mathcal{P} is the linear operator defined in the paper, and assume Fθ=[fθ,1,fθ,2]F_\theta = [f_{\theta,1}, f_{\theta,2}] contains two linearly independent real-valued vectors whose span is invariant under P\mathcal{P}, meaning that PFθ\mathcal{P}F_\theta cannot introduce any vector directions outside the span of FθF_\theta. Then PFθ=FθK\mathcal{P}F_\theta = F_\theta K, where K∈RD×DK \in \mathbb{R}^{D \times D}, with D=2D=2 in this example, giving PFθ=[K11fθ,1+K21fθ,2,  K12fθ,1+K22fθ,2].\mathcal{P}F_\theta = [K_{11}f_{\theta,1}+K_{21}f_{\theta,2},\; K_{12}f_{\theta,1}+K_{22}f_{\theta,2}]. If the vectors in FθF_\theta are eigenvectors of P\mathcal{P}, with eigenvalues ω1\omega_1 and ω2\omega_2, then K=diag⁡(ω1,ω2)K = \operatorname{diag}(\omega_1,\omega_2). However, the vectors in FθF_\theta can also be mixtures of eigenvectors. For instance, let Fθ=[aϕ1r+bϕ2r,  cϕ1r+dϕ2r],F_\theta = [a\phi_1^r+b\phi_2^r,\; c\phi_1^r+d\phi_2^r], where ϕ1r,ϕ2r\phi_1^r,\phi_2^r are real eigenvectors of P\mathcal{P} with distinct eigenvalues ω1,ω2\omega_1,\omega_2, and a,b,c,da,b,c,d are real coefficients satisfying ad−bc≠0ad-bc\neq 0. In this case, KK can have nonzero off-diagonal entries. For example, when a,b≠0a,b\neq 0, the entry K21K_{21} is nonzero. Thus, KK describes how applying P\mathcal{P} combines the learned vectors. Nonzero off-diagonal entries in KK also imply that at least one feature in FθF_{\theta} is not an eigenvector.

Section 3: The discussion of the spectral overlap and Q4. I also don't understand why you have an orthogonality loss?

We have updated the paper to include more discussion of spectral overlap and its relation to neural networks. The orthogonality term is applied to the learned feature basis; it does not require the eigenvectors recovered after unmixing to be orthogonal.

Specifically, without the cross-feature orthogonality term in our loss, the network can learn trivial solutions to the invariance loss. This can lead to significant span collapse and remove spectral overlap that may have existed when the network's output features were initialized.

We discuss this in Section 3.1 and show it empirically through additional baseline comparisons in Figure 3. Specifically, when the cross-feature orthogonality term does not provide a gradient to the network, we observe significantly worse performance across the four drone domains compared with the standard ALR approach, which includes this term.

Q5. Something is wrong in (13). You seem to optimize for KK but JorthJ_{\mathrm{orth}} is not depending on KK. Is that meant to learn any basis of an invariant subspace?

We have updated the paper to make the learning parameters more explicit. In Phase 1, we jointly learn KK and the network parameters θ\theta, which update FθF_\theta. Thus, JorthJ_{\mathrm{orth}} contributes to the optimization through θ\theta, even though it does not depend on KK. Phase 1 is where we learn to increase the span of our features whilst also training them to be approximate invariant to P\mathcal{P}. It is only in Phase 2 that we freeze θ\theta, keeping FθF_\theta fixed, and learn only the small KK matrices. This avoids estimating KK while the feature basis itself is changing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.