acceptodds
Under review as a conference paper at ICLR 2027

Persistent Cross Entropy

Abstract

Persistent entropy is the Shannon entropy of a persistence-based probability measure defined on a persistence diagram. However, its cross-entropy version is not naturally defined because two persistence diagrams generally have different event spaces. To bridge these different event spaces, we utilize a similarity function with persistence weighting to introduce and define an induced probability. The induced probability reflects information from one diagram on the event space of the other diagram and assigns unexplained probability mass to the unexplained event. Using the induced probability, we introduce persistent cross entropy (PCE), a new extension of Shannon cross entropy to persistence diagrams. We establish the main properties of both the induced probability and PCE and prove stability theorems for the induced probability and the entropy excess. Through three numerical studies, we show that PCE distinguishes diagrams that have nearly identical persistent entropy, separates causal directions in a linear dynamical system without constructing a joint persistence diagram, and can be used as a directional topology loss for knowledge distillation.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
topological data analysis, persistent entropy, persistent cross entropy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jJg4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces PCE as a new directional measure for comparing persistence diagrams. It constructs an induced probability distribution using kernel-based similarity and persistence weighting, derives stability properties, and evaluates the measure on synthetic topological examples.

Critical strengths

The paper addresses a potentially useful problem of comparing persistence diagrams with different numbers of features. The proposed construction is mathematically explicit, and the use of PCE as a differentiable topology-related loss is potentially interesting.

Critical weaknesses

My main hesitation is limited novelty and questionable conceptual foundation.

  1. The proposed PCE essentially reduces to a persistence-weighted squared difference between kernel-smoothed diagram representations. The KL-divergence identity and stability results largely follow from the chosen construction and standard Lipschitz arguments, rather than establishing a fundamentally new information-theoretic principle.

  2. The induced probability is constructed by reweighting the reference distribution rather than directly comparing the two persistence-based distributions. Its interpretation as cross entropy is therefore somewhat artificial. More seriously, PCE can be zero for distinct persistence diagrams, so it does not necessarily capture meaningful topological discrepancies.

  3. Also, the paper seems to make unjustified causal interpretation. The asymmetry of PCE does not imply causal directionality. Differences between its forward and reverse values can arise solely from persistence magnitudes or diagram structure, without any causal relationship. The paper provides no theoretical identification argument connecting this asymmetry to causality.

  4. Most theoretical properties characterize the behavior of the specifically engineered measure rather than establish its superiority over existing diagram distances, kernels, or other topological discrepancies.

Questions for authors

Please address my concerns about both the conceptual justification and methodological contribution as listed in the weakness section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer jJg4

To the best of our knowledge, this is the first rigorous formulation of persistent cross entropy (PCE). Our main contribution is to extend persistent entropy to an ordered comparison between persistence diagrams through the induced-probability construction and establish its mathematical properties. We address your questions and concerns below.

  1. The proposed PCE essentially reduces to a persistence-weighted squared difference between kernel-smoothed diagram representations. The KL-divergence identity and stability results largely follow from the chosen construction and standard Lipschitz arguments, rather than establishing a fundamentally new information-theoretic principle.

Our objective is to formulate a cross-entropy extension of persistent entropy. Although the entropy excess can be written as a persistence-weighted squared difference, this does not make it equivalent to the distance used in persistence-weighted kernel methods.

For example, let X={(0,1)}X=\{(0,1)\} and Ya={(0,1),(a,1)}Y_a=\{(0,1),(a,1)\} for a>0a>0, where each pair gives birth time and lifetime. With fixed Gaussian bandwidths and lifetime weights, the common point cancels in the kernel distance. The remaining point has the same lifetime wherever it is placed, so this distance is unchanged as aa varies. PCE instead evaluates the response difference at the point in XX. A nearby additional point contributes more to that response and gives a larger PCE, while moving it farther away reduces PCE. Thus, even with XX fixed, the kernel distance alone does not determine PCE.

The KL identity follows directly from our induced-probability construction. Proposition 3 also shows why bounded weights are required for our diagram-independent stability bounds. With unbounded weights, the unexplained mass can remain strictly positive even as the 11-Wasserstein distance tends to zero.

  1. The induced probability is constructed by reweighting the reference distribution rather than directly comparing the two persistence-based distributions. Its interpretation as cross entropy is therefore somewhat artificial. More seriously, PCE can be zero for distinct persistence diagrams, so it does not necessarily capture meaningful topological discrepancies.

The two persistence-based distributions are defined on different event spaces, DXD_X and DYD_Y. Applying cross entropy therefore requires an additional construction that places the probabilities on a common event space. This is precisely the role of the induced probability. We use information from DYD_Y to construct pXYp_X^Y on DX∪{∂}D_X\cup\{\partial\}, and extend pXp_X to the same space by setting pX(∂)=0p_X(\partial)=0. We then apply the standard definition of cross entropy to these two probability distributions. The reweighting addresses the event-space mismatch and recovers standard persistent entropy when a diagram is compared with itself.

For clarity, we address the zero condition of the entropy excess. It can indeed vanish for distinct persistence diagrams. However, failure to distinguish every pair of diagrams does not by itself make a topological summary uninformative. Persistent entropy, Betti curves, and persistence silhouettes can also agree for distinct diagrams, giving zero differences between their summaries. Our objective is to extend persistent entropy to an ordered comparison, and we explicitly characterize what zero entropy excess means. As stated in Propositions 1(ii) and 2, ΔHXY=0\Delta H_X^Y=0 holds exactly when the weighted kernel responses of the two diagrams agree at every point of DXD_X, equivalently when pXY=pXp_X^Y=p_X. Thus, zero expresses a precisely defined agreement under the chosen comparison. The reverse comparison evaluates the corresponding agreement on DYD_Y.

  1. Also, the paper seems to make unjustified causal interpretation. The asymmetry of PCE does not imply causal directionality. Differences between its forward and reverse values can arise solely from persistence magnitudes or diagram structure, without any causal relationship. The paper provides no theoretical identification argument connecting this asymmetry to causality.

Our motivation follows an established idea in causal inference for coupled deterministic dynamical systems. The response can retain information about the driver, allowing driver states to be reconstructed from response observations. CCM examines this through cross-map reconstruction (Science, 2012), while TC examines the local expansion of mappings between reconstructed state spaces (Physical Review Letters, 2017).

Following the persistent-homology benchmark of Bando et al. (JSIAM Letters, 2022), we investigate this idea through persistence diagrams. In the spring–mass equations, the driver x(t)x(t) can be recovered from the response y(t)y(t) and its second derivative. If the response diagram retains geometric structure from the driver, we expect it to explain the driver’s diagram better than the reverse. This is the hypothesis tested in Section 5.2, and we will make its motivation and scope explicit.

  1. Most theoretical properties characterize the behavior of the specifically engineered measure rather than establish its superiority over existing diagram distances, kernels, or other topological discrepancies.




Our core contribution is the formulation of persistent cross entropy and the proof of its mathematical properties. We establish valid induced probabilities, recovery of persistent entropy under self-comparison, and stability under explicit assumptions. These results give the new concept a rigorous theoretical foundation. Symmetric diagram distances address a different comparison objective. Our experiments are designed to demonstrate the use of PCE in applications requiring ordered comparisons, rather than to establish general superiority over symmetric methods.

AuthorsOpenReview ↗

Follow-up on zero conditions: Appendix C

We have added Appendix C to clarify the zero conditions. It includes an explicit example where one-directional PCE is zero for distinct diagrams. Proposition 4 further proves that, for nonempty finite diagrams, Gaussian similarity with fixed positive bandwidths, and g(ℓ)>0g(\ell)>0 for every ℓ>0\ell>0, $ \Delta H_X^Y=\Delta H_Y^X=0 \quad\Longleftrightarrow\quad D_X=D_Y, $ including multiplicities. The possibility of a zero value in one direction therefore does not invalidate the two-direction comparison.

The authors' clarifications and the additional result on two-direction separation have addressed some of my conceptual concerns. I thank for that. However, I still remain unconvinced that the proposed construction offers sufficient methodological novelty or demonstrated benefit over existing approaches, which are my main concerns, setting aside all the technical details. The causal interpretation also remains a benchmark-specific hypothesis, which further limits the paper’s persuasiveness.

Reviewer_qvWxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces the notion of persistent cross entropy (PCE), extending the existing notion of persistent entropy (a measure of how similar the features in a persistence diagram are) to allow for comparison between two different diagrams. They establish theoretical results concerning the stability of the entropy excess, and demonstrate that PCE can distinguish between diagrams with nearly identical persistent entropy, as well as providing case studies (a linear dynamical system and a topological knowledge distillation experiment) which highlight some of the capabilities of PCE as a directional comparison tool.

Critical strengths

The paper is well written and very clearly presented, and to the best of my knowledge this is a novel extension of the existing literature on persistent entropy. The authors introduce the relevant prerequisite knowledge of persistence diagrams and persistent entropy, discuss the theoretical obstruction which prevents one from just taking the naive extension of persistent entropy to a cross-entropy setting (the fact that the event spaces for two diagrams will not be shared in general) and then present their method for overcoming this (a similarity-based score indicating how well features in one diagram are explained by the other, which are then in turn used to define induced probabilities on a shared event space). The results theorems are a natural extension of existing results for persistent entropy [1] to the PCE setting, and their inclusion gives the paper a strong theoretical grounding.

[1] N. Atienza, R. Gonzalez-Diaz, M. Rucco, Persistent Entropy for Separating Topological Features from Noise in Vietoris-Rips Complexes

Critical weaknesses

While the extension of the persistent entropy literature is novel to me, I'm not completely convinced of the usefulness of the methodology, and in particular I feel that the experimental section of the paper requires significant strengthening to demonstrate either that PCE is definitively better than existing approaches, or that it offers something that other approaches lack.

The first experiment, while a nice example of why PCE can tell us more than simply looking at the persistent entropy, isn't particularly surprising (I see no reason why we would expect diagrams with similar persistent entropy not to differ) and should maybe be moved to earlier in the paper simply as a demonstration of PCE (this is more a matter of personal taste, however).

The second and third experiments, designed to show the performance of PCE against existing approaches, are where I feel additional work is needed. In the spring-mass experiment, PCE is shown to perform better at separating the coupling regimes than the other methods, with it being mentioned that PCE is the only method which does not require construction of the joint persistence diagram. This seems like a unique selling point for PCE which should be highlighted, if in fact there are no other methods which do this. It seems to me though that such a claim is unsupported, so I would require further explanation here to be convinced that PCE is indeed the best choice. In the knowledge distillation experiment, PCE exhibits modest gains over two competing methods, but I would like to see a runtime comparison at least to determine whether PCE is likely to be used in this situation in practice.

Questions for authors
  1. If possible, can you discuss in more detail whether any other existing directional approaches exist and, if they do, compare their performance to that of PCE in the spring-mass experiment. Alternatively, are there any ways to slightly adapt the baselines you have chosen in a different manner to computing the joint diagram?

  2. Could I see details of the runtime of PCE compared against the competing methods in the knowledge distillation example? Also, from the appendix the standard PCE seems to display greater variance than the other methods, I feel this should be included in the main table.

  3. I'd expect your results to be sensitive to the choice of the global scales σb\sigma_b and σl\sigma_l in your Gaussian kernel (from the discussion following Definition 1). While you describe in the appendices the choices you make, it would be useful to have a little more discussion to justify your choice. It might also be interesting to see what happens if we keep one scale fixed and allow the other to vary to see whether PCE discriminates more based on birth values than lifetimes, or vice versa, but that's more just personal interest on my part.

  4. Defining your probabilities in terms of the lifetime of a feature (Section 2.2) could do with some additional justification - you mention that long-lived features are less likely to be noise, but this seems to imply that we don't care at all about the birth times of the features. Maybe this is fine, but I'd like to see if you considered alternative approaches (I can see that this method is used in the existing literature so that's probably the reasoning, but it would be nice to know).

3 comments

To the best of our knowledge, this is the first rigorous formulation of persistent cross entropy (PCE). Our main contribution is to extend persistent entropy to an ordered comparison between persistence diagrams through the induced-probability construction and establish its mathematical properties. We address your questions and concerns below.

  1. If possible, can you discuss in more detail whether any other existing directional approaches exist and, if they do, compare their performance to that of PCE in the spring-mass experiment. Alternatively, are there any ways to slightly adapt the baselines you have chosen in a different manner to computing the joint diagram?



The central contribution of this paper is the formulation of persistent cross entropy between diagrams with different event spaces. As in conventional cross entropy, this formulation assigns different roles to the two inputs and therefore requires an asymmetric construction. Changing the persistence weights or improving a symmetric diagram distance alone does not provide the induced probability needed for this formulation.

Representative examples of existing directional approaches to causal inference in dynamical systems include Topological Causality (TC), introduced by Harnack et al. in Physical Review Letters in 2017, and Convergent Cross Mapping (CCM), introduced by Sugihara et al. in Science in 2012. For the requested comparison, we evaluated both methods using the same spring–mass trajectories and delay embeddings. PCE and TC correctly identified the coupling direction in all 16 one-way settings, while CCM’s full-library scores favored the reverse direction in every case. These results provide additional context for the application, while our main contribution concerns the construction of persistent cross entropy.

Regarding your question about the joint diagram, in Section 5.2 we followed the multivariate delay reconstruction of Cao et al. (1998), as used by Bando at el. (2022). Each individual reconstruction uses four delay coordinates, and the joint reconstruction combines two coordinates from each system to retain dimension four. Alternatively, concatenating both full delay vectors retains all eight coordinates. These choices can produce different joint persistence diagrams. In this respect, PCE offers the advantage of defining both ordered comparisons directly from DAD_A and DBD_B, without requiring an additional choice of joint reconstruction.




  1. Could I see details of the runtime of PCE compared against the competing methods in the knowledge distillation example? Also, from the appendix the standard PCE seems to display greater variance than the other methods, I feel this should be included in the main table.



KD, TopKD, and PCE took 42.9, 45.0, and 51.7 seconds per epoch, respectively, including evaluation. TopKD approximates persistence images with a separately pretrained RipsNet, whose pretraining time is excluded from these timings. PCE computes exact H0H_0 persistence diagrams directly. We will add the runtime details to the appendix and report accuracy as mean ± sample standard deviation across the five seeds in the main table.

  1. I'd expect your results to be sensitive to the choice of the global scales and in your Gaussian kernel (from the discussion following Definition 1). While you describe in the appendices the choices you make, it would be useful to have a little more discussion to justify your choice. It might also be interesting to see what happens if we keep one scale fixed and allow the other to vary to see whether PCE discriminates more based on birth values than lifetimes, or vice versa, but that's more just personal interest on my part.

In the spring–mass experiment, we used σb=0.40\sigma_b=0.40 and σℓ=0.15\sigma_\ell=0.15. For the sensitivity analysis, we varied σb\sigma_b over {0.20,0.30,0.40,0.60,0.80}\{0.20,0.30,0.40,0.60,0.80\} with σℓ=0.15\sigma_\ell=0.15, and varied σℓ\sigma_\ell over {0.075,0.1125,0.15,0.225,0.30}\{0.075,0.1125,0.15,0.225,0.30\} with σb=0.40\sigma_b=0.40. We held the response scale τ\tau fixed throughout. PCE recovered the correct coupling direction in all 16 one-way settings at every tested bandwidth.

We use the same bandwidths across diagram pairs so that the kernel applies a consistent tolerance to differences in birth and lifetime. After normalizing each diagram by its maximum lifetime, the smaller lifetime bandwidth makes the kernel more tolerant of shifts in birth time while limiting the contribution of short-lived features to the response at long-lived features.

For entropy excess, the two-direction separation R2R^2 ranged from 0.7370.737 to 0.7510.751 when varying σb\sigma_b and from 0.6500.650 to 0.8540.854 when varying σℓ\sigma_\ell. Thus, this separation was more sensitive to the lifetime bandwidth in the spring–mass benchmark. The unexplained mass maintained a two-direction separation of R2>0.99R^2>0.99 throughout.

  1. Defining your probabilities in terms of the lifetime of a feature (Section 2.2) could do with some additional justification - you mention that long-lived features are less likely to be noise, but this seems to imply that we don't care at all about the birth times of the features. Maybe this is fine, but I'd like to see if you considered alternative approaches (I can see that this method is used in the existing literature so that's probably the reasoning, but it would be nice to know).

Our goal is to formulate a cross-entropy extension of standard persistent entropy. We therefore retain its event probabilities, which are proportional to feature lifetimes and give greater weight to longer-lived features. When comparing two diagrams, the similarity kernel uses both birth and lifetime coordinates, so birth times affect the induced probabilities and the resulting PCE.

The induced-probability construction also allows other normalized event probabilities, such as uniform probabilities or probabilities incorporating birth-dependent weights, and defines a corresponding cross-entropy for each choice. We use the probabilities from standard persistent entropy so that comparing a diagram with itself recovers its persistent entropy.

Thank you for your responses. On reflection, I think the paper's positioning as a rigorous formulation of persistent cross-entropy, with the experimental section demonstrating its use as a descriptive tool rather than claiming superiority of the methodology over existing methods, make it a useful addition to the literature. I will therefore raise my score.

As an aside (for general interest not a suggestion for the paper) I came across a recent paper ([1]) which uses persistent entropy to detect phase transitions - it seems to me that this could be a potential setting in which PCE could give more insight than their methodology.

[1] M. Gutierrez-del-Pozo, E. Paluzo-Hidalgo, M. Rucco, Persistent Entropy as a Detector of Phase Transitions

Reviewer_KGjGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose a cross entropy for persistence diagrams, motivated and adapted from the persistence entropy for a single persistence diagram. Persistent Cross Entropy is proposed by aligning the event space of two differerent persistence diagrams via the concept of unexplained event. Baseic properties of the induced probability and the stability of PCE are analyzed in a theoretical way. Experiments show that PCE can distinguish persistence diagrams wirh nearly identical PE. Causaliy in a linear dynamical system and PCE's application in knowledge distillsation are also considered.

Critical strengths
  1. The intuition of Persistent Cross Entropy, i.e., aligning support of different Persistence Diagrams, provides an explicit interpretation of the unmatched probability mass. And the joint local stability considers the case where both Persistence diagrams can vary. This establishes a sound math foundatation for the paper.
Critical weaknesses
  1. limited novelty and contribution. The proposed PCE is built on existing ideas of persistence weighting an kernel-based comparsion of persistence diagram points. Although the proposed induced probability has a interpretation w.r.t. information theory, the methodological advantage over existing methods is not clearly established. The direct comparsion with related symmetric methods like persistence landscape on the spring-mass benchmark is very limited, without comparison with more recent method like [1].
  2. The practical evaluation on the knowledge distillation is limited. The authors only compare PCE with simple RipsNet loss LPI\mathcal{L}_{PI}, without comparison with more effective non-topological alternative. And the dataset only involves CIFAR-100. This limited scope can not ground for the advantage of PCE.

[1] Murris J, Stolz B, Borgwardt K. From Persistence to Survival: Hypothesis Testing, Effect Sizes and Vectorisation for Topological Features[J]. arXiv preprint arXiv:2606.11911, 2026.

Questions for authors
  1. What distinction separates PCE from existing perisistence-weighted kernels, beyond its probability interpretation?
  2. What advantge does the direactional formulation provide over a symmetric method?
  3. Any more evidence on comparing PCE with non-topological losses and alternative losses using the same H_0 persistence diagram on more datasets? Such evidence would help show the practical advantage of PCE in a more comprehensive way.
1 comment

To the best of our knowledge, this is the first rigorous formulation of persistent cross entropy (PCE). Our main contribution is to extend persistent entropy to an ordered comparison between persistence diagrams through the induced-probability construction and establish its mathematical properties. We address your questions and concerns below.

  1. What distinction separates PCE from existing perisistence-weighted kernels, beyond its probability interpretation?



Our objective is to formulate a cross-entropy extension of persistent entropy, rather than to improve existing persistence-weighted kernel methods. Each persistence diagram has its own event space, so defining cross entropy requires a construction that brings the two probabilities onto a common event space. We introduce the induced probability pXYp_X^Y on DX∪{∂}D_X\cup\{\partial\} using information from DYD_Y. This makes persistent cross entropy well-defined and recovers standard persistent entropy when a diagram is compared with itself.

Although this construction uses persistence weighting and kernel similarities, DXD_X and DYD_Y play different roles. The event space and persistence probabilities come from DXD_X, while DYD_Y determines the induced probability. Exchanging the diagrams therefore generally changes PCE. This asymmetry enables the cross-entropy formulation, while the kernels and distances used in existing persistence-weighted kernel methods are symmetric.



  1. What advantge does the direactional formulation provide over a symmetric method?



The directional formulation is required by our objective of extending persistent entropy to cross entropy. Cross entropy and KL divergence assign different roles to their two distributions, and PCE preserves this ordered structure when comparing persistence diagrams. A symmetric score remains unchanged when the diagrams are exchanged, so it cannot by itself distinguish the two ordered comparisons. We deliberately chose experiments involving cause and effect in coupled dynamical systems and teacher and student in knowledge distillation to illustrate situations where this distinction matters. We therefore ask that the contribution be assessed in terms of the asymmetric comparison needed to formulate persistent cross entropy, beyond whether it improves performance over a symmetric distance.




  1. Any more evidence on comparing PCE with non-topological losses and alternative losses using the same H_0 persistence diagram on more datasets? Such evidence would help show the practical advantage of PCE in a more comprehensive way.



We will add experiments on benchmark datasets where the two diagrams have structurally different roles and PCE’s asymmetry is expected to be useful. We will compare PCE with existing symmetric methods using identical exact H0H_0 persistence diagrams and include non-topological baselines. These experiments will examine the practical benefit of preserving the direction of comparison in such settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.